Updating mne_bids version

[brunaafl]

Just as pointed at #581 (comment), moabb is using version 0.13 of mne_bids, which conflicts with mne if it's version is >= 1.7 since some functions on mne.io were removed.

I just changed version on poetry's pyproject file and reupdated the lock.

If mne's version is 1.6.1, it still accepts the modified imports from .io returning a warning, but not 1.7. My question is if it is prudent to also update mne's version to get rid of those warnings, but then needing to maybe change some other imports in other files.

Thanks for the attention!

[brunaafl]

@bruAristimunha

Just as pointed at #581 (comment), moabb is using version 0.13 of mne_bids, which conflicts with mne if it's version is >= 1.7 since some functions on mne.io were removed.

I just changed version on poetry's pyproject file and reupdated the lock.

If mne's version is 1.6.1, it still accepts the modified imports from .io returning a warning, but not 1.7. My question is if it is prudent to also update mne's version to get rid of those warnings, but then needing to maybe change some other imports in other files.

Thanks for the attention!

@bruAristimunha

[bruAristimunha]

Hey @brunaafl,

About the mne, I think we can pay this bill now and avoid future issues.

[brunaafl]

KeyErrors building docs. Is failing some auto examples. Ex:

WARNING: /home/runner/work/moabb/moabb/examples/plot_disk_cache.py failed to execute correctly: Traceback (most recent call last):
  File "/home/runner/work/moabb/moabb/examples/plot_disk_cache.py", line 130, in <module>
    _ = paradigm.get_data(dataset, subjects, cache_config=cache_config)
  File "/home/runner/work/moabb/moabb/moabb/paradigms/base.py", line 279, in get_data
    data = [
  File "/home/runner/work/moabb/moabb/moabb/paradigms/base.py", line 280, in <listcomp>
    dataset.get_data(
  File "/home/runner/work/moabb/moabb/moabb/datasets/base.py", line 343, in get_data
    data[subject] = self._get_single_subject_data_using_cache(
  File "/home/runner/work/moabb/moabb/moabb/datasets/base.py", line 508, in _get_single_subject_data_using_cache
    interface.erase()  # remove partial cache
  File "/home/runner/work/moabb/moabb/moabb/datasets/bids_interface.py", line 161, in erase
    path.rm(safe_remove=False)
  File "<decorator-gen-289>", line 12, in rm
  File "/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne_bids/path.py", line 711, in rm
    bids_path.copy()
  File "/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne_bids/path.py", line 12[42](https://github.com/NeuroTechX/moabb/actions/runs/8932160739/job/24550908772#step:9:43), in find_matching_sidecar
    return _find_matching_sidecar(
  File "/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne_bids/path.py", line 1836, in _find_matching_sidecar
    raise RuntimeError(msg)
RuntimeError: Did not find any scans.tsv associated with sub-1_ses-0_task-imagery_run-0_desc-bdedf68b003a1894e1a92bd91d0bbb7c.

The search_str was "/tmp/tmpi4jrw13t/MNE-BIDS-zhou2016/sub-1/**/sub-1_ses-0*scans.tsv"

Import error on tests below. As I could understand, mne 1.7 requires package edfio for writing files in EDF format. However, mne_bids 0.14 still uses EDFlib-Python and will add support for edfio just in 0.15 version (dev right now).

ERROR: test_cache_dataset (moabb.tests.datasets.Test_Datasets)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/mne/utils/check.py", line 399, in _soft_import
    mod = import_module(name)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'edfio'

[bruAristimunha]

@brunaafl, this issue related with stimulus is related with mne-bids version?

[brunaafl]

@brunaafl, this issue related with stimulus is related with mne-bids version?

Yes! As I could notice, the issue related to test_cache_dataset is directly related to incompatibility between mne_bids 0.14 and mne1.7. The problem related to Zhou dataset it could be, although I'm not 100% sure about the source of this error.

I think maybe the best way to proceed would be to go back to enforcing mne<=1.6.1 until the next mne_bids release?

[brunaafl]

I've changed the version of mne to 1.6.1, but not mne_bids (still 0.14). If the problem with Zhou dataset is related to mne_bids 0.14, the error on the example must persist. I'm just interested in checking it.

[bruAristimunha]

Hmmmm, all green

[sylvchev]

Everything seems ok, the doc build correctly, including Zhou examples.

[brunaafl]

That's interesting. I will try to investigate more the source of this problem

[brunaafl]

@brunaafl, this issue related with stimulus is related with mne-bids version?

Yes! As I could notice, the issue related to test_cache_dataset is directly related to incompatibility between mne_bids 0.14 and mne1.7. The problem related to Zhou dataset it could be, although I'm not 100% sure about the source of this error.

I think maybe the best way to proceed would be to go back to enforcing mne<=1.6.1 until the next mne_bids release?

In the end, reproducing the problem, I realized that it is actually a problem also related to the missing edfio package, causing failure while saving the data.

2024-05-16 16:30:01,869 INFO MainThread moabb.datasets.bids_interface Starting caching 'Zhou2016' sub-1 datatype-eeg desc-bdedf68
/home/brunalopes/PycharmProjects/moabb/moabb/datasets/bids_interface.py:349: RuntimeWarning: Converting data files to EDF format
  mne_bids.write_raw_bids(
2024-05-16 16:30:01,900 WARNING MainThread moabb.datasets.base Failed to save 'Zhou2016' sub-1 datatype-eeg desc-bdedf68 to BIDS format:

Importing edfio inside my environment solved the issue and the example finished running. If we want mne=1.7.0 as a requirement, it could be a provisional solution to add edfio as a requirement while mne_bids 0.15 is not released.

[bruAristimunha]

Hi @brunaafl!

Another person reported this issue. Could we integrate this PR this week?

[brunaafl]

Hi @brunaafl!

Another person reported this issue. Could we integrate this PR this week?

I think that, in the end, we still will need to specify python <3.12 because of scikeras dependency. Building the .toml file got me:

The current project's supported Python range (>=3.9) is not compatible with some of the required packages Python requirement:
  - scikeras requires Python >=3.8.0,<3.12.0, so it will not be satisfied for Python >=3.12.0

Because scikeras (0.12.0) requires Python >=3.8.0,<3.12.0
 and no versions of scikeras match >0.12.0,<0.13.0, scikeras is forbidden.
So, because moabb depends on scikeras (^0.12.0), version solving failed.

  • Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties
    
    For scikeras, a possible solution would be to set the `python` property to ">=3.9,<3.12.0"

    https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
    https://python-poetry.org/docs/dependency-specification/#using-environment-markers

[brunaafl]

Unexpected failing examples:
/home/runner/work/moabb/moabb/examples/plot_benchmark_DL.py failed leaving traceback:
Traceback (most recent call last):
  File "/home/runner/work/moabb/moabb/examples/plot_benchmark_DL.py", line 69, in <module>
    results = benchmark(
  File "/home/runner/work/moabb/moabb/moabb/benchmark.py", line 132, in benchmark
    pipeline_configs = parse_pipelines_from_directory(pipelines)
  File "/home/runner/work/moabb/moabb/moabb/pipelines/utils.py", line 112, in parse_pipelines_from_directory
    ppl = create_pipeline_from_config(config_dict["pipeline"])
  File "/home/runner/work/moabb/moabb/moabb/pipelines/utils.py", line 49, in create_pipeline_from_config
    instance = getattr(mod_optm, optm["name"])(**params_optm)
  File "/home/runner/work/moabb/moabb/.venv/lib/python3.9/site-packages/keras/src/optimizers/__init__.py", line 115, in __init__
    raise ImportError(
ImportError: `keras.optimizers.legacy` is not supported in Keras 3. When using `tf.keras`, to continue using a `tf.keras.optimizers.legacy` optimizer, you can install the `tf_keras` package (Keras 2) and set the environment variable `TF_USE_LEGACY_KERAS=True` to configure TensorFlow to use `tf_keras` when accessing `tf.keras`.

keras.optimizers.legacy are not available for Keras 3. I could use the suggestion given by the error and acess Keras 2 where these imports are needed, but I thought I could also update the pipeline codes for this example to use Keras 3 optimizers. What do you think it is best @bruAristimunha ?

[bruAristimunha]

Okay for upgrading @brunaafl!

[brunaafl]

Okay for upgrading @brunaafl!

Since I upgraded Keras version, I'm working now on migrating some code to Keras 3

[bruAristimunha]

thanks @brunaafl!