asteroid-team / asteroid

The PyTorch-based audio source separation toolkit for researchers

Home Page:https://asteroid-team.github.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

KeyError: Options in training stage for recipes

joeoct93 opened this issue Β· comments

πŸ› Bug

When I tried to run a recipe (SuDORMRFNET and SuDORMRFImprovedNet), I always end up running into this problem in stage 2 like the following.

Stage 2: Training
Traceback (most recent call last):
File "/mnt/c/Project/asteroid/egs/librimix/SuDORMRFNet/train.py", line 140, in
arg_dic, plain_args = parse_args_as_dict(parser, return_plain_args=True)
File "/home/cchoi1022/cd/envs/asteroid/lib/python3.10/site-packages/asteroid/utils/parser_utils.py", line 135, in parse_args_as_dict
args_dic["main_args"] = args_dic["optional arguments"]
KeyError: 'optional arguments'

To Reproduce

cd /asteroid/egs/librimix/SuDORMRFImprovedNet
. ./run.sh

or
python train.py

Expected behavior

The training commences

Environment

Package versions

Run asteroid-versions and paste the output here:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 584, in _build_master
ws.require(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 901, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse('PyYAML>=5.4'), {'pytorch-lightning'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/cchoi1022/.local/bin/asteroid-versions", line 33, in
sys.exit(load_entry_point('asteroid', 'console_scripts', 'asteroid-versions')())
File "/home/cchoi1022/.local/bin/asteroid-versions", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.8/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/mnt/c/Project/asteroid/asteroid/scripts/asteroid_versions.py", line 5, in
import pytorch_lightning as pl
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/init.py", line 30, in
from pytorch_lightning.callbacks import Callback # noqa: E402
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/init.py", line 14, in
from pytorch_lightning.callbacks.base import Callback
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/base.py", line 25, in
from pytorch_lightning.utilities.types import STEP_OUTPUT
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/init.py", line 18, in
from pytorch_lightning.utilities.apply_func import move_data_to_device # noqa: F401
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/apply_func.py", line 29, in
from pytorch_lightning.utilities.imports import _compare_version, _TORCHTEXT_LEGACY
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/imports.py", line 22, in
import pkg_resources
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3254, in
def _initialize_master_working_set():
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3237, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3266, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 586, in _build_master
return cls._build_from_requirements(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 599, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse('PyYAML>=5.4'), {'pytorch-lightning'})
(asteroid) cchoi1022@DESKTOP-Q470KUQ:/mnt/c/Project/asteroid/egs/librimix/SuDORMRFImprovedNet$ pyYAML
pyYAML: command not found
(asteroid) cchoi1022@DESKTOP-Q470KUQ:/mnt/c/Project/asteroid/egs/librimix/SuDORMRFImprovedNet$ pip install --upgrade pyyaml
Requirement already satisfied: pyyaml in /home/cchoi1022/cd/envs/asteroid/lib/python3.10/site-packages (5.4.1)
Collecting pyyaml
Using cached PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
Installing collected packages: pyyaml
Attempting uninstall: pyyaml
Found existing installation: PyYAML 5.4.1
Uninstalling PyYAML-5.4.1:
Successfully uninstalled PyYAML-5.4.1
Successfully installed pyyaml-6.0
(asteroid) cchoi1022@DESKTOP-Q470KUQ:/mnt/c/Project/asteroid/egs/librimix/SuDORMRFImprovedNet$ asteroid-versions
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 584, in _build_master
ws.require(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 901, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse('PyYAML>=5.4'), {'pytorch-lightning'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/cchoi1022/.local/bin/asteroid-versions", line 33, in
sys.exit(load_entry_point('asteroid', 'console_scripts', 'asteroid-versions')())
File "/home/cchoi1022/.local/bin/asteroid-versions", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.8/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/mnt/c/Project/asteroid/asteroid/scripts/asteroid_versions.py", line 5, in
import pytorch_lightning as pl
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/init.py", line 30, in
from pytorch_lightning.callbacks import Callback # noqa: E402
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/init.py", line 14, in
from pytorch_lightning.callbacks.base import Callback
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/base.py", line 25, in
from pytorch_lightning.utilities.types import STEP_OUTPUT
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/init.py", line 18, in
from pytorch_lightning.utilities.apply_func import move_data_to_device # noqa: F401
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/apply_func.py", line 29, in
from pytorch_lightning.utilities.imports import _compare_version, _TORCHTEXT_LEGACY
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/imports.py", line 22, in
import pkg_resources
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3254, in
def _initialize_master_working_set():
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3237, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3266, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 586, in _build_master
return cls._build_from_requirements(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 599, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse('PyYAML>=5.4'), {'pytorch-lightning'})

Alternatively, if you cannot install Asteroid or have an old version that doesn't have the asteroid-versions script,
please output the output of:

pip freeze | egrep -i 'pytorch|torch|asteroid'

Additional info

Additional info (environment, custom script, etc...)

Did you try solving the issue?

I tried looking online for solutions, but could not find anything. Apparently this KeyError is telling me that "optional arguments" is not in the dictionary. But I simply installed asteroid, I don't understand why 'optional arguments' is not in the dictionary.

same here any tips?

Sorry, I don't have any bandwidth to debug this.
My tip : have a look at the other recipes and look at what's different in the argument parsing, you'll probably find our error

i also tried SuDORMRFImprovedNet recipe and it gives me the same error any more tips?

Key in argparse module for Python version above 3.10.10 changed from "optional arguments" into "options", hence two possible solutions:

  1. change in asteroid/utils/parser_utils.py args_dic["optional arguments"] to args_dic["options"]
  2. downgrade Python

Thanks for the info, that was my idea !

We can also restrict argparse's version which is much less restrictive than the Python version.

We can also restrict argparse's version which is much less restrictive than the Python version.

My bad, argparse is part of Python -_-

So, change to args_dic["options"] depending on python version here is probably the best. @jbartolewska would you like to do it ?