Issue when running Readme example
zaccharieramzi opened this issue · comments
Hi!
I encounter the following error when running the readme example:
Traceback (most recent call last):
File "/home/zaccharie/workspace/tests/venv/bin/benchopt", line 8, in <module>
sys.exit(main())
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/benchopt/cli.py", line 80, in run
validate_dataset_patterns(benchmark, dataset_names)
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/benchopt/utils/checkers.py", line 35, in validate_dataset_patterns
datasets = list_benchmark_datasets(benchmark)
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/benchopt/util.py", line 104, in list_benchmark_datasets
return _list_benchmark_classes(benchmark_dir, 'Dataset')
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/benchopt/util.py", line 91, in _list_benchmark_classes
classes.append(_load_class_from_module(module_filename, class_name))
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/benchopt/utils/dynamic_modules.py", line 46, in _load_class_from_module
module = _get_module_from_file(module_filename)
File "/home/zaccharie/workspace/tests/venv/lib/python3.6/site-packages/benchopt/utils/dynamic_modules.py", line 22, in _get_module_from_file
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "benchmark_lasso/datasets/simulated.py", line 3, in <module>
from benchopt import BaseDataset
ImportError: cannot import name 'BaseDataset'
From what I understand it's just a problem of installing the development version of benchopt rather than the pypi one.
The CI is done with the development version that's why this error is not caught.
Maybe there should be an addition to the Readme saying that if there are any errors in the execution of the examples, the development version of benchopt should be used (with instructions on how to do so). I can submit a PR for this if you want.
Actually, even with the development version, I still get an error, so it might not be that simple:
/tmp/tmpri_ov0ia:2: no file or folder of this type: benchopt.benchmark.Benchmark
@agramfort yes yes but I mean we might add that to the Readme.
Re the PR: yes
Re the local: it improves things, at least the benchmarks are running now (nice colors) and the results are saved in csv, but I still have an error (for the plotting):
Traceback (most recent call last):
File "/home/zaccharie/workspace/benchmark_lasso/venv/bin/benchopt", line 33, in <module>
sys.exit(load_entry_point('benchopt', 'console_scripts', 'benchopt')())
File "/home/zaccharie/workspace/benchmark_lasso/venv/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/zaccharie/workspace/benchmark_lasso/venv/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/zaccharie/workspace/benchmark_lasso/venv/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/zaccharie/workspace/benchmark_lasso/venv/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/zaccharie/workspace/benchmark_lasso/venv/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/zaccharie/workspace/benchOpt/benchopt/cli.py", line 107, in run
timeout=timeout, plot_result=not no_plot, pdb=pdb
File "/home/zaccharie/workspace/benchOpt/benchopt/runner.py", line 391, in run_benchmark
plot_benchmark(df, benchmark)
File "/home/zaccharie/workspace/benchOpt/benchopt/viz/__init__.py", line 46, in plot_benchmark
output_dir = _get_output_folder(benchmark)
File "/home/zaccharie/workspace/benchOpt/benchopt/utils/files.py", line 5, in _get_output_folder
output_dir = Path(benchmark) / "outputs"
File "/usr/lib/python3.6/pathlib.py", line 1001, in __new__
self = cls._from_parts(args, init=False)
File "/usr/lib/python3.6/pathlib.py", line 656, in _from_parts
drv, root, parts = self._parse_args(args)
File "/usr/lib/python3.6/pathlib.py", line 640, in _parse_args
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not Benchmark
If you suspect this is an IPython 7.16.1 bug, please report it at:
https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@python.org
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.
Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
%config Application.verbose_crash=True
I can try to look into it.
Ok everythiing should be running from master now (both local and in conda env)
Let me know if you see other issues! Thx for the report
Note that you need to install pygithub which is a new dependency for the package (to publish the benchmark results).
Everything working like a charm, do not take into account my last comment on the PR you just merged (deleted it).
Re pygithub, yes indeed, but just reinstalled the whole benchopt with dependencies anyway.
Closing this.