huggingface / evaluate

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Home Page:https://huggingface.co/docs/evaluate

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Metrics that use DownloadConfig not working any more

BramVanroy opened this issue · comments

It seems that some metrics stopped working in some configurations with datasets. I am on datasets==2.11.0 and evaluate==0.4.0.

import evaluate

metric = evaluate.load("bleurt", "BLEURT-20")

But this leads to an error

model_path = dl_manager.download_and_extract(CHECKPOINT_URLS[checkpoint_name])
File ".venv\lib\site-packages\datasets\download\download_manager.py", line 564, in download_and_extract
return self.extract(self.download(url_or_urls))
File ".venv\lib\site-packages\datasets\download\download_manager.py", line 427, in download
downloaded_path_or_paths = map_nested(
    File ".venv\lib\site-packages\datasets\utils\py_utils.py", line 435, in map_nested
return function(data_struct)
File ".venv\lib\site-packages\datasets\download\download_manager.py", line 453, in _download
return cached_path(url_or_filename, download_config=download_config)
File ".venv\lib\site-packages\datasets\utils\file_utils.py", line 195, in cached_path
storage_options=download_config.storage_options,
AttributeError: 'DownloadConfig' object has no attribute 'storage_options'

Related: #440

This is "solved" by downgrading to datasets 2.10.0.

It should be fixed with #447, maybe we can do a release soon

@lhoestq Any chance of doing a new release?

cc @lvwerra who must have the pypi rights