allenai / cached_path

A file utility for accessing both local and remote files through a unified interface.

Home Page:https://cached-path.readthedocs.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Make installing for filetransfer scheme optional

SYDavidBrielbeck opened this issue · comments

Is your feature request related to a problem? Please describe.
I only need files from S3-Buckets and I don't want to install dependencies which handle file-transfers with e.g. google drive.

Describe the solution you'd like
To learn how poetry enables this, please refer to this link. In essence, each subgroup must be specified under dependencies in the pyproject.toml file. I suggest considering the following groups: s3, gc, hf, beaker, and full (which installs all subgroups). Additionally, it’s important to consider a more user-friendly error message if the imported package is not installed. You can use importlib or a try-catch approach to achieve this.

Describe alternatives you've considered
The only alternative i considerd is using a packaged for each file-transfer scheme .

Additional context

Hey @SYDavidBrielbeck we'd be happy to accept a PR for this!

When I have the time I will have a look into it.