Giters

transformerlab / transformerlab-app

Open Source Application for Advanced LLM Engineering: interact, train, fine-tune, and evaluate large language models on your own computer.

Home Page:https://transformerlab.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Downloading datasets throws error saying need to set trust_remote_code=True

dadmobile opened this issue · comments

commented

/home/ali/miniconda3/envs/transformerlab/lib/python3.11/site-packages/datasets/load.py:1454: FutureWarning: The repository for samsum contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/samsum You can avoid this message in future by passing the argument trust_remote_code=True. Passing trust_remote_code=True will be mandatory to load this dataset from the next major release of datasets. warnings.warn(

commented

This is also true for wikipedia dataset:

/Users/tony/.transformerlab/envs/transformerlab/lib/python3.11/site-packages/datasets/load.py:1461: FutureWarning: The repository for wikipedia contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/wikipedia
You can avoid this message in future by passing the argument trust_remote_code=True.

commented

This should be fixed on dataset page and in all training plugins.