mlfoundations / open_lm

A repository for research on medium sized language models.

Improve dataloading.

GeorgiosSmyrnis opened this issue 8 months ago · comments

GeorgiosSmyrnis commented 8 months ago

Some items that need to be addressed:

Clean up the code in data.py.
Make --dataset-resampled and --dataset-manifest the only possible options.
Make --accurate-total-tokens the default.

Achal Dave commented 7 months ago

Should we close this @GeorgiosSmyrnis? :)

GeorgiosSmyrnis commented 7 months ago

Improvement is a continuous process :)

But I agree, closing this thanks to #111 being merged.