huggingface / olm-datasets

Pipeline for pulling and processing online language model pretraining data from the web

huggingface/olm-datasets Issues

Moving large files using `mv`: /bin/mv: Argument list too long
Closed 10 months ago
CC data Language Splits
Updated a year ago3
load wikipedia failed when language is zh
Closed a year ago5
Resource used to produce this version of dataset?
Closed 2 years ago2
olm/wikipedia hangs on tiny wikipedia language
Closed 2 years ago8