huggingface / datasets

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

https://huggingface.co/docs/datasets

Repository from Github https://github.comhuggingface/datasets

huggingface/datasets Issues

deal with python `10_000` legal number in slice syntax
Closed 6 months ago1
Flexible Loader
Updated 6 months ago3
Adding argument to `_get_data_files_patterns`
Closed 6 months ago3
Label casting during `map` process is canceled after the `map` process
Updated 6 months ago4
load_dataset("livecodebench/code_generation_lite", version_tag="release_v2") TypeError: 'NoneType' object is not callable
Updated 6 months ago10
HF_DATASETS_CACHE ignored?
Updated 6 months ago1
Features.from_arrow_schema is destructive
Updated 6 months ago
Is it possible to shard a single-sharded IterableDataset?
Closed 6 months ago4
What is the canonical way to compress a Dataset?
Updated 6 months ago
IterableDataset's state_dict shard_example_idx is always equal to the number of samples in a shard
Updated 6 months ago
load_dataset with streaming hangs on parquet datasets
Updated 6 months ago1
function `load_dataset` can't solve folder path with regex characters like "[]"
Updated 6 months ago1
Adding Core Maintainer List to CONTRIBUTING.md
Closed 6 months ago3
IterableDataset raises FileNotFoundError instead of retrying
Updated 6 months ago5
`datasets.disable_caching` doesn't work
Updated 6 months ago2
Webdataset data format problem
Closed 6 months ago1
Custom split name with the web interface
Closed 6 months ago
Document the HF_DATASETS_CACHE env variable
Updated 6 months ago4
List of images behave differently on IterableDataset and Dataset
Closed 6 months ago2
Problems with local dataset after upgrade from 3.3.2 to 3.4.0
Updated 6 months ago1
Loading the `laion/filtered-wit` dataset in streaming mode fails on v3.4.0
Closed 6 months ago1
.add_faiss_index and .add_elasticsearch_index returns ImportError at Google Colab
Updated 6 months ago6
Cannot load data with different schemas from different parquet files
Closed 6 months ago2
Epochs shortened after resuming mid-epoch with Iterable dataset+StatefulDataloader(persistent_workers=True)
Closed 6 months ago5
pyarrow.lib.ArrowTypeError: Expected dict key of type str or bytes, got 'int'
Updated 6 months ago
Excessive warnings when resuming an IterableDataset+buffered shuffle+DDP.
Updated 6 months ago
index error when num_shards > len(dataset)
Updated 6 months ago1
`drop_last_batch` does not drop the last batch using IterableDataset + interleave_datasets + multi_worker
Updated 6 months ago2
Error in code "Time to slice and dice" from course "NLP Course"
Closed 6 months ago2
Issues with large Datasets
Updated 6 months ago4
`Dataset.map` ignores existing caches and remaps when ran with different `num_proc`
Updated 6 months ago2
Error splitting the input into NAL units.
Updated 6 months ago2
Row indexing a dataset with numpy integers
Updated 6 months ago1
DVC integration broken
Updated 6 months ago1
better correspondence between cached and saved datasets created using from_generator
Updated 7 months ago
pyarrow.lib.arrowinvalid: cannot mix list and non-list, non-null values with map function
Updated 7 months ago4
Import order crashes script execution
Updated 7 months ago
Shard Dataset at specific indices
Updated 7 months ago3
Documentation on multiple media files of the same type with WebDataset
Updated 7 months ago1
Synchronize parameters for various datasets
Updated 7 months ago2
Index Error Invalid Ket is out of bounds for size 0 for code-search-net/code_search_net dataset
Updated 7 months ago
Performance regression in `dataset.filter`
Closed 7 months ago3
Lazy loading of environment variables
Updated 7 months ago1
504 Gateway Timeout when uploading large dataset to Hugging Face Hub
Updated 7 months ago4
Using load_dataset with data_files and split arguments yields an error
Updated 7 months ago
Getting statistics about filtered examples
Closed 7 months ago2
push_to_hub payload too large error when using large ClassLabel feature
Updated 7 months ago1
OSError: [Errno 22] Invalid argument forbidden character
Closed 7 months ago2
AttributeError: module 'pyarrow.lib' has no attribute 'ListViewType'
Updated 7 months ago
Re-add py.typed
Updated 7 months ago