webdataset

webdataset's repositories

A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.

Language:PythonBSD-3-Clause2810 24 364

Fast and simple stream processing of files in tar files, useful for deep learning, big data, and many other applications.

Language:Go127 4 13

A high performance I/O library for deep learning in Julia, based on the PyTorch WebDataset library

Language:JuliaMIT12 30

conversions of standard pytorch datasets to webdataset format

Language:Jupyter Notebook100