webdataset

webdataset

Geek Repo

High Performance I/O for Large Scale Deep Learning

Github PK Tool:Github PK Tool

webdataset's repositories

webdataset

A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.

Language:PythonLicense:BSD-3-ClauseStargazers:1897Issues:22Issues:281

tarp

Fast and simple stream processing of files in tar files, useful for deep learning, big data, and many other applications.

WebDataset.jl

A high performance I/O library for deep learning in Julia, based on the PyTorch WebDataset library

Language:JuliaLicense:MITStargazers:9Issues:3Issues:0

wdsconversions

conversions of standard pytorch datasets to webdataset format

Language:Jupyter NotebookStargazers:0Issues:0Issues:0