whut09 / DeepLoader

Data loaders for deep learning

Deep Loader

Data processing is important in deep learning, it take much time and is very boring to convert data formats among different deep learning frameworks (like caffe, tensorflow, PyTorch, MxNet). The purpose of this project is to make data processing easier.

Main features of the library are:

• Data readers for many common data formats, specially for image classification
• Data batch prefetching in a background thread
• Face recognition evaluation tool , check run_verify for details, which can test models with LFW data
• Data adaptors for different frameworks
• Many utility functions
  • Face alignment
  • A multi-step learning rate scheduler with decay and warmup options
  • Fast image hashing functions like P/A/D/W-Hash, and our MHash
  • Speedometer for tracking program speed
• Tools
  • Image labeling tool for binary classification, it loads multiple images with a handy detail window.
  • Image duplicate removal tool to remove duplicate images
  • A video annotation format and HTML viewer
• ONNX
  • Fixed some bugs for convert MxNet models to ONNX x2onnx
  • Mode forward wrappers for different backends like (Pytorch/MxNet/ONNX) backends

Install

pip install -e .