jmrinaldi / tensorpack

A Neural Net Training Interface on TensorFlow

tensorpack

A neural net training interface based on TensorFlow.

See some examples to learn about the framework:

Vision:

• Multi-GPU training of ResNet on ImageNet
• Generative Adversarial Network(GAN) variants, including DCGAN, InfoGAN, Conditional GAN, WGAN, BEGAN, DiscoGAN, Image to Image, CycleGAN.
• DoReFa-Net: train binary / low-bitwidth CNN on ImageNet
• Fully-convolutional Network for Holistically-Nested Edge Detection(HED)
• Spatial Transformer Networks on MNIST addition
• Visualize CNN saliency maps
• Similarity learning on MNIST

Reinforcement Learning:

• Deep Q-Network(DQN) variants on Atari games, including DQN, DoubleDQN, DuelingDQN.
• Asynchronous Advantage Actor-Critic(A3C) with demos on OpenAI Gym

Speech / NLP:

• LSTM-CTC for speech recognition
• char-rnn for fun
• LSTM language model on PennTreebank

Examples are not only for demonstration of the framework -- you can train them and reproduce the results in papers.

Features:

It's Yet Another TF wrapper, but different in:

1. Not focus on models.

   • There are already too many symbolic function wrappers. Tensorpack includes only a few common models, but you can use any other wrappers within tensorpack, such as sonnet/Keras/slim/tflearn/tensorlayer/....

2. Focus on training speed.

   • Speed comes for free with tensorpack -- it uses TensorFlow in the correct way. Even on a tiny CNN example, the training runs 1.6x faster than the equivalent Keras code.

   • Data-parallel multi-GPU training is off-the-shelf to use. It is as fast as Google's official benchmark.

   • Data-parallel distributed training is off-the-shelf to use. It is as slow as Google's official benchmark.

3. Focus on large datasets.

   • It's painful to read/preprocess data through TF. tensorpack helps you load large datasets (e.g. ImageNet) in pure Python with autoparallelization.

4. Interface of extensible Callbacks. Write a callback to implement everything you want to do apart from the training iterations, and enable it with one line of code. Common examples include:

   • Change hyperparameters during training
   • Print some tensors of interest
   • Monitor GPU utilization
   • Send error rate to your phone

See tutorials to know more about these features.

Install:

Dependencies:

• Python 2 or 3
• TensorFlow >= 1.0.0 (>=1.1.0 for Multi-GPU)
• Python bindings for OpenCV (Optional, but required by a lot of features)

pip install -U git+https://github.com/ppwwyyxx/tensorpack.git
# or add `--user` to avoid system-wide installation.

Besides, if you only want to use tensorpack.dataflow alone as a data processing library, TensorFlow is also optional.