tensorpack / tensorpack

A Neural Net Training Interface on TensorFlow, with focus on speed + flexibility

Tensorpack is a neural network training interface based on graph-mode TensorFlow.

Features:

It's Yet Another TF high-level API, with the following highlights:

1. Focus on training speed.

• Speed comes for free with Tensorpack -- it uses TensorFlow in the efficient way with no extra overhead. On common CNNs, it runs training 1.2~5x faster than the equivalent Keras code. Your training can probably gets faster if written with Tensorpack.

• Scalable data-parallel multi-GPU / distributed training strategy is off-the-shelf to use. See tensorpack/benchmarks for more benchmarks.

2. Squeeze the best data loading performance of Python with tensorpack.dataflow.

• Symbolic programming (e.g. tf.data) does not offer the data processing flexibility needed in research. Tensorpack squeezes the most performance out of pure Python with various autoparallelization strategies.

3. Focus on reproducible and flexible research:

• Built and used by researchers, we provide high-quality reproducible implementation of papers.

4. It's not a model wrapper.

• There are too many symbolic function wrappers already. Tensorpack includes only a few common layers. You can use any TF symbolic functions inside Tensorpack, including tf.layers/Keras/slim/tflearn/tensorlayer/....

See tutorials and documentations to know more about these features.

Examples:

We refuse toy examples. Instead of showing tiny CNNs trained on MNIST/Cifar10, we provide training scripts that reproduce well-known papers.

We refuse low-quality implementations. Unlike most open source repos which only implement papers, Tensorpack examples faithfully reproduce papers, demonstrating its flexibility for actual research.

Vision:

• Train ResNet and other models on ImageNet
• Train Mask/Faster R-CNN on COCO object detection
• Unsupervised learning with Momentum Contrast (MoCo)
• Adversarial training with state-of-the-art robustness
• Generative Adversarial Network(GAN) variants, including DCGAN, InfoGAN, Conditional GAN, WGAN, BEGAN, DiscoGAN, Image to Image, CycleGAN
• DoReFa-Net: train binary / low-bitwidth CNN on ImageNet
• Fully-convolutional Network for Holistically-Nested Edge Detection(HED)
• Spatial Transformer Networks on MNIST addition
• Visualize CNN saliency maps

Reinforcement Learning:

• Deep Q-Network(DQN) variants on Atari games, including DQN, DoubleDQN, DuelingDQN.
• Asynchronous Advantage Actor-Critic(A3C) with demos on OpenAI Gym

Speech / NLP:

• LSTM-CTC for speech recognition
• char-rnn for fun
• LSTM language model on PennTreebank

Install:

Dependencies:

• Python 3.3+.
• Python bindings for OpenCV. (Optional, but required by a lot of features)
• TensorFlow ≥ 1.5
  • TF is not not required if you only want to use tensorpack.dataflow alone as a data processing library
  • When using TF2, tensorpack uses its TF1 compatibility mode. Note that a few examples in the repo are not yet migrated to support TF2.

pip install --upgrade git+https://github.com/tensorpack/tensorpack.git
# or add `--user` to install to user's local directories

Please note that tensorpack is not yet stable. If you use tensorpack in your code, remember to mark the exact version of tensorpack you use as your dependencies.

Citing Tensorpack:

If you use Tensorpack in your research or wish to refer to the examples, please cite with:

@misc{wu2016tensorpack,
  title={Tensorpack},
  author={Wu, Yuxin and others},
  howpublished={\url{https://github.com/tensorpack/}},
  year={2016}
}