Laib / CNTK

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit

The CNTK Documentation has all information on CNTK including setup, examples, etc. Give us feedback through these channels.

Latest news

2017-06-01. CNTK 2.0 The first production release of Cognitive Toolkit v.2.

Highlights:

• CNTK backend for Keras.
• Extremely fast binary convolution with Halide.
• Java API.
• A new set of NuGet Packages.
• Multiple bug fixes.

See more in the Release Notes. Get the Release from the CNTK Releases page.

2017-05-24. CNTK 2.0 Release Candidate 3 Release Candidate 3 is the final preview of Cognitive Toolkit v.2.0.

Highlights:

• API that were previously declared deprecated are now removed. See details in release notes.
• Introduction of CNTK Java API in experimental mode. See details in release notes.
• New operators like to_sequence and sequence.unpack.
• Support of convolution in 1D.
• Support of UDF serialization (available both in Python and native in C++).
• New tools (Crosstalk and RNN Conversion).
• Support of NVIDIA cuDNN v.6.0 when CNTK is built by the user from source code.
• A new set of NuGet Packages.
• Multiple bug fixes.

See more in the Release Notes. Get the Release from the CNTK Releases page.

2017-04-21. CNTK 2.0 Release Candidate 2 With Release Candidate 2 we reacted to customer feedback and improved/added features, functionality, and performance.

Highlights:

• New operators like pow, sequence.reduce_max, sequence.softmax.
• New feature for Linux source builds (GPU Direct RDMA support in distributed gradients aggregation, NCCL support for Python in V2 gradients aggregation).
• Support for Python 3.6 for source and binary installation; see here.
• UserMinibatchSource to write custom minibatch sources; see here.
• New C# APIs: class NDArrayView and methods, SetMaxNumCPUThreads(), GetMaxNumCPUThreads(), SetTraceLevel(), GetTraceLevel()
• A new set of NuGet Packages is provided with this Release.

The release notes contain an overview. Get the release from the CNTK Releases Page.

2017-03-31. V 2.0 Release Candidate 1 With Release Candidate 1 the Microsoft Cognitive Toolkit enters the final set of enhancements before release of the production version of CNTK v.2.0.

Highlights:

• The release candidate contains all changes and improvements introduced in CNTK 2.0 during beta phase.
• Enables Caffe-converted pretrained models on image classification including AlexNet, ResNet, VGG and BN-Inception.
• Slice now supports multiple-axis slicing.
• Improves performance and memory footprint
• Improvements in the device selection API.
• New Python model debugging functions.
• Improvements in Python and C# API. See the release notes for detailed description.
• New file names for CNTK libraries and dlls.

The release notes contain an overview. Get the release from the CNTK Releases Page.

See all news.

What is The Microsoft Cognitive Toolkit

The Microsoft Cognitive Toolkit (https://www.microsoft.com/en-us/research/product/cognitive-toolkit/), is a unified deep-learning toolkit that describes neural networks as a series of computational steps via a directed graph. In this directed graph, leaf nodes represent input values or network parameters, while other nodes represent matrix operations upon their inputs. CNTK allows to easily realize and combine popular model types such as feed-forward DNNs, convolutional nets (CNNs), and recurrent networks (RNNs/LSTMs). It implements stochastic gradient descent (SGD, error backpropagation) learning with automatic differentiation and parallelization across multiple GPUs and servers. CNTK has been available under an open-source license since April 2015. It is our hope that the community will take advantage of CNTK to share ideas more quickly through the exchange of open source working code.

Go to the CNTK Documentation for all information on CNTK including setup, examples, etc.

License: See LICENSE.md in the root of this repository for the full license information.

Tutorial: Microsoft Computational Network Toolkit (CNTK) @ NIPS 2015 Workshops

Blogs:

• Microsoft Computational Network Toolkit offers most efficient distributed deep learning computational performance
• Microsoft researchers win ImageNet computer vision challenge (December 2015)

Performance

Cognitive Toolkit (CNTK) provides significant performance gains compared to other toolkits click here for details. Here is a summary of findings by researchers at HKBU.

• CNTK’s LSTM performance is 5-10x faster than the other toolkits.
• For convolution (image tasks), CNTK is comparable, but note the authors were using CNTK 1.7.2, and current CNTK 2.0 beta 10 is over 30% faster than 1.7.2.
• For all networks, CNTK's performance was superior to TensorFlow performance.

Historically, CNTK has been a pioneer in optimizing performance on multi-GPU systems. We continue to maintain the edge (NVidia news at SuperComputing 2016 and CRAY at NIPS 2016).

CNTK was a pioneer in introducing scalability across multi-server multi-GPU systems. The figure below compares processing speed (frames processed per second) of CNTK to that of four other well-known toolkits. The configuration uses a fully connected 4-layer neural network (see our benchmark scripts) and an effective mini batch size (8192). All results were obtained on the same hardware with the respective latest public software versions as of Dec 3, 2015.

Citation

If you used this toolkit or part of it to do your research, please cite the work as:

Amit Agarwal, Eldar Akchurin, Chris Basoglu, Guoguo Chen, Scott Cyphers, Jasha Droppo, Adam Eversole, Brian Guenter, Mark Hillebrand, T. Ryan Hoens, Xuedong Huang, Zhiheng Huang, Vladimir Ivanov, Alexey Kamenev, Philipp Kranen, Oleksii Kuchaiev, Wolfgang Manousek, Avner May, Bhaskar Mitra, Olivier Nano, Gaizka Navarro, Alexey Orlov, Hari Parthasarathi, Baolin Peng, Marko Radmilac, Alexey Reznichenko, Frank Seide, Michael L. Seltzer, Malcolm Slaney, Andreas Stolcke, Huaming Wang, Yongqiang Wang, Kaisheng Yao, Dong Yu, Yu Zhang, Geoffrey Zweig (in alphabetical order), "An Introduction to Computational Networks and the Computational Network Toolkit", Microsoft Technical Report MSR-TR-2014-112, 2014.

Disclaimer

CNTK is in active use at Microsoft and constantly evolving. There will be bugs.

Microsoft Open Source Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [e-mail] with any additional questions or comments.