huyhoang17 / tutorials

Tutorials for creating and using ONNX models

ONNX Tutorials

Open Neural Network Exchange (ONNX) is an open standard format for representing machine learning models. ONNX is supported by a community of partners who have implemented it in many frameworks and tools.

These images are available for convenience to get started with ONNX and tutorials on this page

• Docker image for ONNX and Caffe2/PyTorch
• Docker image for ONNX, ONNX Runtime, and various converters

Getting ONNX models

• Choose a pre-trained ONNX model from the ONNX Model Zoo. Many pre-trained ONNX models are provided for common scenarios.
• Use services like Azure Custom Vision service that generate customized ONNX models for your data
• Convert models from various frameworks (see below)

Converting to ONNX format

Framework / Tool	Installation	Tutorial
Caffe	apple/coremltools and onnx/onnxmltools	Example
Caffe2	part of caffe2 package	Example
Chainer	chainer/onnx-chainer	Example
Cognitive Toolkit (CNTK)	built-in	Example
CoreML (Apple)	onnx/onnxmltools	Example
Keras	onnx/keras-onnx	Example
LibSVM	onnx/onnxmltools	Example
LightGBM	onnx/onnxmltools	Example
MATLAB	Deep Learning Toolbox	Example
ML.NET	built-in	Example
MXNet (Apache)	part of mxnet package docs github	Example
PyTorch	part of pytorch package	Example, exporting different ONNX opsets, Extending support
SciKit-Learn	onnx/sklearn-onnx	Example
SINGA (Apache) - Github (experimental)	built-in	Example
TensorFlow	onnx/tensorflow-onnx	Examples

Scoring ONNX Models

Once you have an ONNX model, it can be scored with a variety of tools.

Framework / Tool	Installation	Tutorial
Caffe2	Caffe2	Example
Cognitive Toolkit (CNTK)	built-in	Example
CoreML (Apple)	onnx/onnx-coreml	Example
MATLAB	Deep Learning Toolbox Converter	Documentation and Examples
Menoh	Github Packages or from Nuget	Example
ML.NET	Microsoft.ML Nuget Package	Example
MXNet (Apache) - Github	MXNet	API Example
ONNX Runtime	Python (Pypi) - onnxruntime and onnxruntime-gpu C/C# (Nuget) - Microsoft.ML.OnnxRuntime and Microsoft.ML.OnnxRuntime.Gpu	APIs: Python, C#, C, C++ Examples - Python, C#, C
SINGA (Apache) - Github [experimental]	built-in	Example
Tensorflow	onnx-tensorflow	Example
TensorRT	onnx-tensorrt	Example
Windows ML	Pre-installed on Windows 10	API Tutorials - C++ Desktop App, C# UWP App Examples

End-to-End Tutorials

Serving

• Serving ONNX models with MXNet Model Server
• Serving ONNX models with ONNX Runtime Server
• ONNX model hosting with AWS SageMaker and MXNet
• Serving ONNX models with ONNX Runtime on Azure ML
  • FER Facial Expression Recognition
  • MNIST Handwritten Digits
  • Resnet50 Image Classification
• Inferencing ONNX models using ONNX Runtime Python API

ONNX as an intermediary format

• Convert a PyTorch model to Tensorflow using ONNX

Conversion to deployment

• Converting SuperResolution model from PyTorch to Caffe2 with ONNX and deploying on mobile device
• Transferring SqueezeNet from PyTorch to Caffe2 with ONNX and to Android app
• Converting Style Transfer model from PyTorch to CoreML with ONNX and deploying to an iPhone
• Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX
• MXNet to ONNX to ML.NET with SageMaker, ECS and ECR - external link
• Convert CoreML YOLO model to ONNX, score with ONNX Runtime, and deploy in Azure

Other ONNX tools

• Verifying correctness and comparing performance
• Visualizing an ONNX model (useful for debugging)
• Netron: a viewer for ONNX models
• Example of operating on ONNX protobuf
• Float16 <-> Float32 converter

Contributing

We welcome improvements to the convertor tools and contributions of new ONNX bindings. Check out contributor guide to get started.

Use ONNX for something cool? Send the tutorial to this repo by submitting a PR.