ONNX Tutorials

Open Neural Network Exchange (ONNX) is an open standard format for representing machine learning models. ONNX is supported by a community of partners who have implemented it in many frameworks and tools.

Getting ONNX models

• Pre-trained models (validated): Many pre-trained ONNX models are provided for common scenarios in the ONNX Model Zoo
• Pre-trained models (non-validated): Many pre-trained ONNX models are provided for common scenarios in the ONNX Model Zoo.
• Services: Customized ONNX models are generated for your data by cloud based services (see below)
• Convert models from various frameworks (see below)

Services

Below is a list of services that can output ONNX models customized for your data.

• Azure Custom Vision service
• Azure Machine Learning automated ML
• Lobe desktop app

Converting to ONNX format

Framework / Tool	Installation	Tutorial
Caffe	apple/coremltools and onnx/onnxmltools	Example
Caffe2	part of caffe2 package	Example
Chainer	chainer/onnx-chainer	Example
Cognitive Toolkit (CNTK)	built-in	Example
CoreML (Apple)	onnx/onnxmltools	Example
Keras	onnx/tensorflow-onnx	Example
LibSVM	onnx/onnxmltools	Example
LightGBM	onnx/onnxmltools	Example
MATLAB	Deep Learning Toolbox	Example
ML.NET	built-in	Example
MXNet (Apache)	part of mxnet package docs github	Example
PyTorch	part of pytorch package	Example1, Example2, export for Windows ML, Extending support
SciKit-Learn	onnx/sklearn-onnx	Example
SINGA (Apache) - Github (experimental)	built-in	Example
TensorFlow	onnx/tensorflow-onnx	Examples

Scoring ONNX Models

Once you have an ONNX model, it can be scored with a variety of tools.

Framework / Tool	Installation	Tutorial
Caffe2	Caffe2	Example
Cognitive Toolkit (CNTK)	built-in	Example
CoreML (Apple)	onnx/onnx-coreml	Example
MATLAB	Deep Learning Toolbox Converter	Documentation and Examples
Menoh	Github Packages or from Nuget	Example
ML.NET	Microsoft.ML Nuget Package	Example
MXNet (Apache) - Github	MXNet	API Example
ONNX Runtime	See onnxruntime.ai	Documentation
SINGA (Apache) - Github [experimental]	built-in	Example
Tensorflow	onnx-tensorflow	Example
TensorRT	onnx-tensorrt	Example
Windows ML	Pre-installed on Windows 10	API Tutorials - C++ Desktop App, C# UWP App Examples
Vespa.ai	Vespa Getting Started Guide	Real Time ONNX Inference Distributed Real Time ONNX Inference for Search and Passage Ranking

End-to-End Tutorials

Tutorials demonstrating how to use ONNX in practice for varied scenarios across frameworks, platforms, and device types

General

• AI-Serving
• AWS Lambda
• Cortex
• MXNet
  • MXNet Model Server
  • AWS SageMaker and MXNet
  • MXNet to ONNX to ML.NET with SageMaker, ECS and ECR - external link
• ONNX Runtime
  • ONNX Runtime Tutorials
  • Azure ML and ONNX Runtime

Mobile

• Converting SuperResolution model from PyTorch to Caffe2 with ONNX and deploying on mobile device
• Transferring SqueezeNet from PyTorch to Caffe2 with ONNX and to Android app
• Converting Style Transfer model from PyTorch to CoreML with ONNX and deploying to an iPhone
• Deploy ONNX Runtime on Mobile/Edge devices

ONNX Quantization

• HuggingFace Bert Quantization with ONNX Runtime

ONNX as an intermediary format

• Convert a PyTorch model to Tensorflow using ONNX

ONNX Custom Operators

• How to export Pytorch model with custom op to ONNX and run it in ONNX Runtime

Visualizing ONNX Models

• Netdrawer: Visualizing ONNX models
• Netron: Viewer for ONNX models
• Zetane: 3D visualizer for ONNX models and internal tensors

Other ONNX tools

• Verifying correctness and comparing performance
• Example of operating on ONNX protobuf
• Float16 <-> Float32 converter
• Version conversion

Application of ONNX

• Explainable AI for ONNX models

Contributing

We welcome improvements to the convertor tools and contributions of new ONNX bindings. Check out contributor guide to get started.

Use ONNX for something cool? Send the tutorial to this repo by submitting a PR.