This document serves as an index for onnx-mlir documents.

About

• ONNXAI.md gives a short introduction to the project and how to get in touch (Slack channel).

Installation and working environment

• Prerequisite.md lists the software required to build onnx-mlir.
• Installation is covered by README.md, with OS-specific details in BuildOnLinuxOSX.md and BuildOnWindows.md.
• BuildONNX.md describes how to install third_party ONNX for backend tests or to regenerate ONNX operations.
• BuildStandalone.md describes how to build onnx-mlir as a standalone binary, without a full LLVM/MLIR build tree.
• Docker.md describes how to build and develop onnx-mlir using Docker, and DockerInDocker.md covers Docker-in-Docker support.
• Workflow.md describes how to contribute in the github environment.
• This guideline is used to keep documentation and code consistent.
• UpdatingLLVMCommit.md describes how to update the commit of LLVM that onnx-mlir depends on.
• A Jenkins monitor job was setup to help with updating LLVM commit. It locates the next commit we can update to without breaking ONNX-MLIR, as well as the commit that will break ONNX-MLIR. You can see the commit(s) here: s390x, ppc64le, amd64.

Supported ONNX Ops

• CPU support is covered here.
• NNPA support is covered here, with additional operations covered in SupportedOps-NNPA-supplement.md.

Development

• Onnx operations are represented with the ONNX dialect in onnx-mlir. Related dialects used in lowering are Krnl, ZHigh, and ZLow.
• This document tells you how to generate an ONNX operation into the ONNX dialect.
• After an ONNX model is imported into onnx-mlir, several graph-level transformations will be applied. These transformations include operation decomposition, constant propagation, shape inference, and canonicalization.
• Then the ONNX dialect is lowered to Krnl dialect. Documents describing specific lowering/optimization strategies live under optimization-onnx-lowering/: adding a fusion pattern, Conv lowering with Im2Col and MatMul, ConvTranspose decomposition, ConvTranspose with output_shape support, and GridSample bilinear 2D optimization. To help debugging and performance tuning, onnx-mlir supports instrumentation at the ONNX operand level.
• All the passes may be controlled with options.
• How to handle errors can be found here.
• How to support a new accelerator can be found here.
• How to analyze unknown dimensions and query their equality at compile time can be found here.
• How location info is maintained and used during transformation and debugging is covered in LocationInfo.md.
• How onnx-mlir handles the ONNX Sequence type is covered in SequenceType.md.

NNPA Accelerator

• AccelNNPAHowToUseAndTest.md describes how to build and test onnx-mlir with the NNPA accelerator.
• Quantization-NNPA.md describes quantization support on NNPA.
• JsonConfigFile-NNPA.md describes the NNPA-specific JSON configuration options for device placement and quantization.

Execution

The compiled ONNX model can be executed with either a C/C++ driver, python driver, or a java driver.

• UsingPyRuntime.md describes the Python runtime interfaces.
• BuildPyRuntimeLight.md describes building and using PyRuntimeC in lightweight mode.
• PythonPackage.md describes the installable onnxmlir Python package.
• RunTorchModel.md describes installing the torch_onnxmlir package to run a torch model via torch.compile().
• JsonConfigFile.md describes the general JSON configuration file for specifying compile options.

Testing and Debugging

The routine testing for onnx-mlir build is described in this document.

• TestingHighLevel.md covers build trouble-shooting and higher-level testing.
• DebuggingNumericalError.md describes how to debug numerical errors between onnx-mlir and a reference implementation.
• ProfileModel.md describes profiling a compiled ONNX model with utils/profile-model.py.
• PerformanceTesting.md describes gathering and analyzing runtime/compile-time performance statistics with RunONNXModel.py, --profile-ir/--profile-ir-with-sig, and utils/make-report.py.