Compile onnx model for your target machine

Checkout mnist.ir

Step 1:

Generate intermediate code

% onnx2cpp mnist.onnx

Step 2:

Optimize and compile

% /usr/bin/clang++-8 -O3 mnist.cpp -I ../../../include/ -isystem ../../../packages/eigen-eigen-323c052e1731/ -o mnist.exe

Step 3:

Test run

% ./mnist.exe

Step 4:

Run it in the loop

% python demo.mnist.py

More Info

1. batch size 1
2. pytorch version 1.2
3. ONNX IR version 1.4
4. Training colab notebook
5. Watch this video