Image classification getting started - STM32MP1x

The purpose of this package is to enable image classification application on a STM32MPU board.

Directory contents

This repository is structured as follows:

Directory	Content
Application	Python application script + launch script
Resources	All the resources necessary for the application
Optimized_models	Place holder for optimized model from Developer Cloud
STM32MP1	Specific resources for STM32MP1x family + Readme
STM32MP2	Specific resources for STM32MP2x family + Readme
LICENSE.md	Application License file

Before you start

Hardware and Software environment

In order to run this image classification application examples you need to have the following hardware:

• STM32MP157F-DK2 discovery board
• [USB camera] no built-in camera

or

• STM32MP135F-DK discovery board
• [MB1897] GC2145 GalaxyCore camera module

Only these hardwares are supported for now

On software side, this getting started needs X-LINUX-AI expansion package for OpenSTLinux version v6.1.0.

All the information needed to install X-LINUX-AI on your board is available in the following wiki page:

• X-LINUX-AI expansion package : https://wiki.st.com/stm32mpu/wiki/Category:X-LINUX-AI_expansion_package

Deployment

For STM32MPU, application code example is provided as python script to facilitate and accelerate the deployment on the target.

Deploy application code

You should use the deploy.py script to automatically deploy the program on the target (if the hardware is connected to the network).

The deployment script of the model zoo is used to directly populate the target with all the needed files, resources and NN model. The script use the IP address provided in the configuration yaml file. The application source code is available Application\ directory and can be modified easily.

Getting started deep dive

The purpose of this package is to enable image classification application on a STM32MPU board.

Processing workflow

The software executes an image classification on each image captured by the camera. The framerate depends on each step of the processing workflow.

Captured_image: image from the camera

Network_Preprocess - 3 steps:

• ImageResize: rescale the image to fit the resolution needed by the network
• PixelFormatConversion: convert image format (usually RGB565) to fit the network color channels (RGB888 or Grayscale)
• PixelValueConversion: convert to pixel type used by the network (uint8 or int8)

HxWxC: Height, Width and Number of color channels, format defined by the neural network

Network_Inference: call tflite/onnx network

Network_Postprocess: Apply post-processing operations on NN model raw outputs to be able to display inference results on the display

Application workflow

The image classification application can be divided into four main parts :

• Data acquisition and preprocessing which is built with Gstreamer library
• Neural Network inference based on TFLite runtime or ONNX Runtime libraries depending on the model used.
• Neural Network post processing which is handled by common numerical computing tools like Numpy
• Graphical user interface : built with GTK3+ and Cairo libraries

Image processing

The frame captured by the camera is in a standard video format. As the neural network needs to receive a square-shaped image as input, for now only one solutions is provided to reshape the captured frame before running the inference

• ASPECT_RATIO_FIT: the frame is compacted to fit into a square with a side equal to the height of the captured frame. The aspect ratio is modified.

By default the interpolation type used for resizing the input image is simple bilinear interpolation

Limitations

• Supports only X-LINUX-AI latest version v6.1.0.