Semantic Segmentation Prediction

To use the prediction service, users must fill in the 'prediction' section of the user_config.yaml file like the prediction_config.yaml or as shown in the example below:

model:
   model_type: deeplab
   model_path: <path-to-a-Keras-or-TFLite-model-or-ONNX-file>           # Path to the model to use to make predictions

operation_mode: prediction

preprocessing:
   rescaling:
      scale: 1/127.5
      offset: -1
   resizing:
      interpolation: bilinear
      aspect_ratio: False
   color_mode: rgb

dataset:
   dataset_name: person_coco_2017_pascal_voc_2012
   class_names: ["background", "person"]
   prediction_path: ./datasets/my_photos/
prediction:
   target: host # host, stedgeai_host, stedgeai_n6

In the model section, users must provide the type and path to their model file using the model_type and model_path attributes, respectively. The model_type is deeplab. The model_path can point to either a Keras model file with a '.keras' filename extension (float model), a TensorFlow Lite model file with a '.tflite' filename extension (quantized model), or an ONNX model file (.onnx) quantized or not.

The dataset section requires users to provide the names of the classes using the class_names attribute, as there is no way to infer them. Or the classes_name_file pointing to a text file containing the class names.

Users must provide the path to the directory containing the images to predict using the prediction_path attribute, and if users are using a quantized TFLITE or ONNX model, they can decide to do the inferences with the classic Python interpreters (host -> by default), with the C code generated by stedgeai on the PC (stedgeai_host), or with the C code generated by stedgeai on the N6 board directly (stedgeai_n6) using the target attribute.

Finally, once all of these sections have been filled in, users can run the prediction service to generate predictions for their set of images.

Hydra and MLflow settings

The mlflow and hydra sections must always be present in the YAML configuration file. The hydra section can be used to specify the name of the directory where experiment directories are saved and/or the pattern used to name experiment directories. With the YAML code below, every time you run the Model Zoo, an experiment directory is created that contains all the directories and files created during the run. The names of experiment directories are all unique as they are based on the date and time of the run.

hydra:
   run:
      dir: ./tf/src/experiments_outputs/${now:%Y_%m_%d_%H_%M_%S}

The mlflow section is used to specify the location and name of the directory where MLflow files are saved, as shown below:

mlflow:
   uri: ./tf/src/experiments_outputs/mlruns

If you chose to modify the user_config.yaml, you can evaluate the model by running the following command from the UC folder:

python stm32ai_main.py 

If you chose to update the prediction_config.yaml and use it, then run the following command from the UC folder:

python stm32ai_main.py --config-path ./config_file_examples/ --config-name prediction_config.yaml

In the general section, there is an optional parameter display_figures set by default to False. If the user sets display_figures to True, then a figure with the original image and the segmentation prediction is displayed for each image in test_files_path. This allows for a visual analysis of the model behavior.

Of course, if display_figures is kept to False, nothing is displayed on screen. However, whatever display_figures value, a directory called 'prediction' is created under experiments_outputs/<date-and-time> where all the output figures are saved for further analysis or comparison.