Image recognition on Arm Cortex-M with CMSIS-NN

Build image recognition application

The last step is for you to include the neural network in the image acquisition application we have built earlier.

In order to achieve this, we need to make some modifications to ML-examples/cmsisnn-cifar10/camera_demo/camera_app/camera_app.cpp

# In ML-examples/cmsisnn-cifar10/camera_demo/camera_app/
# Rename the camera application 
mv camera_app.cpp camera_with_nn.cpp

First let's include the code that we have just generated through the previous code_gen.py in the camera_with_nn.cpp file.

#include "nn.h"

Add classification capabilities

We need to start by defining a few variables:

1. Label variable in order to convert our final prediction into something meaningful.
2. Output data, as the array that will contain the output of the NN.

const char* cifar10_label[] = {"Plane", "Car", "Bird", "Cat", "Deer", "Dog", "Frog", "Horse", "Ship", "Truck"};
q7_t output_data[10]; //10-classes

We will also need a function to obtain our top prediction out of the softmax:

int get_top_prediction(q7_t* predictions)
{
  int max_ind = 0;
  int max_val = -128;
  for(int i=0;i<10;i++) {
    if(max_val < predictions[i]) {
      max_val = predictions[i];
      max_ind = i;
    }
  }
  return max_ind;
}

Modify main function in camera_with_nn.cpp

It’s now time to add the neural network function call that has been generated by the code_gen.py.

// run neural network 
run_nn((q7_t*)resized_buffer, output_data);
// Softmax: to get predictions
arm_softmax_q7(output_data,IP1_OUT_DIM,output_data);

We also need to identify which one of the classes has the highest probability and for this we add the following:

int top_ind = get_top_prediction(output_data);

Finally, to test the output add code to print the prediction and the confidence:

sprintf(lcd_output_string,"  Prediction: %s       ",cifar10_label[top_ind]);
lcd.DisplayStringAt(0, LINE(8), (uint8_t *)lcd_output_string, LEFT_MODE);
sprintf(lcd_output_string,"  Confidence: %.1f%%   ",(output_data[top_ind]/127.0)*100.0);
lcd.DisplayStringAt(0, LINE(9), (uint8_t *)lcd_output_string, LEFT_MODE);