Build image recognition application
The last step is for you to include the neural network in the image acquisition application we have built earlier.
In order to achieve this, we need to make some modifications to ML-examples/cmsisnn-cifar10/camera_demo/camera_app/camera_app.cpp
# In ML-examples/cmsisnn-cifar10/camera_demo/camera_app/ # Rename the camera application mv camera_app.cpp camera_with_nn.cpp
First let's include the code that we have just generated through the previous code_gen.py in the camera_with_nn.cpp
file.
#include "nn.h"
Add classification capabilities
We need to start by defining a few variables:
- Label variable in order to convert our final prediction into something meaningful.
- Output data, as the array that will contain the output of the NN.
const char* cifar10_label[] = {"Plane", "Car", "Bird", "Cat", "Deer", "Dog", "Frog", "Horse", "Ship", "Truck"}; q7_t output_data[10]; //10-classes
We will also need a function to obtain our top prediction out of the softmax:
int get_top_prediction(q7_t* predictions) { int max_ind = 0; int max_val = -128; for(int i=0;i<10;i++) { if(max_val < predictions[i]) { max_val = predictions[i]; max_ind = i; } } return max_ind; }
Modify main function in camera_with_nn.cpp
It’s now time to add the neural network function call that has been generated by the code_gen.py.
// run neural network run_nn((q7_t*)resized_buffer, output_data); // Softmax: to get predictions arm_softmax_q7(output_data,IP1_OUT_DIM,output_data);
We also need to identify which one of the classes has the highest probability and for this we add the following:
int top_ind = get_top_prediction(output_data);
Finally, to test the output add code to print the prediction and the confidence:
sprintf(lcd_output_string," Prediction: %s ",cifar10_label[top_ind]); lcd.DisplayStringAt(0, LINE(8), (uint8_t *)lcd_output_string, LEFT_MODE); sprintf(lcd_output_string," Confidence: %.1f%% ",(output_data[top_ind]/127.0)*100.0); lcd.DisplayStringAt(0, LINE(9), (uint8_t *)lcd_output_string, LEFT_MODE);