Extend the program

Part of what makes Mbed exciting is how easy it is to pull in dependencies to use in your program. In the next section of this guide, we will show how you can extend the program to make use of the LCD display on the STM32F7.

First, if the LCD driver library has not already been added, we will need to run a command to add the library to our project.

mbed add http://os.mbed.com/teams/ST/code/LCD_DISCO_F746NG/

Once the library has downloaded, we will work with the code and start making changes.

As mentioned, the file command_responder.cc is a convenient integration point. From here, we can extend the program to behave differently depending on which command is detected.

In this case, we will be extending the program to display different information on the LCD display for each command.

In tensorflow/lite/experimental/micro/examples/micro_speech/ open the following command in your text editor:

command_responder.cc

After the license comments, you should see the following code:

#include "tensorflow/lite/experimental/micro/examples/micro_speech/command_responder.h"

// The default implementation writes out the name of the recognized command
// to the error console. Real applications will want to take some custom
// action instead, and should implement their own versions of this function.
void RespondToCommand(tflite::ErrorReporter* error_reporter,
                      int32_t current_time, const char* found_command,
                      uint8_t score, bool is_new_command) {
  if (is_new_command) {
    error_reporter->Report("Heard %s (%d) @%dms", found_command, score,
                           current_time);
  }
}

In the preceding code, you can see how the argument is_new_command indicates whether a new command has been detected. Currently, when a command is detected, the program just writes a debug line to the serial connection. Let’s make things more interesting.

First, we will add a couple of lines to include and instantiate the LCD driver. Next, we will add some logic that checks the found_command pointer to determine which command was detected. In each case, we call lcd.Clear(). This call, clears the LCD and sets the background color to an appropriate value. We then draw a line of text to the LCD.

#include "tensorflow/lite/experimental/micro/examples/micro_speech/command_responder.h"
#include "LCD_DISCO_F746NG.h"
LCD_DISCO_F746NG lcd;

// The default implementation writes out the name of the recognized command
// to the error console. Real applications will want to take some custom
// action instead, and should implement their own versions of this function.
void RespondToCommand(tflite::ErrorReporter* error_reporter,
                      int32_t current_time, const char* found_command,
                      uint8_t score, bool is_new_command) {
  if (is_new_command) {
    error_reporter->Report("Heard %s (%d) @%dms", found_command, score,
                           current_time);
    if(*found_command == 'y') {
      lcd.Clear(0xFF0F9D58);
      lcd.DisplayStringAt(0, LINE(5), (uint8_t *)"Heard yes!", CENTER_MODE);
    } else if(*found_command == 'n') {
      lcd.Clear(0xFFDB4437);
      lcd.DisplayStringAt(0, LINE(5), (uint8_t *)"Heard no :(", CENTER_MODE);
    } else if(*found_command == 'u') {
      lcd.Clear(0xFFF4B400);
      lcd.DisplayStringAt(0, LINE(5), (uint8_t *)"Heard unknown", CENTER_MODE);
    } else {
      lcd.Clear(0xFF4285F4);
      lcd.DisplayStringAt(0, LINE(5), (uint8_t *)"Heard silence", CENTER_MODE);
    }

  }
}

Once you have updated the file, save it and then go back in the tensorflow directory

cd tensorflow/lite/experimental/micro/tools/make/gen/mbed_cortex-m4/prj/micro_speech/mbed 

and run the following command to rebuild the project:

mbed compile -m DISCO_F746NG -t GCC_ARM

Next, copy the binary to the USB storage on your device, using the same method that you used earlier.
Once the board finishes flashing, you should be able to say “yes” and “no” and see the LCD display "Heard yes!", as you can see in the following image:

Heard 'yes' image

Extra: The arguments passed to RespondToCommand contain interesting information about the recognition result. For example, score contains a number that indicates the probability that the result is correct. You might want to modify the code so that the display varies based on this information.

It is easy to change the behavior of our program by writing code, but is it difficult to modify the machine learning model itself? The answer is no, and the next section of this guide, Retrain the machine learning model, will show you how.

Previous Next