Supplementary information: model training
Prepare to build TensorFlow
The code that we used to train our voice model currently depends on some experimental operations that are only available when building TensorFlow from source. We must build TensorFlow.
The easiest way to build TensorFlow from source is to use Docker. Docker is a tool that enables you to run tasks on a virtual machine that is isolated from the rest of your computer, which makes dependency management easier. TensorFlow provides a custom docker image that can be used to build the toolchain from source.
The first step is to follow the instructions to install Docker.
When Docker is installed, run the following command to test that it works:
docker run hello-world
You should see a message starting with “Hello from Docker!”.
When you have installed Docker, use the following command to install the latest TensorFlow development Docker image. This contains the TensorFlow source:
docker pull tensorflow/tensorflow:devel
Visit TensorFlow Docker images for more information.
Now, run the following command to connect to your Docker instance and open a shell:
docker run -it -w /tensorflow_src -v $PWD:/mnt tensorflow/tensorflow:devel bash
You should now be on the command line of the TensorFlow Docker image, in the directory that contains the TensorFlow source code. You must issue the following commands to fetch the very latest code and install some required Python dependencies:
git rebase origin master
pip install -U --user pip six numpy wheel setuptools mock tensorflow_estimator
pip install -U --user keras_applications==1.0.6 --no-deps
pip install -U --user keras_preprocessing==1.0.5 --no-deps
We must now configure the build. Running the following command from the root of the TensorFlow repo starts configuration. You are asked several questions. Just hit return at every prompt to accept the default option.
Once configuration is done, we are ready to go.
Train the model
The following command builds TensorFlow from source and starts training.
The build takes several hours. To save time, you can download the
tiny_conv.pb and skip to the following Convert the model to the TensorFlow Lite format section.
bazel run -c opt --copt=-mavx2 --copt=-mfma tensorflow/examples/speech_commands:train -- --model_architecture=tiny_conv --window_stride=20 --preprocess=micro --wanted_words="up,down" --silence_percentage=25 --unknown_percentage=25 --quantize=1
Notice how the
wanted_words argument contains the words “up” and “down”. You can add any words that you like from the available ten to this field, separated by commas.
On older CPUs, you can leave out the
--copt arguments. These arguments are there to accelerate training on chips that support the extensions.
The process takes a couple of hours. While you wait, you can look at a more detailed overview of the speech model that we are training.
Freeze the model
We must perform a few extra steps to be able to run the model directly on our microcontroller. With your trained model, you should run the following command to create a single “frozen graph” file that represents the trained model.
We must provide our
wanted_words argument again.
bazel run tensorflow/examples/speech_commands:freeze -- --model_qqqcarchitecture=tiny_conv --window_stride=20 --preprocess=micro --wanted_words="up,down" --quantize=1 --output_file=/tmp/tiny_conv.pb --start_checkpoint=/tmp/speech_commands_train/tiny_conv.ckpt-18000
You now have a file,
/tmp/tiny_conv.pb, that represents the model. This is great, but because we deploy the model on a tiny device, we must do everything that we can to make it as small and simple as possible.
Convert the model to the TensorFlow Lite format
To obtain a converted model that can run on the microcontroller, we must run a conversion script, TensorFlow Lite converter. This tool uses clever tricks to make our model as small and efficient as possible and to convert it to a TensorFlow Lite FlatBuffer. To reduce the size of the model, we used a technique called quantization. All weights and activations in the model get converted from 32-bit floating point format to an 8-bit and fixed-point format. This conversion does not only reduce the size of the network, but also avoids floating-point computations, that are more computationally expensive.
Run the following command to perform the conversion:
bazel run tensorflow/lite/toco:toco -- --input_file=/tmp/tiny_conv.pb --output_file=/tmp/tiny_conv.tflite --input_shapes=1,49,40,1 --input_arrays=Reshape_1 --output_arrays='labels_softmax' --inference_type=QUANTIZED_UINT8 --mean_values=0 --std_values=9.8077
You should now have a
/tmp/tiny_conv.tflite file. We now must copy this file from our Docker instance to the host machine. To do this, run the following command:
cp /tmp/tiny_conv.tflite /mnt
This command places the file in the directory that you were in when you first ran the command to connect to Docker. For example, if you ran the command from
~/Desktop, the file is at
To leave the Docker instance and get back to your regular command line, type the following: