Quantizing Neural Networks to 8-bit Using TensorFlow

Train or Fine-tune Your Model

Train your model using your training data and compare the accuracy with the original 32-bit network. The training process varies by model.

To fine-tune an existing model using quantization, load the weights from your trained model into a graph that you use the create_training_graph() function to prepare. The create_training_graph() function is described in the Prerequisites section. When you have loaded the weights, allow the model to continue training for a smaller number of steps. 

When the training is complete, compare the accuracy with the original 32-bit network.

For CifarNet, you can use the following commands to download the training data to /tmp/cifar10 and train the network for 100000 steps:

cd models/research/slim/
bash scripts/train_cifarnet_on_cifar10.sh

The fake quantization layers that tf.contrib.quantize.create_training_graph() adds become active after 90000 steps and ensure that the final network is fine-tuned to work with quantized weights. The training process creates a /tmp/cifarnet-model directory that contains the graph and checkpoint weights.

To view the training progress:

1. Run the following command into TensorBoard:

   tensorboard --logdir=/tmp/cifarnet-model/

2. Open port 6006 on the training server in a browser. If you’re training on your laptop or desktop, this is http://localhost:6006/.

Training takes approximately 40 minutes on a p3.2xlarge EC2 instance.

The final accuracy is approximately 85% for CifarNet.