Tuning performance with OpenCL tuner
You can use the Compute Library OpenCL tuner to find optimum values for GPU acceleration tuning parameters. This involves running the driver twice:
- Once in tuning mode to find a set of good parameters
- Once in normal mode, with the tuned parameters
For more information, please refer to Using the GPU tuner in Arm NN android-nn driver. We assume that the Android NN driver is already transferred onto your phone at
Follow these steps:
- Run the driver in tuning mode.
Tuning could take several minutes, particularly for deeper networks. This means that the OpenCL tuner is difficult to deploy in real-world applications. To overcome this issue, we have introduced three levels of tuning in OpenCL tuner:
- EXHAUSTIVE – This level offers peak performance with a high tuning time.
- RAPID – This level offers the shortest tuning time with reduced performance uplift.
- NORMAL – This level offers the balance between tuning time and a good approximation of the optimal performance.
We use NORMAL level in this example.
The tuned parameter output file must be at a writable location.
- Write the output to a file called
root@android: cd /data/local/tmp
- Create the
root@android: touch tuned_params
When the service starts in the background, you will see the PID of the service printed in the
stdout. Make a note of the PID. We will need to kill the service with this PID when the tuning process finishes.
- Start your app as normal. Because the tuning parameters are being selected, you will notice a delay, between one minute up to five minutes, when processing the frame. After the first frame is processed and displayed in the app, the tuning process is finished.
- Terminate the driver with the PID that you wrote down in step 3, using this code:
root@android: kill <PID>
If you need to find the PID again, run the following command, followed by
root@android: ps -Af | grep <driver>
You can also inspect the tuned parameters with:
root@android: cat tuned_params
- Restart the driver with the tuned parameters that were created in the
tuned_paramsfile, as you can see in the following code:
root@android: ./<driver> --cl-tuned-parameters-file tuned_params -c GpuAcc &
- Restart the app as usual. You should see a noticeable performance boost.