Tuning performance with OpenCL tuner

You can use the Compute Library OpenCL tuner to find optimum values for GPU acceleration tuning parameters. This involves running the driver twice:

  • Once in tuning mode to find a set of good parameters
  • Once in normal mode, with the tuned parameters

For more information, please refer to Using the GPU tuner in Arm NN android-nn driver. We assume that the Android NN driver is already transferred onto your phone at /data/local/tmp.

Follow these steps:

  1. Run the driver in tuning mode.

    Tuning could take several minutes, particularly for deeper networks. This means that the OpenCL tuner is difficult to deploy in real-world applications. To overcome this issue, we have introduced three levels of tuning in OpenCL tuner:
    • EXHAUSTIVE – This level offers peak performance with a high tuning time.
    • RAPID – This level offers the shortest tuning time with reduced performance uplift.
    • NORMAL – This level offers the balance between tuning time and a good approximation of the optimal performance.

    We use NORMAL level in this example.

    The tuned parameter output file must be at a writable location.
  2. Write the output to a file called tuned_params at /data/local/tmp:
    root@android: cd /data/local/tmp
  3. Create the tuned_params file:
    root@android: touch tuned_params

    When the service starts in the background, you will see the PID of the service printed in the stdout. Make a note of the PID. We will need to kill the service with this PID when the tuning process finishes.
  4. Start your app as normal. Because the tuning parameters are being selected, you will notice a delay, between one minute up to five minutes, when processing the frame. After the first frame is processed and displayed in the app, the tuning process is finished.
  5. Terminate the driver with the PID that you wrote down in step 3, using this code:
    root@android: kill <PID>

    If you need to find the PID again, run the following command, followed by “kill <PID>”:
    root@android: ps -Af | grep <driver>

    You can also inspect the tuned parameters with:
    root@android: cat tuned_params
  6. Restart the driver with the tuned parameters that were created in the tuned_params file, as you can see in the following code:
    root@android: ./<driver> --cl-tuned-parameters-file tuned_params -c GpuAcc &
  7. Restart the app as usual. You should see a noticeable performance boost.
Previous Next