When you use the GPU backend, you can improve performance by tuning the application. To improve performance, you:

  1. Set the Arm NN SDK in tuning mode by doing the following:
    1. Create an ICLTunedParameters object using the ICLTunedParameters::Create method. Pass Mode::UpdateTunedParameters as the argument.
    2. Before you create the RunTime object, set the m_ClTunedParameters member on the CreationOptions struct using the tuned parameters object created in step 1.
  2. Optionally load any existing tuning data that you want to use to extend new models with using the IClTunedParameters::Load method.
  3. Run the model with the Arm NN SDK set in tuning mode by:
    1. Loading the model you want to tune the performance of into the RunTime object. For information on how to load the model, see Loading the model into the Arm NN SDK.
    2. Running inference at least once to tune the performance of the model. For information on how to run inference, see Running inference.
    3. Unloading the model.
    4. For every model you want to do performance tuning on, repeat steps 1-3.
    5. Saving the tuned parameters using the ICLTunedParameters::Save method.
Previous Next