Run graph on device

Inference on a compute device is performed using the EnqueueWorkload() function of the context.

This example code runs a single inference on the test image:

// Run a single inference on the test image
std::array<float, 10=""> output;
armnn::Status ret = context->EnqueueWorkload(networkIdentifier,
                                            MakeInputTensors(inputBindingInfo, &input->image[0]),
                                            MakeOutputTensors(outputBindingInfo, &output[0]));</float,>

Here the input and output tensors are bound to data and the loaded network identifier is selected. The result of the inference can be read directly from the output array and compared to the MnistImage label that we read from the data file:

// Convert 1-hot output to an integer label and print
int label = std::distance(output.begin(), std::max_element(output.begin(), output.end()));
std::cout << "Predicted: " << label << std::endl;
std::cout << "   Actual: " << input->label << std::endl;

In this case, the std::distance function is used to find the index of the largest element in the output. This function is the equivalent to NumPy's argmax() function.

Previous Next