Add Android NN support
The Android NN driver is an implementation of the Android NN driver HAL which uses the Arm NN library to execute neural network interfaces.
Support for new operators is introduced in different Android versions. To indicate this, each Android version has a corresponding HAL policy:
- NeuralNetworks HAL 1.0 - The first HAL, present from Android OMR1
- NeuralNetworks HAL 1.1 - Present from Android P
- NeauralNetwork HAL 1.2 - Present from Android Q
The HalPolicy for each HAL version is in /android-nn-driver/1.X/HalPolicy.hpp
where X
represents the version.
Refer to the Android NeuralNetwork documentation to see which HAL policy supports a given operator. Related information includes a link to the Android NeuralNetwork documentation.
- Declare the
Convert<LayerName>
function to eachHalPolicy.hpp
that supports the operator. The following code shows an example declaration for aSoftmaxLayer
function toandroid-nn-driver/1.2/HalPolicy.hpp
: - Define a
Convert<LayerName>()
function in theHalPolicy.cpp
file for each HAL policy that supports the layer. Ensure that this function returns aConvert<LayerName>()
function that is shared with all HAL Policies that support this operator. The following code shows this addition and function creation for an exampleSoftmaxLayer
operator:bool HalPolicy::ConvertOperation(const Operation& operation, const Model& model, ConversionData& data) { switch (operation.type) { ... case V1_2::OperationType::SOFTMAX: return ConvertSoftmax(operation, model, data);... ... bool HalPolicy::ConvertSoftmax(const Operation& operation, const Model& model, ConversionData& data) { ALOGV("hal_1_2::HalPolicy::ConvertSoftmax()"); return ::ConvertSoftmax<hal_1_2::HalPolicy>(operation, model, data); }
- Implement the
Convert<LayerName>()
function in theConversionUtils.hpp
file that corresponds to the earliest HAL policy that supports the operator. The availableConversionUtils.hpp
files are: android-nn-driver/ConversionUtils.hpp
for HAL policy 1.0 and 1.1.android-nn-driver/ConversionUtils.hpp_1_2
for HAL policy 1.2,android-nn-driver/ConversionUtils_1_3.hpp
for HAL policy 1.3.- Run the Android tests for your operator. These tests are in
VTS/NeuralNetworks
. For example, run a VTS for theSoftmaxLayer
operator. The following example code shows starting the driver, running a test and the results that the test prints to the console:
private: static bool ConvertSoftmax(const Operation& operation, const Model& model, ConversionData& data);
The following code shows this implementation for an example SoftmaxLayer
operator:
template<typename HalPolicy, typename HalOperation = typename HalPolicy::Operation, typename HalModel = typename HalPolicy::Model> bool ConvertSoftmax(const HalOperation& operation, const HalModel& model, ConversionData& data) { using HalOperand = typename HalPolicy::Operand; using HalOperandType = typename HalPolicy::OperandType; ALOGV("HalPolicy::ConvertSoftmax()"); LayerInputHandle input = ConvertToLayerInputHandle<HalPolicy>(operation, 0, model, data); if (!input.IsValid()) { return Fail("%s: Operation has invalid inputs", __func__); } const HalOperand* outputOperand = GetOutputOperand<HalPolicy>(operation, 0, model); if (!outputOperand) { return Fail("%s: Operation has no outputs", __func__); } const TensorInfo& outputInfo = GetTensorInfoForOperand(*outputOperand); SoftmaxDescriptor desc; HalOperandType outputType = outputOperand->type; // Read beta value if (outputType == HalOperandType::TENSOR_FLOAT16) { Half value; if (!GetInputScalar<HalPolicy>(operation, 1, HalOperandType::FLOAT16, value, model, data)) { return Fail("%s: Operation has invalid inputs %d", __func__, outputType); } desc.m_Beta = static_cast<float>(value); } else { if (!GetInputFloat32<HalPolicy>(operation, 1, desc.m_Beta, model, data)) { return Fail("%s: Operation has invalid inputs %d", __func__, outputType); } } if (operation.inputs.size() > 2 && !GetInputScalar<HalPolicy>(operation, 2, HalOperandType::INT32, desc.m_Axis, model, data)) { return Fail("%s: Operation has invalid inputs", __func__); } bool isSupported = false; auto validateFunc = [&](const armnn::TensorInfo& outputInfo, bool& isSupported) { FORWARD_LAYER_SUPPORT_FUNC(__func__, IsSoftmaxSupported, data.m_Backends, isSupported, input.GetTensorInfo(), outputInfo, desc); }; if(IsDynamicTensor(outputInfo)) { isSupported = AreDynamicTensorsSupported(); } else { validateFunc(outputInfo, isSupported); } if (!isSupported) { return false; } IConnectableLayer* layer = data.m_Network->AddSoftmaxLayer(desc); assert(layer != nullptr); input.Connect(layer->GetInputSlot(0)); return SetupAndTrackLayerOutputSlot<HalPolicy>(operation, 0, *layer, model, data, nullptr, validateFunc); }
/vendor/bin/hw/android.hardware.neuralnetworks@1.2-service-armnn -v -c CpuRef & /data/nativetest64/VtsHalNeuralnetworksV1_2TargetTest/VtsHalNeuralnetworksV1_2TargetTest --hal_service_instance=android.hardware.neuralnetworks@1.2::IDevice/armnn --gtest_filter="NeuralnetworksHidlTest.softmax_v1_2" Note: Google Test filter = NeuralnetworksHidlTest.softmax_v1_2 [==========] Running 1 test from 1 test suite. [----------] Global test environment set-up. [----------] 1 test from NeuralnetworksHidlTest [ RUN ] NeuralnetworksHidlTest.softmax_v1_2 [ OK ] NeuralnetworksHidlTest.softmax_v1_2 (59 ms) [----------] 1 test from NeuralnetworksHidlTest (61 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test suite ran. (61 ms total) [ PASSED ] 1 test.