Add deserializer support
armnnDeserializer
is a library for loading neural networks, that Arm NN FlatBuffers files define, into the Arm NN runtime.
To add a layer to the armnnDeserializer
:
- Declare and implement a
Parse<LayerName>()
function insrc/armnnDeserializer/Deserializer.hpp
andsrc/armnnDeserializer/Deserializer.cpp
. This function creates the layer and retrieves any necessary information about the layer, for example, descriptors. The following code shows theDeserializer.hpp
for an exampleSoftmaxLayer
: - Register the
Parse<LayerName>()
function inm_Parser_Functions
to map the layer to theenum
that is declared inArmnnSchema.fbs
. The following code shows this registration for an exampleSoftmaxLayer
://Softmax m_ParserFunctions(Layer_MAX+1, &Deserializer::ParseUnsupportedLayer) { // register supported layers m_ParserFunctions[Layer_SoftmaxLayer] = &Deserializer::ParseSoftmax; . . }
- Add the layer to the switch case in
Deserializer::GetBaseLayer
. This switch case retrieves the layer using a layer index from the network graph. The following code shows the addition of an exampleSoftmaxLayer
to a switch case://Softmax Deserializer::LayerBaseRawPtr Deserializer::GetBaseLayer(const GraphPtr& graphPtr, unsigned int layerIndex) { auto layerType = graphPtr->layers()->Get(layerIndex)->layer_type(); switch(layerType) { case Layer::Layer_SoftmaxLayer: return graphPtr->layers()->Get(layerIndex)->layer_as_SoftmaxLayer()->base(); . . } }
- Add a unit test for the serializer in
src/armnnSerializer/test/SerializerTests.cpp
. A serializer unit test creates a simple InputLayer→NewLayer→OutputLayer network. The unit test serializes that network to a string, deserializes the network, and passes the network an implementation of theLayerVerifierBase
, using the visitor pattern. To add the serializer unit test:- Create an implementation of the test class
LayerVerifierBase
orLayerVerifierBaseWithDescriptor
. Use the helper macro functionDECLARE_LAYER_VERIFIER_CLASS()
orDECLARE_LAYER_VERIFIER_CLASS_WITH_DESCRIPTOR
. The following code shows this implementation for an exampleSoftmaxLayer
:BOOST_AUTO_TEST_CASE(SerializeSoftmax) { DECLARE_LAYER_VERIFIER_CLASS_WITH_DESCRIPTOR(Softmax) ...
- Create a simple network that uses the layer being tested. The following code shows an example network that uses an example
SoftmaxLayer
:const std::string layerName("softmax"); const armnn::TensorInfo info({1, 10}, armnn::DataType::Float32); armnn::SoftmaxDescriptor descriptor; descriptor.m_Beta = 1.0f; armnn::INetworkPtr network = armnn::INetwork::Create(); armnn::IConnectableLayer* const inputLayer = network->AddInputLayer(0); armnn::IConnectableLayer* const softmaxLayer = network->AddSoftmaxLayer(descriptor, layerName.c_str()); armnn::IConnectableLayer* const outputLayer = network->AddOutputLayer(0); inputLayer->GetOutputSlot(0).Connect(softmaxLayer->GetInputSlot(0)); softmaxLayer->GetOutputSlot(0).Connect(outputLayer->GetInputSlot(0)); inputLayer->GetOutputSlot(0).SetTensorInfo(info); softmaxLayer->GetOutputSlot(0).SetTensorInfo(info);
- Serialize, then deserialize, the network. Then send the network the visitor test class, which requires deserialization support. The following example code serializes, then deserializes, the network and sends the network the visitor test class:
armnn::INetworkPtr deserializedNetwork = DeserializeNetwork(SerializeNetwork(*network)); BOOST_CHECK(deserializedNetwork); SoftmaxLayerVerifier verifier(layerName, {info}, {info}, descriptor); deserializedNetwork->Accept(verifier);
- Create an implementation of the test class
- Add a unit test for the new layer. To add the test, create a header file in
armnn/src/armnnDeserializer/test/Deserialize<Layer_Name>.hpp
and a source file inarmnn/src/armnnDeserializer/test/Deserialize<Layer_Name>.cpp
. In the unit test, write a JSON string that describes a small network that includes your layer. TheParserFlatbuffersSerializeFixture
class deserializes and tests this JSON string. Ensure that the header of this class is included in the deserializer unit test. You can generate the JSON string using the serializer test. See Step 6 for more information on generating the JSON string using the serializer test. You must create a JSON unit test as this type of test is reliable. - Write the JSON test. Writing this JSON test can be complicated and time consuming. With some changes, you can use Arm NN to create this test using the serialization test. To do this:
- Change the FlatBuffers in
CMakeLists.txt
to build as a shared library and rebuild the FlatBuffers. The following code shows how to make this change: - Change
libflatbuffers.a
tolibflatbuffers.so
inarmnn/GlobalConfig.cmake
. The following code shows the result of this change:# Flatbuffers support for TF Lite and Armnn Serializer if(BUILD_TF_LITE_PARSER OR BUILD_ARMNN_SERIALIZER) # verify we have a valid flatbuffers include path find_path(FLATBUFFERS_INCLUDE_PATH flatbuffers/flatbuffers.h HINTS ${FLATBUFFERS_ROOT}/include /usr/local/include /usr/include) message(STATUS "Flatbuffers headers are located at: ${FLATBUFFERS_INCLUDE_PATH}") find_library(FLATBUFFERS_LIBRARY NAMES libflatbuffers.so flatbuffers HINTS ${FLATBUFFERS_ROOT}/lib /usr/local/lib /usr/lib) message(STATUS "Flatbuffers library located at: ${FLATBUFFERS_LIBRARY}") endif()
- Add
idl.h
to the serializer include and add the following code to theSerialize()
function:#include <flatbuffers/idl.h> ... void Serializer::Serialize(const INetwork& inNetwork) { // Iterate through to network inNetwork.Accept(m_SerializerVisitor); flatbuffers::FlatBufferBuilder& fbBuilder = m_SerializerVisitor.GetFlatBufferBuilder(); // Create FlatBuffer SerializedGraph auto serializedGraph = serializer::CreateSerializedGraph( fbBuilder, fbBuilder.CreateVector(m_SerializerVisitor.GetSerializedLayers()), fbBuilder.CreateVector(m_SerializerVisitor.GetInputIds()), fbBuilder.CreateVector(m_SerializerVisitor.GetOutputIds()), m_SerializerVisitor.GetVersionTable()); // Serialize the graph fbBuilder.Finish(serializedGraph); // Code to be added, delete after use std::string schemafile; flatbuffers::LoadFile(("path/to/ArmnnSchema.fbs"), false, &schemafile); std::string jsongen; flatbuffers::Parser parser; parser.Parse(schemafile.c_str()); GenerateText(parser, fbBuilder.GetBufferPointer(), &jsongen); std::cout << jsongen; }
- Run the serialization test for the layer that you are creating. A working JSON output is printed. You can use this JSON file to make your deserializer test.
option(FLATBUFFERS_BUILD_SHAREDLIB "Enable the build of the flatbuffers shared library" ON)
- Change the FlatBuffers in
- Add the unit test file to
ml/armnn/CMakeLists.txt
://Softmax if(BUILD_ARMNN_SERIALIZER AND ARMNNREF) enable_language(ASM) list(APPEND unittest_sources src/armnnSerializer/test/ActivationSerializationTests.cpp src/armnnSerializer/test/SerializerTests.cpp src/armnnDeserializer/test/Deserialize<Layer_Name>.cpp . .
- Add the new supported layer to the deserializer documentation in
src/armnnDeserializer/
://Softmax The Arm NN SDK Deserialize parser currently supports the following layers: * Abs * Activation ... * Softmax .
// Softmax void ParseSoftmax(GraphPtr graph, unsigned int layerIndex);
The following code shows the Deserializer.cpp
for an example SoftmaxLayer
:
// Softmax void Deserializer::ParseSoftmax(GraphPtr graph, unsigned int layerIndex) { CHECK_LAYERS(graph, 0, layerIndex); Deserializer::TensorRawPtrVector inputs = GetInputs(graph, layerIndex); CHECK_VALID_SIZE(inputs.size(), 1); Deserializer::TensorRawPtrVector outputs = GetOutputs(graph, layerIndex); CHECK_VALID_SIZE(outputs.size(), 1); armnn::SoftmaxDescriptor descriptor; descriptor.m_Beta = graph->layers()->Get(layerIndex)->layer_as_SoftmaxLayer()->descriptor()->beta(); auto layerName = GetLayerName(graph, layerIndex); IConnectableLayer* layer = m_Network->AddSoftmaxLayer(descriptor, layerName.c_str()); armnn::TensorInfo outputTensorInfo = ToTensorInfo(outputs[0]); layer->GetOutputSlot(0).SetTensorInfo(outputTensorInfo); RegisterInputSlots(graph, layerIndex, layer); RegisterOutputSlots(graph, layerIndex, layer); }