AI and Machine Learning on Arm | Add deserializer support

Add deserializer support

armnnDeserializer is a library for loading neural networks, that Arm NN FlatBuffers files define, into the Arm NN runtime.

To add a layer to the armnnDeserializer:

Declare and implement a Parse<LayerName>() function in src/armnnDeserializer/Deserializer.hpp and src/armnnDeserializer/Deserializer.cpp. This function creates the layer and retrieves any necessary information about the layer, for example, descriptors. The following code shows the Deserializer.hpp for an example SoftmaxLayer:

// Softmax
    void ParseSoftmax(GraphPtr graph, unsigned int layerIndex);

The following code shows the Deserializer.cpp for an example SoftmaxLayer:

// Softmax

void Deserializer::ParseSoftmax(GraphPtr graph, unsigned int layerIndex)
{
    CHECK_LAYERS(graph, 0, layerIndex);

    Deserializer::TensorRawPtrVector inputs = GetInputs(graph, layerIndex);
    CHECK_VALID_SIZE(inputs.size(), 1);

    Deserializer::TensorRawPtrVector outputs = GetOutputs(graph, layerIndex);
    CHECK_VALID_SIZE(outputs.size(), 1);

    armnn::SoftmaxDescriptor descriptor;
    descriptor.m_Beta = graph->layers()->Get(layerIndex)->layer_as_SoftmaxLayer()->descriptor()->beta();
    auto layerName = GetLayerName(graph, layerIndex);

    IConnectableLayer* layer = m_Network->AddSoftmaxLayer(descriptor, layerName.c_str());

    armnn::TensorInfo outputTensorInfo = ToTensorInfo(outputs[0]);
    layer->GetOutputSlot(0).SetTensorInfo(outputTensorInfo);

    RegisterInputSlots(graph, layerIndex, layer);
    RegisterOutputSlots(graph, layerIndex, layer);
}

Register the Parse<LayerName>() function in m_Parser_Functions to map the layer to the enum that is declared in ArmnnSchema.fbs. The following code shows this registration for an example SoftmaxLayer:

//Softmax
m_ParserFunctions(Layer_MAX+1, &Deserializer::ParseUnsupportedLayer)
{
    // register supported layers
    m_ParserFunctions[Layer_SoftmaxLayer]                = &Deserializer::ParseSoftmax;    .
    .
}

Add the layer to the switch case in Deserializer::GetBaseLayer. This switch case retrieves the layer using a layer index from the network graph. The following code shows the addition of an example SoftmaxLayer to a switch case:

//Softmax
Deserializer::LayerBaseRawPtr Deserializer::GetBaseLayer(const GraphPtr& graphPtr, unsigned int layerIndex)
{
    auto layerType = graphPtr->layers()->Get(layerIndex)->layer_type();
    switch(layerType)
    {
       case Layer::Layer_SoftmaxLayer:
            return graphPtr->layers()->Get(layerIndex)->layer_as_SoftmaxLayer()->base(); 
       .
       .
    }
}

Add a unit test for the serializer in src/armnnSerializer/test/SerializerTests.cpp. A serializer unit test creates a simple InputLayer→NewLayer→OutputLayer network. The unit test serializes that network to a string, deserializes the network, and passes the network an implementation of the LayerVerifierBase, using the visitor pattern. To add the serializer unit test:

Create an implementation of the test class LayerVerifierBase or LayerVerifierBaseWithDescriptor. Use the helper macro function DECLARE_LAYER_VERIFIER_CLASS() or DECLARE_LAYER_VERIFIER_CLASS_WITH_DESCRIPTOR. The following code shows this implementation for an example SoftmaxLayer:
```
BOOST_AUTO_TEST_CASE(SerializeSoftmax)
{
    DECLARE_LAYER_VERIFIER_CLASS_WITH_DESCRIPTOR(Softmax)

... 
```

Create a simple network that uses the layer being tested. The following code shows an example network that uses an example SoftmaxLayer:

const std::string layerName("softmax");
    const armnn::TensorInfo info({1, 10}, armnn::DataType::Float32);

    armnn::SoftmaxDescriptor descriptor;
    descriptor.m_Beta = 1.0f;

    armnn::INetworkPtr network = armnn::INetwork::Create();
    armnn::IConnectableLayer* const inputLayer   = network->AddInputLayer(0);
    armnn::IConnectableLayer* const softmaxLayer = network->AddSoftmaxLayer(descriptor, layerName.c_str());
    armnn::IConnectableLayer* const outputLayer  = network->AddOutputLayer(0);

    inputLayer->GetOutputSlot(0).Connect(softmaxLayer->GetInputSlot(0));
    softmaxLayer->GetOutputSlot(0).Connect(outputLayer->GetInputSlot(0));

    inputLayer->GetOutputSlot(0).SetTensorInfo(info);
    softmaxLayer->GetOutputSlot(0).SetTensorInfo(info);

Serialize, then deserialize, the network. Then send the network the visitor test class, which requires deserialization support. The following example code serializes, then deserializes, the network and sends the network the visitor test class:

armnn::INetworkPtr deserializedNetwork = DeserializeNetwork(SerializeNetwork(*network));
    BOOST_CHECK(deserializedNetwork);

    SoftmaxLayerVerifier verifier(layerName, {info}, {info}, descriptor);
    deserializedNetwork->Accept(verifier);

Add a unit test for the new layer. To add the test, create a header file in armnn/src/armnnDeserializer/test/Deserialize<Layer_Name>.hpp and a source file in armnn/src/armnnDeserializer/test/Deserialize<Layer_Name>.cpp. In the unit test, write a JSON string that describes a small network that includes your layer. The ParserFlatbuffersSerializeFixture class deserializes and tests this JSON string. Ensure that the header of this class is included in the deserializer unit test. You can generate the JSON string using the serializer test. See Step 6 for more information on generating the JSON string using the serializer test. You must create a JSON unit test as this type of test is reliable.

Write the JSON test. Writing this JSON test can be complicated and time consuming. With some changes, you can use Arm NN to create this test using the serialization test. To do this:

Change the FlatBuffers in CMakeLists.txt to build as a shared library and rebuild the FlatBuffers. The following code shows how to make this change:

option(FLATBUFFERS_BUILD_SHAREDLIB
       "Enable the build of the flatbuffers shared library"
       ON)

Change libflatbuffers.a to libflatbuffers.so in armnn/GlobalConfig.cmake. The following code shows the result of this change:

# Flatbuffers support for TF Lite and Armnn Serializer
if(BUILD_TF_LITE_PARSER OR BUILD_ARMNN_SERIALIZER)
    # verify we have a valid flatbuffers include path
    find_path(FLATBUFFERS_INCLUDE_PATH flatbuffers/flatbuffers.h
              HINTS ${FLATBUFFERS_ROOT}/include /usr/local/include /usr/include)
 
    message(STATUS "Flatbuffers headers are located at: ${FLATBUFFERS_INCLUDE_PATH}")
 
    find_library(FLATBUFFERS_LIBRARY
                 NAMES libflatbuffers.so flatbuffers
                 HINTS ${FLATBUFFERS_ROOT}/lib /usr/local/lib /usr/lib)
 
    message(STATUS "Flatbuffers library located at: ${FLATBUFFERS_LIBRARY}")
endif()

Add idl.h to the serializer include and add the following code to the Serialize() function:

#include <flatbuffers/idl.h>
...
 
void Serializer::Serialize(const INetwork& inNetwork)
{
    // Iterate through to network
    inNetwork.Accept(m_SerializerVisitor);
    flatbuffers::FlatBufferBuilder& fbBuilder = m_SerializerVisitor.GetFlatBufferBuilder();
  
    // Create FlatBuffer SerializedGraph
    auto serializedGraph = serializer::CreateSerializedGraph(
        fbBuilder,
        fbBuilder.CreateVector(m_SerializerVisitor.GetSerializedLayers()),
        fbBuilder.CreateVector(m_SerializerVisitor.GetInputIds()),
        fbBuilder.CreateVector(m_SerializerVisitor.GetOutputIds()),
        m_SerializerVisitor.GetVersionTable());
  
    // Serialize the graph
    fbBuilder.Finish(serializedGraph);
      
    // Code to be added, delete after use
    std::string schemafile;
    flatbuffers::LoadFile(("path/to/ArmnnSchema.fbs"),
                          false, &schemafile);
    std::string jsongen;
    flatbuffers::Parser parser;
    parser.Parse(schemafile.c_str());
    GenerateText(parser, fbBuilder.GetBufferPointer(), &jsongen);
    std::cout << jsongen;
}

Run the serialization test for the layer that you are creating. A working JSON output is printed. You can use this JSON file to make your deserializer test.

Add the unit test file to ml/armnn/CMakeLists.txt:

//Softmax
   if(BUILD_ARMNN_SERIALIZER AND ARMNNREF)
       enable_language(ASM)
       list(APPEND unittest_sources
            src/armnnSerializer/test/ActivationSerializationTests.cpp
            src/armnnSerializer/test/SerializerTests.cpp
            src/armnnDeserializer/test/Deserialize<Layer_Name>.cpp
    .
    .

Add the new supported layer to the deserializer documentation in src/armnnDeserializer/:

//Softmax
The Arm NN SDK Deserialize parser currently supports the following layers:
* Abs
* Activation
...
* Softmax
.

Previous Next