Can't import Performance Advisor JSON into ELK

When trying to import JSON reports that Performance Advisor has generated into an Elasticsearch database, the import fails.

Condition

Affected releases: 2020.1

Cause

Elasticsearch does not process JSON files that are pretty-printed. It also expects an index statement at the start of the JSON file, and a newline at the end. In Arm Mobile Studio 2020.1 and earlier, JSON files are pretty-printed by default, and contain only the JSON data fields for the reported performance data.

Workaround

Download this Python script and run it on your JSON files to convert them to an acceptable format for Elasticsearch:

python3 prepare-json-for-elastic-search.py filename.json

Alternatively, make the following changes to your JSON files:

  1. Elasticsearch expects the following on line 1 of a JSON file: 
    {"index":{}} 
    Alternatively, you can add your own values to tell Elasticsearch which index to add the file to, and an id number: 
     { "index" : { "_index" : "test", "_id" : "1" } }
    If you don't specify an index you will need to specify it with the command you use to import the file, for example:
    curl -X POST "<Elasticsearch_location>/indexname/_bulk?pretty" -H 'Content-Type: application/x-ndjson' --data-binary @<file_name>.json
    If you don't specify an ID number, Elasticsearch allocates a random ID to the document.
  2. On line 2, Elasticsearch expects the JSON data with no newlines.
  3. Add a newline at the end of the doc (line 3). The file should look like the following:
    {"index":{}} 
    {JSON file contents}  
    \n 

Related information