Hi Performance JSON Parsing in Android

Sometimes in Android you connect to a third party API and you need to download gobs of JSON data. One of my current clients has an API endpoint that is used for syncing data to the device. One API call returns hundreds of thousands of lines of JSON resulting in a JSON file (when saved) of around 8MB or more. Processing the entire file by loading it into memory can be painful and can crash a lot memory constrained devices. However, using the Jackson JSON parsing library you can stream the http-request as a Reader into Jacksons Streaming API which will allow you to process each JSON object as soon as it comes down from the pipe. This allows you to process the data as it comes down the pipe instead of loading the entire JSON file.

I use the ObjectMapper from Jackson to deserialize the JSON into a POJO. This allows me to work with domain objects throughout my app, therefore, I don’t have to worry about parsing JSON objects all over the place (or cursors if you move the data into a DB locally on the device). If you were to load the entire JSON file into the object mapper you’d create a humongous object graph which may ballon your memory out of the bounds of what’s available and you might get a OutOfMemoryException. The end goal is to process EACH of the JSON objects as they come down, but also have the benefit of using a POJO to do so. Here’s how you can do it.

The ObjectModel

Below we have a simple customer object that we need to download. Imagine having a large accounting app that has millions of customers. Each customer would be represented as a POJO as defined below.

gistfile1.java

public class Customer {
  String firstName;
  String lastName;
  // 20 Other properties

  public String getFirstName() {
    return firstName;
  }

  public void setFirstName(String firstName) {
    this.firstName = firstName;
  }

  public String getLastName() {
    return lastName;
  }

  public void setLastName(String lastName) {
    this.lastName = lastName;
  }

  // all the other mutators and accessors

}

View on GitHub

Connecting to the API / Consuming the Stream

This actually quite easy. Simply connect to your API endpoint with your http library of choice (In this case I’m using http-request, but you can easily use OkHttp, etc). Using the reader() method on the HttpRequest object I can then pass this reader into the JsonParser (shown below) and then let the loop build each Customer and then I handle each customer accordingly. Comments are inline:

gistfile1.java

// Connect to your API with your http lib, i use http-request: https://github.com/kevinsawicki/http-request
// Then get the reader. The 'request' variable below is just a HttpRequest object
// as shown here: http://kevinsawicki.github.io/http-request/
final Reader reader = request.reader();

final ObjectMapper mapper = new ObjectMapper();
final JsonFactory factory = mapper.getFactory();

// Create a new streaming parser with the reader
JsonParser jp = factory.createParser(reader);

// I'm working with a JSON Array that is returned from the API that I'm connecting to, so I need to advance the
// parser to the start of the array.

// Advance stream to START_ARRAY first:
jp.nextToken();

// Parse each value by looking for each start object. The Mapper will read the object and advance the parser
// until it finds the END_OBJECT at which time the mapper then deserializes into a POJO. Then the loop will
// continue to the next position. This will handle each JSON object as it becomes available and will be very
// light on memory and high in performance.
while(jp.nextToken() == JsonToken.START_OBJECT) {
    // Get customer POJO
    final Customer c = mapper.readValue(jp, Customer.class);

    // Update local db with customer info.
    myDataLayer.updateCustomer(c);
}

View on GitHub

How to Use Jackson

Simply add it as a library or if you’re using Maven add the Maven dependency in your POM and you’re off to the races. 🙂 Now you can download HUGE JSON streams without the worry of exploding your memory footprint. 🙂