Suresh Payankannur

Tuesday, May 20, 2014

Importing JSON with references into Spring Data Repository


Spring Data Repository comes with a nice utility to import JSON using Jackson2RepositoryPopulatorFactoryBean. This class can be configured in the in the application context and when the context is loaded, the provided JSON load scripts will be read and populated into the underlying repository. This is done by first converting the JSON into Java model classes using Jackson and then using the Spring Data Repository to persist the data.
For example, a sample JSON for a User is as follows:
[
  {
    "email" : "user1@nowhere.com",
    "name"  : "John Doe"
  }
]
The Jackson2RepositoryPopulatorFactoryBean implementation requires the name of the class as part of the JSON object using _class property. So in order for the Spring to load this automatically, the JSON has to be modified as follows:
[
  {
    "_class": "org.hb2json.model.User",
    "email" : "user1@nowhere.com",
    "name"  : "John Doe"
  }
]
Now the following Spring configuration will load the JSON using the provided UserRepository


So far, so good...

Now lets create a Blog entity and try to load it from JSON. Jackson has the ability to speficy @id property to JSON data to uniquely identify each instance. This can be used referencing and avoids circular references in JSON.
By annotating the model class with the following, we can use a JSON property @id
@JsonIdentityInfo(generator=IntSequenceGenerator.class, property="@id")
public class User {
    ...
}

Now our JSON becomes:
[
  {
    "@id"   : 1,
    "_class": "org.hb2json.model.User",
    "email" : "user1@nowhere.com",
    "name"  : "John Doe"
  },
  {
    "@id"    : 2,
    "_class" : "org.hb2json.model.Blog",
    "name"   : "My Travel Blog",
    "owner"  : 1
  }
]

But this will not work with Jackson2RepositoryPopulatorFactoryBean. The implementation of this class reads each nodes in the JSON array individually. While reading the second node for Blog, Jackson has no way of knowing what "owner" : 1 means. The provided reference is not available in the current context. Each node is read independently. Resulting in the error.

In order to fix this issue, a couple of steps should be taken:
  1. Create a wrapper object so that the references can be handled
  2. Customize the Jackson2RepositoryPopulatorFactoryBean so that the wrapper object will not get persisted.
Now with the wrapper object, the JSON becomes:
{
  "_class" : "org.hb2json.model.DataSet",

  "users" :[{
    "@id"   : 1,
    "email" : "user1@nowhere.com",
    "name"  : "John Doe"
  }],
  "blogs" :[{
    "@id"    : 2,
    "name"   : "My Travel Blog",
    "owner"  : 1
  }]
}
And the corresponding Java Model. The DataSet is a wrapper class to hold all the data to be loaded.
public class DataSet {
    private Collection<User> users;
    private Collection<Blog> blogs;

    public Collection<User> getUsers() {
        return this.users;
    }
    public void setUsers(Collection<User> list) {
        this.users = list;
    }
    public Collection<Blog> getBlogs() {
        return this.blogs;
    }
    public void setBlogs(Collection<Blog> list) {
        this.blogs = list;
    }
}
In order to achieve this, we have to customize both the Jackson2RepositoryPopulatorFactoryBean and Jackson2ResourceReader. The resource reader is really the class we want to customize to read the given JSON file and skip the nodes corresponding to DataSet class. But the Spring Data implementation of the repository populator is hard wired to use an instance of Jackson2ResourceReader. So we need to customize the Jackson2RepositoryPopulatorFactoryBean so that we can provide our own Jackson2ResourceReader which implements skipping JSON nodes.

Customization of Jackson2RespositoryPopulatorFactoryBean

public class CustomRepositoryPopulatorFactoryBean
    extends Jackson2RepositoryPopulatorFactoryBean {
     
    private static final ObjectMapper DEFAULT_MAPPER = new ObjectMapper();

    static {
        DEFAULT_MAPPER.configure(FAIL_ON_UNKNOWN_PROPERTIES, false);
    }

    @Override
    protected ResourceReader getResourceReader() {
        return new CustomJackson2ResouceReader(DEFAULT_MAPPER);
    }
}

Customization of Jackson2ResourceReader

public class CustomJackson2ResourceReader extends Jackson2ResourceReader {
    private Class<?> entityClass = DataSet.class;
    private String typeKey = "_class";
    private ObjectMapper mapper;
    private DataExtractor extractor = new DataSetExtractor();

    public CustomJackson2ResourceReader(ObjectMapper mpr) {
        super(mpr);
        mapper = mpr;
    }
    @Override
    public Object readFrom(Resource resource, ClassLoader classLoader)
        throws Exception {

        InputStream stream = resource.getInputStream();
        JsonNode node = mapper.reader(JsonNode.class).readTree(stream);

        if (node.isArray()) {

            Iterator<JsonNode> elements = node.elements();
            List<Object> result = new ArrayList<Object>();

            while (elements.hasNext()) {
                JsonNode element = elements.next();
                result.add(readSingle(element, classLoader));
            }

            return result;
        }

        return readSingle(node, classLoader);
    }
    private Object readSingle(JsonNode node, ClassLoader classLoader)
        throws IOException {

        JsonNode typeNode = node.findValue(typeKey);
        String typeName = typeNode == null ? null : typeNode.asText();

        Class<?> type = ClassUtils.resolveClassName(typeName, classLoader);

        Object obj = mapper.reader(type).readValue(node);

        obj = type.equals(DataSet.class)  ?  extractor.getData(obj)  :  obj;

        return obj;
    }
}

Additional classes to extract data from the wrapper DataSet instance

public interface DataExtractor {
    Object getData(Object data);
}

public class DataSetExtractor implements DataExtractor {
    public Object getData(Object obj) {
        DataSet set = (DataSet) obj;
        Collection<Object> data = new ArrayList<Object>();

        for (Object o : set.getUsers()) {
            data.add(o);
       }
       for (Object o : set.getBlogs()) {
            data.add(o);
        }
        return data;
    }
}

And the Spring configuration...

  
    
  

0 comments:

Post a Comment

Blog Archive

Scroll To Top