I don't want to go into too much detail, so I'll try to boil it down as simple as I can. We've got an app (Java, Spring Boot) that generates some information which we then serialize and store in a database. To serialize it, we use ObjectMapper:
final ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JavaTimeModule());
mapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
mapper.enableDefaultTyping(); // default to using DefaultTyping.OBJECT_AND_NON_CONCRETE
mapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
manifestJson = mapper.writeValueAsString(manifest);
And the JSON is then stored in the database. Yippy yay. manifest is a complex object that contains a number of other objects and information that's used to created a final product. The problem is when we go to try to use that info to generate a new object. The FQN of the original class is stored with the information (makes sense). But the read back of the information is taking place in a different location/application. So while the original namespace that was stored with the json was type a.b.c.d.e.f.class1, we're now trying to read it back into class a.b.c.g.h.i.f.class1 ... logically class1 is the same in both namespacesses, in fact it was copied directly from one to the other. The only difference is the intervening package names.
Naturally, the when we then try to deserialize the JSON, it throws an error about not finding the type, the original type is in the original project/applicaiton and the "new" type is in the current project. We could reference the original app from the secondary app, but for functional reasons, we're trying to maintain decoupling between the two.
So the question is, how to get a serialized an object from one project and then deserialize it into another class that is logically identical?
By default, when typing is not used you can generate a JSON payload which you can later deserialise to any model which fits this JSON. Or even to Map or/and List objects if you do not want to create a model at all.
In your case, when JSON payloads are used by two different apps with two different models, you should not attach class info because it does not provide any extra information for another model. One JSON can be deserialised on many ways and used for different purposes so linking it with one model causes problems like you noticed in your example. So, if it is possible disable typing and provide explicitly all informations are needed later to recreate the same object from a JSON.
If it is not possible, just provide your custom TypeIdResolver and map one class from one model to similar class in another model. Simple example which shows an idea:
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.databind.DatabindContext;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonTypeIdResolver;
import com.fasterxml.jackson.databind.jsontype.impl.ClassNameIdResolver;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.type.SimpleType;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class JsonApp {
public static void main(String[] args) throws Exception {
SimpleModule model2Module = new SimpleModule("Model2");
model2Module.setMixInAnnotation(PojoA2.class, CustomJsonTypeIdResolverMixIn.class);
model2Module.setMixInAnnotation(PojoB2.class, CustomJsonTypeIdResolverMixIn.class);
model2Module.setMixInAnnotation(PojoC2.class, CustomJsonTypeIdResolverMixIn.class);
ObjectMapper mapper = new ObjectMapper();
mapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
mapper.registerModule(model2Module);
String json = mapper.writeValueAsString(new PojoA());
System.out.println(json);
System.out.println(mapper.readValue(json, PojoA.class));
System.out.println(mapper.readValue(json, PojoA2.class));
}
}
#JsonTypeInfo(use = JsonTypeInfo.Id.CUSTOM)
#JsonTypeIdResolver(value = Model2ClassNameIdResolver.class)
interface CustomJsonTypeIdResolverMixIn { }
class Model2ClassNameIdResolver extends ClassNameIdResolver {
private final Map<String, JavaType> types = new HashMap<>();
public Model2ClassNameIdResolver() {
super(null, null);
types.put("com.celoxity.PojoA", SimpleType.constructUnsafe(PojoA2.class));
types.put("com.celoxity.PojoB", SimpleType.constructUnsafe(PojoB2.class));
types.put("com.celoxity.PojoC", SimpleType.constructUnsafe(PojoC2.class));
}
#Override
public JavaType typeFromId(DatabindContext context, String id) throws IOException {
JavaType javaType = types.get(id);
if (javaType != null) {
return javaType;
}
return super.typeFromId(context, id);
}
}
class PojoA2 {
private PojoB2 b;
}
class PojoB2 {
private List<PojoC2> c;
}
class PojoC2 {
private String s;
private int i;
}
class PojoA {
private PojoB b;
}
class PojoB {
private List<PojoC> c;
}
class PojoC {
private String s;
private int i;
}
Two models: PojoA, PojoB, PojoC and PojoA2, PojoB2, PojoC2 have the same structure and have only different names. PojoA is the same as PojoA2, etc.
Above code prints:
["com.celoxity.PojoA",{"b":["com.celoxity.PojoB",{"c":["java.util.ArrayList",[["com.celoxity.PojoC",{"s":"Vika","i":22}]]]}]}]
and later:
PojoA{b=PojoB{c=[PojoC{s='Vika', i=22}]}}
PojoA2{b=PojoB2{c=[PojoC2{s='Vika', i=22}]}}
Related
I'm having a class which used to convert POJO to json string and compare it with another json string(read from logs). Recently we added 2 new boolean parameters to json and POJO and now the order of those 2 newly added variables is getting vary time to time.
I know by adding #JsonPropertyOrder(alphabetic = true) or #JsonPropertyOrder({ "att1", "att2", "att3" }) annotations, we can ensure the order. But I just want to know, how we got the same output earlier(with same order. Note: we had that code since 5years back)
Please find the sample code :
package com.....api;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.*;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
public class TestClass {
public String writeAsJson(Object object) throws JsonProcessingException {
JsonMapper.Builder jsonMapperBuilder = JsonMapper.builder();
jsonMapperBuilder.addModule(new Jdk8Module())
.addModule((new JavaTimeModule()))
.addModule((new SimpleModule()))
.propertyNamingStrategy(PropertyNamingStrategy.SNAKE_CASE)
.disable(new SerializationFeature[]{SerializationFeature.WRITE_DATES_AS_TIMESTAMPS})
.enable(new MapperFeature[]{MapperFeature.ACCEPT_CASE_INSENSITIVE_ENUMS})
.disable(new MapperFeature[]{MapperFeature.DEFAULT_VIEW_INCLUSION})
.disable(new DeserializationFeature[]{DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES});
ObjectMapper objectMapper = jsonMapperBuilder.build();
return objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(object);
}
}
As per I checked, we can not ensure the order of the variables in the POJO when it serializing and deserializing json<--> POJO. But I'm curious how we got the same output for such a long period and suddenly how it fail after adding 2 new boolean variables.
I use jackson ObjectMapper to serialize and deserialize some data of mine, which have fields of javaslang Option type. I use JavaslangModule (and Jdk8Module). And when it write the json, Option.None value fields are written as null.
To reduce the json size and provide some simple backward compatibility when later adding new fields, what I want is that:
fields with Option.None value are simply not written,
missing json fields that correspond to data model of Option type, be set to Option.None upon reading
=> Is that possible, and how?
Note:
I think that not-writing/removing null json fields would solve (1). Is it possible? And then, would reading it works (i.e. if model field with Option value is missing in the json, set it None?
Luckily there is a much simpler solution.
1) In your ObjectMapper configuration, set serialization inclusion to only include non absent field:
#Bean
public ObjectMapper objectMapper() {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.registerModules(vavr());
objectMapper.setSerializationInclusion(NON_ABSENT);
return objectMapper;
}
2) Set the default value of your optional fields to Option.none:
#Data
#AllArgsConstructor
#NoArgsConstructor
public class Foo {
private Option<String> bar = Option.none(); // If the JSON field is null or not present, the field will be initialized with none
}
That's it!
And the even better news is that it works for all Iterables, not just for Option. In particular it also works for Vavr List type!
I found a solution that works with immuatble (lombok #Value) models:
add a filter on all Object using mixIn that doesn't write Option.None (see "the solution" below)
my existing ObjectMapper (with JavaslangModule) is already setting None to Option field when the corresponding json entry is missing
The code
import static org.assertj.core.api.Assertions.assertThat;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonFilter;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectWriter;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.ser.PropertyWriter;
import com.fasterxml.jackson.databind.ser.impl.SimpleBeanPropertyFilter;
import com.fasterxml.jackson.databind.ser.impl.SimpleFilterProvider;
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module;
import javaslang.control.Option;
import javaslang.jackson.datatype.JavaslangModule;
import lombok.AllArgsConstructor;
import lombok.Value;
import org.junit.Test;
import java.io.IOException;
import java.lang.reflect.Field;
public class JsonModelAndSerialization {
// Write to Json
// =============
private static ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module())
.registerModule(new JavaslangModule())
// not required but provide forward compatibility on new field
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
static String write(Object data) throws JsonProcessingException {
SimpleBeanPropertyFilter filter = new NoneOptionPropertyFilter();
objectMapper.addMixIn(Object.class, NoneOptionFilter.class);
final SimpleFilterProvider filters = new SimpleFilterProvider().setDefaultFilter(filter);
ObjectWriter writer = objectMapper.writer(filters);
return writer.writeValueAsString(data);
}
// Filter classes
// ==============
#JsonFilter("Filter None")
private static class NoneOptionFilter {}
private static class NoneOptionPropertyFilter extends SimpleBeanPropertyFilter {
#Override
public void serializeAsField(
Object pojo, JsonGenerator jgen,
SerializerProvider provider, PropertyWriter writer) throws Exception{
Field field = pojo.getClass().getDeclaredField(writer.getName());
if(field.getType().equals(Option.class)){
field.setAccessible(true);
Option<?> value = (Option<?>) field.get(pojo);
if(value.isEmpty()) return;
}
super.serializeAsField(pojo, jgen, provider, writer);
}
}
// Usage example
// =============
// **important note**
// For #Value deserialization, a lombok config file should be added
// in the source folder of the model class definition
// with content:
// lombok.anyConstructor.addConstructorProperties = true
#Value
#AllArgsConstructor(onConstructor_={#JsonCreator})
public static class StringInt {
private int intValue;
private Option<String> stringValue;
}
#Value
#AllArgsConstructor(onConstructor_={#JsonCreator})
public static class StringIntPair {
private StringInt item1;
private StringInt item2;
}
#Test
public void readWriteMyClass() throws IOException {
StringIntPair myClass = new StringIntPair(
new StringInt(6 * 9, Option.some("foo")),
new StringInt( 42, Option.none()));
String json = write(myClass);
// {"item1":{"intValue":54,"stringValue":"foo"},"item2":{"intValue":42}}
StringIntPair myClass2 = objectMapper.readValue(json, StringIntPair.class);
assertThat(myClass2).isEqualTo(myClass);
}
}
The advantages:
reduce size of json when having Option.None (thus adding Option fields in the model doesn't cost size when not used)
it provides backward reading compatibility when later adding field with Option type in the model (which will default to None)
The disadvantage:
It is not possible to differentiate correct data with None field value and incorrect data where the field has erroneously been forgotten. I think this is quite acceptable.
I have an XML document that I want to deserialise into an object of a class that I do not know during compilation and the type is determined by both the root element name of the document and a type element text content. The problem is twofold:
I have element lists in my XML document so I can't parse it to JsonNode/Map because in the best case scenario it will only provide me with the last value on the list, the information is lost in the process. All the solutions I found rely on JsonNode API (like convertValue or treeToValue).
I can't use JsonTypeInfo - I don't know the concrete classes that will be deserialised and so I cannot use the annotations. Also I don't think JsonTypeInfo is this flexible as to include both root element name and a property value.
I tried using a custom deserialiser that would wrap the actual parsed JsonNode into another Object with rootElement as an only key (by default this root element is not included in the parsed document). After that I deserialise the document, get the root element from the node, then take the type element value, combine the two, create the name of the class and then convert the JsonNode to the final object. The problem with this approach is that it doesn't handle the lists well and either throws and exception of invalid mapping or provides me with the last element of the list depending of how I write my model or configure the mapper.
package zm.study.xmlserialize.jackson;
import static org.junit.Assert.assertEquals;
import java.io.StringReader;
import java.util.List;
import org.junit.Test;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.dataformat.xml.XmlMapper;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlElementWrapper;
import com.fasterxml.jackson.dataformat.xml.deser.FromXmlParser;
public class JacksonListTest2 {
public static class A < D extends AData > {
public String type;
public D data;
}
public static class AData {}
public static class BRequestData extends AData {
#JacksonXmlElementWrapper(useWrapping = false)
public List < String > bs;
}
public static class BRequest extends A {}
#Test
public void test() throws Exception {
String xml;
xml = "<Request><type>B</type><data><bs>1</bs><bs>2</bs></data></Request>";
XmlMapper mapper = new XmlMapper();
mapper.registerModule(
new SimpleModule()
.addDeserializer(JsonNode.class, new JsonNodeDeserializer() {
#Override
public JsonNode deserialize(JsonParser p, DeserializationContext ctxt) throws java.io.IOException {
String rootName = ((FromXmlParser) p).getStaxReader().getLocalName();
return ctxt.getNodeFactory().objectNode().set(rootName, super.deserialize(p, ctxt));
};
})
);
JsonNode rootNode = mapper.readTree(new StringReader(xml));
String rootName = rootNode.fields().next().getKey();
JsonNode contentNode = rootNode.get(rootName);
String type = contentNode.get("type").asText();
String className = getClass().getCanonicalName() + "$" + type + rootName;
BRequest value = (BRequest) mapper.convertValue(contentNode, Class.forName(className));
System.out.println(value.data.bs);
assertEquals(2, value.data.bs.size());
}
}
My approach results in throwing exception:
Cannot deserialize instance of java.util.ArrayList out of
VALUE_STRING token
I also tried creating a custom deserialiser parsing just enough of the document to find out what type I'm dealing with and then delegating the rest of the actual parsing (the data element) to the default deserialiser but didn't manage to write any sensible code to do it.
I think I need a new approach, any ideas?
I have tried to do something like this:
package org.dnylabs.kosh.data;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import com.google.code.morphia.Datastore;
import com.google.code.morphia.Morphia;
import com.google.code.morphia.annotations.Entity;
import com.google.code.morphia.annotations.Id;
import com.mongodb.Mongo;
import com.mongodb.MongoException;
#Entity
public class Temp {
#Id String _id;
List<Map<String,Object>> strings;
public Temp(){
strings=new LinkedList<Map<String,Object>>();
}
public static void main(String []args) throws UnknownHostException, MongoException{
Mongo mongo=null;
Morphia morphia=null;
Datastore ds=null;
mongo = new Mongo();
morphia = new Morphia();
morphia.map(Temp.class);
ds = morphia.createDatastore(mongo, "test");
Temp t = new Temp();
t._id ="hi";
Map<String, Object> m = new HashMap<String, Object>();
m.put("Hi","1");
m.put("Hi2",2);
t.strings.add(m);
ds.save(t);
t=ds.get(t);
ds.ensureIndexes();
}
}
When I try to do a findAll(9 operation I get this exception:
Caused by: java.lang.RuntimeException: org.mongodb.morphia.mapping.MappingException: Embedded element isn't a DBObject! How can it be that is a class java.lang.String
at org.mongodb.morphia.mapping. here`dedMapper.fromDBObject(EmbeddedMapper.java:172)
at org.mongodb.morphia.mapping.Mapper.readMappedField(Mapper.java:602)
at org.mongodb.morphia.mapping.Mapper.fromDb(Mapper.java:559)
at org.mongodb.morphia.mapping.EmbeddedMapper.readMapOrCollectionOrEntity(EmbeddedMapper.java:256)
at org.mongodb.morphia.mapping.EmbeddedMapper.readCollection(EmbeddedMapper.java:203)
at org.mongodb.morphia.mapping.EmbeddedMapper.fromDBObject(EmbeddedMapper.java:144)
... 16 more
After numerous attempts I have found that the problem is the grafted map.
Can anyone help me understand where I'm wrong? The statement seems correct.
Morphia sees Map as a DB reference to another document rather than seeing it as an embedded class and treating as a document. The solution would be to annotate the Map #Embedded, but this is not possible as you can't edit the Map class.
There is a way to achieve something similar to what you are trying by creating another class and defining the Map as a property of this class and annotate it as #Embedded.
Change the Temp class:
public class Temp {
#Id String _id;
#Embedded // CHANGE HERE
List<MapProxy> strings; // CHANGE HERE
public Temp(){
strings=new LinkedList<MapProxy>(); // CHANGE HERE
}
public static void main(String...args) throws UnknownHostException, MongoException{
Mongo mongo=null;
Morphia morphia=null;
Datastore ds=null;
mongo = new Mongo();
morphia = new Morphia();
morphia.map(Temp.class);
ds = morphia.createDatastore(mongo, "test2");
Temp t = new Temp();
t._id ="hi";
MapProxy mp = new MapProxy(); // CHANGE HERE
mp.m.put("Hi","1"); // CHANGE HERE
mp.m.put("Hi2",2); // CHANGE HERE
t.strings.add(mp); // CHANGE HERE
ds.save(t);
t=ds.get(t);
ds.ensureIndexes();
}
}
and create a new class:
#Embedded
public class MapProxy {
public Map<String,Object> m = new HashMap<String, Object>();
}
I have marked the changes I have made.
The structure that this produces is like this:
{
"_id" : "hi",
"className" : "YOUR CLASS NAME HERE",
"strings" :
[ {
"m" :
{
"Hi" : "1" ,
"Hi2" : 2
}
} ]
}
So here's what's going on. Morphia will attempt to serialize any non-transient, non-static field on a class. Now if that field is of a type annotated with #Entity, morphia will go through and introspect things and set up proper mappings. If not, it will some default basic serializers when constructing the DBObjects to hand over to the Java driver. In this case, you have a raw type (Object) and morphia has to make certain assumptions as to its type and structure. Upon finding out that your structure violates those assumptions, it bails. This is why it works when you break things out like Alex has shown. This is probably fixable in one way or another but as I'm planning on redoing the mapping code top to bottom, I don't foresee trying to fix this in the current code. Hope this helps.
I'm trying to use Jackson to read/write my POJOs to/from Json. As of right now, I've got it configured and working for my classes, except for a 3rd party class. When trying to read in the Json I get the error:
org.codehaus.jackson.map.JsonMappingException: No suitable constructor found for type
After a few quick google searches, it appears that my class needs either a default constructor or to override the default constructor with annotations. Unfortunately, the class in which this is failing is from a 3rd party library and that class does not have a default constructor and I obviously cannot over-write the code.
So my question is, is there anything I can do about this or am I just out of luck?
Thanks.
You could make use of Jackson's Mix-Ins feature, coupled with the Creator feature. The Mix-Ins feature alleviates the need to annotate the original third-party code, and the Creator feature provides a mechanism for custom instance creation.
For yet more customization, it's not too involved to write a custom deserializer.
One approach is to implement a custom JsonDeserializer to create the instance, annotating the fields of the type with #JsonDeserialize. One advantage of this approach over e.g. mixins is that it does not require modifying the ObjectMapper.
The StdNodeBasedDeserializer class allows mapping from a JsonNode representing the value to the desired type.
Type lacking a constructor
public class ThirdPartyType {
private String stringProperty;
private int intProperty;
private Object[] arrayProperty;
public ThirdPartyType(String a, int b, Object[] c) {
this.stringProperty = a;
this.intProperty = b;
this.arrayProperty = c;
}
// Getters and setters go here
}
Custom deserializer
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.deser.std.StdNodeBasedDeserializer;
import java.io.IOException;
import java.util.Spliterator;
import java.util.Spliterators;
import java.util.stream.StreamSupport;
public class ThirdPartyTypeDeserializer
extends StdNodeBasedDeserializer<ThirdPartyType> {
protected ThirdPartyTypeDeserializer() {
super(ThirdPartyType.class);
}
#Override
public ThirdPartyType convert(JsonNode root, DeserializationContext ctxt)
throws IOException {
return new ThirdPartyType(
root.get("stringProperty").asText(null),
root.get("intProperty").asInt(),
StreamSupport.stream(
Spliterators.spliteratorUnknownSize(
root.get("arrayProperty").elements(),
Spliterator.ORDERED),
false).toArray());
}
}
Type containing the third party type
public class EnclosingClass {
#JsonDeserialize(using = ThirdPartyTypeDeserializer.class)
private ThirdPartyType thirdPartyProperty;
// Getters and setters go here
}
Retrieving the value
String json = "{\"thirdPartyProperty\": {"
+ "\"stringProperty\": \"A\", "
+ "\"intProperty\": 5, "
+ "\"arrayProperty\": [1, \"B\", false]"
+ "}}";
ObjectMapper objectMapper = new ObjectMapper();
EnclosingClass enclosingClass =
objectMapper.readValue(json, EnclosingClass.class);