Develop Reference

"Relaxed" fields names for Jackson

I'm working on Jackson configuration and I wonder if there is any option to deserialise different kinds of field patterns.
For example, I have an object:
class DeserializeIt {
String fieldOne;
String fieldOneAndHalf;
String fieldTwo;
String fieldThree;
String fieldFour;
//getters setters etc.
}
And I have below JSON payload:
{
"fieldOne" : "value1",
"field_ONE-and_Half": "value15",
"FIELD_TWO": "value2",
"FIELD_THREE" : "value3",
"field_four": "value4"
}
I would like to deserialize all these field names to camel case without an exception.
I tried to create my custom PropertyNamingStrategy but it goes from another direction: it does not convert delimitered fields to camel case, it tries to convert the objects fields and search for them in the parsed string.
And since I cannot pass a list of possible strings instead of one variation (fieldOne can become field-one, field_one, field-ONE etc.), this does not work.
Do you know what else could I configure for such a relaxed deserialization?

We need to extend com.fasterxml.jackson.databind.deser.BeanDeserializerModifier and com.fasterxml.jackson.databind.deser.BeanDeserializer which deserialises POJO classes. Below solution depends from version you are using because I copied some code from base class which is not ready for intercepting extra functionality. If you do not have any extra configuration for your POJO classes vanillaDeserialize method will be invoked and this one we will try to improve.
In other case you need to debug this deserialiser and updated other places if needed. Below solution uses version 2.9.8.
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.core.JsonTokenId;
import com.fasterxml.jackson.databind.BeanDescription;
import com.fasterxml.jackson.databind.DeserializationConfig;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.deser.BeanDeserializer;
import com.fasterxml.jackson.databind.deser.BeanDeserializerBase;
import com.fasterxml.jackson.databind.deser.BeanDeserializerModifier;
import com.fasterxml.jackson.databind.deser.SettableBeanProperty;
import com.fasterxml.jackson.databind.module.SimpleModule;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
public class JsonApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
SimpleModule relaxedModule = new SimpleModule();
relaxedModule.setDeserializerModifier(new RelaxedBeanDeserializerModifier());
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(relaxedModule);
System.out.println(mapper.readValue(jsonFile, DeserializeIt.class));
}
}
class RelaxedBeanDeserializerModifier extends BeanDeserializerModifier {
#Override
public JsonDeserializer<?> modifyDeserializer(DeserializationConfig config, BeanDescription beanDesc, JsonDeserializer<?> deserializer) {
JsonDeserializer<?> base = super.modifyDeserializer(config, beanDesc, deserializer);
if (base instanceof BeanDeserializer) {
return new RelaxedBeanDeserializer((BeanDeserializer) base);
}
return base;
}
}
class RelaxedBeanDeserializer extends BeanDeserializer {
private Map<String, String> properties = new HashMap<>();
public RelaxedBeanDeserializer(BeanDeserializerBase src) {
super(src);
_beanProperties.forEach(property -> {
properties.put(property.getName().toLowerCase(), property.getName());
});
}
public Object deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
// common case first
if (p.isExpectedStartObjectToken()) {
if (_vanillaProcessing) {
return vanillaDeserialize(p, ctxt, p.nextToken());
}
// 23-Sep-2015, tatu: This is wrong at some many levels, but for now... it is
// what it is, including "expected behavior".
p.nextToken();
if (_objectIdReader != null) {
return deserializeWithObjectId(p, ctxt);
}
return deserializeFromObject(p, ctxt);
}
return _deserializeOther(p, ctxt, p.getCurrentToken());
}
protected Object vanillaDeserialize(JsonParser p, DeserializationContext ctxt, JsonToken t) throws IOException {
final Object bean = _valueInstantiator.createUsingDefault(ctxt);
// [databind#631]: Assign current value, to be accessible by custom serializers
p.setCurrentValue(bean);
if (p.hasTokenId(JsonTokenId.ID_FIELD_NAME)) {
String propName = p.getCurrentName();
do {
String relaxedName = getRelaxedName(propName);
String mappedName = properties.get(relaxedName);
defaultImplementation(p, ctxt, bean, mappedName);
} while ((propName = p.nextFieldName()) != null);
}
return bean;
}
private void defaultImplementation(JsonParser p, DeserializationContext ctxt, Object bean, String propName) throws IOException {
p.nextToken();
SettableBeanProperty prop = _beanProperties.find(propName);
if (prop != null) { // normal case
try {
prop.deserializeAndSet(p, ctxt, bean);
} catch (Exception e) {
wrapAndThrow(e, bean, propName, ctxt);
}
return;
}
handleUnknownVanilla(p, ctxt, bean, propName);
}
private String getRelaxedName(String name) {
return name.replaceAll("[_\\-]", "").toLowerCase();
}
}
Above code prints:
DeserializeIt{fieldOne='value1', fieldOneAndHalf='value15', fieldTwo='value2', fieldThree='value3', fieldFour='value4'}
See also:
Can Jackson check for duplicated properties in a case insensitive way?

From Jackson 2.9, you can provide multiple possible properties names for deserialization using the
#JsonAlias annotation. On your example, it would be like this:
class DeserializeIt {
#JsonAlias("fieldOne")
String fieldOne;
#JsonAlias("field_ONE-and_Half")
String fieldOneAndHalf;
#JsonAlias("FIELD_TWO")
String fieldTwo;
#JsonAlias("FIELD_THREE")
String fieldThree;
// and so on...
}

What worked for myself: I added an AOP component that renames all the fields of incoming object into the Camel case.

JsonMappingException when serializing avro generated object to json

I used avro-tools to generate java classes from avsc files, using:
java.exe -jar avro-tools-1.7.7.jar compile -string schema myfile.avsc
Then I tried to serialize such objects to json by ObjectMapper,
but always got a JsonMappingException saying "not an enum" or "not a union".
In my test I create the generated object using it's builder or constructor.
I got such exceptions for objects of different classes...
Sample Code:
ObjectMapper serializer = new ObjectMapper(); // com.fasterxml.jackson.databind
serializer.register(new JtsModule()); // com.bedatadriven.jackson.datatype.jts
...
return serializer.writeValueAsBytes(avroConvertedObject); // => JsonMappingException
I also tried many configurations using: serializer.configure(...) but still failed.
Versions: Java 1.8, jackson-datatype-jts 2.3,
jackson-core 2.6.5, jackson-databind 2.6.5, jackson-annotations 2.6.5
Any suggestions?
Thanks!

If the SCHEMA member is really the case (we don't see the full error message), then you can switch it off. I use a mixin to do it, like this:
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import org.apache.avro.Schema;
import org.junit.Test;
import java.io.File;
import java.io.IOException;
public class AvroGenerTests
{
abstract class IgnoreSchemaProperty
{
// You have to use the correct package for JsonIgnore,
// fasterxml or codehaus
#JsonIgnore abstract void getSchema();
}
#Test
public void writeJson() throws IOException {
BookAvro b = BookAvro.newBuilder()
.setTitle("Wilk stepowy")
.setAuthor("Herman Hesse")
.build();
ObjectMapper om = new ObjectMapper();
om.enable(SerializationFeature.INDENT_OUTPUT);
om.addMixIn(BookAvro.class, IgnoreSchemaProperty.class);
om.writeValue(new File("plik_z_gen.json"), b);
}
}

My req'ts got changed on me and I was told I needed to convert Avro objects straight to JSON without preserving any of the meta-data. My other answer herein that specified a method convertToJsonString converts the entire Avro object to JSON so that using a de-encoder you can re-create the original Avro object as an Avro object. That isn't what my mgt. wanted anymore so I was back to the old drawing board.
As a Hail Mary pass I tried using Gson and it works to do what I now had to do. It's very simple:
Gson gson = new Gson();
String theJsonString = gson.toJson(object_ur_converting);
And you're done.

2022 Avro field names
abstract class IgnoreSchemaPropertyConfig {
// You have to use the correct package for JsonIgnore,
// fasterxml or codehaus
#JsonIgnore
abstract void getClassSchema();
#JsonIgnore
abstract void getSpecificData();
#JsonIgnore
abstract void get();
#JsonIgnore
abstract void getSchema();
}

After finding the code example at https://www.programcreek.com/java-api-examples/?api=org.apache.avro.io.JsonEncoder I wrote a method that should convert any given Avro object (they extend GenericRecord) to a Json String. Code:
import org.apache.avro.generic.GenericDatumWriter;
import org.apache.avro.generic.GenericRecord;
import org.apache.avro.io.DatumWriter;
import org.apache.avro.io.EncoderFactory;
import org.apache.avro.io.JsonEncoder;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
// ... Class header etc. ...
public static <T extends GenericRecord> String convertToJsonString(T event) throws IOException {
String jsonstring = "";
try {
DatumWriter<T> writer = new GenericDatumWriter<T>(event.getSchema());
OutputStream out = new ByteArrayOutputStream();
JsonEncoder encoder = EncoderFactory.get().jsonEncoder(event.getSchema(), out);
writer.write(event, encoder);
encoder.flush();
jsonstring = out.toString();
} catch (IOException e) {
log.error("IOException occurred.", e);
throw e;
}
return jsonstring;
}

The previous post answers the question correctly. I am just adding on to the previous answer. Instead of writing it to a file I converted it to a string before sending it as body in a POST request.
public class AvroGenerateJSON
{
abstract class IgnoreSchemaProperty
{
// You have to use the correct package for JsonIgnore,
// fasterxml or codehaus
#JsonIgnore abstract void getSchema();
}
public String convertToJson() throws IOException {
BookAvro b = BookAvro.newBuilder()
.setTitle("Wilk stepowy")
.setAuthor("Herman Hesse")
.build();
ObjectMapper om = new ObjectMapper();
om.enable(SerializationFeature.INDENT_OUTPUT);
om.addMixIn(BookAvro.class, IgnoreSchemaProperty.class);
String jsonString = om.writeValueAsString(b);
return jsonString;
}
}

Agree with Shivansh's answer. To add, there might be instances where we need to use the avro-generated pojo in other classes. Under the hood, spring uses jackson library in handling this so we need to override global jackson config by adding a class
#Configuration
public class JacksonConfiguration {
public abstract IgnoreSchemaProperty {
#JsonIgnore abstract void getSchema();
}
#Bean
public ObjectMapper objectMapper() {
ObjectMapper om = new ObjectMapper();
om.addMixIn(SpecificRecordBase.class, IgnoreSchemaProperty.class);
return om;
}
}
SpecificRecordBase -if we want to ignore the schema field from all avro generated classes. In this way, we can serialize/deserialize our avro classes and use it anywhere in our application without getting the issue.

How use jackson ObjectMapper inside custom deserializer?

I try to write custom jackson deserializer. I want "look" at one field and perform auto deserialization to class, see example below:
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.ObjectCodec;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonNode;
import com.mypackage.MyInterface;
import com.mypackage.MyFailure;
import com.mypackage.MySuccess;
import java.io.IOException;
public class MyDeserializer extends JsonDeserializer<MyInterface> {
#Override
public MyInterface deserialize(JsonParser jp, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
ObjectCodec codec = jp.getCodec();
JsonNode node = codec.readTree(jp);
if (node.has("custom_field")) {
return codec.treeToValue(node, MyFailure.class);
} else {
return codec.treeToValue(node, MySuccess.class);
}
}
}
Pojos:
public class MyFailure implements MyInterface {}
public class MySuccess implements MyInterface {}
#JsonDeserialize(using = MyDeserializer.class)
public interface MyInterface {}
And I got StackOverflowError. In understand that codec.treeToValue call same deserializer. Is there a way to use codec.treeToValue or ObjectMapper.readValue(String,Class<T>) inside custome deseralizer?

The immediate problem seems to be that the #JsonDeserialize(using=...) is being picked up for your implementations of MyInterface as well as MyInterface itself: hence the endless loop.
You can fix this my overriding the setting in each implementation:
#JsonDeserialize(using=JsonDeserializer.None.class)
public static class MySuccess implements MyInterface {
}
Or by using a module instead of an annotation to configure the deserialization (and removing the annotation from MyInterface):
mapper.registerModule(new SimpleModule() {{
addDeserializer(MyInterface.class, new MyDeserializer());
}});
On a side-note, you might also consider extending StdNodeBasedDeserializer to implement deserialization based on JsonNode. For example:
#Override
public MyInterface convert(JsonNode root, DeserializationContext ctxt) throws IOException {
java.lang.reflect.Type targetType;
if (root.has("custom_field")) {
targetType = MyFailure.class;
} else {
targetType = MySuccess.class;
}
JavaType jacksonType = ctxt.getTypeFactory().constructType(targetType);
JsonDeserializer<?> deserializer = ctxt.findRootValueDeserializer(jacksonType);
JsonParser nodeParser = root.traverse(ctxt.getParser().getCodec());
nodeParser.nextToken();
return (MyInterface) deserializer.deserialize(nodeParser, ctxt);
}
There are a bunch of improvements to make to this custom deserializer, especially regarding tracking the context of the deserialization etc., but this should provide the functionality you're asking for.

In order to use your own ObjectMapper inside a custom deserializer, you can use Jackson Mix-in Annotations (the DefaultJsonDeserializer interface) to dynamically remove the custom deserializer from the POJO classes, avoiding the StackOverflowError that would otherwise be thrown as a result of objectMapper.readValue(JsonParser, Class<T>).
public class MyDeserializer extends JsonDeserializer<MyInterface> {
private static final ObjectMapper objectMapper = new ObjectMapper();
static {
objectMapper.addMixIn(MySuccess.class, DefaultJsonDeserializer.class);
objectMapper.addMixIn(MyFailure.class, DefaultJsonDeserializer.class);
}
#Override
public MyInterface deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException, JsonProcessingException {
if (jp.getCodec().<JsonNode>readTree(jp).has("custom_field")) {
return objectMapper.readValue(jp, MyFailure.class);
} else {
return objectMapper.readValue(jp, MySuccess.class);
}
}
#JsonDeserialize
private interface DefaultJsonDeserializer {
// Reset default json deserializer
}
}

I find a solution to use object mapper inside custom deserialize
public class DummyDeserializer extends JsonDeserializer<Dummy> {
#Override
public Dummy deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException {
ObjectMapper om = new ObjectMapper();
om.addMixIn(NexusAccount.class, DefaultJsonDeserializer.class);
ObjectCodec oc = jsonParser.getCodec();
JsonNode node = oc.readTree(jsonParser);
String serviceType = node.path("serviceType").asText();
switch (serviceType) {
case "Dummy1":
return om.treeToValue(node, Dumm1.class);
case "Dummy2":
return om.treeToValue(node, Dummy2.class);
case "Dummy3":
return om.treeToValue(node, Dummy3.class);
default:
throw new IllegalArgumentException("Unknown Dummy type");
}
}
#JsonDeserialize
private interface DefaultJsonDeserializer {
// Reset default json deserializer
}
}

This did the trick for me:
ctxt.readValue(node, MyFailure.class)

Jackson: Ignore whitespace in empty #XmlWrapperElement collection

Using Jackson and jackson-dataformat-xml 2.4.4, I'm trying to deserialize a XML document where a collection annotated with #XmlWrapperElement may have zero elements, but where the XML contains whitespace (in my case a line break). Jackson throws a JsonMappingException on this content with the message “Can not deserialize instance of java.util.ArrayList out of VALUE_STRING token”. I cannot change the way the XML is produced.
Example:
static class Outer {
#XmlElementWrapper
List<Inner> inners;
}
static class Inner {
#XmlValue
String foo;
}
ObjectMapper mapper = new XmlMapper().registerModules(new JaxbAnnotationModule());
String xml = "<outer><inners>\n</inners></outer>";
Outer outer = mapper.readValue(xml, Outer.class);
The following workarounds do not work:
Enabling DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY: In this case Jackson wants to instantiate a bogus instance of Inner using the whitespace as content.
Creating setters for this field for both String and the collection type. In this case I get a JsonMappingException (“Conflicting setter definitions for property "inners"”).
In a similar Stackoverflow question it is suggested to downgrade Jackson to 2.2.3. This does not fix the problem for me.
Any suggestions?
Edit: I can work around this issue by wrapping the CollectionDeserializer and checking for a whitespace token. This looks however very fragile to me, e.g. I had to override another method to rewrap the object. I can post the workaround, but a cleaner approach would be better.

A workaround for this problem is to wrap the standard CollectionDeserializer to return an empty collection for tokens containing whitespace and register the new Deserializer. I put the code into a Module so it can be registered easily:
import java.io.IOException;
import java.util.Collection;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.BeanDescription;
import com.fasterxml.jackson.databind.BeanProperty;
import com.fasterxml.jackson.databind.DeserializationConfig;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.deser.BeanDeserializerModifier;
import com.fasterxml.jackson.databind.deser.std.CollectionDeserializer;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.type.CollectionType;
public class XmlWhitespaceModule extends SimpleModule {
private static class CustomizedCollectionDeserialiser extends CollectionDeserializer {
public CustomizedCollectionDeserialiser(CollectionDeserializer src) {
super(src);
}
private static final long serialVersionUID = 1L;
#SuppressWarnings("unchecked")
#Override
public Collection<Object> deserialize(JsonParser jp, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
if (jp.getCurrentToken() == JsonToken.VALUE_STRING
&& jp.getText().matches("^[\\r\\n\\t ]+$")) {
return (Collection<Object>) _valueInstantiator.createUsingDefault(ctxt);
}
return super.deserialize(jp, ctxt);
}
#Override
public CollectionDeserializer createContextual(DeserializationContext ctxt,
BeanProperty property) throws JsonMappingException {
return new CustomizedCollectionDeserialiser(super.createContextual(ctxt, property));
}
}
private static final long serialVersionUID = 1L;
#Override
public void setupModule(SetupContext context) {
super.setupModule(context);
context.addBeanDeserializerModifier(new BeanDeserializerModifier() {
#Override
public JsonDeserializer<?> modifyCollectionDeserializer(
DeserializationConfig config, CollectionType type,
BeanDescription beanDesc, JsonDeserializer<?> deserializer) {
if (deserializer instanceof CollectionDeserializer) {
return new CustomizedCollectionDeserialiser(
(CollectionDeserializer) deserializer);
} else {
return super.modifyCollectionDeserializer(config, type, beanDesc,
deserializer);
}
}
});
}
}
After that you can add it to your ObjectMapper like this:
ObjectMapper mapper = new XmlMapper().registerModule(new XmlWhitespaceModule());

Gson not parsing Class variable

I am using Gson and I have an object that one of its fields is a Class
class A {
…
private Class aClass;
… }
When I parse the instance to Json using default Gson object aClass comes empty.
Any idea why?

You need custom type adapter. Here is example:
package com.sopovs.moradanen;
import java.lang.reflect.Type;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonDeserializationContext;
import com.google.gson.JsonDeserializer;
import com.google.gson.JsonElement;
import com.google.gson.JsonParseException;
import com.google.gson.JsonPrimitive;
import com.google.gson.JsonSerializationContext;
import com.google.gson.JsonSerializer;
public class GsonClassTest {
public static void main(String[] args) {
Gson gson = new GsonBuilder()
.registerTypeAdapter(Class.class, new ClassTypeAdapter())
.setPrettyPrinting()
.create();
String json = gson.toJson(new Foo());
System.out.println(json);
Foo fromJson = gson.fromJson(json, Foo.class);
System.out.println(fromJson.boo.getName());
}
public static class ClassTypeAdapter implements JsonSerializer<Class<?>>, JsonDeserializer<Class<?>> {
#Override
public JsonElement serialize(Class<?> src, Type typeOfSrc, JsonSerializationContext context) {
return new JsonPrimitive(src.getName());
}
#Override
public Class<?> deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context)
throws JsonParseException {
try {
return Class.forName(json.getAsString());
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);
}
}
}
public static class Foo {
Class<?> boo = String.class;
}
}
The output of this code is:
{
"boo": "java.lang.String"
}
java.lang.String

When I parse the instance to Json using default Gson object aClass comes empty.
Any idea why?
In a comment in issue 340, a Gson project manager explains:
Serializing types is actually somewhat of a security problem, so we don't want to support it by default. A malicious .json file could cause your application to load classes that it wouldn't otherwise; depending on your class path loading certain classes could DoS your application.
But it's quite straightforward to write a type adapter to support this in your own app.
Of course, since serialization is not the same as deserialization, I don't understand how this is an explanation for the disabled serialization, unless the unmentioned notion is to in a sense "balance" the default behaviors of serialization with deserialization.