Develop Reference

Jackson Deserializer default to null on failure

I have an issue where an upstream service is making requests and about 5% of the requests that are made are partially malformed. For example, instead of a nullable property coming to me as the object type I am expecting, I get a random string.
data class FooDTO(
val bar: BarDTO?,
val name: String
)
data class BarDTO(
val size: Int
)
But the payload I get looks like
{
"name": "The Name",
"bar": "uh oh random string instead of object"
}
I don't want to fail the request when this happens because the part of the data that is correct is still useful for my purposes, so what I want to do is just default the deserialization failure to null. I also have a few different sub-objects in my FooDTO that do this so I want a generic way to solve it for those specific fields. I know I can write a custom deserializer like the following to solve it on a 1-off basis.
class BarDtoDeserializer #JvmOverloads constructor(vc: Class<*>? = null) : StdDeserializer<BarDTO?>(vc) {
override fun deserialize(jp: JsonParser, ctxt: DeserializationContext): AnalysisDTO? {
return try {
val node = jp.codec.readTree<JsonNode>(jp)
BarDTO(size = node.get("size").numberValue().toInt())
} catch (ex: Throwable) {
null
}
}
}
And then I can decorate my BarDTO object with a #JsonDeserialize(using=BarDtoDeserializer::class) to force it to use that deserializer. What I am hoping to do is have some way to do this in a generic way.

Thanks to this answer https://stackoverflow.com/a/45484422/4161471 I've found that a DeserializationProblemHandler can be used to return 'null' for invalid data.
In your instance, override the handleMissingInstantiator() function. If other payloads have other types of bad data, other overrides may be required.
Also, I thought that CoercionConfig might be the answer, but I couldn't get it to work or find decent documentation. That might be another path.
import com.fasterxml.jackson.core.JsonParser
import com.fasterxml.jackson.databind.DeserializationContext
import com.fasterxml.jackson.databind.deser.DeserializationProblemHandler
import com.fasterxml.jackson.databind.deser.ValueInstantiator
import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper
import com.fasterxml.jackson.module.kotlin.readValue
data class FooDTO(
val name: String,
val bar: BarDTO?,
)
data class BarDTO(
val size: Int
)
fun main() {
val mapper = jacksonObjectMapper()
// mapper
// .coercionConfigFor(LogicalType.Textual)
// .setCoercion(CoercionInputShape.String, CoercionAction.AsNull)
mapper.addHandler(object : DeserializationProblemHandler() {
override fun handleMissingInstantiator(
ctxt: DeserializationContext?,
instClass: Class<*>?,
valueInsta: ValueInstantiator?,
p: JsonParser?,
msg: String?
): Any? {
println("returning null for value, $msg")
return null
}
})
val a: FooDTO = mapper.readValue(
"""
{
"name": "I'm variant A",
"bar": "uh oh random string instead of object"
}
"""
)
val b: FooDTO = mapper.readValue(
"""
{
"name": "I'm variant B",
"bar": {
"size": 2
}
}
"""
)
println(a)
println(b)
assert(a.bar == null)
assert(a.bar?.size == 2)
}
Output:
returning null for value, no String-argument constructor/factory method to deserialize from String value ('uh oh random string instead of object')
FooDTO(name=I'm variant A, bar=null)
FooDTO(name=I'm variant B, bar=BarDTO(size=2))
Process finished with exit code 0

No converter found capable of converting from type org.bson.BsonUndefined

I have mongo driver 3.2.2, spring data mongodb 1.9.1.RELEASE.
Collection :
{
"_id": "5728a1a5abdb9c352cda6432",
"isDeleted": null,
"name": undefined
},
{
"_id": "5728a1a5abdb9c352cda6433",
"isDeleted": null,
"name": null
}
When I try to fetch record with {"name":undefined} I get following exception.
org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type org.bson.BsonUndefined to type java.lang.String
at org.springframework.core.convert.support.GenericConversionService.handleConverterNotFound(GenericConversionService.java:313) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE]
at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:195) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE]
at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:176) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE]
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.getPotentiallyConvertedSimpleRead(MappingMongoConverter.java:821) ~[spring-data-mongodb-1.7.1.RELEASE.jar:?]
How to solve this? I have multiple types which needs to be converted from BsonUndefined like String, Date, PhoneNumber, etc...

We ran into this issue recently, so I'm putting the results of my research here in case someone else has this issue as well.
This is a known error in Spring Data MongoDB:
https://jira.spring.io/browse/DATAMONGO-1439
The workaround from that link is to add an explicit converter for converting from BsonUndefined to null. For example, using Java 8:
#ReadingConverter
public class BsonUndefinedToNullObjectConverterFactory implements ConverterFactory<BsonUndefined, Object> {
#Override
public <T extends Object> Converter<BsonUndefined, T> getConverter(Class<T> targetType) {
return o -> null;
}
}
Without the Lambda (pre-Java 8):
#ReadingConverter
public class BsonUndefinedToNullObjectConverterFactory implements ConverterFactory<BsonUndefined, Object> {
#Override
public <T extends Object> Converter<BsonUndefined, T> getConverter(Class<T> targetType) {
return new Converter<BsonUndefined, T>() {
#Override
public T convert(BsonUndefined source) {
return null;
}
};
}
}

I ran into this exact same problem on my code. I don't know why but for some reason the new mongo-java-drivers do not like having a "null" value at all in the data. If you notice that when you save an object and the value is null it actually doesn't even put the field in the document to start with.
We ended up having 1 collection that was written by a different "nodejs" application but being read by java. When we upgraded our mongo-java-driver to 3.2 version that particular collection started to break.
We ended up having to do an update on all the records in that collection similar to this
db.columnDefinition.update({field: null}, {$unset: {field: 1}}, {multi: true})
Once there were no records containing a "null" at all that was being mapped to a bean in the object, everything started working out just fine.
We did not have a single "undefined" in the collection but I can only guess that it will cause an issue as well.
And we were having the same BsonUndefined exception even though the problem had nothing to do with undefined.
org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [org.bson.BsonUndefined] to type [java.lang.String]
at org.springframework.core.convert.support.GenericConversionService.handleConverterNotFound(GenericConversionService.java:313) ~[spring-core-4.3.1.RELEASE.jar:4.3.1.RELEASE]
at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:195) ~[spring-core-4.3.1.RELEASE.jar:4.3.1.RELEASE]
at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:176) ~[spring-core-4.3.1.RELEASE.jar:4.3.1.RELEASE]
Edit: Also in our case we noticed that it didn't always seem to be a problem. We have other collections where we access them directly and they have "null" fields which are read in just fine. It seems to be related to anything that is being pulled in from a DBRef style. In my case the report had DBRef to column so reading the report broke because the column had a field that was null inside of it. But the report itself has fields that are null but does not break.
This is with spring-data 1.9.2, spring 4.3.1, mongo 3.2.2

For Java it does not really matter whether the field exists or has a null value so you could also replace it just bu null like in the answer above. However if you really want to match the fields that have the exact "undefined" value you can only do it in JS in Mongo console.
In my case I wanted to remove all of such fields from the object so the function doing both the proper check and remove would look like:
function unsetUndefined(obj) {
return Object.keys(obj).filter((k, i) => {
if(obj[k] === undefined) {
delete obj[k];
return true
}
return false;
}).length > 0;
}
Later combining it all together into executable Mongo Shell code it could be something like:
const collectionName = "...";
const query = {...};
function unsetUndefined(obj) {
return Object.keys(obj).some((k, i) => {
if(obj[k] === undefined) {
delete obj[k];
return true;
}
});
}
db.getCollection(collectionName).find(query).forEach(record =>{
const changed = unsetUndefined(record);
if(changed) {
db.getCollection(collectionName).save(record);
}
});

If undefined values are unwanted (i.e. were created due to some error) - you can unset them, by running following mongo script:
var TYPE_UNDEFINED = 6;
db.getCollection("my.lovely.collection").updateMany({ "name": {$type: TYPE_UNDEFINED}}, {
$unset: {"name": 1}
});
or set them to null - if that's preferred solution:
var TYPE_UNDEFINED = 6;
db.getCollection("my.lovely.collection").updateMany({ "name": {$type: TYPE_UNDEFINED}}, {
$set: {"name": null}
});

google gson LinkedTreeMap class cast to myclass

I knew this question has been asked before. Due to my novice skill in java and android. I can't resolve this issue for more than a week.
One of my friend and i developing an android project were there are a couple of things like this.
The most weird part of this things is, it's happening only from when i download and test it from Google play store. Not from local android studio installation or debug mode.
What could be the problem here, or this returning list which is totally wrong ?
My friend convincing that this code returns correctly but from play store installation it's always an error.
Please suggest me where should i keep digging?
#Override
public void promiseMethod(JSONObject object) {
if (object != null) {
if (object.has(DO_SERVICES)) {
vehicleDetails = new ArrayList < Object[] > (1);
List < String > vehicleNoList = new ArrayList < String > (1);
List < String > serviceList = new ArrayList < String > (1);
try {
Gson gson = new Gson();
JSONObject jsonObj = new JSONObject(object.get(DO_SERVICES)
.toString());
servDto = gson.fromJson(jsonObj.toString(),
ServiceDto.class);
if (servDto.getServiceDto() instanceof List) {
List < DoServiceDto > doServiceList = servDto.getServiceDto();
Exception is
java.lang.ClassCastException: com.google.gson.internal.LinkedTreeMap cannot be cast to com.gaurage.dto.DoServiceDto
at com.gaurage.user.User_Test_Main.promiseMethod(Unknown Source)

Serializing and Deserializing Generic Types
When you call toJson(obj), Gson calls obj.getClass() to get information on the fields to serialize. Similarly, you can typically pass MyClass.class object in the fromJson(json, MyClass.class) method. This works fine if the object is a non-generic type. However, if the object is of a generic type, then the Generic type information is lost because of Java Type Erasure. Here is an example illustrating the point:
class Foo<T> { T value;}
Gson gson = new Gson();
Foo<Bar> foo = new Foo<Bar>();
gson.toJson(foo); // May not serialize foo.value correctly
gson.fromJson(json, foo.getClass()); // Fails to deserialize foo.value as Bar
The above code fails to interpret value as type Bar because Gson invokes list.getClass() to get its class information, but this method returns a raw class, Foo.class. This means that Gson has no way of knowing that this is an object of type Foo, and not just plain Foo.
You can solve this problem by specifying the correct parameterized type for your generic type. You can do this by using the TypeToken class.
Type fooType = new TypeToken<Foo<Bar>>() {}.getType();
gson.toJson(foo, fooType);
gson.fromJson(json, fooType);
I have Parent class and it's child class some of them having List types in it. Like this parent class i have 30 files. I solved it like this.
Gson gson = new Gson();
JSONObject jsonObj = new JSONObject(object.get(DO_SERVICES).toString());
Type type = new TypeToken<MyDto>() {}.getType();
servDto = gson.fromJson(jsonObj.toString(),type);
The most important thing is, I can't reproduce this error in local testing from Android studio. This problem pops up only, When i generate signed apk and publish app into PlayStore were the app stops, and the report says Cannot cast LinkedTreeMap to myclass.
It was hard for me to reproduce the same result in my local testing (includes Debug mode).

EngineSense's answer is correct.
However, if you still want to use generics and don't want to pass in the concrete class type as a parameter here's an example of a workaround in Kotlin.
(Note that inline methods with reified type params cannot be called from Java).
May not be the most efficient way to get things done but it does work.
The following is in GsonUtil.kt
inline fun <reified T> fromJson(json: String): T? {
return gson.fromJson(json, object : TypeToken<T>() {}.type)
}
fun <T> mapToObject(map: Map<String, Any?>?, type: Class<T>): T? {
if (map == null) return null
val json = gson.toJson(map)
return gson.fromJson(json, type)
}
Method that retrieves a lightweight list of generic objects.
inline fun <reified T: MyBaseClass> getGenericList(): List<T> {
val json = ...
//Must use map here because the result is a list of LinkedTreeMaps
val list: ArrayList<Map<String, Any?>>? = GsonUtil.fromJson(json)
//handle type erasure
val result = list?.mapNotNull {
GsonUtil.mapToObject(it, T::class.java)
}
return result ?: listOf()
}

Source
In My ListView BaseAdapter facing same issue
JSON format to show
{
results: [
{
id: "10",
phone: "+91783XXXX345",
name: "Mr Example",
email: "freaky#jolly.com"
},
{
id: "11",
phone: "+9178XXXX66",
name: "Mr Foo",
email: "freaky#jolly.com"
}],
statusCode: "1",
count: "2"
}
I was facing this issue
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.hsa.ffgp.hapdfgdfgoon, PID: 25879
java.lang.ClassCastException: com.google.gson.internal.LinkedTreeMap cannot be cast to
com.hsa.......
Then I mapped data using LinkedTreeMap Key Value as below
...
...
#Override
public View getView(final int i, View view, ViewGroup viewGroup) {
if(view==null)
{
view= LayoutInflater.from(c).inflate(R.layout.listview_manage_clients,viewGroup,false);
}
TextView mUserName = (TextView) view.findViewById(R.id.userName);
TextView mUserPhone = (TextView) view.findViewById(R.id.userPhone);
Object getrow = this.users.get(i);
LinkedTreeMap<Object,Object> t = (LinkedTreeMap) getrow;
String name = t.get("name").toString();
mUserName.setText("Name is "+name);
mUserPhone.setText("Phone is "+phone);
return view;
}
...
...

import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
String json = new Gson().toJson("Your Value");
ArrayList<YourClassName> outputList = new Gson().fromJson("Receive Value", new TypeToken<ArrayList<YourClassName>>() {
}.getType());
Log.d("TAG", outputList.get(0).getName);

In my case error occurred while fetching a list of objects from shared preferences.
The error was solved by adding TypeToken as shown below:
public static <GenericClass> GenericClass getListOfObjectsFromSharedPref(Context context,String preferenceFileName, String preferenceKey ){
SharedPreferences sharedPreferences =
context.getSharedPreferences(preferenceFileName, 0);
Type type = new TypeToken<ArrayList<Classname.class>>() {}.getType();
String json = sharedPreferences.getString(preferenceKey, "");
final Gson gson = new Gson();
return gson.fromJson(json, type);
}
....

Jackson: can json specify the target type?

This is what my class looks like -
public class A {
private Map<String, Object> objects = null;
....
}
My json would be like -
{
"f1" : {
"name" : "some name",
"val" : 3
},
"f2" : {
"arg": {
some field/value pairs
}
}
}
What I want is to specify in the JSON itself the type to which it can be deserialized to. So the value for f1 would be converted to an object of class B and f2 would get converted to object of C.
My code will look like this -
Object o = objects.get("f1");
if (o instanceof B) {
...
} else if (o instanceof C) {
...
}
Is there a way to do this? I want the json to control the deserialization.

Yes, Jackson can use a type identifier if JSON document has it. This is usually done by using annotation #JsonTypeInfo.
There are multiple ways to add/use type identifier, both regarding how it is included in JSON document, and as to what kind of id is being used (type name or Java class name?).
The easiest way to see how things match is to actually start with a POJO, add #JsonTypeInfo annotation, and serialize it to see kind of JSON produced. And once you understood how inclusion works you can modify, if necessary, structure of JSON and/or Java class definition.

Configure XStream to dynamically map to different objects

I am hitting a RESTful 3rd party API that always sends JSON in the following format:
{
"response": {
...
}
}
Where ... is the response object that needs to be mapped back to a Java POJO. For instance, sometimes the JSON will contain data that should be mapped back to a Fruit POJO:
{
"response": {
"type": "orange",
"shape": "round"
}
}
...and sometimes the JSON will contain data that should be mapped back to an Employee POJO:
{
"response": {
"name": "John Smith",
"employee_ID": "12345",
"isSupervisor": "true",
"jobTitle": "Chief Burninator"
}
}
So depending on the RESTful API call, we need these two JSON results mapped back to one of the two:
public class Fruit {
private String type;
private String shape;
// Getters & setters for all properties
}
public class Employee {
private String name;
private Integer employeeId;
private Boolean isSupervisor;
private String jobTitle;
// Getters & setters for all properties
}
Unfortunately, I cannot change the fact that this 3rd party REST service always sends back a { "response": { ... } } JSON result. But I still need a way to configure a mapper to dynamically map such a response back to either a Fruit or an Employee.
First, I tried Jackson with limited success, but it wasn't as configurable as I wanted it to be. So now I am trying to use XStream with its JettisonMappedXmlDriver for mapping JSON back to POJOs. Here's the prototype code I have:
public static void main(String[] args) {
XStream xs = new XStream(new JettisonMappedXmlDriver());
xs.alias("response", Fruit.class);
xs.alias("response", Employee.class);
// When XStream sees "employee_ID" in the JSON, replace it with
// "employeeID" to match the field on the POJO.
xs.aliasField("employeeID", Employee.class, "employee_ID");
// Hits 3rd party RESTful API and returns the "*fruit version*" of the JSON.
String json = externalService.getFruit();
Fruit fruit = (Fruit)xs.fromXML(json);
}
Unfortunately when I run this I get an exception, because I have xs.alias("response", ...) mapping response to 2 different Java objects:
Caused by: com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$UnknownFieldException: No such field me.myorg.myapp.domain.Employee.type
---- Debugging information ----
field : type
class : me.myorg.myapp.domain.Employee
required-type : me.myorg.myapp.domain.Employee
converter-type : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
path : /response/type
line number : -1
version : null
-------------------------------
So I ask: what can I do to circumvent the fact that the API will always send back the same "wrapper" response JSON object? The only thing I can think of is first doing a String-replace like so:
String json = externalService.getFruit();
json = json.replaceAll("response", "fruit");
...
But this seems like an ugly hack. Does XStream (or another mapping framework) provide anything that would help me out in this particular case? Thansk in advance.

There are two ways with Jackson:
test manually that the wanted keys are there (JsonNode has the necessary methods);
use JSON Schema; there is one API in Java: json-schema-validator (yes, that is mine), which uses Jackson.
Write a schema matching your first object type:
{
"type": "object",
"properties": {
"type": {
"type": "string",
"required": true
},
"shape": {
"type": "string",
"required": true
}
},
"additionalProperties": false
}
Load this as a schema, validate your input against it: if it validates, you know you need to deserialize against your fruit class. Otherwise, make the schema for the second item type, validate against it as a security measure, and deserialize using the other class.
There are code examples for the API, too (version 1.4.x)

If you do know the actual type, it should be relatively straight-forward with Jackson.
You need to use a generic wrapper type like:
public class Wrapper<T> {
public T response;
}
and then the only trick is to construct type object to let Jackson know what T there is.
If it is statically available, you just do:
Wrapper<Fruit> wrapped = mapper.readValue(input, new TypeReference<Wrapper<Fruit>>() { });
Fruit fruit = wrapped.response;
but if it is more dynamically generated, something like:
Class<?> rawType = ... ; // determined using whatever logic is needed
JavaType actualType = mapper.getTypeFactory().constructGenericType(Wrapper.class, rawType);
Wrapper<?> wrapper = mapper.readValue(input, actualType);
Object value = wrapper.response;
but either way it "should just work". Note that in latter case you may be able to use base types ("? extends MyBaseType"), but in general dynamic type can't be specified.