Develop Reference

How to read a com.fasterxml.jackson.databind.node.TextNode from a Mongo DB and convert to a Map <String, Object>?

We are using SpringDataMongoDB in a Spring-boot app to manage our data.
Our previous model was this:
public class Response implements Serializable {
//...
private JsonNode errorBody; //<-- Dynamic
//...
}
JsonNode FQDN is com.fasterxml.jackson.databind.JsonNode
Which saved documents like so in the DB:
"response": {
...
"errorBody": {
"_children": {
"code": {
"_value": "Error-code-value",
"_class": "com.fasterxml.jackson.databind.node.TextNode"
},
"message": {
"_value": "Error message value",
"_class": "com.fasterxml.jackson.databind.node.TextNode"
},
"description": {
"_value": "Error description value",
"_class": "com.fasterxml.jackson.databind.node.TextNode"
}
},
"_nodeFactory": {
"_cfgBigDecimalExact": false
},
"_class": "com.fasterxml.jackson.databind.node.ObjectNode"
},
...
}
We've saved hundreds of documents like this on the production database without ever the need to read them programmatically as they are just kind of logs.
As we noticed that this output could be difficult to read in the future, we've decided to change the model to this:
public class Response implements Serializable {
//...
private Map<String,Object> errorBody;
//...
}
The data are now saved like so:
"response": {
...
"errorBody": {
"code": "Error code value",
"message": "Error message value",
"description": "Error description value",
...
},
...
}
Which, as you may have noticed is pretty much more simple.
When reading the data, ex: repository.findAll()
The new format is read without any issue.
But we face these issues with the old format:
org.springframework.data.mapping.MappingException: No property v found on entity class com.fasterxml.jackson.databind.node.TextNode to bind constructor parameter to!
Or
org.springframework.data.mapping.model.MappingInstantiationException: Failed to instantiate com.fasterxml.jackson.databind.node.ObjectNode using constructor NO_CONSTRUCTOR with arguments
Of course the TextNode class has a constructor with v as param but the property name is _value and ObjectNode has no default constructor: We simply can't change that.
We've created custom converters that we've added to our configurations.
public class ObjectNodeWriteConverter implements Converter<ObjectNode, DBObject> {
#Override
public DBObject convert(ObjectNode source) {
return BasicDBObject.parse(source.toString());
}
}
public class ObjectNodeReadConverter implements Converter<DBObject, ObjectNode> {
#Override
public ObjectNode convert(DBObject source) {
try {
return new ObjectMapper().readValue(source.toString(), ObjectNode.class);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
}
We did the same for TextNode
But we still got the errors.
The converters are read as we have a ZonedDateTimeConverter that is doing his job.
We can not just wipe out or ignore the old data as we need to read them too in order to study them.
How can we set up a custom reader that will not fail reading the old format ?

As I understood your issue, with the first model, you didn't really have a problem to save or to read in database but, once you wanted to fetch these datas, you noticed that the output is difficult to read. So your problem is to fetch a well readable output then you don't need to change the first model but to extends these classes and overide the toString method to change its behavior while fetching.
There are at least three classes to extends:
TextNode : you can't overide the toString method do that the custom class just print the value
ObjectNode : I can see that there are at least four field inside this class that you want to fecth the value: code, message, description. They are type of TextNode so you can replace them by thier extended classes. Then overide the toString method so that It print fieldName: field.toString() for each field
JsonNode : You can then extend this class and use the custom classes created above, overide the toString method so that It print as you want and use It instead of the common JsonNode
To work like that will make you avoid the way you save or you read the datas but just to fecth on the view.
You can consider it as a little part of the SOLID principle especially the OCP (Open an close principle: avoid to change the class behavoir but extends it to create a custom behavior) and the LSP (Liskov Substitution Principle: Subtypes must be behaviorlly substituable for thier base types).

Since old format is predefined and you know a structure of it you can implement custom deserialiser to handle old and new format at the same time. If errorBody JSON Object contains any of these keys: _children, _nodeFactory or _class you know it is an old format and you need to iterate over keys in _children JSON Object and get _value key to find a real value. Rest of keys and values you can ignore. Simple implementation could look like below:
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.TreeNode;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import lombok.Data;
import lombok.ToString;
import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import java.util.Collections;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
public class JsonMongo2FormatsApp {
public static void main(String[] args) throws IOException {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
JsonMapper mapper = JsonMapper.builder().build();
Response response = mapper.readValue(jsonFile, Response.class);
System.out.println(response.getErrorBody());
}
}
#Data
#ToString
class Response {
#JsonDeserialize(using = ErrorMapJsonDeserializer.class)
private Map<String, String> errorBody;
}
class ErrorMapJsonDeserializer extends JsonDeserializer<Map<String, String>> {
#Override
public Map<String, String> deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
TreeNode root = p.readValueAsTree();
if (!root.isObject()) {
// ignore everything except JSON Object
return Collections.emptyMap();
}
ObjectNode objectNode = (ObjectNode) root;
if (isOldFormat(objectNode)) {
return deserialize(objectNode);
}
return toMap(objectNode);
}
protected boolean isOldFormat(ObjectNode objectNode) {
final List<String> oldFormatKeys = Arrays.asList("_children", "_nodeFactory", "_class");
final Iterator<String> iterator = objectNode.fieldNames();
while (iterator.hasNext()) {
String field = iterator.next();
return oldFormatKeys.contains(field);
}
return false;
}
protected Map<String, String> deserialize(ObjectNode root) {
JsonNode children = root.get("_children");
Map<String, String> result = new LinkedHashMap<>();
children.fields().forEachRemaining(entry -> {
result.put(entry.getKey(), entry.getValue().get("_value").toString());
});
return result;
}
private Map<String, String> toMap(ObjectNode objectNode) {
Map<String, String> result = new LinkedHashMap<>();
objectNode.fields().forEachRemaining(entry -> {
result.put(entry.getKey(), entry.getValue().toString());
});
return result;
}
}
Above deserialiser should handle both formats.

Michal Ziober's answer did not completely solve the problem as we need to tell SpringData MongoDb that we want it to use the custom deserializer
(Annotating the model does not work with Spring data mongodb):
Define the custom deserializer
public class ErrorMapJsonDeserializer extends JsonDeserializer<Map<String, Object>> {
#Override
public Map<String, Object> deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
TreeNode root = p.readValueAsTree();
if (!root.isObject()) {
// ignore everything except JSON Object
return Collections.emptyMap();
}
ObjectNode objectNode = (ObjectNode) root;
if (isOldFormat(objectNode)) {
return deserialize(objectNode);
}
return toMap(objectNode);
}
protected boolean isOldFormat(ObjectNode objectNode) {
final List<String> oldFormatKeys = Arrays.asList("_children", "_nodeFactory", "_class");
final Iterator<String> iterator = objectNode.fieldNames();
while (iterator.hasNext()) {
String field = iterator.next();
return oldFormatKeys.contains(field);
}
return false;
}
protected Map<String, Object> deserialize(ObjectNode root) {
JsonNode children = root.get("_children");
if (children.isArray()) {
children = children.get(0);
children = children.get("_children");
}
return extractValues(children);
}
private Map<String, Object> extractValues(JsonNode children) {
Map<String, Object> result = new LinkedHashMap<>();
children.fields().forEachRemaining(entry -> {
String key = entry.getKey();
if (!key.equals("_class"))
result.put(key, entry.getValue().get("_value").toString());
});
return result;
}
private Map<String, Object> toMap(ObjectNode objectNode) {
Map<String, Object> result = new LinkedHashMap<>();
objectNode.fields().forEachRemaining(entry -> {
result.put(entry.getKey(), entry.getValue().toString());
});
return result;
}
}
Create a Custom mongo converter and pass it the custom deserializer.
Actually we do not pass the serializer directly but by means of an ObjectMapper configured with that Custom deserializer
public class CustomMappingMongoConverter extends MappingMongoConverter {
//The configured objectMapper that will be passed during instantiation
private ObjectMapper objectMapper;
public CustomMappingMongoConverter(DbRefResolver dbRefResolver, MappingContext<? extends MongoPersistentEntity<?>, MongoPersistentProperty> mappingContext, ObjectMapper objectMapper) {
super(dbRefResolver, mappingContext);
this.objectMapper = objectMapper;
}
#Override
public <S> S read(Class<S> clazz, Bson dbObject) {
try {
return objectMapper.readValue(dbObject.toString(), clazz);
} catch (IOException e) {
throw new RuntimeException(dbObject.toString(), e);
}
}
//in case you want to serialize with your custom objectMapper as well
#Override
public void write(Object obj, Bson dbo) {
String string = null;
try {
string = objectMapper.writeValueAsString(obj);
} catch (JsonProcessingException e) {
throw new RuntimeException(string, e);
}
((DBObject) dbo).putAll((DBObject) BasicDBObject.parse(string));
}
}
Create and configure the object mapper then instantiate the custom MongoMappingConverter and add it to Mongo configurations
public class MongoConfiguration extends AbstractMongoClientConfiguration {
//... other configuration method beans
#Bean
#Override
public MappingMongoConverter mappingMongoConverter() throws Exception {
DbRefResolver dbRefResolver = new DefaultDbRefResolver(mongoDbFactory());
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(FAIL_ON_UNKNOWN_PROPERTIES, false);
objectMapper.registerModule(new SimpleModule() {
{
addDeserializer(Map.class, new ErrorMapJsonDeserializer());
}
});
return new CustomMappingMongoConverter(dbRefResolver, mongoMappingContext(), objectMapper);
}
}

How to deserialize a string separated by comma to list with Jackson commonly?

I have a json like:
{
"names": "John, Tom",
"values": "8, 9",
"statuses": "yes, no"
}
and want to deserialize to:
class Bean {
private List<String> names;
private List<Integer> values;
private List<StatusEnum> statuses;
}
I know that implementing StringToStringListDeserializer, StringToIntegerListDeserializer, and StringToStatusEnumListDeserializer separately is practicable. But there are many other content types, including customized types. I tried:
public class StringToListDeserializer<T> extends JsonDeserializer<List<T>> implements ContextualDeserializer
public List<T> deserialize(JsonParser p, DeserializationContext context) throws IOException {
JavaType javaType = property.getType();
if (p.hasToken(JsonToken.VALUE_STRING)) {
String text = p.getText();
if (StringUtils.isBlank(text)) {
return null;
}
List<T> list = new LinkedList<>();
JavaType contentType = javaType.getContentType();
JsonDeserializer<Object> deserializer = context.findNonContextualValueDeserializer(contentType);
for (String s : text.split(DELIMITER)) {
// todo how to deserialize the string to a known type?
}
return list;
}
return context.readValue(p, javaType);
}
and i don't know how to deserialize the string to the known content type. Is there any way to implement a universal deserializer?

To avoid manual deserialisation and handling all possible types we can use a fact that all items on the list are also JSON elements when we wrap them with a quote (") char.
So, we can convert John, Tom to a "John", "Tom", 8, 9 to "8", "9" and so on.
We can use default Jackson behaviour which allows to handle unexpected tokens. In our case whenever: STRING token appears when JSON ARRAY was expected. To handle these cases we can use com.fasterxml.jackson.databind.deser.DeserializationProblemHandler class. It could look like below:
class ComaSeparatedValuesDeserializationProblemHandler extends DeserializationProblemHandler {
#Override
public Object handleUnexpectedToken(DeserializationContext ctxt, JavaType targetType, JsonToken token, JsonParser parser, String failureMsg) throws IOException {
if (token == JsonToken.VALUE_STRING && targetType.isCollectionLikeType()) {
return deserializeAsList(targetType, parser);
}
return super.handleUnexpectedToken(ctxt, targetType, token, parser, failureMsg);
}
private Object deserializeAsList(JavaType listType, JsonParser parser) throws IOException {
String[] values = readValues(parser);
ObjectMapper mapper = (ObjectMapper) parser.getCodec();
JavaType itemType = listType.getContentType();
List<Object> result = new ArrayList<>();
for (String value : values) {
result.add(convertToItemType(mapper, itemType, value));
}
return result;
}
private Object convertToItemType(ObjectMapper mapper, JavaType contentType, String value) throws IOException {
final String json = "\"" + value.trim() + "\"";
return mapper.readValue(json, contentType);
}
private String[] readValues(JsonParser p) throws IOException {
final String text = p.getText();
return text.split(",");
}
}
Example usage:
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.deser.DeserializationProblemHandler;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.google.common.base.Joiner;
import lombok.Data;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
public class ConvertStringToCollectionApp {
public static void main(String[] args) throws IOException {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
ObjectMapper mapper = JsonMapper.builder()
.addHandler(new ComaSeparatedValuesDeserializationProblemHandler())
.build();
Bean bean = mapper.readValue(jsonFile, Bean.class);
print(bean.getNames());
print(bean.getValues());
print(bean.getStatuses());
}
private static void print(List<?> values) {
values.stream().findFirst().ifPresent(value -> System.out.print(value.getClass().getSimpleName() + "s: "));
System.out.println(Joiner.on(", ").join(values));
}
}
#Data
class Bean {
private List<String> names;
private List<Integer> values;
private List<StatusEnum> statuses;
}
enum StatusEnum {
yes, no
}
Above app for your JSON payload prints:
Strings: John, Tom
Integers: 8, 9
StatusEnums: yes, no
I used Lombok and Guava libraries just to make it simple and short but they are not mandatory to make it work.

Your Bean doesn't correctly represents the JSON. The correct version should look something like below
class Bean {
private String names;
private Integer values;
private String statuses;
}
And you can use Object Mapper
ObjectMapper objectMapper = new ObjectMapper();
Bean bean = objectMapper.readValue(json, Bean.class);
Finally, you can break down your Bean object to list of names, values and status for your further usages.

How to convert a json string to java map [duplicate]

I have the following class:
import org.codehaus.jackson.annotate.JsonIgnoreProperties;
import org.codehaus.jackson.annotate.JsonProperty;
import java.io.Serializable;
import java.util.HashMap;
#JsonIgnoreProperties(ignoreUnknown = true)
public class Theme implements Serializable {
#JsonProperty
private String themeName;
#JsonProperty
private boolean customized;
#JsonProperty
private HashMap<String, String> descriptor;
//...getters and setters for the above properties
}
When I execute the following code:
HashMap<String, Theme> test = new HashMap<String, Theme>();
Theme t1 = new Theme();
t1.setCustomized(false);
t1.setThemeName("theme1");
test.put("theme1", t1);
Theme t2 = new Theme();
t2.setCustomized(true);
t2.setThemeName("theme2");
t2.setDescriptor(new HashMap<String, String>());
t2.getDescriptor().put("foo", "one");
t2.getDescriptor().put("bar", "two");
test.put("theme2", t2);
String json = "";
ObjectMapper mapper = objectMapperFactory.createObjectMapper();
try {
json = mapper.writeValueAsString(test);
} catch (IOException e) {
e.printStackTrace();
}
The json string produced looks like this:
{
"theme2": {
"themeName": "theme2",
"customized": true,
"descriptor": {
"foo": "one",
"bar": "two"
}
},
"theme1": {
"themeName": "theme1",
"customized": false,
"descriptor": null
}
}
My problem is getting the above json string to de-serizlize back into a
HashMap<String, Theme>
object.
My de-serialization code looks like this:
HashMap<String, Themes> themes =
objectMapperFactory.createObjectMapper().readValue(json, HashMap.class);
Which de-serializes into a HashMap with the correct keys, but does not create Theme objects for the values. I don't know what to specify instead of "HashMap.class" in the readValue() method.
Any help would be appreciated.

You should create specific Map type and provide it into deserialization process:
TypeFactory typeFactory = mapper.getTypeFactory();
MapType mapType = typeFactory.constructMapType(HashMap.class, String.class, Theme.class);
HashMap<String, Theme> map = mapper.readValue(json, mapType);

You can use TypeReference class which does the type casting for map with user defined types. More documentation at https://github.com/FasterXML/jackson-databind/
ObjectMapper mapper = new ObjectMapper();
Map<String,Theme> result =
mapper.readValue(src, new TypeReference<Map<String,Theme>>() {});

You can make a POJO that extends a Map.
This is important for dealing with nested maps of objects.
{
key1: { nestedKey1: { value: 'You did it!' } }
}
This can be deserialized via:
class Parent extends HashMap<String, Child> {}
class Child extends HashMap<String, MyCoolPojo> {}
class MyCoolPojo { public String value; }
Parent parent = new ObjectMapper().readValue(json, Parent.class);
parent.get("key1").get("nestedKey1").value; // "You did it!"

Mapping JSON response to Java POJO Class using GSON

I am trying to map below JSON to a POJO Class using Gson library. Below is the JSON response and POJO Class and mapping done
import java.util.Map;
import com.google.gson.JsonElement;
public class DataResponse {
private String $status;
private Map<String, JsonElement> $payload;
public String get$status() {
return $status;
}
public void set$status(String $status) {
this.$status = $status;
}
public Map<String, JsonElement> get$payload() {
return $payload;
}
public void set$payload(Map<String, JsonElement> $payload) {
this.$payload = $payload;
}
}
Here is the Sample JSON.
{
"$status": "OK",
"$payload": {
"$nextStart": "123",
"$results": [
{
"$key": "101",
"score": 3,
"to": "Test1"
},
{
"$key": "102",
"score": 4,
"to": "Test2"
},
]
}
}
Below is the mapping done. Is there some problem with POJO class definition. Since I cannot get all the elements of JSON response mapped to the innermost element from the response. Appreciate your support in providing useful suggestions.
Gson gson = new Gson();
DataResponse dataResponse = gson.fromJson(EntityUtils.toString(response.getEntity()),
DataResponse.class);

While working with marshalling and unmarshalling, it is always good to have a model defined as:
public class DataResponse {
private String $status;
private Payload $payload;
// getters and setters
}
class Payload {
private String $nextStart;
private List<Result> $results;
// getters and setters
}
class Result {
private String $key;
private String score;
private String to;
// getters and setters
}
Now when you convert json to POJO as:
Gson gson = new Gson();
DataResponse dataResponse = gson.fromJson(EntityUtils.toString(response.getEntity()), DataResponse.class);
it can easily convert it.
Also, believe me, it is good for processing in your further code!
Update: if you really want to convert json to Map, then you can do something like this:
import java.lang.reflect.Type;
import com.google.gson.reflect.TypeToken;
Type type = new TypeToken<Map<String, String>>(){}.getType();
Map<String, String> myMap = gson.fromJson("{'key':'value'}", type);
Substitute json string there.

ObjectMapper readValue

I load a ressource file json
with the text format
{
"sources": [{
"prop1": "1",
"prop2": "2"
},
{
"prop1": "1",
"prop2": "2"
},
],
"redirection": [{
"prop1": "1",
"prop2": "2"
}
]
}
I have a class with this properties prop1 and prop2
I want to recover with ObjectMapper a list class. What the method ?
This code doesn't work ....
Map<String, Object> mp = mapper.readValue(jsonResource.getInputStream(),new TypeReference<Map<String, Object>>() {});
String sourceText= new ObjectMapper().readTree(jsonResource.getInputStream()).get("sources").asText();
mapper.readValue(sourceText, new TypeReference<List<MyClass>>(){});
Thanks for your help

In your case, I would write a custom JsonDeserializer. Haven't really tested the code, but I think the idea is clear:
final MyClassDeserializer myClassDeserializer = new MyClassDeserializer();
final SimpleModule deserializerModule = new SimpleModule();
deserializerModule.addDeserializer(MyClass.class, myClassDeserializer);
final ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(deserializerModule);
And the code for JsonDeserializer:
public class MyClassDeserializer extends JsonDeserializer<MyClass> {
#Override
public MyClass deserialize(final JsonParser jsonParser, final DeserializationContext context)
throws IOException {
final JsonNode node = jsonParser.getCodec().readTree(jsonParser);
final JsonNode sourcesNode = node.get("sources");
if(node.isArray()) {
final ArrayNode arrayNode = (ArrayNode) node;
final Iterable<JsonNode> nodes = arrayNode::elements;
final Set<Source> set = StreamSupport.stream(nodes.spliterator(), false)
.map(mapper)
.collect(Collectors.toSet());
...
}
...
}

First thing: Your JSON is invalid. There is a comma after the second object in the sources array. This has to be deleted.
Second: I think you didn't choose the right type for your result. What your JSON represents is a map which maps from string to an array of objects. So the type should be something like Map<String, Props[]> (Since you didn't provide the name of your class, I called it Props.
With these considerations you can construct a MapType by using ObjectMappers getTypeFactory() method and deserialize the value using the constructed type like shown below.
ObjectMapper mapper = new ObjectMapper();
TypeFactory typeFactory = mapper.getTypeFactory();
MapType mapType = typeFactory.constructMapType(HashMap.class, String.class, Props[].class);
Map<String, Props[]> map = mapper.readValue(s, mapType);

I actually voted for the other answer, but this is my idea, to create the classes and let jackson do the work :
public class ResourceTest {
#Test
public void test1() throws IOException {
assertTrue(true);
Resource resource = new Resource();
resource.getRedirectrions().add(makeRedirectrion("rprop11", "rprop12"));
resource.getRedirectrions().add(makeRedirectrion("rprop21", "rprop22"));
resource.getSources().add(makeSource("sprop11","sprop12"));
resource.getSources().add(makeSource("sprop21","sprop22"));
String json = new ObjectMapper().writeValueAsString(resource);
System.out.println(json);
Resource resource1 = new ObjectMapper().readValue(json, Resource.class);
System.out.println(resource1);
}
private Source makeSource(String prop1, String prop2) {
Source source = new Source();
source.setProp1(prop1);
source.setProp2(prop2);
return source;
}
private Redirectrion makeRedirectrion(String prop1, String prop2) {
Redirectrion redirectrion = new Redirectrion();
redirectrion.setProp1(prop1);
redirectrion.setProp2(prop2);
return redirectrion;
}
}
Output is:
{"sources":[{"prop1":"sprop11","prop2":"sprop12"},{"prop1":"sprop21","prop2":"sprop22"}],"redirectrions":[{"prop1":"rprop11","prop2":"rprop12"},{"prop1":"rprop21","prop2":"rprop22"}]}
Resource{sources=[Source{prop1='sprop11', prop2='sprop12'}, Source{prop1='sprop21', prop2='sprop22'}], redirectrions=[Source{prop1='rprop11', prop2='rprop12'}, Source{prop1='rprop21', prop2='rprop22'}]}