Develop Reference

How to deserialize json data and get rid of all the excess information

I'm having a problem with deserializing json object into DTO class.
The dto object has the following structure:
#Getter
#Setter
#ToString
#RequiredArgsConstructor
#JsonIgnoreProperties(value = { "charsetinfo" })
public class SingleEngineJsonDto {
public List<SingleEngineDTO> engine;
public List<Timetable> timetable;
public List<Dailytable> dailytable;
}
And the output is:
[SingleEngineJsonDto(engine=null, timetable=null, dailytable=null),
SingleEngineJsonDto(engine=[SingleEngineDTO(name=state, … some data)],
timetable=[Timetable(weekDay=1,some data), more data],
dailytable=[Dailytable(date=2018-05-09, more data), more data])]
How do I get rid of this([SingleEngineJsonDto(engine=null, timetable=null, dailytable=null)) part? Original json contains metadata, that I don't need. The firs object in a big json object is ignored metadata and null fields, the second one is null fields filled in. The only idea that I have is list.get(1). And I was told that this solution is wrong.
UPDATE:
Original json structure:
[{"charsetinfo":{"name": "utf-8"}},{"engine":
[{"NAME": "stock","title": some data}],
"timetable":[{"week_day":1,"is_work_day":1,some data},
more data],"dailytable":[{"date": "2018-05-09","is_work_day":0,"start_time": "10:00:00",data}]}]
Desirialization:
#FeignClient(value = "engine", url = "engine")
public interface EngineClient {
#GetMapping
List<EngineJsonDto> getEngines(URI enginesUri,
#RequestParam(value = "lang", required = false) String lang);
#GetMapping
List<SingleEngineJsonDto> getEngine(URI engineUri, #RequestParam(value = "lang", required = false) String lang);
}
Service gets the data from client and gets a list of dto to work with. I use facory pattern to get pieces of data to work with parameters(engine,timetable,dailytable), and all of them had to look the same:
#Service
public class EngineParamEngine implements EngineFactoryInterface {
#Override
public List<SingleEngineDTO> getEngineObjectPart(List<SingleEngineJsonDto> list){
return list.get(1).getEngine(); //<<this
}
}
I was told to change it to list.get(0).Which gives me (engine=null, timetable=null, dailytable=null). So, I need to make the first object disappear somehow. I'm sorry, English is not my first language and I'm new to programming.

Deserializing JSON objects wrapped inside unnamed root object using Jackson

I have to work with an API that returns all objects wrapped in a unnamed root object. Something like this:
{
"user": {
"firstname":"Tom",
"lastname":"Riddle"
}
}
Here, I am interested in deserializing the user object only. But given the nature of the response, I will have to write a class that wraps the user object if I want to deserialize it successfully.
#Getter
#Setter
#ToString
// Wrapper class
public class Info {
private User user;
}
and then
#Getter
#Setter
#ToString
public class User {
private String firstname;
private String lastname;
}
All responses of the API return the response in this manner, so I am looking for a way to deserialize the response in such a way as to have one generic wrapper class that can be used to extract any type of JSON object.
I have tried this:
#Getter
#Setter
public class ResponseWrapper<T> {
private T responseBody;
}
and then
ResponseWrapper<User> userInfo = objectMapper.readValue(response.body().string(), ResponseWrapper.class);
But this results in the following exception:
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "user" (class com.redacted.response.ResponseWrapper), not marked as ignorable (one known property: "responseBody"])
So, is there any way for me to deserialize this response without having to write separate wrapper classes for each API response like this?

You can do something like this:
JsonNode jsonNode = objectMapper.readTree(response.body().string());
String content = jsonNode.elements().next().toString();
User user = objectMapper.readValue(content, User.class);
Output:
User(firstname=Tom, lastname=Riddle)

How to correctly map huge json file to Java (pojo)?

Large json files are available (from 100 MB to 2 GB).
How I can create java (pojo) from json? I need to get the names of all fields. For example, one class can have more than 1000 fields.
Tried the following libraries & online services:
JsonSchema2Pojo works well, but has a limit of about 50,000 characters at a time. Due to the presence of limits, I cannot completely map the class. Need to partially copy the json each time and then remove the duplicate fields.
http://www.jsonschema2pojo.org/
JsonToJava
https://github.com/astav/JsonToJava
Also tried several other online services. They also have limits or do not work correctly.
Example of json file:
{"type":"FeatureCollection","features":[
{"type":"Feature","geometry":{"type":"Point","coordinates":[27.5618791,53.902334]},"properties":{"ele":"280","name":"Минск","place":"city","capital":"yes","name:ar":"مينسك","name:be":"Мінск","name:bg":"Минск","name:bo":"མིན་སིཀ།","name:cs":"Minsk","name:cu":"Мѣньскъ","name:cv":"Минск","name:de":"Minsk","name:el":"Μινσκ","name:en":"Minsk","name:eo":"Minsko","name:es":"Minsk","name:et":"Minsk","name:fa":"مینسک","name:fi":"Minsk","name:fr":"Minsk","name:ga":"Minsc","name:gl":"Minsk - Мінск","name:he":"מינסק","name:hi":"मिन्‍स्‍क","name:hr":"Minsk","name:hu":"Minszk","name:hy":"Մինսկ","name:ia":"Minsk","name:io":"Minsk","name:is":"Minsk","name:it":"Minsk","name:ja":"ミンスク","name:ka":"მინსკი","name:kk":"Минск","name:kn":"ಮಿನ್ಸ್ಕ್","name:ko":"민스크","name:ku":"Mînsk","name:kv":"Минск","name:ky":"Минск","name:la":"Minscum","name:lt":"Minskas","name:lv":"Minska","name:mk":"Минск","name:ml":"മിൻസ്ക്","name:mr":"मिन्‍स्‍क","name:nl":"Minsk","name:no":"Minsk","name:oc":"Minsk","name:os":"Минск","name:pl":"Mińsk","name:pt":"Minsk","name:ru":"Минск","name:sk":"Minsk","name:sl":"Minsk","name:sr":"Минск","name:sv":"Minsk","name:ta":"மின்ஸ்க்","name:tg":"Минск","name:th":"มินสก์","name:tt":"Минск","name:ug":"مىنىسكى","name:uk":"Мінськ","name:ur":"منسک","name:vi":"Minxcơ","name:vo":"Minsk","name:yi":"מינסק","name:zh":"明斯克","website":"https://minsk.gov.by/","int_name":"Minsk","name:ast":"Minsk","name:ckb":"مینسک","name:csb":"Mińsk","name:jbo":"misk","name:mhr":"Минск","name:myv":"Минск ош","name:nds":"Minsk","name:pnb":"منسک","name:rue":"Мінск","name:sah":"Минскай","name:szl":"Mińsk","name:udm":"Минск","name:wuu":"明斯克","nat_name":"Мінск","old_name":"Менск","wikidata":"Q2280","wikipedia":"ru:Минск","population":"1982444","admin_level":"2","alt_name:be":"Менск","alt_name:vi":"Minsk;Minxcơva","name:prefix":"город","old_name:be":"Менск","addr:country":"BY","name:bat-smg":"Minsks","name:sr-Latn":"Minsk","wikipedia:be":"Мінск","wikipedia:en":"Minsk","wikipedia:pl":"Mińsk","addr:postcode":"220000","is_in:country":"Belarus","name:be-tarask":"Менск","source:name:oc":"Lo Congrès","is_in:continent":"Europe","population:date":"2018-01-01","capital_ISO3166-1":"yes","source:population":"Белстат","is_in:country_code":"BY"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5861178,53.9510657],[27.5861196,53.9510294],[27.5862319,53.9510313],[27.5862302,53.9510676],[27.5861178,53.9510657]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5885075,53.9512721],[27.5885398,53.9511614],[27.5887363,53.9511813],[27.588704,53.9512919],[27.5885075,53.9512721]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5881428,53.9512622],[27.5881704,53.9511135],[27.588421,53.9511296],[27.5883935,53.9512783],[27.5881428,53.9512622]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5892179,53.9516538],[27.5893274,53.9516011],[27.5893778,53.9516374],[27.5894352,53.9516098],[27.5895358,53.9516823],[27.5893689,53.9517626],[27.5892179,53.9516538]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.588949,53.9517486],[27.5891273,53.9516621],[27.5892764,53.9517686],[27.5890981,53.9518551],[27.588949,53.9517486]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5867595,53.9500825],[27.5867648,53.9499951],[27.5869291,53.9499985],[27.5869238,53.9500859],[27.5867595,53.9500825]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5830382,53.9503042],[27.5830422,53.9502013],[27.5832783,53.9502045],[27.5832743,53.9503074],[27.5830382,53.9503042]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5820906,53.9497446],[27.5820973,53.9496721],[27.5822371,53.9496765],[27.5822305,53.949749],[27.5820906,53.9497446]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5477403,53.8940388],[27.5482144,53.8939744],[27.5482459,53.8940791],[27.5482929,53.8940736],[27.5483083,53.8941424],[27.5482593,53.8941471],[27.5482741,53.8942048],[27.5485718,53.8943067],[27.5487113,53.8943028],[27.5487556,53.8943755],[27.548895,53.8944261],[27.5487985,53.894513],[27.5488079,53.8945841],[27.5483385,53.8943889],[27.548317,53.8943202],[27.5481439,53.8942549],[27.5481045,53.8941843],[27.5479863,53.8942063],[27.5479953,53.8942214],[27.5478248,53.8942506],[27.5477403,53.8940388]]]]},"properties":{"name":"Инженерный корпус метрополитена","building":"yes"}}
]}
Thanks in advance!

Thanks a lot for your advices.
Implemented in the following way:
I used JsonAnySetter annotation to set all fields instead creating pojo with all of the possible fields.
Here is my Dto's
FeatureDto
#Getter
#ToString
public class FeatureDto {
#JsonProperty("type")
private String type;
#JsonProperty("geometry")
private GeometryDto geometry;
#JsonProperty("properties")
private PropertiesDto properties;
}
PropertiesDto
#Getter
#ToString
public class PropertiesDto {
private List<FieldDto> fieldDtos = new ArrayList<>();
#JsonAnySetter
public void setProperties(String code, Object value)
{
if(value instanceof ArrayList) {
ArrayList<Object> list = (ArrayList<Object>) value;
StringBuffer stringBuffer = new StringBuffer();
list.forEach(o -> stringBuffer.append(o).append(", "));
fieldDtos.add(new FieldDto(code, stringBuffer.toString()));
return;
}
fieldDtos.add(new FieldDto(code, value.toString()));
}
}
FieldDto
#Data
#ToString
#JsonIgnoreProperties(ignoreUnknown = true)
#AllArgsConstructor
public class FieldDto {
#JsonProperty("code")
private String code;
#JsonProperty("value")
private String value;
}
Here is the json source:
{"type":"FeatureCollection","features":[
{"type":"Feature","geometry":{"type":"Point","coordinates":[27.5618791,53.902334]},"properties":{"ele":"280","name":"Минск","place":"city","capital":"yes","name:ar":"مينسك","name:be":"Мінск","name:bg":"Минск","name:bo":"མིན་སིཀ།","name:cs":"Minsk","name:cu":"Мѣньскъ","name:cv":"Минск","name:de":"Minsk","name:el":"Μινσκ","name:en":"Minsk","name:eo":"Minsko","name:es":"Minsk","name:et":"Minsk","name:fa":"مینسک","name:fi":"Minsk","name:fr":"Minsk","name:ga":"Minsc","name:gl":"Minsk - Мінск","name:he":"מינסק","name:hi":"मिन्‍स्‍क","name:hr":"Minsk","name:hu":"Minszk","name:hy":"Մինսկ","name:ia":"Minsk","name:io":"Minsk","name:is":"Minsk","name:it":"Minsk","name:ja":"ミンスク","name:ka":"მინსკი","name:kk":"Минск","name:kn":"ಮಿನ್ಸ್ಕ್","name:ko":"민스크","name:ku":"Mînsk","name:kv":"Минск","name:ky":"Минск","name:la":"Minscum","name:lt":"Minskas","name:lv":"Minska","name:mk":"Минск","name:ml":"മിൻസ്ക്","name:mr":"मिन्‍स्‍क","name:nl":"Minsk","name:no":"Minsk","name:oc":"Minsk","name:os":"Минск","name:pl":"Mińsk","name:pt":"Minsk","name:ru":"Минск","name:sk":"Minsk","name:sl":"Minsk","name:sr":"Минск","name:sv":"Minsk","name:ta":"மின்ஸ்க்","name:tg":"Минск","name:th":"มินสก์","name:tt":"Минск","name:ug":"مىنىسكى","name:uk":"Мінськ","name:ur":"منسک","name:vi":"Minxcơ","name:vo":"Minsk","name:yi":"מינסק","name:zh":"明斯克","website":"https://minsk.gov.by/","int_name":"Minsk","name:ast":"Minsk","name:ckb":"مینسک","name:csb":"Mińsk","name:jbo":"misk","name:mhr":"Минск","name:myv":"Минск ош","name:nds":"Minsk","name:pnb":"منسک","name:rue":"Мінск","name:sah":"Минскай","name:szl":"Mińsk","name:udm":"Минск","name:wuu":"明斯克","nat_name":"Мінск","old_name":"Менск","wikidata":"Q2280","wikipedia":"ru:Минск","population":"1982444","admin_level":"2","alt_name:be":"Менск","alt_name:vi":"Minsk;Minxcơva","name:prefix":"город","old_name:be":"Менск","addr:country":"BY","name:bat-smg":"Minsks","name:sr-Latn":"Minsk","wikipedia:be":"Мінск","wikipedia:en":"Minsk","wikipedia:pl":"Mińsk","addr:postcode":"220000","is_in:country":"Belarus","name:be-tarask":"Менск","source:name:oc":"Lo Congrès","is_in:continent":"Europe","population:date":"2018-01-01","capital_ISO3166-1":"yes","source:population":"Белстат","is_in:country_code":"BY"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5861178,53.9510657],[27.5861196,53.9510294],[27.5862319,53.9510313],[27.5862302,53.9510676],[27.5861178,53.9510657]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5885075,53.9512721],[27.5885398,53.9511614],[27.5887363,53.9511813],[27.588704,53.9512919],[27.5885075,53.9512721]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5881428,53.9512622],[27.5881704,53.9511135],[27.588421,53.9511296],[27.5883935,53.9512783],[27.5881428,53.9512622]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5892179,53.9516538],[27.5893274,53.9516011],[27.5893778,53.9516374],[27.5894352,53.9516098],[27.5895358,53.9516823],[27.5893689,53.9517626],[27.5892179,53.9516538]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.588949,53.9517486],[27.5891273,53.9516621],[27.5892764,53.9517686],[27.5890981,53.9518551],[27.588949,53.9517486]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5867595,53.9500825],[27.5867648,53.9499951],[27.5869291,53.9499985],[27.5869238,53.9500859],[27.5867595,53.9500825]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5830382,53.9503042],[27.5830422,53.9502013],[27.5832783,53.9502045],[27.5832743,53.9503074],[27.5830382,53.9503042]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5820906,53.9497446],[27.5820973,53.9496721],[27.5822371,53.9496765],[27.5822305,53.949749],[27.5820906,53.9497446]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5477403,53.8940388],[27.5482144,53.8939744],[27.5482459,53.8940791],[27.5482929,53.8940736],[27.5483083,53.8941424],[27.5482593,53.8941471],[27.5482741,53.8942048],[27.5485718,53.8943067],[27.5487113,53.8943028],[27.5487556,53.8943755],[27.548895,53.8944261],[27.5487985,53.894513],[27.5488079,53.8945841],[27.5483385,53.8943889],[27.548317,53.8943202],[27.5481439,53.8942549],[27.5481045,53.8941843],[27.5479863,53.8942063],[27.5479953,53.8942214],[27.5478248,53.8942506],[27.5477403,53.8940388]]]]},"properties":{"name":"Инженерный корпус метрополитена","building":"yes"}}
]}

Java, How can I use Jackson ObjectMapper to deserialize a double nested json list

I'm trying to deserialize json objects with the following structure:
{"results":{
"Assessments":{
"Assessment":[
{
"assessor":"",
"buildingName":"Emerald Palace Project",
"certBody":"",
...
The top level is a single entity named "results" which contains "assessments" which are just a list/array of "assessment."
I've tried multiple combinations of mapper configuration such as:
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationConfig.Feature.FAIL_ON_UNKNOWN_PROPERTIES, false);
mapper.configure(DeserializationConfig.Feature.UNWRAP_ROOT_VALUE, true);
But no matter how I switch it up I keep getting a Results object with null Assessments.
Here are my object classes using Lombok.
#Getter
#Setter
#NoArgsConstructor
#AllArgsConstructor
static class Results {
private Assessments assessments;
}
#Getter
#Setter
#NoArgsConstructor
#AllArgsConstructor
static class Assessments {
private List<Assessment> assessments;
}
#Getter
#Setter
#NoArgsConstructor
#AllArgsConstructor
public class Assessment {
private int parentId;
private String stage;
private String notes;
private String rating;
private String scheme;
}
I feel like I must be missing a piece.
Any insight would be greatly appreciated.

Just to be clear: you wrote that
[…] "assessments" which are just a list/array of "assessment."
and your POJOs reflect that... then your JSON has to be…
{"results":{
"Assessments":{
[
{
"assessor":"",
"buildingName":"Emerald Palace Project",
"certBody":"",
...
…instead of…
{"results":{
"Assessments":{
"Assessment":[
{
"assessor":"",
"buildingName":"Emerald Palace Project",
"certBody":"",
...
Watch out for the list directly in "Assessments".
Also mind that your POJO fields are lowercase and so have to be your JSON keys.
So to match POJO with JSON you have various options:
change your JSON keys to lowercase and camelcase
use #JsonProperty("field-name") to match your JSON keys
…
But in each case you have to be aware of the current clash regarding the Assessments/Assessments list.

So because I had to work with the json given to me by the third party API, altering the json was not an option.
Here's how I got this working.
First off I had to change the Assessments object to have a list with a singular name. "assessment" instead of "assessments" as that is how it was spelled in the json response.
#Getter
#Setter
#NoArgsConstructor
static class Assessments {
private List<Assessment> assessment;
}
Also I set up my ObjectMapper to ignore case and unknown properties. Making sure to use com.fasterxml.jackson.databind.ObjectMapper.
private static ObjectMapper getJsonMapper() {
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
mapper.configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, true);
return mapper;
}
The biggest change however was the introduction of a container class.
#Getter
#Setter
#NoArgsConstructor
private static class JsonContainer {
private Results results;
}
This class was required as it held the top level json object "Results."
After these changes were in place I got the Java objects in the state I expected.

Nesting multiple levels of Jackson WRAPPER_OBJECTs

By no means am I a Jackon/JSON wizard, which is probably evident from the following issue I'm running into:
I have 2 possible data structures I'm receiving.
The first one is called amountTransaction:
{
"amountTransaction": {
"clientCorrelator":"54321",
"endUserId":"tel:+16309700001"
}
}
Which is represented by the following Java object:
#JsonIgnoreProperties(ignoreUnknown = true)
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.WRAPPER_OBJECT)
#JsonTypeName(value = "amountTransaction")
#JsonInclude(JsonInclude.Include.NON_NULL)
public class AmountTransaction {
private String clientCorrelator;
private String endUserId;
...
}
However the amountTransaction object also appears as child element of the paymentTransactionNotification object:
{
"paymentTransactionNotification": {
"amountTransaction": {
"clientCorrelator": "54321",
"endUserId": "tel:+16309700001"
}
}
}
..which I thought would be represented by:
#JsonIgnoreProperties(ignoreUnknown = true)
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.WRAPPER_OBJECT)
#JsonTypeName(value = "paymentTransactionNotification")
#JsonInclude(JsonInclude.Include.NON_NULL)
public class PaymentTransactionNotification {
private AmountTransaction amountTransaction;
...
}
Parsing the JSON with the amountTransaction object alone works fine. It's a pretty straightforward example of a WRAPPER_OBJECT.
However when trying to parse the JSON for the paymentTransactionNotification, I'm getting an exception indicating that it can't properly deal with the amountTransaction as element of the paymentTransactionNotification:
com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id 'clientCorrelator' into a subtype of [simple type, class com.sf.oneapi.pojos.AmountTransaction]
Any thoughts on how I can properly annotate this so my code can properly deal with both stand alone, as well as encapsulated amountTransaction objects?

By default wrapping root node in Jackson is disabled. You can wrap inner objects but if you want to wrap root node you need to enable jackson feature for it (https://jira.codehaus.org/browse/JACKSON-747):
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.enable(SerializationConfig.Feature.WRAP_ROOT_VALUE);
objectMapper.enable(DeserializationConfig.Feature.UNWRAP_ROOT_VALUE);
When you enabled these features you already said Jackson to wrap the root element and you don't need #JsonTypeInfo and #JsonTypeName anymore. You can simple delete them. But now you need to customize the root node name and you can use #JsonRootName for it. Your classes should look like this:
#JsonIgnoreProperties(ignoreUnknown = true)
#JsonRootName("amountTransaction")
#JsonInclude(JsonInclude.Include.NON_NULL)
public class AmountTransaction {
private String clientCorrelator;
private String endUserId;
...............
}
And
#JsonIgnoreProperties(ignoreUnknown = true)
#JsonRootName("paymentTransactionNotification")
#JsonInclude(JsonInclude.Include.NON_NULL)
public class PaymentTransactionNotification {
private AmountTransaction amountTransaction;
.............
}
I've tried and Jackson converted both JSON requests as expected.