I am getting data from an external JSON API and parsing the result with Jackson. Unfortunately, that API returns all fields as String and those are filled with "N/A" when data is unavailable.
I would like to replace those fields with null, especially as those are frustrating when I try to convert JSON String fields to more informative Java fields.
A custom DeserializationProblemHandler worked for Integer fields (see below), but it was useless for Java 8's LocalDate fields. Furthermore, it reacts to a problem rather than anticipating it.
I was unable to find a pre-processor to configure into my ObjectMapper and am uneasy with the idea of overriding the BeanDeserializer.
Do you know of a better/cleaner solution to handle this kind of situations? Thanks!
DeserializationProblemHandler
new DeserializationProblemHandler() {
#Override
public Object handleWeirdStringValue(DeserializationContext ctxt, Class<?> targetType, String valueToConvert, String failureMsg) throws IOException {
return "N/A".equals(valueToConvert) ? null : super.handleWeirdStringValue(ctxt, targetType, valueToConvert, failureMsg);
}
}
Error message when processing "N/A" in LocalDate field
Can not deserialize value of type java.time.LocalDate from String "N/A": Text 'N/A' could not be parsed at index 0
(works fine when there is date in the data)
I feel like there ought to be a better way of doing this, but the following is the only solution I was able to come up with.
Create a new JsonDeserializer that handles "N/A" input. The following example handles strings:
public class EmptyStringDeserializer extends StdScalarDeserializer<String> {
public EmptyStringDeserializer() {
super(String.class);
}
#Override
public String deserialize(JsonParser parser, DeserializationContext ctx) throws IOException {
final String val = parser.getValueAsString();
if ("N/A".equalsIgnoreCase(val))
return null;
return val;
}
}
The class is registered with an ObjectMapper like this:
SimpleModule simpleModule = new SimpleModule().addDeserializer(String.class, new EmptyStringDeserializer());
ObjectMapper om = new ObjectMapper().registerModule(simpleModule);
You'll probably want to collect all your converters in a module named for the API that is making you handle things this way.
Related
I have a JSON string which I would like to translate into POJO using ObjectMapper.readValue method.
The thing is that the input Json string contains keys which I would like to filter out before the deserialization.
I came across DelegatingDeserialization class which according to my understanding allows you to extend it and override one of the deserialize method to reconstruct the json input and then pass it on the chain.
The thing is that I try to enable this custom delegating deserializer by adding the
#JsonDeserialize(using = CustomDelegatingDeserialization.class) on top of my Pojo - is that the right way to instantiate it??
Here is a snippet of my custom delegator:
public static class CustomDeserializer extends DelegatingDeserializer {
public CustomDeserializer() {
super(null);
}
public CustomDeserializer(JsonDeserializer<?> defaultDeserializer) {
super(defaultDeserializer);
}
#Override
protected JsonDeserializer<?> newDelegatingInstance(JsonDeserializer<?> newDelegatee) {
return new CustomDeserializer(newDelegatee);
}
#Override
public Object deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
return super.deserialize(restructure(p), ctxt);
}
private JsonParser restructure(JsonParser jp) throws IOException {
...
return newJsonParser;
}
}
Am I taking the right path or there is a more fitting solution??
THank you!
EDIT 1
Another approach is to have a CustomJsonDeserializer extends JsonDeserializer<T> and override its deserialize method then reconstruct the Node and propagate it by returning codec.treeToValue(jsonNode, Pojo.class); this makes sense BUT it gets me into infinite loop! any idea why?
Assuming your POJO doesn't have a property that you would like to ignore you can use annotation #JsonIgnoreProperties(ignoreUnknown = true)for your class. That tells Jeckson to ignore properties that are not present in your POJO. Read more on the issue how to ignore some properties here: Jackson Unmarshalling JSON with Unknown Properties
Consider json input:
{
companies: [
{
"id": 1,
"name": "name1"
},
{
"id": 1,
"name": "name1"
}
],
nextPage: 2
}
How deserialize this into class:
public class MyClass {
List<String> companies;
Integer nextPage;
}
Where List<String> companies; consists of strings:
{"id": 1,"name": "name1"}
{"id": 1,"name": "name1"}
#JsonRawValue doesn't work for List<String> companies;
Is there a way to configure Jackson serialization to keep companies array with raw json string with annotations only? (E.g. without writing custom deserializator)
There is no annotation-only solution for your problem. Somehow you have to convert JSON Object to java.lang.String and you need to specify that conversion.
You can:
Write custom deserializer which is probably most obvious solution but forbidden in question.
Register custom com.fasterxml.jackson.databind.deser.DeserializationProblemHandler and handle com.fasterxml.jackson.databind.exc.MismatchedInputException situation in more sophisticated way.
Implement com.fasterxml.jackson.databind.util.Converter interface and convert JsonNode to String. It is semi-annotational way to solve a problem but we do not implement the worst part - deserialisation.
Let's go to point 2. right away.
2. DeserializationProblemHandler
Solution is pretty simple:
ObjectMapper mapper = new ObjectMapper();
mapper.addHandler(new DeserializationProblemHandler() {
#Override
public Object handleUnexpectedToken(DeserializationContext ctxt, JavaType targetType, JsonToken t, JsonParser p, String failureMsg) throws IOException {
if (targetType.getRawClass() == String.class) {
// read as tree and convert to String
return p.readValueAsTree().toString();
}
return super.handleUnexpectedToken(ctxt, targetType, t, p, failureMsg);
}
});
Read a whole piece of JSON as TreeNode and convert it to String using toString method. Helpfully, toString generates valid JSON. Downside, this solution has a global scope for given ObjectMapper instance.
3. Custom Converter
This solution requires to implement com.fasterxml.jackson.databind.util.Converter interface which converts com.fasterxml.jackson.databind.JsonNode to String:
class JsonNode2StringConverter implements Converter<JsonNode, String> {
#Override
public String convert(JsonNode value) {
return value.toString();
}
#Override
public JavaType getInputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<JsonNode>() {
});
}
#Override
public JavaType getOutputType(TypeFactory typeFactory) {
return typeFactory.constructType(new TypeReference<String>() {
});
}
}
and now, you can use annotation like below:
#JsonDeserialize(contentConverter = JsonNode2StringConverter.class)
private List<String> companies;
Solutions 2. and 3. solve this problem almost in the same way - read node and convert it back to JSON, but uses different approaches.
If, you want to avoid deserialising and serialising process you can take a look on solution provided in this article: Deserializing JSON property as String with Jackson and take a look at:
How to serialize JSON with array field to object with String field?
How to get a part of JSON as a plain text using Jackson
How to extract part of the original text from JSON with Jackson?
Jackson reads input numbers as Strings. As an example below Student class read name 4567 as a String.
ex: input
{
name: 4567
...
}
Java class
Class Student {
String name;
...
}
Jackson is parsing the JSON text and mapping the number value to the string field, and I don't want the type conversion, i.e. for the number to be converted to a string. In this scenario Jackson converts the value from int (4567) to String("4567").
How this behavior can changes to throw an exception if other type provided to fail ?
Custom deserializer registered for java.lang.String should definitely work and let you prevent conversion. Deserializers will directly see content via JsonParser so they can detect underlying token type.
This will help you:
public class ForceStringDeserializer extends JsonDeserializer<String> {
#Override
public String deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException {
if (jsonParser.getCurrentToken() == JsonToken.VALUE_NUMBER_INT) {
throw deserializationContext.wrongTokenException(jsonParser, JsonToken.VALUE_STRING, "Attempted to parse int to string but this is forbidden");
}
return jsonParser.getValueAsString();
}
}
You can find more info here.
add to application.properties
spring.jackson.mapper.allow-coercion-of-scalars=false
or if you configure object maper like bean
objectMapper.configure(MapperFeature.ALLOW_COERCION_OF_SCALARS, false);
I'm looking into a solution for Jackson deserialization JSON to an instance of a class without breaking the whole process, currently when I do something like:
If Actor.class was like:
#JsonInclude(JsonInclude.Include.NON_EMPTY)
#JsonPropertyOrder(alphabetic = true)
public abstract class BaseDTO {
}
public class Character extend BaseDTO {
private LocalDateTime updatedDate;
private String name;
// Setters and getters
}
and deserialize json {"updatedDate":"N/A", "name": "Jon Snow"} like:
String json = "{\"updatedDate\":\"N/A\", \"name\": \"Jon Snow\"}";
ObjectMapper mapper = new ObjectMapper();
final Character character = mapper.reader().forType(Character.class).readValue(json);
Or as Play Framework directly:
final Character character = Json.fromJson(json, Character.class);
I definitely will get an exception like:
Caused by: com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.time.LocalDateTime` from String "N/A": Text 'N/A' could not be parsed at index 0
And InvalidFormatException actually is a JsonProcessingException, there is also MismatchedInputException and other exceptions, so I need somehow gracefully continue with processing and get the object character and have name value at least instead of stopping it at all.
I prefer to:
Use annotations to config the parser or any solution to be applied to BaseDTO.
Logging the issue in the log file so I know that something wrong happened.
I really can't find the way right now without a huge effort, so I wonder if there is any out-of-box solution do that without re-invent the wheel.
I am assuming that you wannt ot consider string like N/A as null. Using a custom deserializer it can be achieved like this.
class LocalDateTimeDeserializer extends JsonDeserializer<LocalDateTime> {
#Override
public LocalDateTime deserialize(JsonParser p, DeserializationContext context) throws IOException {
if ("N/A".equals(p.getText())) return null;
return LocalDateTime.from(DateTimeFormatter.ISO_OFFSET_DATE_TIME.parse(p.getText()));
}
}
Hope this helps!
Looking through the documentation and source code I don't see a clear way to do this. Curious if I'm missing something.
Say I receive an InputStream from a server response. I create a JsonParser from this InputStream. It is expected that the server response is text containing valid JSON, such as:
{"iamValidJson":"yay"}
However, if the response ends up being invalid JSON or not JSON at all such as:
Some text that is not JSON
the JsonParser will eventually throw an exception. In this case, I would like to be able to extract the underlying invalid text "Some text that is not JSON" out of the JsonParser so it can be used for another purpose.
I cannot pull it out of the InputStream because it doesn't support resetting and the creation of the JsonParser consumes it.
Is there a way to do this?
If you have the JsonParser then you can use jsonParser.readValueAsTree().toString().
However, this likely requires that the JSON being parsed is indeed valid JSON.
I had a situation where I was using a custom deserializer, but I wanted the default deserializer to do most of the work, and then using the SAME json do some additional custom work. However, after the default deserializer does its work, the JsonParser object current location was beyond the json text I needed. So I had the same problem as you: how to get access to the underlying json string.
You can use JsonParser.getCurrentLocation.getSourceRef() to get access to the underlying json source. Use JsonParser.getCurrentLocation().getCharOffset() to find the current location in the json source.
Here's the solution I used:
public class WalkStepDeserializer extends StdDeserializer<WalkStep> implements
ResolvableDeserializer {
// constructor, logger, and ResolvableDeserializer methods not shown
#Override
public MyObj deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException,
JsonProcessingException {
MyObj myObj = null;
JsonLocation startLocation = jp.getCurrentLocation();
long charOffsetStart = startLocation.getCharOffset();
try {
myObj = (MyObj) defaultDeserializer.deserialize(jp, ctxt);
} catch (UnrecognizedPropertyException e) {
logger.info(e.getMessage());
}
JsonLocation endLocation = jp.getCurrentLocation();
long charOffsetEnd = endLocation.getCharOffset();
String jsonSubString = endLocation.getSourceRef().toString().substring((int)charOffsetStart - 1, (int)charOffsetEnd);
logger.info(strWalkStep);
// Special logic - use JsonLocation.getSourceRef() to get and use the entire Json
// string for further processing
return myObj;
}
}
And info about using a default deserializer in a custom deserializer is at How do I call the default deserializer from a custom deserializer in Jackson
5 year late, but this was my solution:
I converted the jsonParser to string
String requestString = jsonParser.readValueAsTree().toString();
Then I converted that string into a JsonParser
JsonFactory factory = new JsonFactory();
JsonParser parser = factory.createParser(requestString);
Then I iterated through my parser
ObjectMapper objectMapper = new ObjectMapper();
while(!parser.isClosed()){
JsonToken jsonToken = parser.nextToken();
if(JsonToken.FIELD_NAME.equals(jsonToken)){
String currentName = parser.getCurrentName();
parser.nextToken();
switch (currentName) {
case "someObject":
Object someObject = objectMapper.readValue(parser, Object.class)
//validate someObject
break;
}
}
I needed to save the original json string for logging purposes, which is why I did this in the first place. Was a headache to find out, but finally did it and I hope i'm helping someone out :)
Building my own Deserialiser in which I wanted to deserialise a specific field as text i.s.o. a proper DTO, this is the solution I came up with.
I wrote my own JsonToStringDeserializer like this:
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.TreeNode;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import lombok.NoArgsConstructor;
import org.apache.commons.lang3.StringEscapeUtils;
import java.io.IOException;
/**
* Deserialiser to deserialise any Json content to a String.
*/
#NoArgsConstructor
public class JsonToStringDeserializer extends JsonDeserializer<String> {
/**
* Deserialise a Json attribute that is a fully fledged Json object, into a {#link String}.
* #param jsonParser Parsed used for reading JSON content
* #param context Context that can be used to access information about this deserialization activity.
* #return The deserialized value as a {#link String}.
* #throws IOException
*/
#Override
public String deserialize(JsonParser jsonParser, DeserializationContext context) throws IOException {
final TreeNode node = jsonParser.getCodec().readTree(jsonParser);
final String unescapedString = StringEscapeUtils.unescapeJava(node.toString());
return unescapedString.substring(1, unescapedString.length()-1);
}
}
Annotate the field you want to deserialize like this:
#JsonDeserialize(using = JsonToStringDeserializer.class)
I initially followed advice that said to use a TreeNode like this:
final TreeNode treeNode = jsonParser.getCodec().readTree(jsonParser);
return treeNode.toString();
But then you get a Json String that contains escape characters.
What you are trying to do is outside the scope of Jackson (and most, if not all other Java JSON libraries out there). What you want to do is fully consume the input stream into a string, then attempt to convert that string to a JSON object using Jackson. If the conversion fails then do something with the intermediate string, else proceed normally. Here's an example, which utilizes the excellent Apache Commons IO library, for convenience:
final InputStream stream ; // Your stream here
final String json = IOUtils.toString(stream);
try {
final JsonNode node = new ObjectMapper().readTree(json);
// Do something with JSON object here
} catch(final JsonProcessingException jpe) {
// Do something with intermediate string here
}