I am not a Java programmer but keen to know how to achieve a feature which would ignore one or more invalid fields while de-serializing. In my case, the byte[] is send by a legacy system and hence no control over data correction (on encoding side or to send default in case of missing). This means, I need to anticipate invalid data/missing data within the byte[].
Following code serialize payloads but throws exception upon receiving invalid (wrong encoding, null etc).
public class LDeserializer<T> implements Deserializer<T> {
private final ObjectMapper objectMapper;
private final Class<T> myType;
public LDeserializer(final ObjectMapper objectMapper, final Class<T> myType) {
this.objectMapper = objectMapper;
this.myType = myType;
}
public LDeserializer(final Class<T> myType) {
this(new ObjectMapper(), myType);
}
#Override
public T deserialize(final String sometext, final byte[] bytes) {
if (bytes == null) {
return null;
}
try {
return objectMapper.readValue(bytes, myType);
} catch (final IOException e) {
throw new SerializationException(e);
}
}
}
Question: In my case, I can safely ignore invalid values while serialization. However, I am not sure how to instruct my serializer to ignore invalid fields (irrespective of its type). So that the serializer would emit object even thou the constructed object is not complete.
Note: ObjectMapper is the type I am using here to support serialization. However, I am free to use any helper types that can be used instead ObjectMapper. The only concern here is how to ignore invalid fields (irrespective of its type).
Please advise
Related
I managed to configure Jackson to serialize a class without any getters or annotations on the private fields inside the class by just using
objectMapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
But I don't manage to make a JSON string of this non-static class to get deserialized into an object without any parameterized constructor or setters. Is this possible at all - I think it should through reflection but I don't get exactly how...
Here are 2 tests which need to pass to achieve what I need:
public class ObjectMapperTest {
private ObjectMapper mapper;
#Before
public void init() {
mapper = new ObjectMapper();
mapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
}
#Test
public void serialize() throws Exception {
Payload payloadToSerialize = new Payload();
payloadToSerialize.data = "testData";
String serializedPayload = mapper.writeValueAsString(payloadToSerialize);
assertThat(serializedPayload, is("{\"data\":\"testData\"}"));
// --> OK
}
#Test
public void deserialize() throws Exception {
Payload deserializedPayload = mapper.readValue("{\"data\":\"testData\"}", Payload.class);
assertThat(deserializedPayload.data, is("testData"));
// com.fasterxml.jackson.databind.JsonMappingException:
// No suitable constructor found for type [simple type, class ...ObjectMapperTest$Payload]:
// can not instantiate from JSON object (missing default constructor or creator, or perhaps need to add/enable type information?)
// at [Source: {"data":"testData"}; line: 1, column: 2]
}
public class Payload {
private String data;
public Payload() {
// empty constructor for Jackson
}
}
}
Making the Payload class static would fix the test but static classes are not an option for me as I am not working with inner payload classes in the project. Any ideas how to fix it through object mapper configuration change?
EDIT
As I am using the Jackson object mapper in a Spring MVC application to serialize / deserialize under the hood I need a solution which changes or extends the object mapper configuration only.
You can write your own deserializer to parse the JSON and create the instance of Payload, and then set the data value using reflection.
Exemple :
#Before
public void init() {
mapper = new ObjectMapper();
mapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
mapper.registerModule(
new SimpleModule()
.addDeserializer(Payload.class, new JsonDeserializer<Payload>() {
#Override
public Payload deserialize(JsonParser parser, DeserializationContext ctx)
throws IOException, JsonProcessingException {
JsonNode obj = parser.readValueAsTree(); // Read the JSON as a node
Payload payload = new Payload();
if (obj.isObject() && obj.has("data")) { // The node is an object and has a "data" field
try {
// Use reflection to set the value
Field dataField = payload.getClass().getDeclaredField("data");
dataField.setAccessible(true);
dataField.set(payload, obj.get("data").asText());
} catch (NoSuchFieldException | SecurityException | IllegalArgumentException | IllegalAccessException ex) {
throw new IOException("Reflection error", ex);
}
}
return payload;
}
}));
}
Edit: If you want something more "generic" you can try to create the instance yourself and change the accessibility of all the fields. Then you tell Jackson to update the values using the JSON.
public <T> T deserialize(final String json, final T instance) throws Exception {
for (Field field : instance.getClass().getDeclaredFields()) {
field.setAccessible(true);
}
mapper.readerForUpdating(instance).readValue(json);
return instance;
}
#Test
public void deserializeUpdate() throws Exception {
Payload deserializedPayload = deserialize("{\"data\":\"testData\"}", new Payload());
assertThat(deserializedPayload.data, is("testData"));
}
I tested this on your Payload class, maybe it doesn't work on more complex objects.
I am getting data from an external JSON API and parsing the result with Jackson. Unfortunately, that API returns all fields as String and those are filled with "N/A" when data is unavailable.
I would like to replace those fields with null, especially as those are frustrating when I try to convert JSON String fields to more informative Java fields.
A custom DeserializationProblemHandler worked for Integer fields (see below), but it was useless for Java 8's LocalDate fields. Furthermore, it reacts to a problem rather than anticipating it.
I was unable to find a pre-processor to configure into my ObjectMapper and am uneasy with the idea of overriding the BeanDeserializer.
Do you know of a better/cleaner solution to handle this kind of situations? Thanks!
DeserializationProblemHandler
new DeserializationProblemHandler() {
#Override
public Object handleWeirdStringValue(DeserializationContext ctxt, Class<?> targetType, String valueToConvert, String failureMsg) throws IOException {
return "N/A".equals(valueToConvert) ? null : super.handleWeirdStringValue(ctxt, targetType, valueToConvert, failureMsg);
}
}
Error message when processing "N/A" in LocalDate field
Can not deserialize value of type java.time.LocalDate from String "N/A": Text 'N/A' could not be parsed at index 0
(works fine when there is date in the data)
I feel like there ought to be a better way of doing this, but the following is the only solution I was able to come up with.
Create a new JsonDeserializer that handles "N/A" input. The following example handles strings:
public class EmptyStringDeserializer extends StdScalarDeserializer<String> {
public EmptyStringDeserializer() {
super(String.class);
}
#Override
public String deserialize(JsonParser parser, DeserializationContext ctx) throws IOException {
final String val = parser.getValueAsString();
if ("N/A".equalsIgnoreCase(val))
return null;
return val;
}
}
The class is registered with an ObjectMapper like this:
SimpleModule simpleModule = new SimpleModule().addDeserializer(String.class, new EmptyStringDeserializer());
ObjectMapper om = new ObjectMapper().registerModule(simpleModule);
You'll probably want to collect all your converters in a module named for the API that is making you handle things this way.
Is there a way where we can add ObjectMapper for a particular class through annotation.
#JsonRootName("employee")
public class Sample implements Serializable{
private String name;
private String id;
// Getters and setters
}
In the RestController i have RequestMapping and a method like:-
#ResponseBody
public Sample (#RequestBody Sample sample){
//some logic
return sample;
}
My input payload to this will be like
{
"employee":{
"name":"abcd",
"id":"1234"
}
}
My desired output would be
{
"name":"abcd",
"id":"1234"
}
1)Is there a way i can use the same class to fulfill the input and the output.
2) I have added #JsonRootName at the top of the class which requires ObjectMapper's Serialization feature enable to WRAP_ROOT_VALUE like :-
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.WRAP_ROOT_VALUE);
mapper.enable(DeserializationFeature.UNWRAP_ROOT_VALUE);
where this can be added to reflect in only this class.
Maybe just leave the default serialization behavior? Then, at deserialization you would still pull out the "employee" wrapper, but at serialization you would write it without the wrapper.
ObjectMapper mapper = new ObjectMapper();
//mapper.enable(SerializationFeature.WRAP_ROOT_VALUE);
mapper.enable(DeserializationFeature.UNWRAP_ROOT_VALUE);
With your input, I got the desired serialization output:
{"name":"abcd","id":"1234"}
EDIT
As for where to put this code, I'd recommend a singleton or class with static methods that handle your (de)serialization. You could have two different mappers than perform the "normal" or "wrapped" behavior. Here's an outline of the static method approach:
public class SerializationUtil {
private static ObjectMapper normalObjectMapper;
private static ObjectMapper wrappedObjectMapper;
static {
/* configure different (de)serialization strategies */
normalObjectMapper = new ObjectMapper();
wrappedObjectMapper = new ObjectMapper();
wrappedObjectMapper.enable(SerializationFeature.WRAP_ROOT_VALUE);
wrappedObjectMapper.enable(DeserializationFeature.UNWRAP_ROOT_VALUE);
}
public static <T> T normalDeserialize(String json, Class<T> clazz) throws Exception {
return normalObjectMapper.readValue(json, clazz);
}
public static String normalSerialize(Object bean) throws Exception {
return normalObjectMapper.writeValueAsString(bean);
}
public static <T> T deserializeWrappedObject(String json, Class<T> clazz) throws Exception {
return wrappedObjectMapper.readValue(json, clazz);
}
public static String serializeWrappedObject(Object bean) throws Exception {
return wrappedObjectMapper.writeValueAsString(bean);
}
}
The benefit of this method is it allows the caller to decide the serialization behavior. So if there are portions of your code where you need to handle it differently you can call another method. Note that the wrapping/unwrapping are both enabled. So to get your desired behavior, you would call these methods like so:
public static void main(String[] args) {
Bean bean = SerializationUtil.deserializeWrappedObject(jsonInput, Bean.class);
String jsonOutput = SerializationUtil.normalSerialize(bean);
}
If this does not appeal to you, you could alternatively detect the special case and handle it in the same method call:
public static <T> T deserialize(String json, Class<T> clazz) throws Exception {
if (clazz instanceof Bean) {
return wrappedObjectMapper.readValue(json, clazz);
} else {
return normalObjectMapper.readValue(json, clazz);
}
}
In Spring MVC project, I have controllers which return data in the form of various objects. Later these objects are serialized to JSON, and JSON returned as a response to a client.
This is achieved by registering custom MappingJackson2HttpMessageConverter with configureMessageConverters() in application config. The converter uses
jackson.databind.ObjectWriter.writeValue(jackson.core.JsonGenerator, object)
for object serialization.
Now I need to implement XSS protection. Since all data goes through the step of serialization, I want to filter here String values, removing all symbols except those in whitelist (alphanumeric and some punctuation signs).
Is there a way to provide Jackson with a filter which will edit String values during serialization?
I have an instance of MappingJackson2HttpMessageConverter. There, in method writeInternal(Object object, HttpOutputMessage outputMessage), I create an instance of my custom mapper: mapper = new KJsonMapper(); This mapper is then used there to generate JSON.
KJsonMapper.java:
public class KJsonMapper extends ObjectMapper {
public KJsonMapper() {
enableAntiXSS();
}
private void enableAntiXSS() {
SimpleModule module = new SimpleModule("Anti-XSS Serializer",
new Version(1, 0, 0, "FINAL", "klab", "klab.anti-xss-serializer"));
module.addSerializer(String.class, new KJsonAntiXssSerializer());
registerModule(module);
}
}
The mapper itself uses custom JsonSerializer, which is to provide anti-xss filtering.
KJsonAntiXssSerializer.java:
public class KJsonAntiXssSerializer extends JsonSerializer<String> {
public void serialize(String value, JsonGenerator jsonGenerator, SerializerProvider serializerProvider)
throws IOException {
if (value == null) {
return;
}
String encodedValue = antiXss(value);
jsonGenerator.writeString(encodedValue);
}
private String antiXss(String value) {
// return processed value from here
}
}
I am trying to be able to define the following code:
public class MyObject {
private String name;
... // Other attributes
}
#Path(...)
#Stateless
public class MyRestResource {
#POST
#Consumes(MediaType.APPLICATION_JSON)
public Response create(List<MyObject> myObjects) {
// Do some stuff there
}
}
I know that I need to use:
DeserializationConfig.Feature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true
to setup correctly my object mapper to be able to accept single value as array on my rest resources. I succeed to setup that part.
My problem with this approach is that the following content is not differentiable:
{
"name": "a name",
... // other attributes
}
and
[{
"name": "a name",
... // other attributes
}]
will result into a list (List) of size one. Then, in the method create(List myObjects), I will not be able to do the difference between the List and the Single Object sent to the Rest Resource.
Then, my question is how to do something like that. The idea is to have only one #POST that accepts both Arrays and Single values?
Ideally, I will get rid of the configuration of the ObjectMapper to avoid letting the possibility to set Single Object into the other level of the JSON document. For example, I do not want to allow that:
{
...
"attributes": {
...
}
}
where normally this format should be mandatory:
{
...
"attributes": [{
...
}]
}
Based on that, I tried to put in place an object wrapper of my List to set if I am able to the difference between the list and the object. With something like that:
public class ObjectWrapper<T> {
private List<T> list;
private T object;
public boolean isObject() {
return list == null;
}
}
with the resource that becomes:
#Path(...)
#Stateless
public class MyRestResource {
#POST
#Consumes(MediaType.APPLICATION_JSON)
public Response create(ObjectWrapper myObjects) {
// Do some stuff there
}
}
and trying to put in place the deserialization of my content through the JAX-RS/Jersey/Jackson mechanisms. If I let the solution as it is now, the deserialization fails due to the fact that the JSON format expected is the following:
{
"list": [{
"name": "a name",
... // other attributes
}]
}
Then I tried to write a custom deserializer but I am a bit lost in this task. I have something like that:
public class ObjectWrapperDeserializer<T> extends JsonDeserializer<T> {
#Override
public T deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException, JsonProcessingException {
... // What to put there to deserialize Array or Object
}
}
I just want to deserialize the root level to set the content deserialized into the object wrapper. I also want to keep the feature configured in a class annotated with #ApplicationPath when the configuraiton of the different #Provider are done.
I hope that all the info will give a sufficient picture of what I want to do and what I already tested.
Waiting for suggestion on how to do a resource that accept Arrays or Objects on the same path.
Thanks a lot in advance.
Ok, finally I succeed to put in place a mechanism that do exactly what I am looking for. But, I am not sure if there are negative consequences such the performance or such things.
First, I defined a class that can accept both List or Single Object:
public class RootWrapper<T> {
private List<T> list;
private T object;
}
Then, I need a custom deserializer that is able to know which kind of T type to deserialize and to handle the collection or the single object.
public class RootWrapperDeserializer extends JsonDeserializer<CollectionWrapper<?>> {
private Class contentType;
public RootWrapperDeserializer(Class contentType) {
this.contentType = contentType;
}
#Override
public RootWrapper deserialize(JsonParser jp, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
// Retrieve the object mapper and read the tree.
ObjectMapper mapper = (ObjectMapper) jp.getCodec();
JsonNode root = mapper.readTree(jp);
RootWrapper wrapper = new RootWrapper();
// Check if the root received is an array.
if (root.isArray()) {
List list = new LinkedList();
// Deserialize each node of the array using the type expected.
Iterator<JsonNode> rootIterator = root.getElements();
while (rootIterator.hasNext()) {
list.add(mapper.readValue(rootIterator.next(), contentType));
}
wrapper.setList(list);
}
// Deserialize the single object.
else {
wrapper.setObject(mapper.readValue(root, contentType));
}
return wrapper;
}
}
As far as I know, I try to only deserialize the root level manually and then let Jackson take the next operations in charge. I only have to know which real type I expect to be present in the Wrapper.
At this stage, I need a way to tell Jersey/Jackson which deserializer to use. One way I found for that is to create a sort of deserializer registry where are stored the type to deserialize with the right deserializer. I extended the Deserializers.Base class for that.
public class CustomDeserializers extends Deserializers.Base {
// Deserializers caching
private Map<Class, RootWrapperDeserializer> deserializers = new HashMap<>();
#Override
public JsonDeserializer<?> findBeanDeserializer(JavaType type,
DeserializationConfig config, DeserializerProvider provider,
BeanDescription beanDesc, BeanProperty property) throws JsonMappingException {
// Check if we have to provide a deserializer
if (type.getRawClass() == RootWrapper.class) {
// Check the deserializer cache
if (deserializers.containsKey(type.getRawClass())) {
return deserializers.get(type.getRawClass());
}
else {
// Create the new deserializer and cache it.
RootWrapperDeserializer deserializer =
new RootWrapperDeserializer(type.containedType(0).getRawClass());
deserializers.put(type.getRawClass(), deserializer);
return deserializer;
}
}
return null;
}
}
Ok, then I have my deserializers registry that create new deserializer only on demand and keep them once created. What I am not sure about that approach is if there is any concurrency issue. I know that Jackson do a lot of caching and do not call every time the findBeanDeserializer once it was called a first time on a specific deserialization context.
Now I have created my different classes, I need to do some plumbing to combine everything together. In a provider where I create the ObjectMapper, I can setup the deserializers registry to the created object mapper like below:
#Provider
#Produces(MediaType.APPLICATION_JSON)
public class JsonObjectMapper implements ContextResolver<ObjectMapper> {
private ObjectMapper jacksonObjectMapper;
public JsonObjectMapper() {
jacksonObjectMapper = new ObjectMapper();
// Do some custom configuration...
// Configure a new deserializer registry
jacksonObjectMapper.setDeserializerProvider(
jacksonObjectMapper.getDeserializerProvider().withAdditionalDeserializers(
new RootArrayObjectDeserializers()
)
);
}
#Override
public ObjectMapper getContext(Class<?> arg0) {
return jacksonObjectMapper;
}
}
Then, I can also define my #ApplicationPath that is my REST application like following:
public abstract class AbstractRestApplication extends Application {
private Set<Class<?>> classes = new HashSet<>();
public AbstractRestApplication() {
classes.add(JacksonFeature.class);
classes.add(JsonObjectMapper.class);
addResources(classes);
}
#Override
public Set<Class<?>> getClasses() {
return classes;
}
#Override
public Set<Object> getSingletons() {
final Set<Object> singletons = new HashSet<>(1);
singletons.add(new JacksonJsonProvider());
return singletons;
}
private void addResources(Set<Class<?>> classes) {
classes.add(SomeRestResource.class);
// ...
}
}
Now, everything is in place and I can write a REST resource method like that:
#POST
#Path("somePath")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public Response create(RootWrapper<SpecificClass> wrapper) {
if (wrapper.isObject()) {
// Do something for one single object
SpecificClass sc = wrapper.getObject();
// ...
return Response.ok(resultSingleObject).build();
}
else {
// Do something for list of objects
for (SpecificClass sc = wrapper.getList()) {
// ...
}
return Response.ok(resultList).build();
}
}
That's all. Do not hesitate to comment the solution. Feedbacks are really welcome especially around the way of deserialization process where I am really not sure that it is safe for performance and concurrency.