Ignore JsonIgnore in Elasticsearch

Ignore JsonIgnore in Elasticsearch - java

I am developing an application which uses Spring-boot, a relational database and Elasticsearch.
I use JSON serialization at 2 differents places in the code:
In the response of the REST API.
When the code interacts with Elasticsearch.
There are some properties that I need in Elasticsearch but that I want to hide to the application user (e.g. internal ids coming from the relational database).
Here is an example of entity :
#Document
public class MyElasticsearchEntity {
#Id
private Long id; //I want to hide this to the user.
private String name;
private String description;
}
Problem : When the object it persisted in Elasticsearch, it gets serialized as JSON. Hence, fields with #JsonIgnore are ignored when serialized to Elasticsearch.
Up to now, I found 2 unsatisfying solutions :
Solution 1 : Use #JsonProperty like this :
#Id
#JsonProperty(access = JsonProperty.Access.READ_ONLY)
private Long id;
The id gets written in Elasticsearch and is nullified in the JSON response :
{
"id" : null,
"name" : "abc",
"description" : null
}
So it works but the application user still sees that this property exists. This is messy.
Solution 2 : Cutomize the object mapper to ignore null values
Spring-boot has a built-in option for that :
spring.jackson.serialization-inclusion=NON_NULL
Problem : it suppresses all non-null properties, not only those that I want to ignore. Suppose that the field description of the previous entity is empty, the JSON response will be :
{
"name" : "abc"
}
And this is problematic for the UI.
So is there a way to ignore such field only in the JSON response?

You could use Jackson JsonView for your purpose. You can define one view which will be used to serialize pojo for the application user :
Create the views as class, one public and one private:
class Views {
static class Public { }
static class Private extends Public { }
}
Then uses the view in your Pojo as an annotation:
#Id
#JsonView(Views.Private.class) String name;
private Long id;
#JsonView(Views.Public.class) String name;
private String publicField;
and then serialize your pojo for the application user using the view:
objectMapper.writeValueUsingView(out, beanInstance, Views.Public.class);
This is one example of many others on how view can fit your question. Eg. you can use too objectMapper.configure(SerializationConfig.Feature.DEFAULT_VIEW_INCLUSION, false); to exclude field without view annotation and remove the Private view.

Related

Spring Data Elasticsearch: Convert a String to an Object (and vice versa) using ValueConverter and dot-notation

I have kind of a combination-follow up question to [1] and [2].
I have a POJO with a field I want to persist in - and read from - Elasticsearch:
#Document
public class MyPojo {
private String level3;
// getters/setters...
}
For convenience and because the property is also being persisted (flatly) into postgres, the property level3 should be a String, however it should be written into ES as a nested object (because the ES index is defined elsewhere).
The current solution is unsatisfactory:
#Document
#Entity
public class MyPojo {
#Column(name = "level3)
#Field(name = "level3", type = FieldType.Keyword)
#ValueConverter(MyConverter.class)
private String level3;
// getters/setters...
}
with the object path "level1.level2.level3" hardcoded within MyConverter, which converts from Map<String, Object> to String (read) and from String to Map<String, Object> (write). Because we potentially need to do this on multiple fields, this is not a really viable solution.
I'd rather do something like this:
#Document
#Entity
public class MyPojo {
#Column(name = "level3)
#Field(name = "level1.level2.level3", type = FieldType.Keyword)
#ValueConverter(MyConverter2.class)
private String level3;
// getters/setters...
}
which does not work (writing works fine, while reading we get the "null is not a map" error from [2]).
Is this at all possible (if I understood [2] correctly, no)? If not, is there another way to achieve what I want without hardcoding and an extra converter per field?
Can I somehow access the #Field annotation within MyConverter (e.g. the name), or can I somehow supply additional arguments to MyConverter?
[1] Spring data elasticsearch embedded field mapping
[2] Spring Elasticsearch: Adding fields to working POJO class causes IllegalArgumentException: null is not a Map

Springboot: Can DTO be changed at runtime with null values not being present in the object returned from api?

I have a springboot application which is hitting raw api's of the datasource. Now suppose I have a Customer entity with approx 50 fields and I have a raw api for it in which I pass names of the columns and I get the values for that column. Now I am implementing api in springboot which consumes raw api.
I need to implement different api's in springboot for different combinations of the fields of Customer entity and return only those fields setted in object for which user had queried and remove the null valued fields from the object. One way is to implement different dto's for different combinations of the columns of Customer entity. Is there any other way to implement the same in which I don't need to define different dto's for different combinations of the columns of Customer entity in Spring boot ???

You can configure the ObjectMapper directly, or make use of the #JsonInclude annotation:
mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
OR
#JsonInclude(JsonInclude.Include.NON_NULL)
#JsonInclude(JsonInclude.Include.NON_NULL)
public class Customer {
private Long id;
private String name;
private String email;
private String password;
public Customer() {
}
// getter/setter ..
}
You can see how to do it with this sample code:
Customer customer = new Customer();
customer.setId(1L);
customer.setName("Vikas");
customer.setEmail("info#vikas.com");
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
String valueAsString = objectMapper.writeValueAsString(customer);
Since the password is left null, you will have an object that does not exist password.
{
"id": 1,
"name": "Vikas",
"email": "info#vikas.com"
}

with Jackson 2.0 serialization you can specify data inclusion on non nulls at different levels, i.e. on the object mapper (with constructor options), the DTO class or DTO class fields (with annotations).
See Jackson annotations here

This can be done using #JsonInclude inside the DTO class. Please refer following code block for ignoring null values.
#JsonInclude(Include.NON_NULL) // ignoring null values
#Data //lombock
#Builder //builder pattern
public class Customer {
private Long id;
private String name;
private String email;
private String password;
}

Jackson: referencing an object as a property

In my java spring application, I am working with hibernate and jpa, and i use jackson to populate data in DB.
Here is the User class:
#Data
#Entity
public class User{
#Id
#GeneratedValue
Long id;
String username;
String password;
boolean activated;
public User(){}
}
and the second class is:
#Entity
#Data
public class Roles {
#Id
#GeneratedValue
Long id;
#OneToOne
User user;
String role;
public Roles(){}
}
In the class Roles i have a property of User
and then i made a json file to store the data:
[ {"_class" : "com.example.domains.User", "id": 1, "username": "Admin", "password": "123Admin123","activated":true}
,
{"_class" : "com.example.domains.Roles", "id": 1,"user":1, "role": "Admin"}]
Unfortunately, when i run the app it complains with:
.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.example.domains.User: no int/Int-argument constructor/factory method to deserialize from Number value (1)
at [Source: N/A; line: -1, column: -1] (through reference chain: com.example.domains.Roles["user"])
The problem comes from
{"_class" : "com.example.domains.Roles", "id": 1,"user":1, "role": "Admin"}
and when i remove the above line the app works well.
I think, it complains because it cannot make an instance of user.
So, how can i fix it?

Do yourself a favor and stop using your Entities as DTOs!
JPA entities have bidirectional relations, JSON objects don't, I also believe that the responsibilities of an Entity is very different from a DTO, and although joining these responsibilities into a single Java class is possible, in my experience it is a very bad idea.
Here are a couple of reasons
You almost always need more flexibility in the DTO layer, because it is often related to a UI.
You should avoid exposing primary keys from your database to the outside, including your own UI. We always generate an additional uniqueId (UUID) for every publicly exposed Entity, the primary key stays in the DB and is only used for joins.
You often need multiple views of the same Entity. Or a single view of multiple entities.
If you need to add a new entity to a relation with an existing, you will need find the existing one in the database, so posting the new and old object as a single JSON structure has no advantage. You just need the uniqueId of the existing, and then new.
A lot of the problems developers have with JPA, specifically with regards to merging comes from the fact that they receive a detached entity after their json has been deserialized. But this entity typically doesn't have the OneToMany relations (and if it does, it's the parent which has a relation to the child in JSON, but in JPA it is the child's reference to the parent which constitutes the relationship). In most cases you will always need to load the existing version of the entity from the database, and then copy the changes from your DTO into the entity.
I have worked extensively with JPA since 2009, and I know most corner cases of detachment and merging, and have no problem using an Entity as a DTO, but I have seen the confusion and types of errors that occur when you hand such code over to some one who is not intimately familiar with JPA. The few lines you need for a DTO (especially since you already use Lombok), are so simple and allows you much more flexibility, than trying to save a few files and breaking the separation of concerns.

Jackson provide ObjectIdResolver interface for resolving the objects from ids during de-serialization.
In your case you want to resolve the id based from the JPA/hibernate. So you need to implement a custom resolver to resolve id by calling the JPA/hierbate entity manager.
At high level below are the steps:
Implement a custom ObjectIdResolver say JPAEntityResolver (you may extends from SimpleObjectIdResolver). During resolving object it will call JPA entity manager class to find entity by given id and scope(see. ObjectIdResolver#resolveId java docs)
//Example only;
#Component
#Scope("prototype") // must not be a singleton component as it has state
public class JPAEntityResolver extends SimpleObjectIdResolver {
//This would be JPA based object repository or you can EntityManager instance directly.
private PersistentObjectRepository objectRepository;
#Autowired
public JPAEntityResolver (PersistentObjectRepository objectRepository) {
this.objectRepository = objectRepository;
}
#Override
public void bindItem(IdKey id, Object pojo) {
super.bindItem(id, pojo);
}
#Override
public Object resolveId(IdKey id) {
Object resolved = super.resolveId(id);
if (resolved == null) {
resolved = _tryToLoadFromSource(id);
bindItem(id, resolved);
}
return resolved;
}
private Object _tryToLoadFromSource(IdKey idKey) {
requireNonNull(idKey.scope, "global scope does not supported");
String id = (String) idKey.key;
Class<?> poType = idKey.scope;
return objectRepository.getById(id, poType);
}
#Override
public ObjectIdResolver newForDeserialization(Object context) {
return new JPAEntityResolver(objectRepository);
}
#Override
public boolean canUseFor(ObjectIdResolver resolverType) {
return resolverType.getClass() == JPAEntityResolver.class;
}
}
Tell Jackson to use a custom id resolver for a class, by using annotation JsonIdentityInfo(resolver = JPAEntityResolver.class). For e.g.
#JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class,
property = "id",
scope = User.class,
resolver = JPAObjectIdResolver.class)
public class User { ... }
JPAObjectIdResolver is a custom implementation and will have dependency on other resources( JPA Entity Manager) which might not be known to Jackson. So Jackson need help to instantiate resolver object. For this purpose, you need to supply a custom HandlerInstantiator to ObjectMapper instance. (In my case I was using spring so I asked spring to create instance of JPAObjectIdResolver by using autowiring)
Now de-serialization should work as expected.
Hope this helps.

I have changed the json file to :
[
{"_class" : "com.example.domains.User",
"id": 1,
"username": "Admin",
"password": "123Admin123",
"activated":true
},
{
"_class" : "com.example.domains.Roles",
"id": 1,
"user":{"_class" : "com.example.domains.User",
"id": 1,
"username": "Admin",
"password": "123Admin123",
"activated":true
},
"role": "Admin"
}
]
But i still think, the best ways is using a foreign key to user record.
Any solution is welcomed

If your bean doesn't strictly adhere to the JavaBeans format, Jackson has difficulties.
It's best to create an explicit #JsonCreator constructor for your JSON model bean, e.g.
class User {
...
#JsonCreator
public User(#JsonProperty("name") String name,
#JsonProperty("age") int age) {
this.name = name;
this.age = age;
}
..
}

1-1 mapping of fields works well , but when it comes to complex object mapping , better to use some API.
You can use Dozer Mapping or Mapstruct to map Object instances.
Dozer has spring integration also.

You could specify non default constructors and then use a custom deserialiser.
Something like this should work (it has not been tested).
public class RolesDeserializer extends StdDeserializer<Roles> {
public RolesDeserializer() {
this(null);
}
public RolesDeserializer(Class<?> c) {
super(c);
}
#Override
public Roles deserialize(JsonParser jp, DeserializationContext dsctxt)
throws IOException, JsonProcessingException {
JsonNode node = jp.getCodec().readTree(jp);
long id = ((LongNode) node.get("id")).longValue();
String roleName = node.get("role").asText();
long userId = ((LongNode) node.get("user")).longValue();
//Based on the userId you need to search the user and build the user object properly
User user = new User(userId, ....);
return new Roles(id, roleName, user);
}
}
Then you need to register your new deserialiser (1) or use the #JsonDeserialize annotation (2)
(1)
ObjectMapper mapper = new ObjectMapper();
SimpleModule module = new SimpleModule();
module.addDeserializer(Item.class, new RolesDeserializer());
mapper.registerModule(module);
Roles deserializedRol = mapper.readValue(yourjson, Roles.class);
(2)
#JsonDeserialize(using = RolesDeserializer.class)
#Entity
#Data
public class Roles {
...
}
Roles deserializedRol = new ObjectMapper().readValue(yourjson, Roles.class);

public class Roles {
#Id
#GeneratedValue
Long id;
#JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "id")
#JsonIdentityReference(alwaysAsId = true)
#OneToOne
User user;
String role;
public Roles(){}
}

JPA / Jackson - Exclude fields when deserialize and include them when serialize

I have a JPA entity with a couple of fields (the real ones are more complex). I'm receiving some data via REST (POST operation in a Spring controller) and storing it right away in the JPA entities; I want to see if there is a possibility to exclude some field(s) when the request is sent, Jackson deserializes it, and constructs the object. But at the same time I want those fields to be included when I send back (object gets serialized) the response.
#Table("key_card")
public final class KeyCard {
private String username; // Don't want this to be sent as input,
// but want to be able to send it back
// in the response
#NotBlank
private final char[] password;
}
I'm just trying not to model it twice (for the request and response) if there is a way to solve this.

You can use JSON views: http://wiki.fasterxml.com/JacksonJsonView
Class Views {
static class AlwaysInclude { }
static class OnlyOnSerialize extends AlwaysInclude { }
}
And then on your view:
#Table("key_card")
public final class KeyCard {
#JsonView(Views.OnlyOnSerialize.class)
private String username;
#JsonView(Views.AlwaysInclude.class)
#NotBlank
private final char[] password;
}

To exclude a Java object property only from Json deserialization and to include instead its value during serialization you can use an appropriate combination of #JsonIgnore and #JsonProperty annotations.
In particular you should:
annotate with #JsonIgnore the property itself
annotate with #JsonIgnore its set method
annotate with #JsonProperty its get method
Here you can find an in-depth explanation and an example: Jackson: using #JsonIgnore and #JsonProperty annotations to exclude a property only from JSON deserialization

Serialize Jackson field in two different ways

I am doing POJO serialization / deserialization using Jackson.
Here is a POJO exemple :
public class Pojo {
public String productId;
public String name;
}
I have to read the field productId in this JSON :
{"productId":"1","name":"exemple"}
But also in :
{"_id":"1","name":"exemple"}
To make it short, I would like to use the same object to read the field in a JSON file found somewhere and to save the object as this in MongoDB, using productId as the primary key, which has to be named _id.
Since I am using Jackson (fasterxml) both to read from the file and to write to the database, I can not find a way to do so, except by creating a new class with the same fields (or inheritance) and fill them one by one. Basically, I would like to find a way to put 2 #JsonProperty annotations on productId.

Works with both strings:
public class Pojo {
#JsonProperty("_id")
public String productId;
public String name;
#JsonProperty("productId")
public void setProductId(String id) {
productId = id;
}
}

Develop Reference

Java is a programming language and computing platform first released by Sun Microsystems in 1995.