Develop Reference

Gson custom deserializer for specific fields to String

I have a variety of JSON files (with slightly different schemas) flowing on Kinesis. Their Schema is really complex. They are user raw hits and their schema is super complex. I would like to create a Single POJO to represent all underneath messages (Something Spark does internally by creating a Single schema). I was trying to use GSON library but the only way to accomplish this is, by writing a custom deserializer and in that case, I will end up writing deserialization logic for all the fields in those JSONs.
Is there any way where we can only overwrite the deserialization of a few fields only and the rest of the fields can still be deserialized by GSON as a default way?
JSON-1
{
"first_col":"abc",
"second_col":false
}
JSON-2
{
"first_col":"abc",
"second_col":"false-String"
}
JSON-3
{
"first_col":"abc",
"second_col":{
"col1":"xyz",
"col2":123
}
}
Common POJO
import com.google.gson.annotations.Expose;
import com.google.gson.annotations.SerializedName;
public class Example {
private String firstCol;
private String secondCol;
public String getFirstCol() {
return firstCol;
}
public void setFirstCol(String firstCol) {
this.firstCol = firstCol;
}
public Boolean getSecondCol() {
return secondCol;
}
public void setSecondCol(String secondCol) {
this.secondCol = secondCol;
}
}
So basically second_col could be boolean, string, or complex object. first_col is always a string. So I don't want to write deserialize logic for first_col. I want to write deserialize logic for only second_col and deserialize it to string only and the downstream consumer will take care of converting it to the right type before consuming this field.

Jackson: How do I post-process JsonNode during serialization?

I am attempting to implement the HL7 FHIR spec's assertion that JSON representing a FHIR model will not have empty objects nor empty arrays. For the sake of not making the lives of my consumers any harder, I'm not strictly enforcing this during deserialization, but I want to ensure the serialized JSON produced by my library conforms as specified. I am using Java and Jackson ObjectMapper to serialize Objects into JSON. My understanding from writing a custom serializer is that the Object is at one point represented as JsonNode, regardless of what you are converting to.
What I would like to do is intercept the JsonNode as it exits the serializer, make some adjustments to it (find and remove empty arrays and objects), and then let it continue on its way. I need to do this in an environment where I can't tweak the ObjectMapper, because I don't have access to it. And further, the complex hierarchy of models in this library use Jackson's default serialization with annotations etc. heavily, and I cannot eliminate this.
If I go the route of defining a custom serializer for the base type, let's say "Resource", then I have a problem, because I still need the original serializer's output in order to generate my modified output. And further, that needs to accommodate any custom serializers that may already exist on various types within the model.
I got pretty far with the above option using https://www.baeldung.com/jackson-call-default-serializer-from-custom-serializer and the last option, implementing BeanSerializerModifier, but I ran into the issue where I can't control the ObjectMapper that my library consumers use.
Example POJOs (Using Lombok for accessors):
#Data
#JsonInclude(JsonInclude.Include.NON_EMPTY)
#JsonIgnoreProperties(ignoreUnknown = true)
abstract class Resource {
private FhirString id;
private List<Extension> extension;
#JsonProperty(access = JsonProperty.Access.READ_ONLY)
public abstract ResourceType getResourceType();
}
#Data
#Builder
class SomethingElse extends Resource {
FhirUri someProperty;
CodeableConcept someCode;
List<Reference> someReferences;
#Override
public ResourceType getResourceType() {
return ResourceType.SOMETHING_ELSE;
}
}
And an example instance of the SomethingElse class:
SomethingElse somethingElse = SomethingElse.builder()
.someProperty(FhirUri.from("some-simple-uri"))
.someCode(new CodeableConcept())
.someReference(List.of(new Reference()))
.build();
somethingElse.setId(FhirString.randomUuid());
somethingElse.setExtension(new ArrayList<>());
When I tell any mapper (or, for example, use a Spring service) to map the SomethingElse class into JsonNode, I can, for example, end up with empty objects and arrays, like this:
ObjectMapper mapper = getUntouchableMapper();
JsonNode somethingElseNode = mapper.valueToTree(somethingElse);
System.out.println(somethingElseNode.toString());
Becomes:
{
"resourceType": "SomethingElse",
"id": "00000000-0002-0004-0000-000000000000",
"someProperty": "some-simple-uri",
"someCode": {},
"someReferences": [{}],
"extension": []
}
According to FHIR, this should actually look like:
{
"resourceType": "SomethingElse",
"id": "00000000-0002-0004-0000-000000000000",
"someProperty": "some-simple-uri"
}
To summarize
How do I preserve the serialization mechanisms already in place, regardless of the ObjectMapper used, and somehow remove empty lists and objects from outgoing JSON produced by the Jackson serialization process?
Edit:
I also tried #JsonInclude(JsonInclude.Include.NON_EMPTY), which did omit empty list implementations. However, the vast majority of data in this library is represented by POJOs that serialize to maps and primitives, and this annotation only works if they are represented directly by maps and primitives in the model.

The solution is to use a custom #JsonInclude, which is new in Jackson 2.9. Thank you #dai for pointing me back towards this functionality.
On the base Resource class, this looks like:
#JsonInclude(value = JsonInclude.Include.CUSTOM, valueFilter = FhirJsonValueFilter.class)
class Resource implements FhirTypeInterface {
...
#Override
public boolean isEmpty() {
//Details omitted for simplicity
}
}
For visibility, the interface used above:
interface FhirTypeInterface {
boolean isEmpty();
}
And my custom definition for FhirJsonValueFilter implements all of the functionality of JsonInclude.Include.NON_EMPTY but also adds functionality for checking against a method implemented by FHIR types (implementation of this is not relevant to the answer).
public class FhirJsonValueFilter {
#Override
public boolean equals(Object value) {
return !getWillInclude(value);
}
/**
* Returns true for an object that matched filter criteria (will be
* included) and false for those to omit from the response.
*/
public boolean getWillInclude(Object value) {
//Omit explicit null values
if (null == value) {
return false;
}
//Omit empty collections
if (Collection.class.isAssignableFrom(value.getClass())) {
return !((Collection) value).isEmpty();
}
//Omit empty maps
if (Map.class.isAssignableFrom(value.getClass())) {
return !((Map) value).isEmpty();
}
//Omit empty char sequences (Strings, etc.)
if (CharSequence.class.isAssignableFrom(value.getClass())) {
return ((CharSequence) value).length() > 0;
}
//Omit empty FHIR data represented by an object
if (FhirTypeInterface.class.isAssignableFrom(value.getClass())) {
return !((FhirTypeInterface) value).isEmpty();
}
//If we missed something, default to include it
return true;
}
}
Note that the custom omission filter uses Java's Object.equals functionality, where true means to omit the property, and I've used a second method to reduce confusion in this answer.

Handle Json non existent keys spring boot

I am creating a requestModel and let say a person doesn't send me some keys.
If that key is not present I want to put null if i get the value of the key.
I don't want to investigate if a key is present or not .
public class CustomerModel {
private Optional<String> s3Bucket;
private Optional<String> docType;
public String getS3Bucket() {
if(s3Bucket.isPresent()) {
return s3Bucket.get();
} else {
return null;
}
}
public void setS3Bucket(Optional<String> s3Bucket) {
this.s3Bucket = s3Bucket;
}
public Optional<String> getDocType() {
return docType;
}
public void setDocType(Optional<String> docType) {
this.docType = docType;
}
}
Do we have any library or something where.
1. If i get the key and it is not present in the coming request json, i will get the null out of it and if the key is present and has value . It will be stored as value.
2. When writing the getter for s3bucket (getS3Bucket), i dont want to write it for everykey value. Is there a automatic way to do this.
I looked at lot of posts but the scenario is not there.
P.S - I am new to java

I believe Jackson is exactly what you need. And if you are using Spring - it already uses Jackson under the hood I guess.
Here you can find some examples and documentation of how JSON mapping on to model class is done.
If you need to customize some behavior, you can use annotations like #JsonProperty (there are many).
If properties in your model class have the same names as properties in JSON, most probably you won't need to provide any further configs.
Here is a simple example:
public class User {
#JsonProperty("userName")
private String name;
private int age;
// getters and setters
}
And if you have JSON like this:
{
"userName" : "Foo Bar",
"age" : 18
}
Jackson will do all the magic for you unless you need something very specific.
If something is not in JSON you get (let's say you received JSON without age) - corresponding property in model class will be null if it is object type and default value (0, false, etc.) for primitives (in our case age would be 0).

How to generate an example POJO from Swagger ApiModelProperty annotations?

We are creating a REST API which is documented using Swagger's #ApiModelProperty annotations. I am writing end-to-end tests for the API, and I need to generate the JSON body for some of the requests. Assume I need to post the following JSON to an endpoint:
{ "name": "dan", "age": "33" }
So far I created a separate class containing all the necessary properties and which can be serialized to JSON using Jackson:
#JsonIgnoreProperties(ignoreUnknown = true)
public class MyPostRequest {
private String name;
private String age;
// getters and fluid setters omitted...
public static MyPostRequest getExample() {
return new MyPostRequest().setName("dan").setAge("33");
}
}
However, we noticed that we already have a very similar class in the codebase which defines the model that the API accepts. In this model class, the example values for each property are already defined in #ApiModelProperty:
#ApiModel(value = "MyAPIModel")
public class MyAPIModel extends AbstractModel {
#ApiModelProperty(required = true, example = "dan")
private String name;
#ApiModelProperty(required = true, example = "33")
private String age;
}
Is there a simple way to generate an instance of MyAPIModel filled with the example values for each property? Note: I need to be able to modify single properties in my end-to-end test before converting to JSON in order to test different edge cases. Therefore it is not sufficient to generate the example JSON directly.
Essentially, can I write a static method getExample() on MyAPIModel (or even better on the base class AbstractModel) which returns an example instance of MyAPIModel as specified in the Swagger annotations?

This does not seem to be possible as of the time of this answer. The closest possibilities I found are:
io.swagger.converter.ModelConverters: The method read() creates Model objects, but the example member in those models is null. The examples are present in the properties member in String form (taken directly from the APIModelParameter annotations).
io.swagger.codegen.examples.ExampleGenerator: The method resolveModelToExample() takes the output from ModelConverters.read(), and generates a Map representing the object with its properties (while also parsing non-string properties such as nested models). This method is used for serializing to JSON. Unfortunately, resolveModelToExample() is private. If it were publicly accessible, code to generate a model default for an annotated Swagger API model class might look like this:
protected <T extends AbstractModel> T getModelExample(Class<T> clazz) {
// Get the swagger model instance including properties list with examples
Map<String,Model> models = ModelConverters.getInstance().read(clazz);
// Parse non-string example values into proper objects, and compile a map of properties representing an example object
ExampleGenerator eg = new ExampleGenerator(models);
Object resolved = eg.resolveModelToExample(clazz.getSimpleName(), null, new HashSet<String>());
if (!(resolved instanceof Map<?,?>)) {
// Model is not an instance of io.swagger.models.ModelImpl, and therefore no example can be resolved
return null;
}
T result = clazz.newInstance();
BeanUtils.populate(result, (Map<?,?>) resolved);
return result;
}
Since in our case all we need are String, boolean and int properties, there is at least the possibility to parse the annotations ourselves in a crazy hackish manner:
protected <T extends MyModelBaseClass> T getModelExample(Class<T> clazz) {
try {
T result = clazz.newInstance();
for(Field field : clazz.getDeclaredFields()) {
if (field.isAnnotationPresent(ApiModelProperty.class)) {
String exampleValue = field.getAnnotation(ApiModelProperty.class).example();
if (exampleValue != null) {
boolean accessible = field.isAccessible();
field.setAccessible(true);
setField(result, field, exampleValue);
field.setAccessible(accessible);
}
}
}
return result;
} catch (InstantiationException | IllegalAccessException e) {
throw new IllegalArgumentException("Could not create model example", e);
}
}
private <T extends MyModelBaseClass> void setField(T model, Field field, String value) throws IllegalArgumentException, IllegalAccessException {
Class<?> type = field.getType();
LOGGER.info(type.toString());
if (String.class.equals(type)) {
field.set(model, value);
} else if (Boolean.TYPE.equals(type) || Boolean.class.equals(type)) {
field.set(model, Boolean.parseBoolean(value));
} else if (Integer.TYPE.equals(type) || Integer.class.equals(type)) {
field.set(model, Integer.parseInt(value));
}
}
I might open an Issue / PR on Github later to propose adding functionality to Swagger. I am very surprised that nobody else has seemed to request this feature, given that our use case of sending exemplary model instances to the API as a test should be common.

Runtime annotations design and performance

I have a java api which performs an external resource lookup and then maps the values to a Pojo. To do this, the api needs the field names of the Pojo as string values, something like:
public <F> F populatePojoFields(String primaryField, String secondaryField);
This works fine, however passing the pojo field names as String to the api does not feel right. I was able to change this by writing marker annotations for the pojo, so now it is like
public class POJO {
#Primary //custom marker annotation
private int mojo;
#Secondary //custom marker annotation
private String jojo;
}
String primaryField = getFieldNameUsingReflection(Pojo.class, Primary.class)
String secondryField = getFieldNameUsingReflection(Pojo.class, Secondary.class)
Pojo pojo = populatePojoFields(primaryField, secondaryField);
This way I don't have to keep track of string values, I can just add marker annotations to the Pojo fields. This works fine, but I'm worried about performance. Is this a standard way to do things? as keeping hardcoded string values is more efficient than looking up the field names every time we need to call the api. Is there a better way to do this?

If you call getFieldNameUsingReflection often you can think to cache the result of this call.
You can use a singleton class with internal Map with a code like the following:
public class SingletonMapPrimarySecondary {
Map<Class, String> mapPrimary;
Map<Class, String> mapSecondary;
// TODO: Handle mapPrimary and mapSecondary creation and singleton pattern
public String getPrimary(Class clazz) {
String primary = mapPrimary.get(clazz);
if (primary == null) {
primary = getFieldNameUsingReflection(clazz, Primary.class);
mapPrimary.put(clazz, primary);
}
return primary;
}
public String getSecondary(Class clazz) {
// TODO: Similar to getPrimary
}
}