I have a variety of JSON files (with slightly different schemas) flowing on Kinesis. Their Schema is really complex. They are user raw hits and their schema is super complex. I would like to create a Single POJO to represent all underneath messages (Something Spark does internally by creating a Single schema). I was trying to use GSON library but the only way to accomplish this is, by writing a custom deserializer and in that case, I will end up writing deserialization logic for all the fields in those JSONs.
Is there any way where we can only overwrite the deserialization of a few fields only and the rest of the fields can still be deserialized by GSON as a default way?
JSON-1
{
"first_col":"abc",
"second_col":false
}
JSON-2
{
"first_col":"abc",
"second_col":"false-String"
}
JSON-3
{
"first_col":"abc",
"second_col":{
"col1":"xyz",
"col2":123
}
}
Common POJO
import com.google.gson.annotations.Expose;
import com.google.gson.annotations.SerializedName;
public class Example {
private String firstCol;
private String secondCol;
public String getFirstCol() {
return firstCol;
}
public void setFirstCol(String firstCol) {
this.firstCol = firstCol;
}
public Boolean getSecondCol() {
return secondCol;
}
public void setSecondCol(String secondCol) {
this.secondCol = secondCol;
}
}
So basically second_col could be boolean, string, or complex object. first_col is always a string. So I don't want to write deserialize logic for first_col. I want to write deserialize logic for only second_col and deserialize it to string only and the downstream consumer will take care of converting it to the right type before consuming this field.
Related
I have some json object that looks like this:
{
"make":"Volvo",
"model":"240",
"metadata":{
"color":"white",
"year":"1986",
"previousOwner":"Joe",
"condition":"good"
}
}
And I want to turn this JSON into List<Car>, which is comprised of the following objects:
public class Car {
private String make;
private String model;
private CarMetadata carMetadata;
}
public class CarMetadata {
private Body body;
private History history;
}
public class Body {
private String color;
private String condition;
}
public class History {
private String previousOwner;
private String year;
}
So essentially the point is that the object I want to turn it into (Car) is very nested, whereas my JSON is not very nested. In reality the "Car" object is actually much more nested than this example I'm showing.
I was thinking of two options:
Create a CarDTO object to represent my input JSON, do objectMapper.readValue(json, CarDTO.class), then map CarDTO to Car to create my List<Car>.
Just parse the JSON and create the final List<Car> object in the first place.
I don't want to create an unnecessary DTO, but I also don't want to mess with parsing this JSON.
Is there a best practice in this scenario, and would this even be a valid use of a DTO?
Use a DTO.
Although you can deserialize from json directly to your domain class, their structure differs so you would have to create a custom deserializer... DO NOT TRY THIS AT HOME. I've been there and it's completely not worth the hassle.
Use the DTO to parse the json into a POJO, then map the DTO to the domain object.
This will decouple the transport from your domain object, allowing both to change freely with only the mapping code being affected. It's also way easier to write, understand, test and debug.
I am attempting to implement the HL7 FHIR spec's assertion that JSON representing a FHIR model will not have empty objects nor empty arrays. For the sake of not making the lives of my consumers any harder, I'm not strictly enforcing this during deserialization, but I want to ensure the serialized JSON produced by my library conforms as specified. I am using Java and Jackson ObjectMapper to serialize Objects into JSON. My understanding from writing a custom serializer is that the Object is at one point represented as JsonNode, regardless of what you are converting to.
What I would like to do is intercept the JsonNode as it exits the serializer, make some adjustments to it (find and remove empty arrays and objects), and then let it continue on its way. I need to do this in an environment where I can't tweak the ObjectMapper, because I don't have access to it. And further, the complex hierarchy of models in this library use Jackson's default serialization with annotations etc. heavily, and I cannot eliminate this.
If I go the route of defining a custom serializer for the base type, let's say "Resource", then I have a problem, because I still need the original serializer's output in order to generate my modified output. And further, that needs to accommodate any custom serializers that may already exist on various types within the model.
I got pretty far with the above option using https://www.baeldung.com/jackson-call-default-serializer-from-custom-serializer and the last option, implementing BeanSerializerModifier, but I ran into the issue where I can't control the ObjectMapper that my library consumers use.
Example POJOs (Using Lombok for accessors):
#Data
#JsonInclude(JsonInclude.Include.NON_EMPTY)
#JsonIgnoreProperties(ignoreUnknown = true)
abstract class Resource {
private FhirString id;
private List<Extension> extension;
#JsonProperty(access = JsonProperty.Access.READ_ONLY)
public abstract ResourceType getResourceType();
}
#Data
#Builder
class SomethingElse extends Resource {
FhirUri someProperty;
CodeableConcept someCode;
List<Reference> someReferences;
#Override
public ResourceType getResourceType() {
return ResourceType.SOMETHING_ELSE;
}
}
And an example instance of the SomethingElse class:
SomethingElse somethingElse = SomethingElse.builder()
.someProperty(FhirUri.from("some-simple-uri"))
.someCode(new CodeableConcept())
.someReference(List.of(new Reference()))
.build();
somethingElse.setId(FhirString.randomUuid());
somethingElse.setExtension(new ArrayList<>());
When I tell any mapper (or, for example, use a Spring service) to map the SomethingElse class into JsonNode, I can, for example, end up with empty objects and arrays, like this:
ObjectMapper mapper = getUntouchableMapper();
JsonNode somethingElseNode = mapper.valueToTree(somethingElse);
System.out.println(somethingElseNode.toString());
Becomes:
{
"resourceType": "SomethingElse",
"id": "00000000-0002-0004-0000-000000000000",
"someProperty": "some-simple-uri",
"someCode": {},
"someReferences": [{}],
"extension": []
}
According to FHIR, this should actually look like:
{
"resourceType": "SomethingElse",
"id": "00000000-0002-0004-0000-000000000000",
"someProperty": "some-simple-uri"
}
To summarize
How do I preserve the serialization mechanisms already in place, regardless of the ObjectMapper used, and somehow remove empty lists and objects from outgoing JSON produced by the Jackson serialization process?
Edit:
I also tried #JsonInclude(JsonInclude.Include.NON_EMPTY), which did omit empty list implementations. However, the vast majority of data in this library is represented by POJOs that serialize to maps and primitives, and this annotation only works if they are represented directly by maps and primitives in the model.
The solution is to use a custom #JsonInclude, which is new in Jackson 2.9. Thank you #dai for pointing me back towards this functionality.
On the base Resource class, this looks like:
#JsonInclude(value = JsonInclude.Include.CUSTOM, valueFilter = FhirJsonValueFilter.class)
class Resource implements FhirTypeInterface {
...
#Override
public boolean isEmpty() {
//Details omitted for simplicity
}
}
For visibility, the interface used above:
interface FhirTypeInterface {
boolean isEmpty();
}
And my custom definition for FhirJsonValueFilter implements all of the functionality of JsonInclude.Include.NON_EMPTY but also adds functionality for checking against a method implemented by FHIR types (implementation of this is not relevant to the answer).
public class FhirJsonValueFilter {
#Override
public boolean equals(Object value) {
return !getWillInclude(value);
}
/**
* Returns true for an object that matched filter criteria (will be
* included) and false for those to omit from the response.
*/
public boolean getWillInclude(Object value) {
//Omit explicit null values
if (null == value) {
return false;
}
//Omit empty collections
if (Collection.class.isAssignableFrom(value.getClass())) {
return !((Collection) value).isEmpty();
}
//Omit empty maps
if (Map.class.isAssignableFrom(value.getClass())) {
return !((Map) value).isEmpty();
}
//Omit empty char sequences (Strings, etc.)
if (CharSequence.class.isAssignableFrom(value.getClass())) {
return ((CharSequence) value).length() > 0;
}
//Omit empty FHIR data represented by an object
if (FhirTypeInterface.class.isAssignableFrom(value.getClass())) {
return !((FhirTypeInterface) value).isEmpty();
}
//If we missed something, default to include it
return true;
}
}
Note that the custom omission filter uses Java's Object.equals functionality, where true means to omit the property, and I've used a second method to reduce confusion in this answer.
I am trying to use Jackson to serialize the same DTO object in 2 different ways, depending on the #JsonView.
I want to use 2 different names for the same field. In one case I want to name the json property myField (just like the class field name), in the other I want it to be named myInternalApiField.
As a result I would like to see outcomes similar to the presented below:
Usage 1 (External API View):
{
"myField": "value1",
"myOtherField": "otherValue"
}
Usage 2 (Internal API View):
{
"myInternalApiField": "value1",
"myOtherField": "otherValue"
}
In my implementation in Java to achieve that I used the combination of custom getters, setters and #JsonView annotation as below:
public class CustomDTO {
#JsonView(Views.ExternalApiView)
private String myField;
// Other fields here
#JsonView(Views.InternalApiView)
public String getMyInternalApiField() { return myField; }
#JsonView(Views.InternalApiView)
public void setMyInternalApiField(String value) { this.myField = value; }
#JsonView(Views.ExternalApiView)
public String getMyField() { return myField; }
#JsonView(Views.ExternalApiView)
public void setMyField(String value) { this.myField = value }
}
However I don't know how to properly achieve the same result in Kotlin.
I was thinking about using something like:
data class CustomDTO(
#get:[JsonView(Views.ExternalApiView) JsonProperty("myField")]
#get:[JsonView(Views.InternalApiView) JsonProperty("myInternalApiField")]
#set:[JsonView(Views.InternalApiView) JsonProperty("myField")]
#set:[JsonView(Views.InternalApiView) JsonProperty("myInternalApiField")]
var myField: String,
val myOtherField: String,
val myDifferentField: String
)
But this is not allowed in Kotlin.
Do you have any suggestions how to utilize the #JsonView in Kotlin in the similar way as I did it in Java?
How about something like:
data class CustomDTO(
#JsonView(ExternalApiView::class)
var myField: String,
val myOtherField: String,
val myDifferentField: String
) {
val myExternalField: String
#JsonView(InternalApiView::class)
get() {
return myField
}
}
It looks like there are ways that don't require creating computed properties in the DTO, like:
Using Jackson Mixins
Creating a custom serializer for a particular invocation
Combining a custom serializer with custom annotations
But these have their own complexity, even if that complexity isn't in the DTO class. I'm not sure these are much more appealing to me but you could see if they appeal to you.
I have a java api which performs an external resource lookup and then maps the values to a Pojo. To do this, the api needs the field names of the Pojo as string values, something like:
public <F> F populatePojoFields(String primaryField, String secondaryField);
This works fine, however passing the pojo field names as String to the api does not feel right. I was able to change this by writing marker annotations for the pojo, so now it is like
public class POJO {
#Primary //custom marker annotation
private int mojo;
#Secondary //custom marker annotation
private String jojo;
}
String primaryField = getFieldNameUsingReflection(Pojo.class, Primary.class)
String secondryField = getFieldNameUsingReflection(Pojo.class, Secondary.class)
Pojo pojo = populatePojoFields(primaryField, secondaryField);
This way I don't have to keep track of string values, I can just add marker annotations to the Pojo fields. This works fine, but I'm worried about performance. Is this a standard way to do things? as keeping hardcoded string values is more efficient than looking up the field names every time we need to call the api. Is there a better way to do this?
If you call getFieldNameUsingReflection often you can think to cache the result of this call.
You can use a singleton class with internal Map with a code like the following:
public class SingletonMapPrimarySecondary {
Map<Class, String> mapPrimary;
Map<Class, String> mapSecondary;
// TODO: Handle mapPrimary and mapSecondary creation and singleton pattern
public String getPrimary(Class clazz) {
String primary = mapPrimary.get(clazz);
if (primary == null) {
primary = getFieldNameUsingReflection(clazz, Primary.class);
mapPrimary.put(clazz, primary);
}
return primary;
}
public String getSecondary(Class clazz) {
// TODO: Similar to getPrimary
}
}
I have this specific problem with JSON deserialization. Let's have this JSON structure:
{
"header":{
"objects":[
{"field":"value1"},
{"field":"value2"}
]
}
}
The JSON structure can't be altered as it comes from a 3rd party system.
Now let's have this simple POJO:
#JsonDeserialize(using=PojoDeserializer.class)
public class Pojo {
private string field;
//...getter, setter
}
The mentioned PojoDeserializer takes {"field": "value"} json string and deserializes it to the Pojo instance. So I can simply do the deserialization like this
Pojo instance = new
ObjectMapper().readValue("{\"field\":
\"value\"}", Pojo.class);
And here's my problem. Let's have another deserializer PojosCollectionDeserializer which takes the mentioned structure and deserializes it to a Collection of Pojo instances. I'd like to use it in a similar fashion as in the previous example:
Collection<Pojo> pojos = new ObjectMapper().readValue("{...}", Collection.class);
But this doesn't work as there is not defined that Collection should be created using the PojosCollectionDeserializer. Is there any way to achieve it?
I am not sure why are trying to explicitly specify deserializers, as it would all work just fine with something like:
public class Message {
public Header header; // or, if you prefer, getter and setter
}
public class Header {
public List<Pojo> objects;
}
public class Pojo {
public String field;
}
Message msg = objectMapper.readValue(json, Message.class);
without any additional configuration or annotations. There is no need to construct custom serializers or deserializers for simple cases like this.