I'm refactoring a camel route to hopefully be a little more generic. (I'm also using spring boot, if that helps for any possible bean injection solutions)
from(fromKafka)
.routeId("Rest Models")
.removeHeaders("*")
.aggregate(new GroupedBodyAggregationStrategy())
.constant(true)
.completionTimeout(batchingInterval)
.process(new ListOfJsonToJsonArray())
.unmarshal().json(JsonLibrary.Jackson, InputArrayPojo.class)
.enrich("seda:rest", mergeRestResult)
the processor ListOfJsonToJsonArray() takes the json string representation of the kafka message, and joins everything, comma separated, with a {[ ]} on the outside.
The InputArrayPojo.class is thus a wrapper for the array of objects that are coming in from kafka. I need to bundle the objects in order to mini-batch to the REST interface in the enrichment. The objects contained are of format InputPojo.class (effectively just a schema, but also performs some basic data quality checks)
I need a way to generify InputPojo.class such that for our new jobs, we can run the same route, but supply a different InputPojo.class.
I've tried to apply polymorphism and create an interface for InputPojo, however this runs into an error when trying to construct the interface.
#JsonSubTypes({
#JsonSubTypes.Type(value=InputPojo.class, name = "online")
})
public interface InputPojoInterface {
}
I also tried some parameterisation, but I had no luck there either because it would not apply the constructor of the bean, none of the methods then existed.
I've also included
com.fasterxml.jackson.databind.exc.InvalidDefinitionException - Cannot construct instance of `InputPojoInterface` (no Creators, like default construct, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
at [Source: (ByteArrayInputStream); line: 1, column: 10] (through reference chain: InputArrayPojo["data"]->java.util.ArrayList[0])]
#JsonInclude(JsonInclude.Include.NON_NULL)
#JsonPropertyOrder({
"data"
})
public class InputArrayPojo{
#JsonProperty("data")
private List<InputPojo> data = null;
#JsonIgnore
private Map<String, Object> additionalProperties = new HashMap<String, Object>();
#JsonProperty("data")
public List<InputPojo> getData() {
return data;
}
#JsonProperty("data")
public void setData(List<InputPojo> data) {
this.data = data;
}
#JsonAnyGetter
public Map<String, Object> getAdditionalProperties() {
return this.additionalProperties;
}
#JsonAnySetter
public void setAdditionalProperty(String name, Object value) {
this.additionalProperties.put(name, value);
}
}
The enrichment also needs to implement some type of generifying logic
#Override
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
List<IngestionOutPojo> originalMessages = oldExchange.getIn().getBody(IngestionOutArrayPojo.class).getData();
List<PredictionPojo> enrichmentMessages = newExchange.getIn().getBody(PredictionArrayPojo.class).getData();
List<ModelResultPojo> outputList = new ArrayList<>();
for (int i = 0; i < originalMessages.size(); ++i) {
ModelResultPojo output = new ModelResultPojo();
IngestionOutPojo raw = originalMessages.get(i);
PredictionPojo enrich = enrichmentMessages.get(i);
/*
enrichment logic to create modelResult
*/
outputList.add(modelResult)
}
newExchange.getIn().setBody(outputList);
return newExchange
}
I ended up coming up with a solution by doing the following:
unmarshalled to the default type: Map<String,Object> (without specifying a class, it camel unmarshalls to a Map<String,Object>)
After that I wrote an abstract class that implements a processor. In this processor I take the Map, and apply an abstract editFields() function to the Map.
thus I now have polymorphic handling of business logic through a Map instead of through a POJO.
Related
Following is the body of POST request to my endpoint:
{
"availabilityMap":
{
"2021-07-18":["9AM-10AM", "OT"],
"2021-07-19":["9AM-10AM", "OPD"],
.
.
}
}
and following is the skeleton of my controller:
#PostMapping(value = "/appointment", consumes = MediaType.APPLICATION_JSON_VALUE,
produces= MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity newAppointment(#RequestBody /* WHAT TO PUT HERE?*/) {
}
How do I define an entity so that the #RequestBody is automatically converted to an entity? Is that possible?
In general, we prepare dto or resource for request and response. If your requests or responses are related to the database, it should be dto, otherwise you should mark it as a resource and you can perform automatic mapping operations using MapStruct.
#Data
public class RequestDto {
public Map<String, List<String>> availabilityMap;
}
public class RequestVO {
public Map<String, List<String>> availabilityMap;
public Map<String, List<String>> getAvailabilityMap() {
return availabilityMap;
}
public void setAvailabilityMap(Map<String, List<String>> availabilityMap) {
this.availabilityMap = availabilityMap;
}
}
{} can be represented as an object or a map, and [] can be represented as an implementation class of a Collection. (Commonly used are List and Set, which are also interfaces. The specific classes are specified by the sequence number framework. You can also use implementation classes such as HashMap.) For {}, using an object or a map depends on whether the field is fixed or not. In the example, availabilityMap is a fixed-name field, so the object is used. 2021-07-18 is an indefinite date or other class, so use Map. For one object, if its field is another object, you may need to write a new java class.
I have several YAML config files I want to deserialize into a class. The YAML in the files consists of simple name value pairs with no nesting. There's a handful of properties that will have explicit fields, but the rest I just want dumped into a Map.
This all works fine, but I also want all the values of the properties that get deserialized into the Map through .add() to be run through a custom deserializer. I've tried using #JsonDeserialize on the setter value parameter and the setter method itself but Jackson seems to ignore it altogether.
Here's how it's set up:
public class ConfigData {
private Map<String, Object> dynamicConfig = new LinkedHashMap<>();
#JsonAnyGetter
public Map<String, Object> getConfig() {
return dynamicConfig;
}
#JsonAnySetter
public void add(String name, #JsonDeserialize(using = FooDeserializer.class) Object value) {
dynamicConfig.put(name, value);
}
#JsonProperty("some_special_property")
public String setSomeSpecialProperty(String value) {
add("some_special_property", value);
}
}
And to deserialize:
public static ConfigData getConfig(URL configResource) throws IOException {
try (InputStream stream = configResource.openStream()) {
ObjectMapper mapper = new YAMLMapper();
return mapper.readValue(new InputStreamReader(stream, StandardCharsets.UTF_8), ConfigData.class);
}
}
I discovered the problem was that I was specifying the deserializer class with the using property of the #JsonDeserialize annotation. For this specific use case I needed to use the contentUsing property instead, which is used for things like the value field of a Map entry.
This is what my setter looks like now:
#JsonAnySetter
#JsonDeserialize(contentUsing = FooDeserializer.class)
public void add(String name, Object value) {
dynamicConfig.put(name, value);
}
Now all the values will be serialized using FooDeserializer, except for "some_special_property" which has its own setter.
I have a problem with a generic function. I want to use a function to which I assign a certain class / type to first generate the corresponding result from a rest response and then return it.
public class TimerService {
[...]
public <T extends myObjInterface> RestMessageContainer<T> send(String endpointPath, Map<String, String> parameters, Class<T> clazz) {
[...]
Response response = webTarget.request(MediaType.APPLICATION_JSON_TYPE).get();
RestMessageContainer<T> container = response.readEntity(new GenericType<RestMessageContainer<T>>() {});
return container;
}
}
public class RestMessageContainer<T extends myObjInterface> {
[...]
#XmlAttribute(name = "data")
private List<T> data;
[...]
}
I get the following error message at runtime.
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `com.test.myObjInterface` (no Creators, like default construct, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
14:47:41,982 ERROR [stderr] (EJB default - 2) at [Source: (org.jboss.resteasy.client.jaxrs.internal.ClientResponse$InputStreamWrapper); line: 3, column: 14] (through reference chain: com.test.RestMessageContainer["data"]->java.util.ArrayList[0])
The error is output for the line RestMessageContainer<T> container = response.readEntity(new GenericType<RestMessageContainer<T>>() {});
Is my approach correct at all or how should I solve my problem?
Thanks for your advice,
I have several subclasses. There is no information about the type in the JSON strings. The type results from the reqeust address. I couldn't configure Jackson to recognize the subtype. There is no unique field in the JSON strings that I could use as a type.
I am not able to change the web service that delivers the JSON strings.
[UPDATE]
I have found a solution. I no longer let the JAX-RS client convert the JSON string. I have the JSON string returned to me as a string and convert it independently using Jackson.
public <T extends myObjInterface> RestMessageContainer<T> send(String endpointPath, Map<String, String> parameters, Class<T> clazz) {
[...]
Response response = webTarget.request(MediaType.APPLICATION_JSON_TYPE).get();
ObjectMapper mapper = new ObjectMapper();
mapper.setVisibility(PropertyAccessor.FIELD, Visibility.ANY);
mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
RestMessageContainer<T> container = mapper.readValue(response.readEntity(String.class), mapper.getTypeFactory().constructParametricType(RestMessageContainer.class, clazz));
return container;
}
You cannot create an instance of an abstract class. However, you can solve the problem with a simple annotation – #JsonDeserialize on the abstract class:
#JsonDeserialize(as = Cat.class)
abstract class Animal {...}
In your case, the abstract class would be myObjInterface.
Note: If you have more than one subtype of the abstract class, then you should consider including subtype information as shown in this post.
I am trying to de-serialize this JSON object using Jackson 2.8 as part of Retrofit response. Here is the JSON response I get from the server.
{
"id":"8938209912"
"version":"1.1"
"cars":{
"mercedes":[
{
"property":"color"
},
{
"property":"price"
},
{
"property":"location"
}
],
"tesla":[
{
"property":"environment"
}
]
}
}
Based on the query, the cars above may have one or more models returned. I cannot create a class each for each model as these get created/removed arbitrarily. For each model of the car (say tesla), there may be one or more property key-value pairs.
I am new to Jackson. I have been looking at several examples and looks like a custom #JsonDeserialize is the best way to go. So, I created Root class and Cars class like this:
// In file Root.java
public class Root {
#JsonProperty("id")
private String id = null;
#JsonProperty("version")
private String version = null;
#JsonProperty("cars")
private Cars cars = null;
}
// In file Cars.java
public class Cars {
public Cars(){}
#JsonDeserialize(using = CarDeserializer.class)
private Map<String, List<Property>> properties;
public Map<String, List<Property>> getProperties() {
return properties;
}
public void setProperties(Map<String, List<Property>> properties) {
this.properties = properties;
}
}
// Property.java
public class Property {
#JsonProperty("property")
private String property;
}
My de-serializer is below. However, even though the empty constructor gets called, the parse method itself is not called at all!
// CarDeserializer.class
public class RelationshipDeserializer extends StdDeserializer<Map<String, List<Action>>>{
protected RelationshipDeserializer(){
super(Class.class);
}
#Override
public Map<String, List<Action>> deserialize(JsonParser parser, DeserializationContext ctx)
throws IOException, JsonProcessingException
{
// This method never gets invoked.
}
}
My questions:
Is this the right approach in the first place?
Why do you think the execution never gets to the deserialize()? (I checked, the cars object is present in JSON.
Are there better approaches to parse this JSON using Jackson?
The "properties" deserializer is never called because that does not match anything in that JSON. The field name in the JSON is "property" and it does not match Map<String, List<Property>>. It looks like it would be closer to List<Property>
Do you control the in coming JSON? It would be better for the car name/type to be in its own field rather than the name of the object. Then you can use a generic object. What you have now is going to break. Any time they add a new name/type and you do not have a matching object for it.
I'm using Amazon's DynamoDBMapper Java class to save data to a DynamoDB table. This code needs to work for data structured in multiple different ways, so I would like to stay away from writing particularly structure-specific code. For this reason, I store the code as JSON objects in Java -- which are basically glorified HashMaps.
I would like to store these JSON objects into DynamoDB as Dynamo's relatively new JSON Document type.
The way the DynamoDBMapper API works is essentially that you write a Java class (typically a POJO), then add some annotations, then pass your objects of that class into DynamoDBMapper so that it can then put items into the database with the structure of the Java class. This works well for many aspects of what I'm doing, but not with the fact that I want these classes to contain arbitrarily-structured JSON documents. This is the way you're meant to store JSON documents using DynamoDBMapper, and as you can see, it doesn't allow for the structure of the documents to be arbitrary.
I realize I could use Dynamo's putItem() to pass the jsons as Strings into Item objects -- I just wanted to see if what I want to do is possible with DynamoDBMapper before I shift my approach.
You can try using the DynamoDB Java document SDK instead of the object mapper. This allows you to serialize and deserialize JSON strings using the fromJSON and toJSON methods in the Item class. Check out http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/JavaDocumentAPIItemCRUD.html.
Here's how I came up with my answer of how to store arbitrary Map objects in DynamoDB. This is extremely useful for archiving REST API responses that have been unmarshaled to foreign objects. I'm personally using this to archive REST responses from the PayPal Payment API. I don't care what variables they use in their REST API or the structure of their POJO / beans. I just want to make sure I save everything.
#DynamoDBTable(tableName = "PaymentResponse")
#JsonIgnoreProperties(ignoreUnknown = true)
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY)
#JsonSubTypes({
#JsonSubTypes.Type(value = PayPalPaymentResponse.class, name = "PayPalPaymentResponse"),
#JsonSubTypes.Type(value = BatchPayPalPaymentResponse.class, name = "BatchPayPalPaymentResponse")}
)
public abstract class PaymentResponse {
// store any arbitrary REST resrponse data in map form so we don't have to worry about the
// structure or the actual response itself
protected Map<String, String> paymentResponseData = Maps.newHashMap();
public PaymentResponse(PaymentResponseType paymentResponseType) {
this.paymentResponseType = paymentResponseType;
}
public Map<String, String> getPaymentResponseData() { return paymentResponseData; }
public void setPaymentResponseData(Map<String, String> paymentResponseData) { this.paymentResponseData = paymentResponseData; }
#Override
public String toString() {
return Arrays.toString(paymentResponseData.entrySet().toArray());
}
}
public class ConverterUtils {
public static BatchPayPalPaymentResponse getBatchPayPalPaymentResponse(PayoutBatch payoutBatch) throws IOException {
//read in the PayoutBatch response data and convert it first to a JSON string and then convert the
//JSON string into a Map<String, String>
Map<String, String> responseData = objectMapper.readValue(objectMapper.writeValueAsString(payoutBatch), new TypeReference<Map<String, String>>() {});
BatchPayPalPaymentResponse batchPayPalPaymentResponse = new BatchPayPalPaymentResponse(responseData);
return batchPayPalPaymentResponse;
}
public static PayPalPaymentResponse getSinglePayPalPaymentResponse(PayoutItemDetails payoutItemDetails) throws IOException {
//read in the paypal PayoutItemDetails response data and convert it first to a JSON string and then convert the
//JSON string into a Map<String, String>
Map<String, String> responseData = objectMapper.readValue(objectMapper.writeValueAsString(payoutItemDetails), new TypeReference<Map<String, String>>() {});
PayPalPaymentResponse payPalPaymentResponse = new PayPalPaymentResponse(responseData);
return payPalPaymentResponse;
}
}
public class BatchPayPalPaymentResponse extends PaymentResponse {
public BatchPayPalPaymentResponse(Map<String, String> responseData) {
super(responseData);
}
....
....
....
}
public class PayPalPaymentResponse extends PaymentResponse {
public PayPalPaymentResponse(Map<String, String> responseData) {
super(responseData);
}
....
....
....
}
Now you can just call mapper.save(instanceOfPaymentResponse). Note that my code also includes how to use a Jackson parser to pick and choose which sub-class of PaymentResponse to unmarshal too. That's because I use a DynamoDBTypeConverter to marshal my class to a string before putting it into the database.
Finally, I'll throw in my converter for completeness so it all hopefully makes sense.
public class PaymentResponseConverter implements DynamoDBTypeConverter<String, PaymentResponse> {
private static final ObjectMapper objectMapper = new ObjectMapper();
static {
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
}
#Override
public String convert(PaymentResponse object) {
try {
return objectMapper.writeValueAsString(object);
} catch (JsonProcessingException e) {
throw new IllegalArgumentException(String.format("Received invalid instance of PaymentResponse and cannot marshal it to a string (%s)", e.getMessage()));
}
}
#Override
public PaymentResponse unconvert(String object) {
try {
return objectMapper.readValue(object, PaymentResponse.class);
} catch (IOException e) {
throw new IllegalArgumentException(String.format("Unable to convert JSON to instance of PaymentResponse. This is a fatal error. (%s)", e.getMessage()));
}
}
}
I had the same problem and went the route of serializing and deserializing objects to json string by myself and then just store them as strings. The whole Document concept of DynamoDB is IMHO just a glorified object serializer. Only if you need to access attributes inside your object in dynamodb actions (eg. scans, projections) it makes sense to use the json document type. If our data is opaque to dynamodb, you are better off with strings.