How to save a Map<String, LocalDateTime> to DynamoDB? - java

I'm trying to add a Map<String, LocalDateTime> attribute to an already existing DAO. Seems that AWS SDK doesn't know how to convert that since I keep getting this error :
.DynamoDBMappingException: not supported; requires #DynamoDBTyped or #DynamoDBTypeConverted
I wrote a DynamoDBTypeConverter to try to resolve this, but it's not showing the data in the correct format:
public static class StringLocalDateTimeMapConverter
implements DynamoDBTypeConverter<String, Map<String, LocalDateTime>> {
#Override
public String convert(Map<String, LocalDateTime> map) {
try {
if (map != null) {
ObjectMapper mapper = new ObjectMapper();
return mapper.writeValueAsString(map);
} else {
throw new Exception("map is empty");
}
} catch (Exception e) {
LOGGER.error(String.format("Error converting map to Dynamo String. Reason - {%s}",
e.getMessage()));
return "";
}
}
#Override
public Map<String, LocalDateTime> unconvert(String string) {
try {
ObjectMapper mapper = new ObjectMapper();
return mapper.readValue(string, Map.class);
} catch (Exception e) {
LOGGER.error(String.format("Error unconverting Dynamo String to map. Reason - {%s}",
e.getMessage()));
return new HashMap<>();
}
}
}
However, this doesn't look as desired - in DDB the map ended up looking like:
{\"1234567890\":{\"year\":2019,\"month\":\"SEPTEMBER\",\"monthValue\":9,\"dayOfMonth\":20,\"hour\":15,\"minute\":13,\"second\":26,\"nano\":98000000,\"dayOfWeek\":\"FRIDAY\",\"dayOfYear\":263,\"chronology\":{\"calendarType\":\"iso8601\",\"id\":\"ISO\"}}}
I'm not sure what the best way to model this data so DDB is happy with it. Any suggestions?

maybe parse the localDateTime to string before saving to DDB and make sure to parse it back to LocalDateTime once reading to from DDB
You can save it in ISO format

This seem's an issue with the ObjectMapper and not with DynamoDb or with your CustomMapper.
I suppose that you use the Jackson provided by the AWS library, probably you need to add the JavaTimeModule into your project and then.
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.registerModule(new JavaTimeModule());
And all works fine.
This the link for the library
https://mvnrepository.com/artifact/com.fasterxml.jackson.datatype/jackson-datatype-jsr310

Related

Retrieve value from returned JSON object in Java

I am a receiving a JSON object and I need to save the values to my DB. But I'm having an issue figuring out how to retrieve the particular values in the JSON object.
In this case, I want to retrieve the values of 'originationNumber' and 'messageBody'
The response object -
{"originationNumber":"***","destinationNumber":"***","messageKeyword":"KEYWORD_***","messageBody":"Answer ","previousPublishedMessageId":"1slamq6mdpucd8q4i7iabf1sikc629ga253tr6o0","inboundMessageId":"88bc02fc-aff3-4277-ac1d-f27b6d3b6abb"}
Method to receive message -
public String getReceivedMessages(Messaging receivedMessage) {
BasicAWSCredentials awsCredentials = new BasicAWSCredentials(awsAccessKey, awsSecretKey);
AmazonSQS sqsClient = AmazonSQSClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
.withRegion(String.valueOf(awsRegion)).build();
StringBuilder sb = new StringBuilder();
String queueUrl = "https://sqs.us-east-1.amazonaws.com/1234567/GetReceivedMessages";
List<Message> messages = sqsClient.receiveMessage(new ReceiveMessageRequest(queueUrl)
.withMaxNumberOfMessages(1).withWaitTimeSeconds(20)).getMessages();
for (Message message : messages) {
sb.append(message.getBody());
sqsClient.deleteMessage(queueUrl, message.getReceiptHandle());
}
// Save messages to DB
String userId = connectionRequestRepository.getUserId();
Date date = new Date();
Timestamp now = new Timestamp(date.getTime());
receivedMessage.setUserId(userId);
receivedMessage.setOriginationNumber("");
receivedMessage.setDestinationNumber("***");
receivedMessage.setMessageBody("");
receivedMessage.setMessageType("RECEIVED");
receivedMessage.setCreatedAt(now);
messagingRepository.save(receivedMessage);
System.out.println(sb); <--- Prints response object to console
return sb.toString();
}
You can use jackson library for that.
Solution 1: You can use ObjectMapper as below:
Message Class to map JSON to Java Object:
public class Message {
private String originationNumber;
private String messageBody;
// public getter and setters methods
}
Create Object From JSON String:
Message message = null;
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
try {
message = objectMapper.readValue(payload, Message.class);
} catch (JsonProcessingException e) {
// Log Or do some action as per need
}
Here message will have those values. DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES is used here to skip exception as there are other fields also in JSON and not present in Java Class (as we don't need them).
Solution 2: Alternatively you can use JsonNode from same Jackson library and read nodes one by one as below:
ObjectMapper mapper = new ObjectMapper();
JsonNode actualObj = mapper.readTree("{\"originationNumber\":\"***\",\"destinationNumber\":\"***\",\"messageKeyword\":\"KEYWORD_***\",\"messageBody\":\"Answer \",\"previousPublishedMessageId\":\"1slamq6mdpucd8q4i7iabf1sikc629ga253tr6o0\",\"inboundMessageId\":\"88bc02fc-aff3-4277-ac1d-f27b6d3b6abb\"}");
String originationNumber = actualObj.get("originationNumber");
String messageBody = actualObj.get("messageBody");
In this approach you won't need to create Message class.
You can convert the json string into a json object
https://www.javatpoint.com/how-to-convert-string-to-json-object-in-java
String string = "{\"originationNumber\":\"***\",\"destinationNumber\":\"***\",\"messageKeyword\":\"KEYWORD_***\",\"messageBody\":\"Answer \",\"previousPublishedMessageId\":\"1slamq6mdpucd8q4i7iabf1sikc629ga253tr6o0\",\"inboundMessageId\":\"88bc02fc-aff3-4277-ac1d-f27b6d3b6abb\"}";
JSONObject json = new JSONObject(string);
System.out.println(json.toString());
String destinationNumber = json.getString("destinationNumber");
System.out.println(destinationNumber);
where ur json like
{"originationNumber":"***",
"destinationNumber":"***",
"messageKeyword":"KEYWORD_***",
"messageBody":"Answer","previousPublishedMessageId":"1slamq6mdpucd8q4i7iabf1sikc629ga253tr6o0",
"inboundMessageId":"88bc02fc-aff3-4277-ac1d-f27b6d3b6abb"
}
it like
"key":Value
i think ur code will be like
receivedMessage.setUserId(userId);
receivedMessage.setOriginationNumber("originationNumber");
receivedMessage.setDestinationNumber("destinationNumber");
receivedMessage.setMessageBody("messageBody");
receivedMessage.setMessageType("RECEIVED");
receivedMessage.setCreatedAt(now);
messagingRepository.save(receivedMessage);

JSON Response Has Escaped Quotations Using Jackson and JAX-RS Exception Mapper

I have a simple requirement where, if application encounters an exception, my JAX-RS Rest endpoint should return a custom JSON response with 500 HTTP header status.
Data needed to construct the response comes from an object with several properties (see below). The problem is, I am only interested in one or two values from each property (out of several dozens). And I cannot modify any of these models/classes (some have a Jackson annotation for JSON processing, e.g. null properties should be discarded during serialization).
public class MainObject {
private FirstProperty firstProperty;
private SecondProperty secondProperty;
private ThirdProperty thirdProperty;
// other codes
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
public class FirstProperty {
private boolean bol = true;
private double dob = 5.0;
private List<String> subProperty;
// other properties
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
#JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)
public class SecondProperty {
private String str;
private List<String> subProperty;
// other properties
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
#JsonSerialize(include=JsonSerialize.Inclusion.NON_NULL)
public class ThirdProperty {
private int intProp = true;
private List<String> subProperty;
// other properties
public String toString() {
ObjectMapper mapper = new ObjectMapper();
try { return mapper.writeValueAsString(this); }
catch (Exception e) { return null; }
}
}
The expected JSON that I should be seeing coming back is on the client side (say, a browser -- testing in Edge):
{
"firstProperty" : { "subProperty" : [ "val1" ] },
"secondProperty" : { "str" : "val2", "subproperty" : [ "val3", "val6" ] },
"thirdProperty" : { "subProperty" : [ "val4" ] }
}
Instead, all my field names and their values have their quotations escaped, and extra double quotes around the entire value, e.g.:
{
"firstProperty" : "{ \"subProperty\" : [ \"val1\" ] }",
"secondProperty" : "{ \"str\" : \"val2\", \"subproperty\" : [ \"val3\", \"val6\" ] }",
"thirdProperty" : "{ \"subProperty\" : [ \"val4\" ] }"
}
Please note the extra " before and after the curly brackets. My environment is:
Java 1.8.45
FasterXML Jackson 2.9.8
Spring Boot 2.0.1
RestEasy (JBoss) JAX-RS
JBoss 6.4
I eliminated the majority of "noise" in the code to see at what point this happens. This is the controller:
#Path("/")
public class MainController {
#GET
#Produces(MediaType.APPLICATION_JSON)
#Path("/rest/path")
public MainObject getMainObject throws MyCustomException {
// A service call that throws MyCustomException
}
}
And JAX-RS ExceptionMapper where I send the response back:
#Provider
public class MyCustomExceptionMapper extends ExceptionMapper<MyCustomException> {
#Override
public Response toResponse(MyCustomException ex) {
HashMap<String, Object> responseBody = new HashMap<>();
String strEx = ex.getStrEx(); // Comes from SecondProperty.str stored in MyCustomException, not that it matters
// Instantiate an empty object that contains
MainObject obj = new MainObject();
obj.getFirstProperty().setSubProperty(ex.getStrs());
obj.getSecondProperty().setStr(strEx);
obj.getSecondProperty().setSubProperty(ex.getStrs());
obj.getThirdProperty().setSubProperty(ex.getStrs());
responseBody.put("firstProperty", serializeFirstProperty(obj.getFirstProperty()));
responseBody.put("secondProperty", serializeSecondProperty(obj.getSecondProperty()));
responseBody.put("thirdProperty", serializeThirdProperty(obj.getThirdProperty()));
Response response = Response.status(/* 500 status */).entity(responseBody).build();
return response;
}
}
Since I only need to send back a very small subset of overall properties from each of my types, I created a custom StdSerializer that would only populate a needed property. For brevity, I only do serializeFirstProperty() but they are all more or less identical:
private StdSerializer<FirstProperty> getFPSerializer(FirstProperty firstProperty) {
return new StdSerializer<FirstProperty>(FirstProperty.class) {
#Override
public void serialize(FirstProperty value, JsonGenerator gen, SerializerProvider provider) throws IOException {
gen.writeStartObject();
if (/* there are items in FirstProperty.subProperty */) {
gen.writeArrayFieldStart("subProperty");
for (String str : value.getSubProperty()) {
gen.writeString(str);
}
gen.writeEndArray();
}
gen.writeEndObject();
}
}
private <T> ObjectMapper getCustomOM(StdSerializer<?> serializer) {
ObjectMapper mapper = new ObjectMapper();
SimpleModule sm = new SimpleModule();
sm.addSerializer(serializer);
mapper.registerModule(module);
return mapper;
}
Then use these helper methods like:
private String serializeFirstProperty(FirstProperty firstProperty) {
ObjectMapper mapper = getCustomOM(getFPSerializer(firstProperty));
String ser = null;
try { ser = mapper.writeValueAsString(firstProperty); }
catch (JsonProcessingException e) { return null; }
return ser;
}
I have tried countless of configurations with ObjectMapper, e.g. disable(JsonParser.Feature.ALLOW_BACKLASH_ESCAPING_ANY_CHARACTER) (couldn't find any relevant flag for JsonGenerator which I really want to disable in a similar fashion).
Or explicitly returning Object from serializeFirstProperty(), or replacing all the \" with " in serializeFirstProperty() when ser is returned.
Or set custom StdSerializer's JsonGenerator.setCharacterEscapes(new CharacterEscapes() { //... } or play around with JAX-RS Response at no avail. I always seem to get a "string" value with quotations, e.g.:
"firstProperty" : "{ \"subProperty\" : [ \"val1\" ] }"
If I simply just do
responseBody.put("firstProperty", mapper.writeValueAsString(obj.getFirstProperty()));
somehow this produces the right JSON output, however, it includes a lot of unnecessary properties which I don't want in this exception handling case.
Funny thing is, when I peer into response (or responseBody map), everything looks right (I don't see values having double quotations).
Please also note that not only I can't modify the models, but some of their properties are instantiated during creation with default values, so not-null inclusion doesn't work, and they will appear in the final JSON if I don't use a custom serialization.
Does anyone know what's causing this escaped and extra quotations?
I think I misunderstood the question in the first attempt of answering it.
The problem is that you serialize a property as string (using mapper.writeValueAsString(this) and then add it to the responseBody which you think of being string to json object map but it is a string to Java object map. In your case at runtime it is a string mapping to another string (the serialized json object is represented as Java string) and a Java string is a Java object as well.
What you want to do instead is to construct a Java object responseBody instead of a map. It should act as a DTO having all the specific properties etc and then serializing it in in one action using the mapper. Because if you first serialize a property to a json string than it it is just a string from the Java point of view and the mapper has no chance to interpret it as a json object.

Spring Boot 2 WebClient response convert JSON to HashMap

I would like to get a response from WebClient and convert it to just a Map without creating any Class for the response. Is it possible? So I would like something like this below. The code below is not a valid code, it is just an idea of what I want.
public Map<String, String> someFunction() {
return webClient.post()
.uri("/some/path")
.retrieve()
.bodyToFlux(HashMap.class)
.block();
If you're interested in saving a LOC, you may want to look at a core Spring Framework class: ParameterizedTypeReference<T>, found here.
public Map<String, String> someFunction() {
return webClient.post()
.uri("/some/path")
.retrieve()
.bodyToFlux(new ParameterizedTypeReference<Map<String,String>>(){})
.block();
}
Cheers.
I would first try getting the response object into a String and also make sure I am accepting JSON type in return. Once I get the respone into a String, you can try using fasterxml's jackson databind library which can convert a JSON string into Hashmap.
For example
ObjectMapper mapper = new ObjectMapper();
String json = "{\"name\":\"abc\", \"age\":25}";
Map<String, Object> map = new HashMap<String, Object>();
// convert JSON string to Map
map = mapper.readValue(json, new TypeReference<Map<String, String>>(){});
System.out.println(map);
Here is the databind library and core library java docs link
jackson-databind
jackson-core
I solved it like this:
public Map<String, String> someFunction() {
return webClient.post()
.uri("/some/path")
.retrieve()
.bodyToFlux(TypedMap.class)
.block();
}
private static class TypedMap extends HashMap<String, String>{}

JSON file printed with bars separating fields

I need to send a minified JSON to a rest service, it seemed all right, however, when printing my JSON into a file several "/" slashes appear separating the fields from the file, as I am new to this type of implementation I was confused ...
Basically, I add objects to a list and at the end, I convert the generic list to Json, this is my method that converts the List object to JSON
public String convertListToJson(List obj) throws IOException {
String jsonInString = null;
ObjectMapper mapper = new ObjectMapper();
mapper.setDefaultPropertyInclusion(JsonInclude.Include.ALWAYS);
mapper.configure(SerializationFeature.USE_EQUALITY_FOR_OBJECT_ID, true);
jsonInString = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(obj);
JsonNode jsonNode = mapper.readValue(jsonInString, JsonNode.class);
String json = jsonNode.toString();
return json;
}
String payload = convertListToJson(object);
System.out.println(payload);
This is the result of printing on the console
[{"id":12,"nomePeriodo":"2012","inicio":"2012-01-01T02:00:00Z","fim":"2012-12-31T02:00:00Z"},{"id":13,"nomePeriodo":"2013","inicio":"2013-01-01T02:00:00Z","fim":"2013-12-31T02:00:00Z"},{"id":14,"nomePeriodo":"2014","inicio":"2014-01-01T02:00:00Z","fim":"2014-12-31T02:00:00Z"},{"id":15,"nomePeriodo":"2015","inicio":"2015-01-01T02:00:00Z","fim":"2015-12-31T02:00:00Z"},{"id":16,"nomePeriodo":"2016","inicio":"2016-01-01T02:00:00Z","fim":"2016-12-31T02:00:00Z"},{"id":17,"nomePeriodo":"2017","inicio":"2017-01-01T02:00:00Z","fim":"2017-12-31T02:00:00Z"},{"id":18,"nomePeriodo":"2018","inicio":"2018-01-01T02:00:00Z","fim":"2018-12-31T02:00:00Z"}]
This is the method I use for printing to file
printJson(payload);
private void printJson(String payload) {
ObjectMapper mapper = new ObjectMapper();
try {
mapper.writeValue(new File("c:\\server\\JSON.json"), payload);
} catch (IOException e) {
e.printStackTrace();
}
}
When I print to a file this is the result:
"[{\"id\":12,\"nomePeriodo\":\"2012\",\"inicio\":\"2012-01-01T02:00:00Z\",\"fim\":\"2012-12-31T02:00:00Z\"},{\"id\":13,\"nomePeriodo\":\"2013\",\"inicio\":\"2013-01-01T02:00:00Z\",\"fim\":\"2013-12-31T02:00:00Z\"},{\"id\":14,\"nomePeriodo\":\"2014\",\"inicio\":\"2014-01-01T02:00:00Z\",\"fim\":\"2014-12-31T02:00:00Z\"},{\"id\":15,\"nomePeriodo\":\"2015\",\"inicio\":\"2015-01-01T02:00:00Z\",\"fim\":\"2015-12-31T02:00:00Z\"},{\"id\":16,\"nomePeriodo\":\"2016\",\"inicio\":\"2016-01-01T02:00:00Z\",\"fim\":\"2016-12-31T02:00:00Z\"},{\"id\":17,\"nomePeriodo\":\"2017\",\"inicio\":\"2017-01-01T02:00:00Z\",\"fim\":\"2017-12-31T02:00:00Z\"},{\"id\":18,\"nomePeriodo\":\"2018\",\"inicio\":\"2018-01-01T02:00:00Z\",\"fim\":\"2018-12-31T02:00:00Z\"}]"
So you have these \ bars separating the items in json...
Am I doing something wrong or is this common?
If possible I would like to remove them from my archive

JsonParseException: Unexpected character ('i' (code 105)): was expecting double-quote

I am trying to integrate Spring Cloud streams and publishing a custom Java Object across services with RabbitMQ as broker. The object I am publishing looks like:
public class AppMessageEnvelope implements Serializable {
...
private Object messageBody;
private Date sentAt = new Date();
...
// setters and getters
}
This is just a wrapper object and the original object is put in messageBody. The object I am putting in messageBody looks like:
public class Job {
...
private String message;
private Map<MyEnum, String> myMap;
...
}
Note that both AppMessageEnvelope and Job are in a different model project which is imported as a Maven dependency in the publisher and subscriber Spring Boot projects, so models are exactly the same.
In producer, I publish the object as:
#EnableBinding(Source.class)
public class JobDistributor {
private final Source jobQueue;
#Autowired
public JobDistributor(Source jobQueue) {
this.jobQueue = jobQueue;
}
public AppMessageEnvelope publishJob(AppMessageEnvelope message) {
LOG.info("Sending message: {}.", message);
jobQueue.output().send(MessageBuilder.withPayload(message).build());
return message;
}
}
In consumer, I get the message as:
#Component
#EnableBinding(Sink.class)
public class JobConsumer {
private final JobManager jobManager;
private final ObjectMapper objectMapper;
#Autowired
public JobConsumer(
JobManager jobManager, ObjectMapper objectMapper) {
this.jobManager = jobManager;
this.objectMapper = objectMapper;
}
#StreamListener(target = Sink.INPUT)
public void processData(AppMessageEnvelope messageEnvelope) {
LOG.info("Envelope received: {}.", messageEnvelope);
try {
TypeReference<Job> mapType = new TypeReference<Job>() {};
Job job = objectMapper.readValue(messageEnvelope.getMessageBody().toString(), mapType);
jobManager.processRequest(job);
} catch (Exception ex) {
LOG.error("Couldn't convert to correct object for processing: {}.", ex);
}
}
}
I try to use TypeReference to convert the internal object into a correct object but I get an error as:
JobConsumer - Couldn't convert to correct object for processing: {}.
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('i' (code 105)): was expecting double-quote to start field name
at [Source: (StringReader); line: 1, column: 3]
Before the message is converted, I log it:
JobConsumer - Envelope received: AppMessageEnvelope{..., messageBody={id=5bf3a7302dbe9c7cf9927c60, jobId=8c0bfcb0b21248e694b5cd52337a1f9e, submittedAt=2018-11-20T06:18:24+0000, lastUpdatedOn=null, message=null, ..., fileContentMap={FILE_BYTES=JVBERi0xLjUKJb/3ov}}, sentAt=Tue Nov 20 11:48:24 IST 2018}
I tried configuring ObjectMapper as:
#Autowired
private ObjectMapper objectMapper() {
JsonFactory factory = new JsonFactory();
factory.enable(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES);
return new ObjectMapper(factory);
}
I tried enabling unquoting fields with this too:
objectMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
I tried solutions provided by this blog and some similar SO problem but nothing solved. What am I missing?
Use below json conversion to convert JSON to Model / Entity
ObjectMapper objectMapper = new ObjectMapper();
Job job = objectMapper.convertValue(messageEnvelope.getMessageBody().toString(), Job.class);
This may be funny error please copy your json and past into online json formatter.
if the validation wrong please rewrite the json manually or encode it.
Error cause: character encoding issue at time of copy from various sources
Here the problem:
Job job = objectMapper.readValue(messageEnvelope.getMessageBody().toString(), mapType);
Just like this:
Job job = objectMapper.readValue(objectMapper.writeValueAsString(messageEnvelope.getMessageBody()), mapType);

Categories

Resources