Develop Reference

How to deserialize/serialize Azure Service Bus using spring.cloud in Java

I am using spring.cloud to connect to Azure Service Bus in Java. Here is maven dependency I am using:
<dependency>
<groupId>com.azure.spring</groupId>
<artifactId>spring-cloud-azure-starter-integration-servicebus</artifactId>
<version>4.5.0</version>
</dependency>
I am able to consume the message from the queue as byte array and it converts the message to string. Here is my main code after receiving a message from queue:
#ServiceActivator(inputChannel = INPUT_CHANNEL)
public void messageReceiver(byte[] payload, #Header(AzureHeaders.CHECKPOINTER) Checkpointer checkpointer) {
String message = new String(payload);
LOGGER.info("New message received: '{}'", message);
checkpointer.success()
.doOnSuccess(s -> LOGGER.info("Message '{}' successfully checkpointed", message))
.doOnError(e -> LOGGER.error("Error found", e))
.block();
}
And here is my example data in JSON as short version:
{
"serverId": 123,
"message": "{some message}"
}
What I would like to do is to create a Java object like this:
public class ExampleMessage {
private final Integer serverId;
private final String message;
and when a message from queue is consumed, it will convert the message to my Java object. I am used to using DataTypeProvider in order to use custom Java object for AMQP message consumption which will convert and validate the conversion behind the scene. Does spring.cloud.azure has built-in method/functionality for deserialization? or Do I manually deserialize and do validation for a consumed message?

Here I was able to convert the Json object to java object using Gson class.
I am just reading a message from the service bus and converting it to java object.
my pom.xml (dependencies)
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-servicebus</artifactId>
<version>7.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.projectreactor/reactor-core -->
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-core</artifactId>
<version>3.3.11.RELEASE</version>
</dependency>
My test class :
public class TestClass {
public String name ;
public int version;
TestClass(String n , int v)
{
this.name = n ;
this.version = v ;
}
}
The main class :
#SpringBootApplication
public class ServicebustestApplication {
public static void main(String[] args) throws InterruptedException {
SpringApplication.run(ServicebustestApplication.class, args);
String conn = " Endpoint=sb://v-mganorkarjsonobject.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=HglLVGlgMsYZGQMOtUfp4g2oka1CpCbVR0YEHgly7jA= ";
CountDownLatch countdownLatch1 = new CountDownLatch(1);
ServiceBusProcessorClient processorClient = new ServiceBusClientBuilder()
.connectionString("<Your COnnection String >")
.processor()
.queueName("test")
.processMessage(ServicebustestApplication::processMessage)
.processError(context -> processError(context,countdownLatch1))
.buildProcessorClient();
processorClient.start();
TimeUnit.SECONDS.sleep(10);
processorClient.close();
}
private static void processMessage(ServiceBusReceivedMessageContext context) {
ServiceBusReceivedMessage message = context.getMessage();
System.out.printf("Processing message. Session: %s, Sequence #: %s. Contents: %s%n", message.getMessageId(),
message.getSequenceNumber(), message.getBody());
Gson gson = new Gson();
TestClass testobject = gson.fromJson(String.valueOf(message.getBody()),TestClass.class);
System.out.println("name: "+testobject.name +" version: "+ testobject.version+"");
}
private static void processError(ServiceBusErrorContext context, CountDownLatch countdownLatch) {
}
}
Here the callback to process message will process the message and then we can use the GSON to convert the json string to java object.
Gson gson = new Gson();
TestClass testobject = gson.fromJson(String.valueOf(message.getBody()),TestClass.class);
output of the code :

How can I deserialize JSON with u' unicode markup in Java?

I have this HTTP response body to de-serialize:
String response = "result : {'url': u'https://somedomain/', 'fields': {'policy':
u'eyJjb25kaXRpb25zIjogW1siYfgfhudGVudC1sZjMyWiJ9', 'AWSAccessKeyId':
u'ASIccccccNA', 'x-amz-security-token': 'FQofgF', 'key': u'bbb.file',
'signature': u'rm9gdflkjfs='}}"
I am using the jackson.core (2.9.0) java package and lib (have also tried GSON) and get this error:
Exception in thread "main" com.fasterxml.jackson.core.JsonParseException:
Unrecognized token 'u': was expecting ('true', 'false' or 'null')
Deserialization code:
MyResponse deserializedResponse = new ObjectMapper()
.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true)
.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true)
.configure(JsonGenerator.Feature.ESCAPE_NON_ASCII, true)
.readValue(response, MyResponse.class);
I have considered something like this but it feels like there should be a better / safer way:
String sanitizedResponse = response.replaceAll("u'", "'");
--
Using Java 1.8.
Any help appreciated.

As python caused this problem I think the best solution is to let python fix it ;-). Fortunately with jython you can stick with a pure java implementation.
First you need to add the jython standalone dependency in your pom.xml:
<dependencies>
<dependency>
<groupId>org.python</groupId>
<artifactId>jython-standalone</artifactId>
<version>2.7.1</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.5</version>
</dependency>
</dependencies>
(As you can see I also used apache commons io for my example so I added it as well)
I put your (invalid) json string into the text file "c:/temp/json.txt" which has the following content:
{'url': u'https://somedomain/', 'fields': {'policy':
u'eyJjb25kaXRpb25zIjogW1siYfgfhudGVudC1sZjMyWiJ9', 'AWSAccessKeyId':
u'ASIccccccNA', 'x-amz-security-token': 'FQofgF', 'key': u'bbb.file',
'signature': u'rm9gdflkjfs='}}
Now here is the code to read the json file, set up the Python Interpreter and handover the json to clean it up:
String content = FileUtils.readFileToString(new File("c:/temp/json.txt"), "UTF-8");
PythonInterpreter pi = new PythonInterpreter();
pi.exec("import json");
pi.exec("jsondata = " + content);
pi.exec("jsonCleaned = json.dumps(jsondata)");
PyObject jsonCleaned = (PyObject) pi.get("jsonCleaned");
System.out.println(jsonCleaned.asString());
pi.close();
The output is:
{"url": "https://somedomain/", "fields": {"signature": "rm9gdflkjfs=", "AWSAccessKeyId": "ASIccccccNA", "x-amz-security-token": "FQofgF", "key": "bbb.file", "policy": "eyJjb25kaXRpb25zIjogW1siYfgfhudGVudC1sZjMyWiJ9"}}
When you put that in a json validator (https://jsonlint.com/) you can see that it is a valid json now.
I can't tell if the performance is good enough for your use case so you have to test that out.
Remark:
In Eclipse there seems to be a bug with that jython version. It shows the following error:
console: Failed to install '': java.nio.charset.UnsupportedCharsetException: cp0.
Although it works nevertheless you can get rid of it by adding the following VM-Argument to your Run-Configuration:
-Dpython.console.encoding=UTF-8
Remark2: For the sake of completeness and to fully answer that question - here is how you can deserialize the cleaned JSON:
Add GSON Dependency to your pom.xml:
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.2</version>
</dependency>
Create representing classes:
Info class
public class Info {
private String url;
private Fields fields;
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url;
}
public Fields getFields() {
return fields;
}
public void setFields(Fields fields) {
this.fields = fields;
}
}
Fields class
import com.google.gson.annotations.SerializedName;
public class Fields {
private String signature;
private String AWSAccessKeyId;
#SerializedName("x-amz-security-token")
private String x_amz_security_token;
private String key;
private String policy;
public String getSignature() {
return signature;
}
public void setSignature(String signature) {
this.signature = signature;
}
public String getAWSAccessKeyId() {
return AWSAccessKeyId;
}
public void setAWSAccessKeyId(String aWSAccessKeyId) {
AWSAccessKeyId = aWSAccessKeyId;
}
public String getX_amz_security_token() {
return x_amz_security_token;
}
public void setX_amz_security_token(String x_amz_security_token) {
this.x_amz_security_token = x_amz_security_token;
}
public String getKey() {
return key;
}
public void setKey(String key) {
this.key = key;
}
public String getPolicy() {
return policy;
}
public void setPolicy(String policy) {
this.policy = policy;
}
}
Finally add the following code after you get your cleaned JSON:
Gson gson = new Gson();
Info info = gson.fromJson(jsonCleaned.asString(), Info.class);

You need to use the following regex that replaces u' from beginning of word boundaries to replace with '
String regexPattern = "(\\bu')";

Cloudant Library adding unnecessary properties while saving document

Most probably this issue is because of JSONObject(org.json.JSONObject) is incompatible with cloudant library.
Is any alternative way to use any other Object?
I am using below cloudant libraries,
<dependency>
<groupId>com.cloudant</groupId>
<artifactId>cloudant-client</artifactId>
<version>2.6.2</version>
</dependency>
Here is my code
package data.repositories;
import org.json.JSONObject;
import com.cloudant.client.api.*;
import com.cloudant.client.api.CloudantClient;
import com.cloudant.client.api.Database;
import com.cloudant.client.api.model.Response;
import util.Config;
public class DatabaseRepository {
CloudantClient client = ClientBuilder.account(Config.CLOUDANT_ACCOUNT_NAME)
.username(Config.CLOUDANT_USER_NAME)
.password(Config.CLOUDANT_PASSWORD).build();
public DatabaseRepository() {
JSONObject
}
public void Save(String dbName) {
Database db = client.database("dbTempName", true);
JSONObject jsonObject = new JSONObject("{hello: data}");
db.save(jsonObject);
}
}
Document saved in cloudant database is,
{
"_id": "1c7f223f74a54e7c9f4c8a713feaa537",
"_rev": "1-a3cd12379eec936b61f899c8278c9d62",
"map": {
"hello": "data"
}
}

I'm not familiar with cloudant but my guess is JsonObject has a property called "map" that holds your json string data (probably there's a myArray property too), and cloudant serializes it into json, thus adding those unnecessary values.
my suggestions:
1) try to save your json string directly like db.save("{hello: data}") to avoid serialization
2) if you really need to create a JsonObject try to customize cloudant's serialization process to avoid that extra fields.
in response to comment:
from what I read here, then I think you need a pojo, which when serialized into json would look like:
{ 'hello' : 'data' }
which is something like:
public class MyClass implements Serializable {
String hello;
public MyClass(String hello) {
this.hello = hello;
}
public String getHello() {
return hello;
}
}
then save it like:
db.save(new MyClass("data"));
or you can use a hashmap instead of a pojo:
Map<String, Object> map = new Hashmap ...
map.put("hello", "data");
db.save(map);

Look at the example in the README for the repo. It shows that you want a POJO, but you don't have to implement Serializable. Just create a class that has _id and _rev properties that are Strings. Then add Javascript object compatible properties as desired.
// A Java type that can be serialized to JSON
public class ExampleDocument {
private String _id = "example_id";
private String _rev = null;
private boolean isExample;
public ExampleDocument(boolean isExample) {
this.isExample = isExample;
}
public String toString() {
return "{ id: " + _id + ",\nrev: " + _rev + ",\nisExample: " + isExample + "\n}";
}
}
// Create an ExampleDocument and save it in the database
db.save(new ExampleDocument(true));
Although I haven't tried it, the Hashmap approach may work also, as discussed in this tutorial: https://www.ibm.com/blogs/bluemix/2014/07/cloudant_on_bluemix/.
// create a simple doc to place into your new database
Map<String, Object> doc = new HashMap<String, Object>();
doc.put("_id", UUID.randomUUID().toString());
doc.put("season", "summer");
doc.put("climate", "arid");
dbc.create(doc);

In question It seems org.json.JSONObject used And it is not compatible with cloudant client library. I tried with google object it is working good for me.
Issue got resolved by using google com.google.gson.JsonObject instead of org.json.JSONObject.
Correct Full code is given below,
Database db = client.database("dbTempName", true);
// Used google.gson.JsonObject instead of org.json.JSONObject.
com.google.gson.JsonParser parser = new com.google.gson.JsonParser();
com.google.gson.JsonObject jsonObject = parser.parse("{\"hello\": \"data\"}").getAsJsonObject();
db.save(jsonObject);

Bson Document to Json in Java

This is my code:
MongoDBSingleton dbSingleton = MongoDBSingleton.getInstance();
MongoDatabase db;
try {
db = dbSingleton.getTestdb();
MongoIterable<String> mg = db.listCollectionNames();
MongoCursor<String> iterator=mg.iterator();
while (iterator.hasNext()) {
MongoCollection<Document> table = db.getCollection(iterator.next());
for (Document doc: table.find()) {
System.out.println(doc.toJson());
}
}
}
This the output of toJson:
"modified" : { "$date" : 1475789185087}
This is my output of toString:
{"modified":"Fri Oct 07 02:56:25 IST 2016"}
I want String date format in Json, how to do it?

Sadly, IMO, MongoDB Java support is broken.
That said, there is a #deprecated class in the mongo-java-driver that you can use:
String json = com.mongodb.util.JSON.serialize(document);
System.out.println("JSON serialized Document: " + json);
I'm using this to produce fasterxml (jackson) compatible JSON from a Document object that I can deserialize via new ObjectMapper().readValue(json, MyObject.class).
However, I'm not sure what they expect you to use now that the JSON class is deprecated. But for the time being, it is still in the project (as of v3.4.2).
I'm importing the following in my pom:
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-driver-async</artifactId>
<version>3.4.2</version>
</dependency>
<!-- Sadly, we need the mongo-java-driver solely to serialize
Document objects in a sane manner -->
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</artifactId>
<version>3.4.2</version>
</dependency>
I'm using the async driver for actually fetching and pushing updates to mongo, and the non-async driver solely for the use of the JSON.serialize method.

No, it is not possible to produce the plain JSON. Please refer this link.
However, it can produce JSON in two modes.
1) Strict mode - Output that you have already got
2) Shell mode
Shell Mode:-
JsonWriterSettings writerSettings = new JsonWriterSettings(JsonMode.SHELL, true);
System.out.println(doc.toJson(writerSettings));
Output:-
"createdOn" : ISODate("2016-07-16T16:26:51.951Z")
MongoDB Extended JSON

In theory we are supposed to use toJSON() per...
https://jira.mongodb.org/browse/JAVA-1770
However, it seems that, at least up through 3.6, toJSON() isn't supported on various types the old JSON.serialize() method handled without issue, such as the AggregateIterable<Document> objects output by aggregate().

Here is a 2020 update to answer exactly your question, i.e. getting this exact format:
"modified":"2016-07-16T16:26:51.951Z"
You have to use writerSettings like notionquest suggested, but with a custom date converter and DateTimeFormatter.ISO_INSTANT:
public class JsonDateTimeConverter implements Converter<Long> {
private static final Logger LOGGER = LoggerFactory.getLogger(JsonDateTimeConverter.class);
static final DateTimeFormatter DATE_TIME_FORMATTER = DateTimeFormatter.ISO_INSTANT
.withZone(ZoneId.of("UTC"));
#Override
public void convert(Long value, StrictJsonWriter writer) {
try {
Instant instant = new Date(value).toInstant();
String s = DATE_TIME_FORMATTER.format(instant);
writer.writeString(s);
} catch (Exception e) {
LOGGER.error(String.format("Fail to convert offset %d to JSON date", value), e);
}
}
}
Use it like this:
doc.toJson(JsonWriterSettings
.builder()
.dateTimeConverter(new JsonDateTimeConverter())
.build())

if the bson.jar version is > 3.0.0 you may try document.toJson()

I used following
try {
MongoDatabase db = mongoClient.getDatabase("dbname");
MongoCollection<Document> collection = db.getCollection("nameofcollect");
Gson gson = new Gson();
ArrayList<JsonObject> array = new ArrayList<JsonObject>();
String jsonString = null;
/*WARNING Gson lib serialize string ->means add slash if you convert "json string" into "json string"*/
for (Document doc : collection.find()) {
jsonString = gson.toJson(doc);
array.add(new Gson().fromJson(jsonString, JsonObject.class));
}
//String finalarry = gson.toJson(array);
Map<Object, ArrayList<JsonObject>> seedMap = new HashMap<Object, ArrayList<JsonObject>>();
// String encode = coCoRy.encryptAndEncode(jsonString);
seedMap.put("seed", array);
String seedJsonString = gson.toJson(seedMap);
mongoClient.close();
return seedJsonString;
} catch (MongoException | ClassCastException e) {
e.printStackTrace();
return null;
}
Result will be like following
{
"seed": [
{
"_id": {
"timestamp": 1590914828,
"counter": 10457170,
"randomValue1": 5587428,
"randomValue2": -25784
},
"FIR_EVID_NUM": "3436345346",
"FIR_REG_NUM": "89678967",
"LOGIN_ID": "pc_admin",
"MEDIA_PATH": "C:\\Users\\ALPHAMALE\\Documents\\ShareX\\Screenshots\\2020-05\\1590211570.png"
},
{
"_id": {
"timestamp": 1590924463,
"counter": 7254997,
"randomValue1": 5012578,
"randomValue2": 24700
},
"FIR_EVID_NUM": "999999",
"FIR_REG_NUM": "888888",
"LOGIN_ID": "32323",
"MEDIA_PATH": "C:/uploads/c46847c7e2d130ffd746c789c0f0932e.png"
}
]
}

try this:
final JsonWriterSettings settings = JsonWriterSettings.builder( ).outputMode( JsonMode.SHELL ).build( );
System.out.println(doc.toJson(settings));
You can change the JsonMode is you wish

Spring/Java - Encode HTML Entities + XHR

When using regular JSP forms for printing to the client, configuring the web.xml properly works for me (http://stackoverflow.com/questions/2147958/how-do-i-prevent-people-from-doing-xss-in-java).
Is there any "best practice" on how to escape/entityze strings which will be send via JSON to a jQuery function, which then populates the DOM with these values? Any recommended libraries or Spring Web Framework build-ins?
jQuery $.ajax-call to Spring MVC
Spring MVC responds in JSON
(magic encoding happens, e.g. <a> becomes <a> ) <= this one
jQuery receives the JSON and populates the DOM XSS-safe
Thanks in advance!
edit: I am also sometimes sending HTML on purpose, so the solution would need to be able to only handle the user input. It will probably turn out that every user-poisoned string will have to be sanitized manually?

try this class which I wrote for my use .
it may be useful check wether any case is missing . . . as no detailed testing is done on this yet.
If any issue arise please let me know. . .
(add corresponding jar Apache commons and net.sf.json)
package myutil;
import java.util.Iterator;
import net.sf.json.JSONArray;
import net.sf.json.JSONObject;
import org.apache.commons.lang.StringEscapeUtils;
public class JSONCleaner {
public static void main(String[] args) {
// TODO Auto-generated method stub
JSONObject jsonchild2=new JSONObject();
jsonchild2.put("subchlidkey1", "subchildvalue1");
jsonchild2.put("subchlidkey2", "subchildvalue2");
jsonchild2.put("subchlidkey3", "subchildvalue3");
JSONObject jsonchild=new JSONObject();
jsonchild.put("chlidkey1", "childvalue1");
jsonchild.put("chlidkey2", "childvalue2");
jsonchild.put("chlidkey3", "childvalue3");
JSONArray jsonarray=new JSONArray();
jsonarray.add("option1");
jsonarray.add("<p>option2</p>");
jsonarray.add(jsonchild2);
JSONObject json=new JSONObject();
json.put("name", "<b>nirbhay</b>");
json.put("age", 23);
json.put("jsonChildObject", jsonchild);
json.put("weight", 65);
json.put("array", jsonarray);
System.out.println(cleanJSONObject(json));
//System.out.println(json.getString("name"));
}
public static JSONObject cleanJSONObject(JSONObject jsonObject)
{
JSONObject returnJson=new JSONObject();
Iterator<?> keys = jsonObject.keys();
while( keys.hasNext() ){
String key = (String)keys.next();
//System.out.println(jsonObject.get(key));
if(jsonObject.optJSONObject(key)==null)
{
if(jsonObject.optJSONArray(key)!=null)
{
returnJson.put(key, cleanJSONArray(jsonObject.getJSONArray(key)));
}
else
{
returnJson.put(key, StringEscapeUtils.escapeHtml(jsonObject.getString(key)));
}
}
else
{
returnJson.put(key,cleanJSONObject(jsonObject.optJSONObject(key)));
}
}
return returnJson;
}
private static JSONArray cleanJSONArray(JSONArray array)
{
JSONArray returnArray=new JSONArray();
for(int i=0,j=array.size();i<j;i++)
{
if(array.optJSONObject(i)==null)
{
if(array.optJSONArray(i) != null)
{
returnArray.add(cleanJSONArray((JSONArray) array.get(i)));
}
else
{
returnArray.add(StringEscapeUtils.escapeHtml(array.getString(i)));
}
}
else
{
returnArray.add(cleanJSONObject((JSONObject) array.get(i)));
}
}
return returnArray;
}
}