I have this HTTP response body to de-serialize:
String response = "result : {'url': u'https://somedomain/', 'fields': {'policy':
u'eyJjb25kaXRpb25zIjogW1siYfgfhudGVudC1sZjMyWiJ9', 'AWSAccessKeyId':
u'ASIccccccNA', 'x-amz-security-token': 'FQofgF', 'key': u'bbb.file',
'signature': u'rm9gdflkjfs='}}"
I am using the jackson.core (2.9.0) java package and lib (have also tried GSON) and get this error:
Exception in thread "main" com.fasterxml.jackson.core.JsonParseException:
Unrecognized token 'u': was expecting ('true', 'false' or 'null')
Deserialization code:
MyResponse deserializedResponse = new ObjectMapper()
.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true)
.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true)
.configure(JsonGenerator.Feature.ESCAPE_NON_ASCII, true)
.readValue(response, MyResponse.class);
I have considered something like this but it feels like there should be a better / safer way:
String sanitizedResponse = response.replaceAll("u'", "'");
--
Using Java 1.8.
Any help appreciated.
As python caused this problem I think the best solution is to let python fix it ;-). Fortunately with jython you can stick with a pure java implementation.
First you need to add the jython standalone dependency in your pom.xml:
<dependencies>
<dependency>
<groupId>org.python</groupId>
<artifactId>jython-standalone</artifactId>
<version>2.7.1</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.5</version>
</dependency>
</dependencies>
(As you can see I also used apache commons io for my example so I added it as well)
I put your (invalid) json string into the text file "c:/temp/json.txt" which has the following content:
{'url': u'https://somedomain/', 'fields': {'policy':
u'eyJjb25kaXRpb25zIjogW1siYfgfhudGVudC1sZjMyWiJ9', 'AWSAccessKeyId':
u'ASIccccccNA', 'x-amz-security-token': 'FQofgF', 'key': u'bbb.file',
'signature': u'rm9gdflkjfs='}}
Now here is the code to read the json file, set up the Python Interpreter and handover the json to clean it up:
String content = FileUtils.readFileToString(new File("c:/temp/json.txt"), "UTF-8");
PythonInterpreter pi = new PythonInterpreter();
pi.exec("import json");
pi.exec("jsondata = " + content);
pi.exec("jsonCleaned = json.dumps(jsondata)");
PyObject jsonCleaned = (PyObject) pi.get("jsonCleaned");
System.out.println(jsonCleaned.asString());
pi.close();
The output is:
{"url": "https://somedomain/", "fields": {"signature": "rm9gdflkjfs=", "AWSAccessKeyId": "ASIccccccNA", "x-amz-security-token": "FQofgF", "key": "bbb.file", "policy": "eyJjb25kaXRpb25zIjogW1siYfgfhudGVudC1sZjMyWiJ9"}}
When you put that in a json validator (https://jsonlint.com/) you can see that it is a valid json now.
I can't tell if the performance is good enough for your use case so you have to test that out.
Remark:
In Eclipse there seems to be a bug with that jython version. It shows the following error:
console: Failed to install '': java.nio.charset.UnsupportedCharsetException: cp0.
Although it works nevertheless you can get rid of it by adding the following VM-Argument to your Run-Configuration:
-Dpython.console.encoding=UTF-8
Remark2: For the sake of completeness and to fully answer that question - here is how you can deserialize the cleaned JSON:
Add GSON Dependency to your pom.xml:
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.2</version>
</dependency>
Create representing classes:
Info class
public class Info {
private String url;
private Fields fields;
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url;
}
public Fields getFields() {
return fields;
}
public void setFields(Fields fields) {
this.fields = fields;
}
}
Fields class
import com.google.gson.annotations.SerializedName;
public class Fields {
private String signature;
private String AWSAccessKeyId;
#SerializedName("x-amz-security-token")
private String x_amz_security_token;
private String key;
private String policy;
public String getSignature() {
return signature;
}
public void setSignature(String signature) {
this.signature = signature;
}
public String getAWSAccessKeyId() {
return AWSAccessKeyId;
}
public void setAWSAccessKeyId(String aWSAccessKeyId) {
AWSAccessKeyId = aWSAccessKeyId;
}
public String getX_amz_security_token() {
return x_amz_security_token;
}
public void setX_amz_security_token(String x_amz_security_token) {
this.x_amz_security_token = x_amz_security_token;
}
public String getKey() {
return key;
}
public void setKey(String key) {
this.key = key;
}
public String getPolicy() {
return policy;
}
public void setPolicy(String policy) {
this.policy = policy;
}
}
Finally add the following code after you get your cleaned JSON:
Gson gson = new Gson();
Info info = gson.fromJson(jsonCleaned.asString(), Info.class);
You need to use the following regex that replaces u' from beginning of word boundaries to replace with '
String regexPattern = "(\\bu')";
Related
I am using spring.cloud to connect to Azure Service Bus in Java. Here is maven dependency I am using:
<dependency>
<groupId>com.azure.spring</groupId>
<artifactId>spring-cloud-azure-starter-integration-servicebus</artifactId>
<version>4.5.0</version>
</dependency>
I am able to consume the message from the queue as byte array and it converts the message to string. Here is my main code after receiving a message from queue:
#ServiceActivator(inputChannel = INPUT_CHANNEL)
public void messageReceiver(byte[] payload, #Header(AzureHeaders.CHECKPOINTER) Checkpointer checkpointer) {
String message = new String(payload);
LOGGER.info("New message received: '{}'", message);
checkpointer.success()
.doOnSuccess(s -> LOGGER.info("Message '{}' successfully checkpointed", message))
.doOnError(e -> LOGGER.error("Error found", e))
.block();
}
And here is my example data in JSON as short version:
{
"serverId": 123,
"message": "{some message}"
}
What I would like to do is to create a Java object like this:
public class ExampleMessage {
private final Integer serverId;
private final String message;
and when a message from queue is consumed, it will convert the message to my Java object. I am used to using DataTypeProvider in order to use custom Java object for AMQP message consumption which will convert and validate the conversion behind the scene. Does spring.cloud.azure has built-in method/functionality for deserialization? or Do I manually deserialize and do validation for a consumed message?
Here I was able to convert the Json object to java object using Gson class.
I am just reading a message from the service bus and converting it to java object.
my pom.xml (dependencies)
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-servicebus</artifactId>
<version>7.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.projectreactor/reactor-core -->
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-core</artifactId>
<version>3.3.11.RELEASE</version>
</dependency>
My test class :
public class TestClass {
public String name ;
public int version;
TestClass(String n , int v)
{
this.name = n ;
this.version = v ;
}
}
The main class :
#SpringBootApplication
public class ServicebustestApplication {
public static void main(String[] args) throws InterruptedException {
SpringApplication.run(ServicebustestApplication.class, args);
String conn = " Endpoint=sb://v-mganorkarjsonobject.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=HglLVGlgMsYZGQMOtUfp4g2oka1CpCbVR0YEHgly7jA= ";
CountDownLatch countdownLatch1 = new CountDownLatch(1);
ServiceBusProcessorClient processorClient = new ServiceBusClientBuilder()
.connectionString("<Your COnnection String >")
.processor()
.queueName("test")
.processMessage(ServicebustestApplication::processMessage)
.processError(context -> processError(context,countdownLatch1))
.buildProcessorClient();
processorClient.start();
TimeUnit.SECONDS.sleep(10);
processorClient.close();
}
private static void processMessage(ServiceBusReceivedMessageContext context) {
ServiceBusReceivedMessage message = context.getMessage();
System.out.printf("Processing message. Session: %s, Sequence #: %s. Contents: %s%n", message.getMessageId(),
message.getSequenceNumber(), message.getBody());
Gson gson = new Gson();
TestClass testobject = gson.fromJson(String.valueOf(message.getBody()),TestClass.class);
System.out.println("name: "+testobject.name +" version: "+ testobject.version+"");
}
private static void processError(ServiceBusErrorContext context, CountDownLatch countdownLatch) {
}
}
Here the callback to process message will process the message and then we can use the GSON to convert the json string to java object.
Gson gson = new Gson();
TestClass testobject = gson.fromJson(String.valueOf(message.getBody()),TestClass.class);
output of the code :
I want to make a POST request with URL Query Params set to the values of an object.
For example
http://test/data?a=1&b=2&c=3
I want to make a post request to this URL with a class like this:
public class Data {
private Integer a;
private Integer b;
private Integer c;
}
I do NOT want to do each field manually, like this:
public void sendRequest(Data data) {
String url = UriComponentsBuilder.fromHttpUrl("http://test/")
.queryParam("a", data.getA())
.queryParam("b", data.getB())
.queryParam("c", data.getC())
.toUriString();
restTemplate.postForObject(url, body, Void.class);
}
Instead, I want to use the entire object:
public void sendRequest(Data data) {
String url = UriComponentsBuilder.fromHttpUrl("http://test/")
.queryParamsAll(data) //pseudo
.toUriString();
restTemplate.postForObject(url, body, Void.class);
}
Your requirement is like QS in js. Thx qianshui423/qs . It is implementation QS in java. It is coded by a Chinese guy. At first git clone it and use below cmd to build. You will get a jar called "qs-1.0.0.jar" in build/libs (JDK required version 8)
# cd qs directory
./gradlew build -x test
Import it, I do a simple demo as below. For your requirement, you can build class to transfer your Obj into QSObject. Besides toQString, QS can parse string to QSObject. I think it powerful.
import com.qs.core.QS;
import com.qs.core.model.QSObject;
public class Demo {
public static void main(String[] args) throws Exception{
QSObject qsobj = new QSObject();
qsobj.put("a",1);
qsobj.put("b",2);
qsobj.put("c",3);
String str = QS.toQString(qsobj);
System.out.println(str); // output is a=1&b=2&c=3
}
}
Most probably this issue is because of JSONObject(org.json.JSONObject) is incompatible with cloudant library.
Is any alternative way to use any other Object?
I am using below cloudant libraries,
<dependency>
<groupId>com.cloudant</groupId>
<artifactId>cloudant-client</artifactId>
<version>2.6.2</version>
</dependency>
Here is my code
package data.repositories;
import org.json.JSONObject;
import com.cloudant.client.api.*;
import com.cloudant.client.api.CloudantClient;
import com.cloudant.client.api.Database;
import com.cloudant.client.api.model.Response;
import util.Config;
public class DatabaseRepository {
CloudantClient client = ClientBuilder.account(Config.CLOUDANT_ACCOUNT_NAME)
.username(Config.CLOUDANT_USER_NAME)
.password(Config.CLOUDANT_PASSWORD).build();
public DatabaseRepository() {
JSONObject
}
public void Save(String dbName) {
Database db = client.database("dbTempName", true);
JSONObject jsonObject = new JSONObject("{hello: data}");
db.save(jsonObject);
}
}
Document saved in cloudant database is,
{
"_id": "1c7f223f74a54e7c9f4c8a713feaa537",
"_rev": "1-a3cd12379eec936b61f899c8278c9d62",
"map": {
"hello": "data"
}
}
I'm not familiar with cloudant but my guess is JsonObject has a property called "map" that holds your json string data (probably there's a myArray property too), and cloudant serializes it into json, thus adding those unnecessary values.
my suggestions:
1) try to save your json string directly like db.save("{hello: data}") to avoid serialization
2) if you really need to create a JsonObject try to customize cloudant's serialization process to avoid that extra fields.
in response to comment:
from what I read here, then I think you need a pojo, which when serialized into json would look like:
{ 'hello' : 'data' }
which is something like:
public class MyClass implements Serializable {
String hello;
public MyClass(String hello) {
this.hello = hello;
}
public String getHello() {
return hello;
}
}
then save it like:
db.save(new MyClass("data"));
or you can use a hashmap instead of a pojo:
Map<String, Object> map = new Hashmap ...
map.put("hello", "data");
db.save(map);
Look at the example in the README for the repo. It shows that you want a POJO, but you don't have to implement Serializable. Just create a class that has _id and _rev properties that are Strings. Then add Javascript object compatible properties as desired.
// A Java type that can be serialized to JSON
public class ExampleDocument {
private String _id = "example_id";
private String _rev = null;
private boolean isExample;
public ExampleDocument(boolean isExample) {
this.isExample = isExample;
}
public String toString() {
return "{ id: " + _id + ",\nrev: " + _rev + ",\nisExample: " + isExample + "\n}";
}
}
// Create an ExampleDocument and save it in the database
db.save(new ExampleDocument(true));
Although I haven't tried it, the Hashmap approach may work also, as discussed in this tutorial: https://www.ibm.com/blogs/bluemix/2014/07/cloudant_on_bluemix/.
// create a simple doc to place into your new database
Map<String, Object> doc = new HashMap<String, Object>();
doc.put("_id", UUID.randomUUID().toString());
doc.put("season", "summer");
doc.put("climate", "arid");
dbc.create(doc);
In question It seems org.json.JSONObject used And it is not compatible with cloudant client library. I tried with google object it is working good for me.
Issue got resolved by using google com.google.gson.JsonObject instead of org.json.JSONObject.
Correct Full code is given below,
Database db = client.database("dbTempName", true);
// Used google.gson.JsonObject instead of org.json.JSONObject.
com.google.gson.JsonParser parser = new com.google.gson.JsonParser();
com.google.gson.JsonObject jsonObject = parser.parse("{\"hello\": \"data\"}").getAsJsonObject();
db.save(jsonObject);
The program I have is checking for most of the file types, how ever I have a .properties file when run with program it says text/plain rather than properties file. How do I customize my program to figure of mime type(based on extension and content) of properties file:
Code below:
public class TikaFileTypeDetector {
private final Tika tika = new Tika();
public TikaFileTypeDetector() {
super();
}
public String probeContentType(Path path) throws IOException {
// Check contents first
String fileContentDetect = tika.detect(path.toFile());
if (!fileContentDetect.equals(MimeTypes.OCTET_STREAM)) {
return fileContentDetect;
}
// Try file name only if content search was not successful
String fileNameDetect = tika.detect(path.toString());
if (!fileNameDetect.equals(MimeTypes.OCTET_STREAM)) {
return fileNameDetect;
}
return null;
}
public static void main(String[] args) throws IOException {
Tika tika = new Tika();
if (args.length != 1) {
printUsage();
return;
}
Path path = Paths.get(args[0]);
TikaFileTypeDetector detector = new TikaFileTypeDetector();
String contentType = detector.probeContentType(path);
System.out.println("File is of type - " + contentType);
}
public static void printUsage() {
System.out.print("Usage: java -classpath ... "
+ TikaFileTypeDetector.class.getName()
+ " ");
}
}
Following is custom xml for mime-type:
<?xml version="1.0" encoding="UTF-8"?>
<mime-info>
<mime-type type="application/hello">
<glob pattern="*.properties"/>
</mime-type>
</mime-info>
In the apache docs, it says to add a new mime type in xml as above and to add it to the code base. How do I add this to my code now? I couldn't find much info on this
Note: I am using apache tika
Tika 1.10 or later versions addresses this issue by adding a new mime-type named text/x-java-properties to its tika-mimetypes.xml
Use the following dependency so that you don't have to get your hands dirty by adding a custom mimetype.xml
<dependency>
<groupId>org.apache.tika</groupId>
<artifactId>tika-core</artifactId>
<version>1.10</version>
</dependency>
And I would also like to add that, As per the documentation, the project team welcomes patches to add any standard mime type If it is missing from org/apache/tika/mime/tika-mimetypes.xml.
I was asked to include math expression inside a string
say: "price: ${price}, tax: ${price}*${tax)"
the string is given at run-time and a Map values too
I used Velocity for this:
maven:
<properties>
<velocity.version>1.6.2</velocity.version>
<velocity.tools.version>2.0</velocity.tools.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.velocity</groupId>
<artifactId>velocity</artifactId>
<version>${velocity.version}</version>
</dependency>
<dependency>
<groupId>org.apache.velocity</groupId>
<artifactId>velocity-tools</artifactId>
<version>${velocity.tools.version}</version>
</dependency>
</dependencies>
java:
public class VelocityUtils {
public static String mergeTemplateIntoString(String template, Map<String,String> model)
{
try
{
final VelocityEngine ve = new VelocityEngine();
ve.init();
final VelocityContext context = new VelocityContext();
context.put("math", new MathTool());
context.put("number", new NumberTool());
for (final Map.Entry<String, String> entry : model.entrySet())
{
final String macroName = entry.getKey();
context.put(macroName, entry.getValue());
}
final StringWriter wr = new StringWriter();
final String logStr = "";
ve.evaluate(context, wr, logStr,template);
return wr.toString();
} catch(Exception e)
{
return "";
}
}
}
test class:
public class VelocityUtilsTest
{
#Test
public void testMergeTemplateIntoString() throws Exception
{
Map<String,String> model = new HashMap<>();
model.put("price","100");
model.put("tax","22");
String parsedString = VelocityUtils.mergeTemplateIntoString("price: ${price} tax: ${tax}",model);
assertEquals("price: 100 tax: 22",parsedString);
String parsedStringWithMath = VelocityUtils.mergeTemplateIntoString("price: $number.integer($math.div($price,2))",model);
assertEquals("price: 50",parsedStringWithMath);
}
}
would it be better to use SPel instead?
I agree that this is kind of off topic, but I think it merits an answer nonetheless.
The whole idea of using a templating engine is that you need to have access to the templates at runtime. If that is the case, then sure, Velocity is a good choice. Then you could provide new versions of the HTML, and assuming you didn't change the variables that were used, you would not have to provide a new version of the application itself (recompiled).
However, if you are just using Velocity to save yourself time, it's not saving much here: you could do this with the StringTokenizer in only a few more lines of code.