Develop Reference

How to remove extra Escape characters from a text column in spark dataframe

My Data in a json looks like -
{"text": "\"I have recently taken out a 12 month mobile phone contract with Virgin but despite two calls to customer help I still am getting a message on my phone indicating \\\"No Service\\\" although intermittently I do get connected.\"", "created_at": "\"2018-08-27 16:58:30\"", "service_id": "51870", "category_id": "249"}
I read this JSON Using -
val complaintsSourceRaw = spark.read.json("file:///complaints.jsonl")
When i read the data in dataframe, it looks like
|249 |"2018-08-27 16:58:30"|51870 |"I have recently taken out a 12 month mobile phone contract with Virgin but despite two calls to customer help I still am getting a message on my phone indicating **\"No Service\"** although intermittently I do get connected."
Issue is
**\"No Service\"** need to be like **"No Service"**
How i am trying -
complaintsSourceRaw.withColumn("text_cleaned", functions.regexp_replace(complaintsSourceRaw.col("text"), "\", ""));
However \ character excapes my " and code breaks. Any idea how to acieve this?

You need to escape the "\" character, so in your regexp_replace you should look for two backslash ("\\") characters, not one.

How do I access a list of maps nesting in ${body} of Apache Camel in Java

I am trying to access data in the incoming {body} of my incoming Json I have done the unmarshaling with Jackson and mapped it to a Java Map with
`.unmarshal().json(JsonLibrary.Jackson, java.util.Map.class)
`
My Json data is something like this after unmarshal step above
`{ "projectId" : 12345,
"title" : “12345 - Plant 1 Processing",
"partners": [{"partnerName": "partnerJV1", "partnerLocation": "JA"},
{"partnerName": "partnerJV2", "partnerLocation": "FL"},
{"partnerName": "partnerJV3", "partnerLocation": "OH"}
]`
The last part can have 0-N number of partnerName, partnerLocation maps in the partners List.
Now I am having to insert this data into a SQL table with
.to("sql:classpath:sql/sql_queries.sql")
My sql_queries.sql has the following query in it to insert data fields into the table:
`INSERT INTO MY_TABLE(PID, TITLE, PartnerName1, PartnerLocation1, PartnerName2, PartnerLocation2, PartnerName3, PartnerLocation3) VALUES(:#${body['projectId']}, :#${body['title']}, :#${body['partners[0]']['partnerName']}, :#${body['partners[0]']['partnerLocation']} )'
I cannot figure out the syntax for the last part which is a List of Maps. If it should be
`:#${body['partners[0]']['partnerName']}`
or something else in order for me to get that value.
Any hints would help, thank you!

What worked for me in the end was this:
:#${body['partners'][0]['partnerName']}
but I would love to find a way to iterate over the values like a list in Java if I don't know the size of it to begin with.

How to print a payload text from resultset in java

I have sql queried for the particular data using jdbc in java.
Now I have payload text in the result set. Anyone please help how to read the payload text from resultset in JAVA. Below is my code giving exception
System.out.println(set.getString(columnumber));
I have payload text like below,
<tag1>Data1</tag1>
<tag2>Data2</tag2>
<tag3>Data3</tag3>
My issue is that the payload have 'lines' as mentioned above that cannot print using,
System.out.println(set.getString(columnumber));
Please help me to print the payload text from resultset in JAVA.

I found the solution for this issue. If the database column is payload text, to get it as a string in the resultset, added the below function in the SQL query
columnname.getCLOBVal()
This returns the payload text as string and I can able to print it in Java by using
System.out.println(set.getString(columnumber));
Thanks everyone for your valuable time to suggest idea on this.

Java: JSONParseException

All I'm trying to do is to parse very simple json line, even its valid i dont know why its throwing an error
the line is
com.mongodb.util.JSONParseException:
{publish_status:'active',activation_date:{$lt:new Date()},expiration_date:{$gt:new Date()}}
^
what is wrong with the new Date() as a value?

That's not valid JSON at all. JSON syntax is defined on json.org, and it's always a string key with a value that's one of a string, number, boolean, null, array, or object. You're writing a Mongo query from Java. You should reformulate your question and retag appropriately.

I tried using the new date() in mongo DB 2.2.3 directly and it worked .. it created a value of ISODate.
You may try using this:
{publish_status:'active',activation_date:new Date(),expiration_date:new Date()}

Reading Json String using Gson results error "not a JSON Array"

In my project i have a complex json response. I want to read it by GSon.
JSON : {'FoodMenuRS':{'Results':[{'Items':{'Item':[{'#Id':'24'},{'#Id':'24'}]}}, {'Items':{'Item':{'#Id':'24'}}}]}}
It contains a JSONArray with first "Item" and JSONObject with second one. Hence its call results in error,
failed to deserialize json object {"#Id":"24"} given the type java.util.List<com.servlet.action.ItemInfo> and java.lang.IllegalStateException: This is not a JSON Array.
Please help how i should handle this scenario. Thanks.

The string you are showing is a JSONObject not a JSONArray. So, in this case you first of all have to get the JSONObject and perform further decoding on that JSONObject.
JSONObject - {}
JSONArray - []
And indeed JSONObject or JSONArray should be encoded using Double-quotes(")

Your JSON is valid, but not for the doble quotes (") because JSON supports simple quotes (') and no quotes in the key name. See http://sites.google.com/site/gson/gson-user-guide#TOC-Serializing-and-Deserializing-Colle
However this JSON have key names that begin with #. For JSON strings this character is valid at the beginning of the name (see right column http://www.json.org/) but for Java this names are illegal (see Naming section http://download.oracle.com/javase/tutorial/java/nutsandbolts/variables.html). Specifically, names started with # are annotations and you can't use annotations tags to declare variables, fields, methods, etc.

This is not a valid JSON object. Strings in JSON are always encapsulated in double quotes ("). Contact the producer of that JSON and tell him to use a correct encoder.