I am having a json which is somethink like {"Header" : {"name" : "TestData", "contactNumber" : 8019071740}}
If i insert this to mongoDB it will be something like
{"_id" : ObjectId("58b7e55097989619e4ddb0bb"),"Header" : {"name" : "TestData","contactNumber" : NumberLong(8019071743)}
When i read this data back and try to convert to java object using Gson it throws exception com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a long but was BEGIN_OBJECT at line 1 column 109 path $.Header.contactNumber
I have found this, But i was wondering if i have very complex json structure then i might need to manipulate many json nodes in this approach.
Do anyone has any better alternatives on this.
Edit:1
I am reading querying and converting json as below
Document MongoDocument = mycollection.find(searchCondition);
String resultJson = MongoDocument.toJson();
Gson gson = new Gson();
Model model= gson.fromJson(resultJson, ItemList.class);
We can use below code:
Document doc = documentCursor.next();
JsonWriterSettings relaxed = JsonWriterSettings.builder().outputMode(JsonMode.RELAXED).build();
CustomeObject obj = gson.fromJson(doc.toJson(relaxed), CustomeObject.class);
Take a look at: converting Document objects in MongoDB 3 to POJOS
I had the same problem. The workaround with com.mongodb.util.JSON.serialize(document) does the trick.
Mongo db uses Bson format with its own types which follows json standards but it can't be parsed by json library without writing the custom wrapper/codec.
You can use third party framework/plugins to make the library take care of converting between document and pojo.
If that is not an option for you, you will have to do mapping yourself.
Document mongoDocument = mycollection.find(searchCondition);
Model model= new Model();
model.setProperty(mongoDocument.get("property");
Document.toJson() gives bson but not json.
I.e. for Long field myLong equals to xxx of Document it produces json like:
"myLong" : { "$numberLong" : "xxx"}
Parsing of such with Gson will not give myLong=xxx evidently.
To convert Document to Pojo using Gson you may do next:
Gson gson = new Gson();
MyPojo pojo = gson.fromJson(gson.toJson(document), MyPojo.class);
val mongoJsonWriterSettings: JsonWriterSettings = JsonWriterSettings.builder.int64Converter((value, writer) => writer.writeNumber(value.toString)).build
def bsonToJson(document: Document): String = document toJson mongoJsonWriterSettings
Related
SearchRequest searchRequest = Requests.searchRequest(indexName);
SearchSourceBuilder builder = new SearchSourceBuilder();
Gson gson = new Gson();
QueryBuilder querybuilder = QueryBuilders.wrapperQuery(query);
query : {
"range": {
"timecolumn": {
"gte":"2022-10-07T09:45:13Z",
"lte":"2022-10-07T09:50:50Z"
}
}
}
While passing the above Query I am getting Parser Exception , I cannot change the date format as data in DB is getting inserted in same format .
Need Advice on :
How can we parse this kind of timestamp in ElasticSearch Java , if not
How can we control pattern updation during data insertion like my column in defined as text which takes date format "2022-10-07T09:45:13Z" as text .
either I have to pass this format in ES Parser or I have to change format to 2022-10-07 09:45:13 during insertion itself .
I cannot convert for each row after inserting because we have lakhs of data
As you are mentioning, Elasticsearch storing timecolumn as text type hence i will suggest to change mapping of timecolumn to date type and you will be able to use range query with date. Because if you store date as text and applied range then it will not return a expected result.
{
"mappings": {
"properties": {
"timecolumn": {
"type": "date"
}
}
}
}
Now coming to your Java code issue, You can use below code for creating range query in Java as you are using Java client.
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
QueryBuilder query = QueryBuilders.rangeQuery("timecolumn").gte("2022-10-07T09:45:13Z").lte("2022-10-07T09:50:50Z");
searchSourceBuilder.query(query);
searchRequest.source(searchSourceBuilder);
Regarding your concern about reindexing data:
I cannot convert for each row after inserting because we have lakhs of
data
You can use Reindex API to move data from original index to temp index and delete your original index. Then, defined the index with proper mapping and again use same Reindex API to copy data from temp index to original index with new mapping.
Is it possible in kryo to serialize an object along with the data schema, or to get the schema from the data serialized in the standard way ? I Need to make sure that the client side doesn't need a class in classpath. to load it from serialized data and then use reflection to subtract its fields, or deserialize all data in Maps, Lists, primitive types etc same as JSON or XML
Saves SampleBean as JSON string
val conf = new SparkConf()
.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
.set("spark.kryo.registrationRequired", "true")
.registerKryoClasses(Array(classOf[SampleBean], classOf[InternalRow]
, classOf[Array[InternalRow]]
, classOf[WriteTaskResult]
, classOf[FileCommitProtocol.TaskCommitMessage]
, classOf[ExecutedWriteSummary],
classOf[BasicWriteTaskStats]))
val spark = SparkSession.builder.master("local[*]")
.config(conf)
.getOrCreate
import spark.implicits._
val df = List(SampleBean("A", "B")).toDF()
df.write.mode(SaveMode.Overwrite).json("src/main/resources/kryoTest")
df.printSchema()
reads the data simple JSON
val sparkNew = Constant.getSparkSess
val dfNew = sparkNew.read.json("src/main/resources/serialisedJavaObj.json").toDF()
dfNew.printSchema()
I am converting JSON Values to XML. Instead of getting JSON properties as elements of XML I am getting "title":"source". The output I wanted is <title>source</title>. What is the mistake I am doing? I am writing this code in JavaScript function.
I am using x2js plugin for conversion and I have included it using script tag.
My code to convert dynatree to JSON and JSON to XML is:
var x2js = new X2JS();
var tree = $("#source").dynatree("getTree").toDict();
alert(" tree:"+tree);
var jsonObject = JSON.stringify(tree);//dynatree to JSON
alert(" jsonObject :"+jsonObject);
var xmlAsStr = x2js.json2xml_str( jsonObject );//JSON to XML
alert("xml "+xmlAsStr);
Try to not use JSON.stringify(tree); this escapes the string.
Set var xmlAsStr = x2js.json2xml_str(tree);
I have a C# web service which returns soap with a JSON string which looks like
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<getAndroidVersionResponse xmlns="http://development.com/">
<getAndroidVersionResult>{"Valid":false,"Url":"http://localhost/test","Av":[{"Valid":true,"Url":"Som!##$%^&*()_+{}:<>?\"''\\\"\\\"eURL","Av":null},{"Valid":false,"Url":"SomeURLSeond","Av":null}]}</getAndroidVersionResult>
</getAndroidVersionResponse>
</s:Body>
</s:Envelope>
The object is Serialized using Newtonsoft.Json.JsonConvert.SerializeObject(obj);
For normal non-weird strings Gson and Jackson work just fine, but when I do a combination of weird characters then both of them cannot deserialize it.
When I validate the string online with Json Parser Online and JSONLint, there is no problem with it.
{"Valid":false,"Url":"http://localhost/test","Av":[{"Valid":true,"Url":"Som!##$%^&*()_+{}:<>?\"''\\\"\\\"eURL","Av":null},{"Valid":false,"Url":"SomeURLSeond","Av":null}]}
Json Parser Online
JSONLint
The code I am using for them:
/*GSON*/
com.google.gson.Gson gson = new com.google.gson.Gson();
AndroidVersion v = gson.fromJson(subList.item(0).getNodeValue(), AndroidVersion.class);
/*Jackson*/
com.fasterxml.jackson.databind.ObjectMapper mapper = new com.fasterxml.jackson.databind.ObjectMapper();
mapper.setPropertyNamingStrategy(PropertyNamingStrategy.PASCAL_CASE_TO_CAMEL_CASE);
mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
AndroidVersion v = mapper.readValue(subList.item(0).getNodeValue(), AndroidVersion.class);
Any other approach I can take with this? I have come across Boon-JSON but it does not look promising.
I thought by now Newtonsoft would have Redone Newtonsoft.JSON for Java because that library is just excellent!
Write your models like for example:
public class AndroidVersion {
public boolean Valid;
public String Url;
public List <AndroidVersion> Av;
}
And then use Gson like below:
String jsonString = "Your json string here";
AndroidVersion v = gson.fromJson(jsonString, AndroidVersion.class);
I am trying to deserialize the following incoming JSON data:
{"TimeTable":[{"PersonID":"100649771",
..,..,..,"xx":null},
{"PersonID":"100631701",
..,..,..,"xx":{"abc":1234,"xyz":5678}}],
"xxx":"","xxxx":0,"xxxxx":false}
But I am facing a problem while parsing using a custom deserialization block made up of:
jParser.nextToken();
while ((jParser.nextToken() != JsonToken.END_ARRAY)) {
String innerField = jParser.getCurrentName();
jParser.nextToken();
But in this way I am skipping the array contents While parsing for the second row in the array (as illustrated in the JSON sample above^).
UPDATE: Here is the method(PasteBin Link) which is trying to parse the JSON data coming in the described format. Is there a way I can bind the JSON data directly to my bean? (IMO it appeared way more complex to me as because of the JSON structure; Moreover I can't change the JSON structure nor the bean structure. So, I just dropped the idea of binding directly :| ) Anyways here(PasteBin Link) is the bean as well.
Following is the sample of the incoming JSON Data:
{"Schedules":[{"PersonID":"100649771",
"HasSchedule":false,
"TripType":null,
"StickerNumber":null,
"VehicleRegNo":null,
"ExpectedStartDate":null,
"ActualStartDate":null,
"ActualEndDate":null,
"PersonScheduledDate":null,
"Shift":null,
"ColdCall":null,
"PickupLocationCoord":null},
{"PersonID":"100631701",
"HasSchedule":true,
"TripType":"P",
"StickerNumber":"PC0409",
"VehicleRegNo":"ASJHAHSP1758",
"ExpectedStartDate":"16 Aug 2013, 10:00:00",
"ActualStartDate":"16 Aug 2013, 10:02:52",
"ActualEndDate":"16 Aug 2013, 14:14:12",
"PersonScheduledDate":null,
"Shift":"02:30 PM",
"ColdCall":"N",
"PickupLocationCoord":{"Latitude":92.01011101,"Longitude":48.01011101}}],
"ErrorMessage":"","ErrorCode":0,"HasError":false}
Please can anyone fireup some pointers for me here in order- to deserialize 'em correctly?
Thanks
UPDATED
Among other things, you are mixing up the two aproaches to read a JSON stream: using readTree() to get all your JSON data in a memory tree (like XML's DOM) but also using a JsonParser to read a JSON stream token by token (like XML's JAX). The following is a method that does almost the same using readTree(), which seems to be more appropriate to you as you are reading JSON already loaded in a String:
public List<VehicleInformationBean> getAllVehiclesInTree(String response) {
List<VehicleInformationBean> vehicleList = new ArrayList<VehicleInformationBean>();
try {
PersonInformationBean mPersonInformationBean;
DatabaseHelper mDatabaseHelper = DatabaseHelper.getInstance(sContext);
ObjectMapper mapper = new ObjectMapper();
ObjectNode root = (ObjectNode) mapper.readTree(response);
if ((root.get(ServiceConstant.ErrorCode).asInt()) != 0 || !root.has(ServiceConstant.Schedules)) {
return vehicleList;
}
for(JsonNode element: root.get(ServiceConstant.Schedules)) {
VehicleInformationBean lstVehicleInformation = new VehicleInformationBean();
if (element.has(ServiceConstant.PersonID)) {
String personId = element.get(ServiceConstant.PersonID).asText();
mPersonInformationBean = mDatabaseHelper.getPersonDetailById(personId);
lstVehicleInformation.setPersonID(personId);
lstVehicleInformation.setName(mPersonInformationBean.getName());
lstVehicleInformation.setPickupLocation(mPersonInformationBean.getPickupLocation());
lstVehicleInformation.setDropLocation(mPersonInformationBean.getDropLocation());
}
lstVehicleInformation.setTripType(element.get(ServiceConstant.TripType).textValue());
lstVehicleInformation.setStickerNumber(element.get(ServiceConstant.StickerNumber).textValue());
lstVehicleInformation.setVehicleRegNo(element.get(ServiceConstant.VehicleRegNo).textValue());
lstVehicleInformation.setExpectedStartDate(element.get(ServiceConstant.ExpectedStartDate).textValue());
lstVehicleInformation.setActualStartDate(element.get(ServiceConstant.ActualStartDate).textValue());
lstVehicleInformation.setActualEndDate(element.get(ServiceConstant.ActualEndDate).textValue());
lstVehicleInformation.setPersonScheduledDate(element.get(ServiceConstant.PersonScheduledDate).textValue());
lstVehicleInformation.setShift(element.get(ServiceConstant.Shift).textValue());
if (element.has("PickupLocationCoord")) {
JsonNode coords = element.get("PickupLocationCoord");
if(coords.has(ServiceConstant.Latitude)) {
lstVehicleInformation.setLatitude(coords.get(ServiceConstant.Latitude).asDouble());
}
if(coords.has(ServiceConstant.Longitude)) {
lstVehicleInformation.setLongitude(coords.get(ServiceConstant.Longitude).asDouble());
}
} else if (element.has(ServiceConstant.ColdCall)) {
lstVehicleInformation.setColdCall(element.get(ServiceConstant.ColdCall).textValue());
}
vehicleList.add(lstVehicleInformation);
}
} catch (Exception e) {
// TODO doing something with exception or throw it if it can't be handled here
e.printStackTrace();
}
return vehicleList;
}
There are some validations and extra code you need to add to this method for it to do exactly what your original method does. This method only shows you the main idea of how to do it.