I have defined an example JSON below-
{ "common" : {
"data" : [ {
"ref" : "data1"
"x" : "${variable1}",
}, {
"ref" : "data2"
"y" : "${variable2}"
}
]
},
"section1" : {
"variables" : {
"${variable1}" : "var1_section1",
"${variable2}" : "var2_section1",
},
"data" : ["data1", "data2"]
},
"section2" : {
"variables" : {
"${variable1}" : "var1_section2"
"${variable2}" : "var2_section2",
},
"data" : ["data1", "data2"]
}
}
I have a common section and two more sections (section1 and section2). Data which is common for both sections is defined in the common section.
I have to deserialize this JSON into a POJO. Based on the section which will be passed as a parameter, the value of the variables should be deserialized for those sections. If I pass section1 as the parameter, the data should look like-
data : [ "x" : "var1_section1", "y" :"var2_section1" ]
And if I pass section2 as the parameter, the data should look like-
data : [ "x" : "var1_section2", "y" :"var2_section2" ]
I want to know if this is achievable using any JSON annotations in Java?
Related
I have one Json array which is consist 2 or 3 json objects, now I need to combine both the json objects into single json object, 1st json object is coming from one method 2nd is coming from another method let me explain with example
response[
{
"id" : 1,
"name" : "Hi",
"no.of slots" [
{
"Mrng" : 10:30,
"Evening" : 11:20
},
{
"Mrng" : 12:00,
"Evening" : 4:00
}
]
},
{
"email" : "abc#gmail.com",
"address" : "abc district"
"no.of slots" [
{
"Mrng" : 10:30,
"Evening" : 11:20
}
]
}
]
then I need output like
response: {
"id" : 1,
"name" : "Hi",
"no.of slots" [
{
"Mrng" : 10:30,
"Evening" : 11:20
},
{
"Mrng" : 12:00,
"Evening" : 4:00
}
],
"email" : "abc#gmail.com",
"address" : "abc district"
}
In no of slots If I have duplicates need to remove or if unique need to combine, any help will be appreciate.
Thanks in advance
You can have a class representing Mrng and Evening properties and override that class's equals and hash methods for comparing for these attributes. Then have a Set and generate instances and put elements to this.
Finally you will get a unique list and have a json string with some little research. This should be done to no.of.slots only
I have the following document:
{
"_index" : "testdb",
"_type" : "artWork",
"_id" : "0",
"_version" : 4,
"found" : true,
"_source":{"uuid":0,
"StatusHistoryList":[
{
"ArtWorkDate":"2015-08-28T15:52:03.030+05:00",
"ArtworkStatus":"ACTIVE"
},
{
"ArtWorkDate":"2015-08-28T15:52:03.030+05:00",
"ArtworkStatus":"INACTIVE"
}
]
}
and here is the mapping of the document:
{
"testdb" : {
"mappings" : {
"artWork" : {
"properties" : {
"StatusHistoryList" : {
"type" : "nested",
"properties" : {
"ArtWorkDate" : {
"type" : "string",
"store" : true
},
"ArtworkStatus" : {
"type" : "string",
"store" : true
}
}
},
"uuid" : {
"type" : "integer",
"store" : true
}
}
}
}
}
}
Now I want to access the values of StatusHistoryList. I got null values if I do it like this:
val get = client.prepareGet("testdb", "artWork", Id.toString()).setOperationThreaded(false)
.setFields("uuid",,"StatusHistoryList.ArtworkStatus","StatusHistoryList.ArtWorkDate","_source")
.execute()
.actionGet()
var artworkStatusList= get.getField("StatusHistoryList.ArtworkStatus").getValues.toArray()
var artWorkDateList= get.getField("StatusHistoryList.ArtWorkDate").getValues.toArray()
then I got null values from the code but my document contains the values then I found this question
so after that i tried to do it like this
var smap = get.getSource.get("StatusHistoryList").asInstanceOf[Map[String,Object]]
but then a ClassCastException is thrown
java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.util.Map
Please help me how can I get the values of StatusHistoryList 's ArtworkStatus and ArtWorkDate values please guide me I will be very thankfull to you.
You have almost derived the solution. The GET request has retrieved the response but the problem is in parsing the response.
Let's see the problem. Below is the document that the elastic search returns as source
{
"uuid":0,
"StatusHistoryList":[
{
"ArtWorkDate":"2015-08-28T15:52:03.030+05:00",
"ArtworkStatus":"ACTIVE"
},
{
"ArtWorkDate":"2015-08-28T15:52:03.030+05:00",
"ArtworkStatus":"INACTIVE"
}
]
}
When we do get.getSource.get("StatusHistoryList") it returns List ArtWork objects and not a Map. That is the reason for the classCastException exception.
So if you cast the response to list of objects your problem will be solved.
But this would not be an ideal solution. Some of the libraries like Jackson-Faterxml does the job for you. Using the fasterxml library you can bind the json to equivalent POJO Object.
I have been reading a lot about Apache Avro these days and I am more inclined towards using it instead of using JSON. Currently, what we are doing is, we are serializing the JSON document using Jackson and then writing that serialize JSON document into Cassandra for each row key/user id.
Then we have a REST service that reads the whole JSON document using the row key and then deserialize it and use it further.
Now while reading on the web it looks like, Avro requires a schema beforehand... I am not sure how to come up with a schema in Apache Avro for my JSON document.
Below is my JSON document that I am writing into Cassandra after serializing it using Jackson. Now how to come up with an Avro schema for the below JSON?
{
"lv" : [ {
"v" : {
"site-id" : 0,
"categories" : {
"321" : {
"price_score" : "0.2",
"confidence_score" : "0.5"
},
"123" : {
"price_score" : "0.4",
"confidence_score" : "0.2"
}
},
"price-score" : 0.5,
"confidence-score" : 0.2
}
} ],
"lmd" : 1379231624261
}
Can anyone provide a simple example on this, how to come up with a schema in Avro basis on my above JSON document? Thanks for the help.
The simplest way to define an avro schema as you have outlined above would be to start from what they call IDL. IDL is a high-level language than the Avro schema (json) and makes writing avro schema much more straight-forward..
See avro IDL here: http://avro.apache.org/docs/current/idl.html
To define what you've got above in JSON, you're going to define a set of records in IDL that look like this:
#namespace("com.sample")
protocol sample {
record Category {
union {null, string} price_score = null;
union {null, string} confidence_score = null;
}
record vObject {
int site_id = 0;
union {null, map<Category>} categories = null;
union {null, float} price_score = null;
union {null, float} confidence_score = null;
}
record SampleObject {
union {null, array<vObject>} lv = null;
long lmd = -1;
}
}
When you run the compiler tool (as listed on that website above), you will get an avro schema generated like so:
{
"protocol" : "sample",
"namespace" : "com.sample",
"types" : [ {
"type" : "record",
"name" : "Category",
"fields" : [ {
"name" : "price_score",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "confidence_score",
"type" : [ "null", "string" ],
"default" : null
} ]
}, {
"type" : "record",
"name" : "vObject",
"fields" : [ {
"name" : "site_id",
"type" : "int",
"default" : 0
}, {
"name" : "categories",
"type" : [ "null", {
"type" : "map",
"values" : "Category"
} ],
"default" : null
}, {
"name" : "price_score",
"type" : [ "null", "float" ],
"default" : null
}, {
"name" : "confidence_score",
"type" : [ "null", "float" ],
"default" : null
} ]
}, {
"type" : "record",
"name" : "SampleObject",
"fields" : [ {
"name" : "lv",
"type" : [ "null", {
"type" : "array",
"items" : "vObject"
} ],
"default" : null
}, {
"name" : "lmd",
"type" : "long",
"default" : -1
} ]
} ],
"messages" : {
}
}
Using whatever language you'd like, you can now generate a set of objects and the default "toString" operation is to output in JSON form as you have above. However, the true power of Avro comes with it's compression capabilities. You should truly write out in avro binary format to see the real benefits of avro.
Hope this helps!
I am facing a trouble in the use of ElasticSearch for my java application.
I explain myself, I have a mapping, which is something like :
{
"products": {
"properties": {
"id": {
"type": "long",
"ignore_malformed": false
},
"locations": {
"properties": {
"category": {
"type": "long",
"ignore_malformed": false
},
"subCategory": {
"type": "long",
"ignore_malformed": false
},
"order": {
"type": "long",
"ignore_malformed": false
}
}
},
...
So, as you can see, I receive a list of products, which are composed of locations. In my model, this locations are all the categories' product. It means that a product can be in 1 or more categories. In each of this category, the product has an order, which is the order the client wants to show them.
For instance, a diamond product can have a first place in Jewelry, but the third place in Woman (my examples are not so logic ^^).
So, when I click on Jewelry, I want to show this products, ordered by the field locations.order in this specific category.
For the moment, when I search all the products on a specific category the response for ElasticSearch that I receive is something like :
{"id":5331880,"locations":[{"category":5322606,"order":1},
{"category":5883712,"subCategory":null,"order":3},
{"category":5322605,"subCategory":6032961,"order":2},.......
Is it possible to sort this products, by the element locations.order for the specific category I am searching for ? For instance, if I am querying the category 5322606, I want the order 1 for this product to be taken.
Thank you very much beforehand !
Regards,
Olivier.
First a correction of terminology: in Elasticsearch, "parent/child" refers to completely separate docs, where the child doc points to the parent doc. Parent and children are stored on the same shard, but they can be updated independently.
With your example above, what you are trying to achieve can be done with nested docs.
Currently, your locations field is of type:"object". This means that the values in each location get flattened to look something like this:
{
"locations.category": [5322606, 5883712, 5322605],
"locations.subCategory": [6032961],
"locations.order": [1, 3, 2]
}
In other words, the "sub" fields get flattened into multi-value fields, which is of no use to you, because there is no correlation between category: 5322606 and order: 1.
However, if you change locations to be type:"nested" then internally it will index each location as a separate doc, meaning that each location can be queried independently, using the dedicated nested query and filter.
By default, the nested query will return a _score based upon how well each location matches, but in your case you want to return the highest value of the order field from any matching children. To do this, you'll need to use a custom_score query.
So let's start by creating the index with the appropriate mapping:
curl -XPUT 'http://127.0.0.1:9200/test/?pretty=1' -d '
{
"mappings" : {
"products" : {
"properties" : {
"locations" : {
"type" : "nested",
"properties" : {
"order" : {
"type" : "long"
},
"subCategory" : {
"type" : "long"
},
"category" : {
"type" : "long"
}
}
},
"id" : {
"type" : "long"
}
}
}
}
}
'
The we index your example doc:
curl -XPOST 'http://127.0.0.1:9200/test/products?pretty=1' -d '
{
"locations" : [
{
"order" : 1,
"category" : 5322606
},
{
"order" : 3,
"subCategory" : null,
"category" : 5883712
},
{
"order" : 2,
"subCategory" : 6032961,
"category" : 5322605
}
],
"id" : 5331880
}
'
And now we can search for it using the queries we discussed above:
curl -XGET 'http://127.0.0.1:9200/test/products/_search?pretty=1' -d '
{
"query" : {
"nested" : {
"query" : {
"custom_score" : {
"script" : "doc[\u0027locations.order\u0027].value",
"query" : {
"constant_score" : {
"filter" : {
"and" : [
{
"term" : {
"category" : 5322605
}
},
{
"term" : {
"subCategory" : 6032961
}
}
]
}
}
}
}
},
"score_mode" : "max",
"path" : "locations"
}
}
}
'
Note: the single quotes within the script have been escaped as \u0027 to get around shell quoting. The script actually looks like this: "doc['locations.order'].value"
If you look at the _score from the results, you can see that it has used the order value from the matching location:
{
"hits" : {
"hits" : [
{
"_source" : {
"locations" : [
{
"order" : 1,
"category" : 5322606
},
{
"order" : 3,
"subCategory" : null,
"category" : 5883712
},
{
"order" : 2,
"subCategory" : 6032961,
"category" : 5322605
}
],
"id" : 5331880
},
"_score" : 2,
"_index" : "test",
"_id" : "cXTFUHlGTKi0hKAgUJFcBw",
"_type" : "products"
}
],
"max_score" : 2,
"total" : 1
},
"timed_out" : false,
"_shards" : {
"failed" : 0,
"successful" : 5,
"total" : 5
},
"took" : 9
}
Just add a more updated version related to sorting parent by child field.
We can query parent doc type sorted by child field ('count' e.g.) similar as follows.
https://gist.github.com/robinloxley1/7ea7c4f37a3413b1ca16
I am having the mongo document as below:
{
"_id" : ObjectId("506e9e54a4e8f51423679428"),
"description" : "ffffffffffffffff",
"menus" : [
{
"_id" : ObjectId("506e9e5aa4e8f51423679429"),
"description" : "ffffffffffffffffffff",
"items" : [
{
"name" : "xcvxc",
"description" : "vxvxcvxc",
"text" : "vxcvxcvx",
"menuKey" : "0",
"onSelect" : "1",
"_id" : ObjectId("506e9f07a4e8f5142367942f")
} ,
{
"name" : "abcd",
"description" : "qqq",
"text" : "qqq",
"menuKey" : "0",
"onSelect" : "3",
"_id" : ObjectId("507e9f07a4e8f5142367942f")
}
]
}
]
}
Now i want to change this to :
{
"_id" : ObjectId("506e9e54a4e8f51423679428"),
"description" : "ffffffffffffffff",
"menus" : [
{
"_id" : ObjectId("506e9e5aa4e8f51423679429"),
"description" : "ffffffffffffffffffff",
"items" : {
{
"name" : "xcvxc",
"description" : "vxvxcvxc",
"text" : "vxcvxcvx",
"menuKey" : "0",
"onSelect" : "1",
"_id" : ObjectId("506e9f07a4e8f5142367942f")
} ,
{
"name" : "abcd",
"description" : "qqq",
"text" : "qqq",
"menuKey" : "0",
"onSelect" : "3",
"_id" : ObjectId("507e9f07a4e8f5142367942f")
}
}
}
]
}
Is this possible in mongo? In the first schema, updating is not possible atomically becoz we can't use two "$" while updating deep layer. So i thought to change schema as same as second one, how can i achieve it?
For first one i have used "$push" for adding items inside menus...
Any help would be great..
Your update is changing 'menu' object, so I would suggest changing the schema so that the menu is the top level document, rather than an array in another document.
Menu could either have a field referencing the top level object (in another collection) that it belongs to, or you can denormalize the fields of the top level object into each menu document.
Without knowing complete requirements of the application, it's difficult to know when the schema is "good enough".