When Calling an external API I am receiving a complex json structure, the same I need to transform into a simple json as per our need. I found jolt has the capability to transform a json but not able to come up with the jolt spec.
My input Json array --
{
"attribute": [
"teamalloc",
"prodAlloc"
],
"item": {
"id": "abcde",
"name": "Champak Kumar",
"allocDetails": {
"updatedAt": "2020-08-10T14:26:48-07:00",
"token": 1134,
"items": [
{
"allocation": 0.2,
"team": {
"id": 90,
"name": "Some Team Name 1",
"createdAt": "2010-01-19T10:52:52-07:00"
}
},
{
"allocation": 0.9,
"team": {
"id": 80,
"name": "Some Team Name 2",
"createdAt": "2010-01-19T10:52:52-07:00",
"product": {
"id": 20,
"name": "Some Product Name 1",
"otherDetails": {
"key": "Id",
"value": "GEC"
}
}
}
},
{
"allocation": 0.1,
"team": {
"id": 10,
"name": "Some Team Name 3",
"createdAt": "2010-01-19T10:52:52-07:00",
"product": {
"id": 22,
"name": "Some Product Name 2",
"otherDetails": {
"key": "Id1",
"value": "GEC1"
}
}
}
}
]
}
}
}
My output Json structure should look like --
{
"name": "Champak Kumar",
"allocDetails": [
{
"allocation": 0.2,
"team": {
"id": 90,
"name": "Some Team Name 1"
}
},
{
"allocation": 0.9,
"team": {
"id": 80,
"name": "Some Team Name 2",
"product": {
"id": 20,
"name": "Some Product Name 1"
}
}
},
{
"allocation": 0.1,
"team": {
"id": 10,
"name": "Some Team Name 3",
"product": {
"id": 22,
"name": "Some Product Name 2"
}
}
}
]
}
I tried multiple jolt spec but couldn't come up with the desired output. What should be the ideal jolt spec for this?
This spec should using shift operation should work:
[
{
"operation": "shift",
"spec": {
"item": {
"name": "name",
"allocDetails": {
"items": {
"*": {
"allocation": "allocDetails[&1].allocation",
"team": {
"id": "allocDetails[&2].team.id",
"name": "allocDetails[&2].team.name",
"product": "allocDetails[&2].team.product"
}
}
}
}
}
}
}
]
Edit #1:
The explanation:
The shift operation spec defines where do we want to put the values from the input json in the result json.
Each key - value pair in the spec defines the source - target relation.
For example (let's start from the simplest one): value from the ["item"]["name"] will land under the ["name"] key in the output JSON.
The
"items": {
"*": {
...
}
}
part says: "For each element of the array under the 'items' key do ..."
The ampersand operator "&X" lets us reference the key that is X levels above the LHS (the key in question). For example:
Let's imagine we have the 57th element of the items array (counting from 0):
...
"name": "allocDetails[&2].team.name"
...
says: "Place the value found under the "name" key (that is in item["items"][56]["team"]["name"] in the 57th element of the array placed under the "allocDetails" key.
'*' matches the 57th element. '&2' lets us find out that the element of the array we are working on right now is the 57th element.
Have a look at the shift operation javadocs, especially at the "&" and "*" wildcards
Edit #2:
considering the otherDetails comment:
You could also approach it like this:
...
"team": {
"id": "allocDetails[&2].team.id",
"name": "allocDetails[&2].team.name",
"product": {
"otherDetails": null,
"*": "allocDetails[&3].team.product.&"
}
}
...
the above: puts all of the products' part keys (matched by the "*") to the key with the same name ("&") placed in the "allocDetails[&3].team.product" output json's key... with the exception of the "otherDetails" key.
Working jolt spec --
{
"operation": "shift",
"spec": {
"item": {
"name": "name",
"allocDetails": {
"items": {
"*": {
"allocation": "allocDetails[&1].allocation",
"team": {
"id": "allocDetails[&2].team.id",
"name": "allocDetails[&2].team.name",
"product": {
"id": "allocDetails[&3].team.product.id",
"name": "allocDetails[&3].team.product.name"
}
}
}
}
}
}
}
}
]```
Related
I'am new using jolt tool and i wonder if there a way to take several json objects and put them to an array with a new field named as follows:
Input :
{
"userId": 1,
"age": 20,
"desc1": "value desc1",
"desc2": "value desc2",
"desc3": "value desc3"
}
JSON spec :
[
{
"operation": "shift",
"spec": {
"userId": "ID",
"age": "age",
"*": "additionalInformation"
}
}
]
Expected result :
{
"ID": 1,
"age": 20,
"additionalInformation": [
{
"code": "desc1",
"value": "value desc1"
},
{
"code": "desc2",
"value": "value desc2"
},
{
"code": "desc3",
"value": "value desc3"
}
]
}
Using the spec above i can only obtain this result :
{
"ID": 1,
"test": 20,
"additionalInformation": [
"value desc1",
"value desc2",
"value desc3"
]
}
Any suggestion what i have missed ?
You can use $ and # wildcards for code and value, respectively while combining the elements under common factor by using &. as prefixes for them in the first step. Then combine elements other than ID and age within the same array in the last step such as
[
{
"operation": "shift",
"spec": {
"userId": "ID",
"age": "age",
"*": {
"$": "&.code",
"#": "&.value"
}
}
},
{
"operation": "shift",
"spec": {
"ID": "&",
"age": "&",
"*": "additionalInformation[]"
}
}
]
Edit(due to the comment) : If you need to pick some individual elements such as desc1 and desc3 rather than all("*"), then replace the first spec with following
{
"operation": "shift",
"spec": {
"userId": "ID",
"age": "age",
"desc1|desc3": {
"$": "&.code",
"#": "&.value"
}
}
}
I am using jsonpath to filter.
Json(Dummy json just to explain) source String, which is basically a list of Operating systems and details of its programs etc. In this example, the OS whose id = 1403 is a windows 10 OS and has 2 features acchritecture and browser. There are more details to the browser feature as shown in json
[
{
"id": 1403,
"os": "window 10",
"features": [
{
"id": 1356,
"name": "architecture",
"value": [
{
"id": 1308,
"feature": [
{
"id": 1262,
"key": "name",
"value": "amd64"
}
]
}
],
"category": "cat1"
},
{
"id": 1357,
"name": "browser",
"value": [
{
"id": 1309,
"feature": [
{
"id": 1263,
"key": "name",
"value": "Firefox"
},
{
"id": 1265,
"key": "version",
"value": "187"
}
]
}
],
"category": "cat2"
}
]
},
{
"id": 2804,
"os": "window 7",
"features": [
{
"id": 2764,
"name": "architecture",
"value": [
{
"id": 2719,
"feature": [
{
"id": 2679,
"key": "name",
"value": "amd64"
}
]
}
],
"category": "cat1"
},
{
"id": 2765,
"name": "browser",
"value": [
{
"id": 2722,
"feature": [
{
"id": 2685,
"key": "name",
"value": "Chrome"
},
{
"id": 2684,
"key": "version",
"value": "87.0.4280.88"
}
]
}
],
"category": "cat2"
}
]
}
]
I want to be able to filter the json such that
features[*].name == 'browser' and features[*].value[*].feature[*].value == 'chrome'
What will be the JsonPath string that can help me achieve above query? The above query uses similar syntax used by JsonPath string but doesn't do the job. Its just to explain.
There is another example here gets Movie Title Given 'Starring' field
And would like to get the full OS json that fulfils this condition. In this case a array of OS which contains only one OS i.e. with id= 2804
[
{
"id": "2804",
...
}
]
I am stuck much before what aim to achieve. Here is my code to get all the OS that have "name=browser". I get the array but it only contains value[] items. I want it get the full json. It returns object with IDs- 1357, 2765.
List<Map<String, Object>> expensive = JsonPath.parse(jsonDataSourceString)
.read("$[*].features[*].[?(#.name == 'browser')]");
To get the outer array you need to use the filter like $[?(...)]
For your current use case, we need to use nested array filters. There is an open issue in JsonPath for filter on children level. (Refer here).
Luckily, there is a workaround suggested to use contains over here.
we can use the below expression to filter:
List<Object> expensive = JsonPath.parse(jsonDataSourceString)
.read("$[?(#.features[?(#.name == 'browser')].value[*].feature[*].value contains 'Chrome')]");
Prints the below output
{id=2804, os=window 7, features=[{"id":2764,"name":"architecture","value":[{"id":2719,"feature":[{"id":2679,"key":"name","value":"amd64"}]}],"category":"cat1"},{"id":2765,"name":"browser","value":[{"id":2722,"feature":[{"id":2685,"key":"name","value":"Chrome"},{"id":2684,"key":"version","value":"87.0.4280.88"}]}],"category":"cat2"}]}
I am trying to do JOLT shift operation with below spec which is not working. Not sure what mistake I have done. Need help in this case. Output JSON is coming as an object instead of Array and shift also not working as expected.
Input : [
{
"Header": {
"Number": 1,
"Id": "JO"
},
"Name": "John"
},
{
"Header": {
"Number": 2,
"Id": "JS"
},
"Name": "Justin"
}
]
Spec : [
{
"operation": "shift",
"spec": {
"*": {
"Header": "Header",
"Name": "Header.Name"
}
}
}
]
Expected Output : [
{
"Header": {
"Number": 1,
"Id": "JO",
"Name": "John"
}
},
{
"Header": {
"Number": 2,
"Id": "JS",
"Name": "Justin"
}
}
]
Actual Output : {
"Header" : [ {
"Number" : 1,
"Id" : "JO",
"Name" : "John"
}, {
"Number" : 2,
"Id" : "JS"
} ]
}
You have to also specify that the "Header" object is inside the array.
Moreover, the index of the array where you place the "Header" object for each of the element of the array.
That's what the spec below does (using the [&1] - apmersand wildcard combined with array):
[
{
"operation": "shift",
"spec": {
"*": {
"Header": "[&1].Header",
"Name": "[&1].Header.Name"
}
}
}
]
Sources:
Shiftr.java javadocs:
The arrays
The ampersand wildcard
Other answer: "How do I transform an array using Jolt?"
Demo application linked in the jolt repo to test the spec
I need to extract a json in required format from input json. i'm using jayway json path library. How to achieve it ?
Input Json:
{
"ccid": [
{
"id": 13,
"src": {
"sname": "XA-SXXD",
"lname": "John",
"identifier": 2,
"StatusCode": "C"
}
},
{
"id": 14,
"src": {
"sname": "XB-SXXD",
"lname": "Cena",
"identifier": 3,
"StatusCode": "C",
}
}
]
}
Required Format:
[ {
"id": "13",
"sources": {
"sname": "XA-SXXD",
"lname": "John",
"identifier": 2
}
},
{
"id": "14",
"sources": {
"sname": "XB-SXXD",
"lname": "Cena",
"identifier": 3
}
}]
Query that i use:
$.ccid[*].src[?(#.identifier!=null)].['identifier','sname']
Output that i get:
[
{
"identifier" : 2,
"sname" : "XA-SXXD"
},
{
"identifier" : 3,
"sname" : "XB-SXXD"
}
]
Kindly help me to modify my query to get the required format. The string "sources" in the required format can be hardcoded.
I think I managed to solve this :)
$.ccid[*].[?(#.src.identifier!=null)].['id', 'src']
Give it a try.
Input tested on:
{
"ccid": [
{
"id": 13,
"src": {
"sname": "XA-SXXD",
"lname": "John",
"StatusCode": "C"
}
},
{
"id": 14,
"src": {
"sname": "XB-SXXD",
"lname": "Cena",
"identifier": null,
"StatusCode": "C",
}
}
]
}
Output received:
[
{
"id" : 13,
"src" : {
"sname" : "XA-SXXD",
"lname" : "John",
"StatusCode" : "C"
}
}
]
Only problem is saw is if identifier tag is not available it's essentially treated as not-null. Hence we are getting 13 as an output. But if value is explicitly null then it's fine. So the needs to be enhanced a bit more.
Hope this helps.
I have a JSONArray within a JSONArray, I want to apply JSONPath expression on this in such a way that i get JSONObject or JSONArray as a result when a condition is satsified on the inner JSONArray.
Eg:
{
"A": [
{
"B": [
{
"id": 1
},
{
"id": 2
},
{
"id": 3
}
],
"C": {
"id": 10,
"name": "PQR"
},
"id": 25,
"name": "XYZ"
},
{
"B": [
{
"id": 4
},
{
"id": 5
},
{
"id": 6
}
],
"C": {
"id": 15,
"name": "PQR"
},
"id": 20,
"name": "XYZ"
}
]
}
if i want all elements of A where C.id = 10, I would use: $.A[?(#.C.id == 10)]
Now, What predicate is to be used to obtain all the objects within A, where B.id = 1? Note: B is an array of JSON objects.
I had success with $.A[?(#.B[?(#.id == 1)])]
but only when using Scala's Gatling implementation:
http://jsonpath.herokuapp.com/
The Jayway implementation seems to totally ignore the inner filter and according to an issue on their GitHub, that's a bug.