Develop Reference

Transform array to sub-values using jolt

I'm struggling to trans my input response to a specific output using jolt, below is my example
The input that describe my message
{
"firstAttribute": true,
"secondAttribute": "12",
"data": {
"propertyKey1": "1",
"propertyKey2": [
"a"
],
"propertyKey3": "2",
"propertyKey4": "3",
"propertyKey_test": [
"option1",
"option2",
"option3"
],
"propertyKey5": "4",
"propertyKey6": "87.0"
},
"Keytest1": "value1",
"KeyTest2": "value2"
}
The used jolt Spec
[
{
"operation": "shift",
"spec": {
"data": {
"propertyKey2": {
"0": "propertyKey2"
},
"*": {
"#": "data.&"
}
},
"*": {
"#": "&"
}
}
}
]
My actual output after a jolt transfomration
{
"firstAttribute" : true,
"secondAttribute" : "12",
"data" : {
"propertyKey1" : "1",
"propertyKey3" : "2",
"propertyKey4" : "3",
"propertyKey_test" : [ "option1", "option2", "option3" ],
"propertyKey5" : "4",
"propertyKey6" : "87.0"
},
"propertyKey2" : "a",
"Keytest1" : "value1",
"KeyTest2" : "value2"
}
The desired output is to convert every array fields to separated fields such as propertyKey_test as below
{
"firstAttribute" : true,
"secondAttribute" : "12",
"data" : {
"propertyKey1" : "1",
"propertyKey3" : "2",
"propertyKey4" : "3",
"propertyKey_test": "option1",
"propertyKey5" : "4",
"propertyKey6" : "87.0"
},
"propertyKey2" : "a",
"Keytest1" : "value1",
"KeyTest2" : "value2"
}
Any help would be appreciated. Thanks in advance
PS: we can receive a dynamic array fields (ex: today we have propertyKey_test:["option1","option2","option3"] tomorrow we could receive another array field for example new_field:["a","b","c"].
My goal is to check everytime if the field is an array and take just the first element as described above

The current desired output is not a valid JSON value, since there may not exist more than one attribute with identical key within a common object. But, if you need to differentiate the components of the "propertyKey_test" array, then using the indexes of the array as below might be preferred such as
[
{
"operation": "shift",
"spec": {
"data": {
"propertyKey2": {
"0": "&1"
},
"*": {
"#": "&2.&"
},
"propertyKey_*": {
"*": {
"#": "&3.&2&1"
}
}
},
"*": {
"#": "&"
}
}
}
]
in order to get
{
"firstAttribute" : true,
"secondAttribute" : "12",
"data" : {
"propertyKey1" : "1",
"propertyKey3" : "2",
"propertyKey4" : "3",
"propertyKey_test0" : "option1",
"propertyKey_test1" : "option2",
"propertyKey_test2" : "option3",
"propertyKey5" : "4",
"propertyKey6" : "87.0"
},
"propertyKey2" : "a",
"Keytest1" : "value1",
"KeyTest2" : "value2"
}
as output.

Java Jolt: can't find proper spec for my transformation

I'm struggling with that Jolt transformation:
Here is my input JSON
{
"bloc1-1": {
"bloc1-2": [
{
"key": "key1",
"value": "value1-1"
},
{
"key": "key2",
"value": "value1-2"
}
]
},
"bloc2-1": {
"bloc2-2": [
{
"key": "key1",
"value": "value2-1"
},
{
"key": "key2",
"value": "value2-2"
},
{
"key": "key3",
"value": "value2-3"
}
]
}
}
Here is what I'm expecting
{
"bloc1-key1" : "value1-1",
"bloc1-key2" : "value1-2",
"bloc2-key1" : "value2-1",
"bloc2-key2" : "value2-2",
"bloc2-key3" : "value2-3"
}
I have tried the following spec, but I cannot figure out how to prefix the key in the RHS (the # should be the first character)
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"value": "#(1,key)"
}
}
}
}
}
]
and got that
{
"key1" : [ "value1-1", "value2-1" ],
"key2" : [ "value1-2", "value2-2" ],
"key3" : "value2-3"
}
Any help would be appreciated

The spec below should do the trick:
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"$2": "keyParts1",
"key": "keyParts2",
"value": "values"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"keyParts1": {
"*": "[#1].part1"
},
"keyParts2": {
"*": "[#1].part2"
},
"values": {
"*": "[#1].value"
}
}
},
{
"operation": "modify-default-beta",
"spec": {
"*": {
"newKey": "=split('-', #(1,part1))",
"newKeyFirst": "#(1,newKey[0])",
"newKeyComplete": "=concat(#(1,newKeyFirst),'-',#(1,part2))"
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"value": "#(1,newKeyComplete)"
}
}
}
]
Try to apply each of the operations separately (for example in the demo app).
Check out the links to the jolt docs and the examples from this answer
Step by step explanation:
First "shift" transformation places the components of the final result in the separate arrays:
{
"keyParts1" : [ "bloc1-1", "bloc1-1", "bloc2-1", "bloc2-1", "bloc2-1" ],
"keyParts2" : [ "key1", "key2", "key1", "key2", "key3" ],
"values" : [ "value1-1", "value1-2", "value2-1", "value2-2", "value2-3" ]
}
The above operation let us have each of the components of the end result positioned in the right index of the desired result.
Note that each array has the same size.
In the second operation we group each of the array elements by its index using the # operator. Looking at the docs [#1] can be translated to:
"Go up one level and ask that node how many matches it has had before finding the current element".
The node above is an array, so for the 3rd array element it will be the array index 2.
[
...
{
"part1" : "bloc2-1",
"part2" : "key1",
"value" : "value2-1"
}
...
]
In the 3rd operation I have assumed that the key of the desired result is the concatenation of the "blocX" substring of the "blocX-Y" key concatenated with the "key" value. That's why I've used the "modify-default-beta". Have a look at the example from the demo app
Each of five elements of the array is transformed similarly to the 3rd array element:
[
...
{
"part1" : "bloc2-1",
"part2" : "key1",
"value" : "value2-1",
"newKey" : [ "bloc2", "1" ],
"newKeyFirst" : "bloc2",
"newKeyComplete" : "bloc2-key1"
}
...
]
The last operation produces the desired result from the 3rd operation's output.

You may consider another library Josson.
https://github.com/octomix/josson
Josson josson = Josson.fromJsonString(inputJSON);
JsonNode node = josson.getNode(
"entries()" + // Field "key" generated by entries() is the same as the inner one.
".field(k:key)" + // So, rename it to "k".
".unwind(value.*)" + // Unwind the "value" generated by entries(), not the inner "value".
".map(concat(k.keepBefore('-'),'-',key)::value)" +
".mergeObjects()");
System.out.println(node.toPrettyString());
Output
{
"bloc1-key1" : "value1-1",
"bloc1-key2" : "value1-2",
"bloc2-key1" : "value2-1",
"bloc2-key2" : "value2-2",
"bloc2-key3" : "value2-3"
}

Jolt transform several object to array with new field name

I'am new using jolt tool and i wonder if there a way to take several json objects and put them to an array with a new field named as follows:
Input :
{
"userId": 1,
"age": 20,
"desc1": "value desc1",
"desc2": "value desc2",
"desc3": "value desc3"
}
JSON spec :
[
{
"operation": "shift",
"spec": {
"userId": "ID",
"age": "age",
"*": "additionalInformation"
}
}
]
Expected result :
{
"ID": 1,
"age": 20,
"additionalInformation": [
{
"code": "desc1",
"value": "value desc1"
},
{
"code": "desc2",
"value": "value desc2"
},
{
"code": "desc3",
"value": "value desc3"
}
]
}
Using the spec above i can only obtain this result :
{
"ID": 1,
"test": 20,
"additionalInformation": [
"value desc1",
"value desc2",
"value desc3"
]
}
Any suggestion what i have missed ?

You can use $ and # wildcards for code and value, respectively while combining the elements under common factor by using &. as prefixes for them in the first step. Then combine elements other than ID and age within the same array in the last step such as
[
{
"operation": "shift",
"spec": {
"userId": "ID",
"age": "age",
"*": {
"$": "&.code",
"#": "&.value"
}
}
},
{
"operation": "shift",
"spec": {
"ID": "&",
"age": "&",
"*": "additionalInformation[]"
}
}
]
Edit(due to the comment) : If you need to pick some individual elements such as desc1 and desc3 rather than all("*"), then replace the first spec with following
{
"operation": "shift",
"spec": {
"userId": "ID",
"age": "age",
"desc1|desc3": {
"$": "&.code",
"#": "&.value"
}
}
}

How to collect all other values for a tag within JSON even when the exception `com.jayway.jsonpath.PathNotFoundException` is thrown?

Consider the code snippet below-
{
"header": {
"systemId": "1"
},
"body": {
"approvalType": "S",
"requester": "CRM",
"approver": "V",
"additionalInfoList": [
{
"additionalInfoItem": {
"value": [
{
"secret": [
{
"question": "1"
}
]
},
{
"secret": [
{
"question": "2"
}
]
},
{
"secret": [
{
"question": "3"
}
]
}
]
}
},
{
"additionalInfoItem": {
"name": "key2",
"value": [
{
"secret": [
{
"question": "00"
}
]
},
{
"secret": [
{
"question": "002"
}
]
},
{
"secret": [
{
"question": "003"
}
]
}
]
}
}
]
}
}
For this json path
$.body.additionalInfoList[*].additionalInfoItem.value[*].secret[*].question
the API gives
[
"1",
"2",
"3",
"00",
"002",
"003"
]
I am using configuration REQUIRE_PROPERTIES option which configures JsonPath to require properties defined in path when an indefinite path is evaluated.
If in the above JSON, one of the question is not sent in request, an exception as below will be thrown - No results for path: $['body']['additionalInfoList'][1]['additionalInfoItem']['value'][0]['secret'][0]['question']
I have a requirement to collect all other values for the question tag even when the exception com.jayway.jsonpath.PathNotFoundException was thrown. How can I achieve this?
On the other hand, if I use the option SUPPRESS_EXCEPTIONS, how can I know if there is a missing path?

I do not have a great answer here; however, to get it done I would suggest processing the response twice:
once with SUPPRESS_EXCEPTIONS (or simply no option might work as well),
and then with REQUIRE_PROPERTIES to detect the errors.
This should allow you to handle the scenario as described.

Jolt Transform JSON Spec for Array Input

I am trying to do JOLT shift operation with below spec which is not working. Not sure what mistake I have done. Need help in this case. Output JSON is coming as an object instead of Array and shift also not working as expected.
Input : [
{
"Header": {
"Number": 1,
"Id": "JO"
},
"Name": "John"
},
{
"Header": {
"Number": 2,
"Id": "JS"
},
"Name": "Justin"
}
]
Spec : [
{
"operation": "shift",
"spec": {
"*": {
"Header": "Header",
"Name": "Header.Name"
}
}
}
]
Expected Output : [
{
"Header": {
"Number": 1,
"Id": "JO",
"Name": "John"
}
},
{
"Header": {
"Number": 2,
"Id": "JS",
"Name": "Justin"
}
}
]
Actual Output : {
"Header" : [ {
"Number" : 1,
"Id" : "JO",
"Name" : "John"
}, {
"Number" : 2,
"Id" : "JS"
} ]
}

You have to also specify that the "Header" object is inside the array.
Moreover, the index of the array where you place the "Header" object for each of the element of the array.
That's what the spec below does (using the [&1] - apmersand wildcard combined with array):
[
{
"operation": "shift",
"spec": {
"*": {
"Header": "[&1].Header",
"Name": "[&1].Header.Name"
}
}
}
]
Sources:
Shiftr.java javadocs:
The arrays
The ampersand wildcard
Other answer: "How do I transform an array using Jolt?"
Demo application linked in the jolt repo to test the spec