I want the jolt transform for the given input . Your help in this is highly appreciated . thanks
i am providing the input and expected output.
in input json Photos array is dynamic in nature. Here it is 3 , it can be 3 or 4 or5 any .
INPUT JSON .
{
"Entity": {
"card": {
"cardNo":"123456789",
"cardStatus":"10",
"cardAddress":"UK",
"cardAddress1":"US",
"cardCity":"mk" ,
"name": "RAM",
"lastName": "ABU",
"name1": "RAM1",
"lastName1": "ABU1"
},
"Photos": [
{
"Id": "327703",
"Caption": "TEST>> photo 1",
"Url": "http://bob.com/0001/327703/photo.jpg"
},
{
"Id": "327704",
"Caption": "TEST>> photo 2",
"Url": "http://bob.com/0001/327704/photo.jpg"
},
{
"Id": "327704",
"Caption": "TEST>> photo 2",
"Url": "http://bob.com/0001/327704/photo.jpg"
}
]
}
}
OUTPUT GETTING after jolt transform
{
"tab": {
"text": "123456789"
},
"address": [
{
"add": "UK",
"add2": "US",
"city": "mk"
}
],
"Photos": [
{
"no": "327703",
"caption": "TEST>> photo 1"
},
{
"no": "327704",
"caption": "TEST>> photo 2"
},
{
"no": "327704",
"caption": "TEST>> photo 2"
}
]
}
WHAT WILL BE THE CORRECT JOLT TRANSFORM FOR THIS?
jolt spec that i have used IS
[
{
"operation": "shift",
"spec": {
"Entity": {
"card": {
"cardNo": "tab.text",
"cardAddress": "address[0].add",
"cardAddress1": "address[0].add2",
"cardCity": "address[0].city",
"name": "Photos[&1].no",
"lastName": "Photos[&1].caption",
"name1": "Photos[&1].no",
"lastName1": "Photos[&1].caption"
},
"Photos": {
"*": {
"Id": "Photos[&1].no",
"Caption": "Photos[&1].caption"
}
}
}
}
}
]
EXPECTED OUTPUT:
{
"tab": {
"text": "123456789"
},
"address": [
{
"add": "UK",
"add2": "US",
"mk": "mk"
}
],
"Photos": [
{
"no": "RAM",
"caption2": "ABU"
},
{
"no": "RAM1",
"caption2": "ABU1"
},
{
"no": "327703",
"caption2": "TEST>> photo 1"
},
{
"no": "327704",
"caption2": "TEST>> photo 2"
},
{
"no": "327704",
"caption2": "TEST>> photo 2"
}
]
}
I am very new to jolt transform. Your help is highly appreciated. Thanks
You can use two consecutive shift transformation specs, in the first determine the groupings as desired such as in the following
[
{
"operation": "shift",
"spec": {
"Entity": {
"card": {
"cardNo": "tab.text",
"cardAddress": "address[0].add",
"cardAddress1": "address[0].add2",
"cardC*": "address[0].mk",
"nam*": "Photos.no",
"lastNam*": "Photos.caption2"
},
"Photos": {
"*": {
"Id": "Photos.no",
"Caption": "Photos.caption2"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&", // "else" case --> the arrrays/objects/attributes other than "Photos"
"Photos": {
"*": {
"*": {
"#": "&3[&1].&2"
}
}
}
}
}
]
Related
I am not able to transform the jolt properly using jolt spec. Thanks in photos array elements can be any like in above two are there, so it can be 3 or 4 or 5 any.
My jolt spec:
Input JSON :
{
"Entity": {
"card": {
"name": "RAM",
"lastName": "ABU"
},
"Photos": [
{
"Id": "327703",
"Caption": "TEST>> photo 1",
"Url": "http://bob.com/0001/327703/photo.jpg"
},
{
"Id": "327704",
"Caption": "TEST>> photo 2",
"Url": "http://bob.com/0001/327704/photo.jpg"
}
]
}
}
Jolt Spec :
[
{
"operation": "shift",
"spec": {
"Entity": {
"card": {
"name": "Photos[0].no",
"lastName": "Photos[0].caption2."
},
"Photos": {
"*": {
"Id": "Photos[&1].no",
"Caption": "Photos[&1].caption2"
}
}
}
}
}
]
Current output :
{
"Photos" : [ {
"no" : [ "RAM", "327703" ],
"caption2" : [ "ABU", "TEST>> photo 1" ]
}, {
"no" : "327704",
"caption2" : "TEST>> photo 2"
} ]
}
Expected output :
{
"Photos" : [
{
"no" : "Ram",
"caption2" : "ABU"
},
{
"no" : "327703" ,
"caption2" : "TEST>> photo 1"
},
{
"no" : "327704",
"caption2" : "TEST>> photo 2"
}
]
}
You can construct a shift transformation spec which will rearrange the desired elements to remain as two individual arrays nested within an object tagged Photos, and then put them into their respective (three) objects such as
[
{
"operation": "shift",
"spec": {
"Entity": {
"card": {
"name": "Photos.no",
"lastName": "Photos.caption2"
},
"Photos": {
"*": {
"Id": "Photos.no",
"Caption": "Photos.caption2"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"#": "&3[&1].&2" // grouped by "Photos" which is grabbed after going the tree 3 leves up(&3) and binded as array by their respective indexes([&1] which yields 0,1,2) and thier key names(&2)
}
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
Introduction
I am currently working on a project to achieve a bulk insertion in a Couchbase database.
For this, I chose Quarkus Reactive to get data from the client. And I used a Flux to insert all the data to database.
For each data inserted, I send a response, example of a response:
[
{
"data": {
"id": "1"
"created": "2022-02-22T00:28:34Z",
"lastName": "DUPOND",
"firstName": "Marianne",
"gender": "FEMALE",
"title": "MRS",
"addresses": [
{
"street1": "rue du Testeur",
"zipCode": "34000",
"city": "MONTPELLIER",
"country": "France",
"typeKey": "PERSONAL"
}
],
"phones": [
{
"number": "0610529856",
"typeKey": "PERSONAL"
}
],
"emails": [
{
"email": "marianne.dupond#test.com",
"typeKey": "PERSONAL"
}
],
"birthday": "1987-11-13",
"languages": [
"FR",
"EN"
],
"socialNetworks": [
{
"socialNetworkId": "marianne.dupond#twitter",
"typeKey": "PERSONAL"
}
],
"customFields": [
{
"customFieldKey": "MAIDEN_NAME",
"value": "DURAND"
}
]
},
"result": "201",
"id":"1"
},
{
"data": {
"id" : "2"
"created": "2022-02-22T00:28:34Z",
"lastName": "BOND",
"firstName": "James",
"gender": "MALE",
"title": "MR",
"addresses": [
{
"street1": "boulevard des services secrets",
"zipCode": "10000",
"city": "LONDRES",
"country": "Royaume-Uni",
"typeKey": "PERSONAL"
}
],
"phones": [
{
"number": "0600700700",
"typeKey": "PERSONAL"
}
],
"emails": [
{
"email": "james.bond#test.com",
"typeKey": "PERSONAL"
}
],
"birthday": "1950-07-07",
"languages": [
"EN"
],
"socialNetworks": [
{
"socialNetworkId": "james.bond#facebook",
"typeKey": "PERSONAL"
}
],
"customFields": [
{
"customFieldKey": "VIP",
"value": "1"
}
]
},
"result": "201",
"id": "2"
}
]
Problematic
But currently the returned response is:
[
{
"data": {
"id": "1"
"created": "2022-02-22T00:37:39Z",
"lastName": "DUPOND",
"firstName": "Marianne",
"gender": "FEMALE",
"title": "MRS",
"addresses": [
{
"street1": "rue du Testeur",
"zipCode": "34000",
"city": "MONTPELLIER",
"country": "France",
"typeKey": "PERSONAL"
}
],
"phones": [
{
"number": "0610529856",
"typeKey": "PERSONAL"
}
],
"emails": [
{
"email": "marianne.dupond#test.com",
"typeKey": "PERSONAL"
}
],
"birthday": "1987-11-13",
"languages": [
"FR",
"EN"
],
"socialNetworks": [
{
"socialNetworkId": "marianne.dupond#twitter",
"typeKey": "PERSONAL"
}
],
"customFields": [
{
"customFieldKey": "MAIDEN_NAME",
"value": "DURAND"
}
]
},
"result": "201"
},
{
"data": {
"created": "2022-02-22T00:37:39Z",
"lastName": "BOND",
"firstName": "James",
"gender": "MALE",
"title": "MR",
"addresses": [
{
"street1": "boulevard des services secrets",
"zipCode": "10000",
"city": "LONDRES",
"country": "Royaume-Uni",
"typeKey": "PERSONAL"
}
],
"phones": [
{
"number": "0600700700",
"typeKey": "PERSONAL"
}
],
"emails": [
{
"email": "james.bond#test.com",
"typeKey": "PERSONAL"
}
],
"birthday": "1950-07-07",
"languages": [
"EN"
],
"socialNetworks": [
{
"socialNetworkId": "james.bond#facebook",
"typeKey": "PERSONAL"
}
],
"customFields": [
{
"customFieldKey": "VIP",
"value": "1"
}
]
},
"result": "201"
}
]
The id field inside the data and the response field do not appear, and sometimes the id field inside the data of the first element appears, sometimes not.
Affected code
Bulk insertion(flux of single insertion) code:
public Flux<SingleResponseBulk> bulkInsertCustomerProfile(Multi<SingleResponseBulk> singleResponses) {
return Flux.from(singleResponses)
.doOnEach(signal -> {
if (signal.hasValue()) {
SingleResponseBulk singleResponseBulk = Objects.requireNonNull(signal.get());
insertCustomerProfileWithSingleResponseBulk(singleResponseBulk);
}
});
}
private void insertCustomerProfileWithSingleResponseBulk(SingleResponseBulk response) {
try {
insertCustomerProfile(response.getData()).subscribe();
response.setId(response.getData().getId());
response.setResult("201");
} catch (CouchbaseException e) {
response.setResult("400");
response.setError(e.getMessage());
}
}
Single insertion code:
public Mono<CustomerProfile> insertCustomerProfile(CustomerProfile customerProfile) {
customerProfile.setCreated(LocalDateTime.now());
return getNextId().flatMap(counterResult -> {
String id = String.valueOf(counterResult.content());
customerProfile.setId(id);
String key = getDocumentKey(id);
return collection.insert(key, customerProfile)
.flatMap(result -> {
customerProfile.setCas(result.cas());
return Mono.just(customerProfile);
})
.doOnError(CouchbaseException.class, mapCouchbaseExceptionConsumer(id));
});
}
Id generation code:
private Mono<CounterResult> getNextId() {
return collection.binary()
.increment("counter" + COLLECTION_DELIMITER + CUSTOMER_PROFILE_COLLECTION)
.doOnError(CouchbaseException.class, error -> {
String errorMessage = "An exception occurred during id generation.";
throw new RepositoryException(errorMessage, error);
});
}
Question
So I was wondering if it would be possible to solve the problem. If so, how can we do it?
I have a requirement to transform the Nested Object in a Json structure.
Here's the Input JSON
Input JSON
{
"data": {
"PRODUCTS": {
"ProductID": "1234-5678",
"ModelNumber": "B550",
"Price": "199",
"Quantity": "1",
"ATTRIBUTES": {
"ProductID": "1234-5678",
"Height": "25",
"Width": "75"
}
}
}
}
Required Output
{
"data": {
"products": [
{
"productId": "1234-5678",
"modelNumber": "B550",
"unitPrice": "199",
"quantity": "1",
"attributes": [
{
"productId": "1234-5678",
"height": "25",
"width": "75"
}
]
}
]
}
}
My JSON Spec:
[
{
"operation": "modify-overwrite-beta",
"spec": {
"data": {
"PRODUCTS": "=toList"
}
}
},
{
"operation": "shift",
"spec": {
"data": {
"PRODUCTS": {
"*": {
"ProductID": "data.products[&1].productId",
"ModelNumber": "data.products[&1].modelNumber",
"Price": "data.products[&1].unitPrice",
"Quantity": "data.products[&1].quantity",
"ATTRIBUTES": {
"ProductID": "data.products[&1].attributes[&1].productId",
"Height": "data.products[&1].attributes[&1].height",
"Width": "data.products[&1].attributes[&1].width"
}
}
}
}
}
},
{
"operation": "default",
"spec": {
"data": {
"*": {}
}
}
}
]
Current Output
{
"data" : {
"products" : [ {
"productId" : "1234-5678",
"modelNumber" : "B550",
"unitPrice" : "199",
"quantity" : "1"
} ]
}
}
I want to convert the ATTRIBUTES nested object to a list and also the nodes inside the ATTRIBUTES object as per the expected output. Can someone throw some light as to how can I achieve this?
You can apply three steps of shift transformations;
To rename all keys as desired
To create the innermost array (attributes)
To create the outermost array (products)
such as
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"ProductID": "&2.products.productID",
"ModelNumber": "&2.products.modelNumber",
"Price": "&2.products.unitPrice",
"Quantity": "&2.products.quantity",
"*": {
"ProductID": "&3.products.attributes.productId",
"Height": "&3.products.attributes.height",
"Width": "&3.products.attributes.width"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"attributes": "&2.&1.&[]",
"*": "&2.&1.&"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "&1.&[]"
}
}
}
]
I have to transform one JSON to another JSON,
I am new to jolt.if you have any other methods in java please let me know.
Input can have many other attributes in a nested manner.
I have to make generic code which can consume all the fields in JSON and transform it into the desired output which I have mentioned.
Input
{
"id": "123456789",
"OrderType": "ABC",
"Abc": [
{
"Name": "Pluto",
"Value": "Charon"
},
{
"Name": "Earth",
"Value": "Moon"
}
]
}
Desired Output
"MyFieldList": [
{
"Footer": "My Footer",
"fieldList": [
{
"label": "id",
"fieldName": "id",
"fieldValue": "123456789",
"editable": false,
"dataType": "STRING"
},
{
"label": "OrderType",
"fieldName": "OrderType",
"fieldValue": "ABC",
"editable": false,
"dataType": "STRING"
},
{
"label": "Pluto",
"fieldName": "Pluto",
"fieldValue": "Charon",
"editable": false,
"dataType": "STRING"
},
{
"label": "Earth",
"fieldName": "Earth",
"fieldValue": "Moon",
"editable": false,
"dataType": "STRING"
}]
}
]
I have tried using this jolt spec but , I cannot figure out the nested part how to flatten it .
{
"operation": "shift",
"spec": {
"*": {
"$": "[#2].fieldName",
"#": "[#2].fieldValue",
"#false": "[#2].editable",
"# ": "[#2].Size",
"#STRING": "[#2].dataType"
}
}
}
The important part is creating a array of arrays, before turning into the fieldList:
[
{
"operation": "shift",
"spec": {
"id": {
"$": "[#1].[#1].fieldName",
"#": "[#1].[#1].fieldValue",
"#false": "[#1].[#1].editable",
"#STRING": "[#1].[#1].dataType"
},
"OrderType": {
"$": "[#2].[#1].fieldName",
"#": "[#2].[#1].fieldValue",
"#false": "[#2].[#1].editable",
"#STRING": "[#2].[#1].dataType"
},
"Abc": {
"*": {
"Name": "[#3].[&1].fieldName",
"Value": "[#3].[&1].fieldValue",
"#false": "[#3].[&1].editable",
"#STRING": "[#3].[&1].dataType"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "MyFieldList.fieldList.[]"
}
}
},
{
"operation": "default",
"spec": {
"MyFieldList": {
"Footer": "My Footer"
}
}
}
]
I want a conditional transformation where I need to add a property in output if the value of a specific field in input matches my condition. Below is my input and output required.
Input
{
"attr": [
{
"name": "first",
"validations": [
{
"type": "Required",
"value": true
}
]
},
{
"name": "last",
"validations": [
{
"type": "lenght",
"value": "10"
}
]
},
{
"name": "email",
"validations": [
{
"type": "min",
"value": 10
}
]
}
]
}
Output
{
"out": [
{
"name": "first",
"required": "yes"
},
{
"name": "last"
},
{
"name": "email"
}
]
}
So I am able to get till the condition, but inside condition, & and # are being respective to the input rather than to the output. Can anybody help me out with the transformation? Below is the spec I have written so far.
[
{
"operation": "shift",
"spec": {
"attr": {
"*": {
"name": "out.&1.name",
"validations": {
"*": {
"type": {
"Required": {
"#(2,value)": "out.&1.req"
}
}
}
}
}
}
}
}
]
This spec does the transform.
[
{
"operation": "shift",
"spec": {
"attr": {
"*": {
"name": "out[&1].name",
"validations": {
"*": {
"type": {
"Required": {
"#yes": "out[&5].required"
}
}
}
}
}
}
}
}
]
However, I think you meant to grab the "value" : true that is a sibling of the "Required" : true, rather than have the output be "yes".
If so swap in this bit.
"Required": {
"#(2,value)": "out[&5].required"
}