I am trying to generate query dynamically based on the inputs but in the generated query i can see there are only two aggregations are getting generated how can i make each fields to have the separate aggregations below is the code what i have tried and the response what i'm getting.
From main() i'm calling
buildSearchCriteria("1");
Here i am setting the aggregation type and respective values:
public static void buildSearchCriteria(String... exceptionId) {
SearchCriteria searchCriteria = new SearchCriteria();
Map<String, List<FieldNameAndPath>> stringListMap = new HashMap<>();
stringListMap.put("nested", asList(new FieldNameAndPath("nested", "recommendations",
"recommendations", null, emptyList(), 1)));
stringListMap.put("filter", asList(new FieldNameAndPath("filter", "exceptionIds", "recommendations.exceptionId.keyword",
asList(exceptionId),
asList(new NestedAggsFields("terms", "exceptionIdsMatch")), 2)));
stringListMap.put("terms", asList(new FieldNameAndPath("terms", "by_exceptionId", "recommendations.exceptionId.keyword", null, emptyList(), 3),
new FieldNameAndPath("terms", "by_item", "recommendations.item.keyword", null, emptyList(), 4),
new FieldNameAndPath("terms", "by_destination", "recommendations.location.keyword", null, emptyList(), 5),
new FieldNameAndPath("terms", "by_trans", "recommendations.transportMode.keyword", null, emptyList(), 6),
new FieldNameAndPath("terms", "by_sourcelocation", "recommendations.sourceLocation.keyword", null, emptyList(), 7),
new FieldNameAndPath("terms", "by_shipdate", "recommendations.shipDate", null, emptyList(), 8),
new FieldNameAndPath("terms", "by_arrival", "recommendations.arrivalDate", null, emptyList(), 9)));
stringListMap.put("sum", asList(new FieldNameAndPath("sum", "quantity", "recommendations.transferQuantity", null, emptyList(), 10),
new FieldNameAndPath("sum", "transfercost", "recommendations.transferCost", null, emptyList(), 11),
new FieldNameAndPath("sum", "revenueRecovered", "recommendations.revenueRecovered", null, emptyList(), 12)));
System.out.println(stringListMap);
searchCriteria.setStringListMap(stringListMap);
aggregate(searchCriteria);
}
Below is the aggregate function which will get the the above information and builds query:
public static void aggregate(SearchCriteria searchCriteria) throws IOException {
Map<String, List<FieldNameAndPath>> map = searchCriteria.getStringListMap();
List<FieldNameAndPath> nesteds = map.get("nested");
List<FieldNameAndPath> filter = map.get("filter");
List<FieldNameAndPath> terms = map.get("terms");
List<FieldNameAndPath> sums = map.get("sum");
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
AggregationBuilder aggregationBuilder = new SamplerAggregationBuilder("parent");
nesteds.stream().forEach(l -> buildAggregations(l, aggregationBuilder));
filter.stream().forEach(l -> buildAggregations(l, aggregationBuilder));
terms.stream().forEach(l -> buildAggregations(l, aggregationBuilder));
sums.stream().forEach(l -> buildAggregations(l, aggregationBuilder));
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("index");
searchRequest.types("type");
sourceBuilder.aggregation(aggregationBuilder);
searchRequest.source(sourceBuilder);
System.out.println(searchRequest.source().toString());
}
buildAggregations method:
private static AggregationBuilder buildAggregations(FieldNameAndPath fieldNameAndPath , AggregationBuilder parentAggregationBuilder) {
if(fieldNameAndPath.getAggType().equals("nested")){
parentAggregationBuilder = AggregationBuilders.nested(fieldNameAndPath.getFieldName(), fieldNameAndPath.getFieldPath());
}
if(fieldNameAndPath.getAggType().equals("filter")){
parentAggregationBuilder.subAggregation(AggregationBuilders
.filter(fieldNameAndPath.getFieldName(),
QueryBuilders.termsQuery(fieldNameAndPath.getNestedAggs()
.stream().map(nestedAggsFields -> nestedAggsFields.getFieldName()).findFirst().get(), fieldNameAndPath.getFieldValues())));
}
if(fieldNameAndPath.getAggType().equals("terms")){
parentAggregationBuilder.subAggregation(AggregationBuilders.terms(fieldNameAndPath.getFieldName())
.field(fieldNameAndPath.getFieldPath()));
}
if(fieldNameAndPath.getAggType().equals("sum")){
parentAggregationBuilder.subAggregation(AggregationBuilders.
sum(fieldNameAndPath.getFieldName()).field(fieldNameAndPath.getFieldPath()));
}
return parentAggregationBuilder;
}
SearchCriteria class:
#Data
public class SearchCriteria {
Map<String, List<FieldNameAndPath>> stringListMap;
private List<String> searchFields;
}
And the DTO FieldNameAndPath:
public class FieldNameAndPath{
private String aggType;
private String fieldName;
private String fieldPath;
private List<String> fieldValues;
private List<NestedAggsFields> nestedAggs;
private int order;
}
And the query output from the above code is:
{
"aggregations": {
"parent": {
"sampler": {
"shard_size": 100
},
"aggregations": {
"exceptionIds": {
"filter": {
"terms": {
"exceptionIdsMatch": [
"1"
],
"boost": 1
}
}
},
"by_exceptionId": {
"terms": {
"field": "recommendations.exceptionId.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
},
"by_item": {
"terms": {
"field": "recommendations.item.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
},
"by_destination": {
"terms": {
"field": "recommendations.location.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
},
"by_trans": {
"terms": {
"field": "recommendations.transportMode.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
},
"by_sourcelocation": {
"terms": {
"field": "recommendations.sourceLocation.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
},
"by_shipdate": {
"terms": {
"field": "recommendations.shipDate",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
},
"by_arrival": {
"terms": {
"field": "recommendations.arrivalDate",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
}
},
"quantity": {
"sum": {
"field": "recommendations.transferQuantity"
}
},
"transfercost": {
"sum": {
"field": "recommendations.transferCost"
}
},
"revenueRecovered": {
"sum": {
"field": "recommendations.revenueRecovered"
}
}
}
}
}
}
Expected Query is:
{
"size": 0,
"aggregations": {
"exceptionIds": {
"nested": {
"path": "recommendations"
},
"aggregations": {
"exceptionIdsMatch": {
"filter": {
"terms": {
"recommendations.exceptionId.keyword": [
"1"
],
"boost": 1
}
},
"aggregations": {
"by_exceptionId": {
"terms": {
"field": "recommendations.exceptionId.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"by_item": {
"terms": {
"field": "recommendations.item.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"by_destination": {
"terms": {
"field": "recommendations.location.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"by_trans": {
"terms": {
"field": "recommendations.transportMode.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"by_sourcelocation": {
"terms": {
"field": "recommendations.sourceLocation.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"by_shipdate": {
"terms": {
"field": "recommendations.shipDate",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"by_arrival": {
"terms": {
"field": "recommendations.arrivalDate",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": [
{
"_count": "desc"
},
{
"_key": "asc"
}
]
},
"aggregations": {
"quantity": {
"sum": {
"field": "recommendations.transferQuantity"
}
},
"transfercost": {
"sum": {
"field": "recommendations.transferCost"
}
},
"revenueRecovered": {
"sum": {
"field": "recommendations.revenueRecovered"
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
Related
I'm trying to fetch users from ES based on the status of some of the fields.
I have 5 fields whose status I want to check and if any of these fields have the failed status I want to fetch that record. Since it's an OR condition between these 5 fields I was trying to use should in ES and adding terms to it. But it returns records of those users who don't match the criteria as well.
{
"from": 0,
"size": 50,
"query": {
"bool": {
"must": [
{
"nested": {
"query": {
"bool": {
"must": [
{
"range": {
"segment_status.updated_at": {
"from": "2021-01-24",
"to": null,
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
}
],
"should": [
{
"terms": {
"segment_status.bse_status": [
2,
3,
4
],
"boost": 1
}
},
{
"terms": {
"segment_status.nse_status": [
2,
3
],
"boost": 1
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"path": "segment_status",
"ignore_unmapped": false,
"score_mode": "avg",
"boost": 1
}
}
],
"must_not": [
{
"term": {
"marked_failed_manually": {
"value": true,
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"sort": [
{
"segment_status.updated_at": {
"order": "asc",
"mode": "min",
"nested_filter": {
"term": {
"segment_status.segment_type": {
"value": "CASH",
"boost": 1
}
}
},
"nested_path": "segment_status"
}
}
]
}
That is the query generated by the code. I'm using spring boot to build the query.
Just for reference, I tried this query and it seems to work.
{
"from": 0,
"size": 50,
"query": {
"bool": {
"must": [
{
"nested":{
"query":{
"bool":{
"must" : [
{
"range" : {
"segment_status.updated_at" : {
"from" : "2021-08-30",
"to" : null,
"include_lower" : true,
"include_upper" : true,
"boost" : 1.0
}
}
}
]
}
},
"path" : "segment_status",
"ignore_unmapped" : false,
"score_mode" : "avg",
"boost" : 1.0
}
},
{
"nested": {
"query": {
"bool": {
"should": [
{
"terms": {
"segment_status.bse_status": [
2,
3
],
"boost": 1
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"path": "segment_status",
"ignore_unmapped": false,
"score_mode": "avg",
"boost": 1
}
}
],
"must_not": [
{
"term": {
"marked_failed_manually": {
"value": true,
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"sort": [
{
"segment_status.updated_at": {
"order": "asc",
"mode": "min",
"nested_filter": {
"term": {
"segment_status.segment_type": {
"value": "CASH",
"boost": 1
}
}
},
"nested_path": "segment_status"
}
}
]
}
I am using rest high level client and getting the above exception but works fine from the postman here is the java source code and this is built using rest high level client.
suspecting the last three sum aggregations are causing the problem please correct me if the order is not correct in the java code query construction.
AggregationBuilder recommendationDataAgg = AggregationBuilders.nested("recommendations", "events.recommendationData");
AggregationBuilder exceptionIdAgg = AggregationBuilders
.filter("exceptionId",
QueryBuilders.termsQuery("events.recommendationData.exceptionId", exactMatchThese));
AggregationBuilder itemAgg = AggregationBuilders.terms("by_item").field("events.recommendationData.item.keyword");
AggregationBuilder destinationAgg = AggregationBuilders.terms("by_destination").field("events.recommendationData.location.keyword");
AggregationBuilder recommendationsAgg = AggregationBuilders.nested("recommendations", "events.recommendationData.recommendations");
AggregationBuilder transportAgg = AggregationBuilders.terms("by_trans").field("events.recommendationData.recommendations.stockTransfer.transportMode.keyword");
AggregationBuilder sourceAgg = AggregationBuilders.terms("by_sourcelocation").field("events.recommendationData.recommendations.stockTransfer.sourceLocation.keyword");
AggregationBuilder shipDateAgg = AggregationBuilders.terms("by_shipdate").field("events.recommendationData.recommendations.stockTransfer.shipDate.keyword");
AggregationBuilder arrivalDateAgg = AggregationBuilders.terms("by_arrival").field("events.recommendationData.recommendations.stockTransfer.arrivalDate.keyword");
AggregationBuilder quantityAgg = AggregationBuilders.sum("quantity").field("events.recommendationData.recommendations.stockTransfer.transferQuantity");
AggregationBuilder transfercostAgg = AggregationBuilders.sum("transfercost").field("events.recommendationData.recommendations.stockTransfer.transferCost");
AggregationBuilder revenueAgg = AggregationBuilders.sum("revenueRecovered").field("events.recommendationData.recommendations.stockTransfer.revenueRecovered");
arrivalDateAgg = recommendationDataAgg.subAggregation(
exceptionIdAgg.subAggregation(itemAgg.subAggregation(destinationAgg.subAggregation(recommendationsAgg
.subAggregation(transportAgg.subAggregation(
sourceAgg.subAggregation(shipDateAgg.subAggregation(arrivalDateAgg))))))));
AggregationBuilder aggregation = arrivalDateAgg
.subAggregation(quantityAgg)
.subAggregation(transfercostAgg)
.subAggregation(revenueAgg);
searchSourceBuilder.aggregation(aggregation);
searchRequest.source(searchSourceBuilder);
This is working query from postman
{
"aggs": {
"recommendations": {
"nested": {
"path": "events.recommendationData"
},
"aggs": {
"exception": {
"filter": {
"terms": {
"events.recommendationData.exceptionId": [
"1"
]
}
},
"aggs": {
"by_item": {
"terms": {
"field": "events.recommendationData.item.keyword"
},
"aggs": {
"by_destination": {
"terms": {
"field": "events.recommendationData.location.keyword"
},
"aggs": {
"recommendations": {
"nested": {
"path": "events.recommendationData.recommendations"
},
"aggs": {
"by_trans": {
"terms": {
"field": "events.recommendationData.recommendations.stockTransfer.transportMode.keyword"
},
"aggs": {
"by_sourcelocation": {
"terms": {
"field": "events.recommendationData.recommendations.stockTransfer.sourceLocation.keyword"
},
"aggs": {
"by_shipdate": {
"terms": {
"field": "events.recommendationData.recommendations.stockTransfer.shipDate.keyword"
},
"aggs": {
"by_arrival": {
"terms": {
"field": "events.recommendationData.recommendations.stockTransfer.arrivalDate.keyword"
},
"aggs": {
"quantity": {
"sum": {
"field": "events.recommendationData.recommendations.stockTransfer.transferQuantity"
}
},
"transferCost": {
"sum": {
"field": "events.recommendationData.recommendations.stockTransfer.transferCost"
}
},
"revenueRecovered": {
"sum": {
"field": "events.recommendationData.recommendations.stockTransfer.revenueRecovered"
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
This is the mapping
{
"mappings": {
"recommendations": {
"properties": {
"events": {
"type": "nested",
"properties": {
"recommendationData": {
"type": "nested",
"properties": {
"recommendations": {
"type": "nested",
"properties": {
"recommendationType": {
"type": "keyword"
}
}
}
}
}
}
}
}
}
}
}
This is the document
{
"clusterId": "1",
"rank": 1,
"events": [
{
"eventId": "2",
"eventType": "Delayed",
"metaInfo": {
"batch_id": "batch_1"
},
"recommendationData": [
{
"exceptionId": "1",
"item": "KitKat",
"location": "DC1",
"dueDate": "2019-01-10T05:30:00.000+0530",
"quantity": 100,
"metaInfo": {
"batch_id": "batch_1",
"dummy_id": "dummy_1"
},
"rank": 1,
"recommendations": [
{
"rank": 1,
"recommendationType": "Out Of Stock",
"customerName": "Walmart",
"stockTransfer": {
"primaryRecommendation": true,
"priority": 1,
"sourceLocation": "DC1",
"transferQuantity": 100,
"metaInfo": 40,
"shipDate": "01/01/2020",
"arrivalDate": "10/01/2020",
"transportMode": "Air",
"transferCost": 1000,
"maxQtyAvailableForTransfer": 40,
"totalQtyAtSource": 1,
"revenueRecovered": 12000
},
"expedite": null
},
{
"rank": 1,
"recommendationType": "Out Of Stock",
"customerName": "Walmart",
"stockTransfer": {
"primaryRecommendation": true,
"priority": 1,
"sourceLocation": "DC1",
"transferQuantity": 100,
"metaInfo": 40,
"shipDate": "01/01/2020",
"arrivalDate": "10/01/2020",
"transportMode": "Air",
"transferCost": 1000,
"maxQtyAvailableForTransfer": 40,
"totalQtyAtSource": 1,
"revenueRecovered": 12000
},
"expedite": null
},
{
"rank": 1,
"recommendationType": "Out Of Stock",
"customerName": "Walmart",
"stockTransfer": {
"primaryRecommendation": true,
"priority": 1,
"sourceLocation": "DC1",
"transferQuantity": 100,
"metaInfo": 40,
"shipDate": "01/01/2020",
"arrivalDate": "10/01/2020",
"transportMode": "Air",
"transferCost": 1000,
"maxQtyAvailableForTransfer": 40,
"totalQtyAtSource": 1,
"revenueRecovered": 12000
},
"expedite": null
}, {
"rank": 1,
"recommendationType": "Out Of Stock",
"customerName": "Walmart",
"stockTransfer": {
"primaryRecommendation": true,
"priority": 1,
"sourceLocation": "DC1",
"transferQuantity": 100,
"metaInfo": 40,
"shipDate": "01/01/2020",
"arrivalDate": "10/01/2020",
"transportMode": "Road",
"transferCost": 1000,
"maxQtyAvailableForTransfer": 40,
"totalQtyAtSource": 1,
"revenueRecovered": 12000
},
"expedite": null
},
{
"rank": 1,
"recommendationType": "Out Of Stock",
"customerName": "Walmart",
"stockTransfer": {
"primaryRecommendation": true,
"priority": 1,
"sourceLocation": "DC1",
"transferQuantity": 100,
"metaInfo": 40,
"shipDate": "01/01/2020",
"arrivalDate": "10/01/2020",
"transportMode": "Rail",
"transferCost": 1000,
"maxQtyAvailableForTransfer": 40,
"totalQtyAtSource": 1,
"revenueRecovered": 12000
},
"expedite": null
}
]
}
]
}
]
}
Indeed, the last three sum aggregations are not added correctly. You need to add them all to the arrivalDateAgg aggregation
...
arrivalDateAgg
.subAggregation(quantityAgg)
.subAggregation(transfercostAgg)
.subAggregation(revenueAgg)
I'm trying to get the counts group by the repetitive items in array without distinct, use aggs terms but not work
GET /my_index/_search
{
"size": 0,
"aggs": {
"keywords": {
"terms": {
"field": "keywords"
}
}
}
}
documents like:
"keywords": [
"value1",
"value1",
"value2"
],
but the result is:
"buckets": [
{
"key": "value1",
"doc_count": 1
},
{
"key": "value2",
"doc_count": 1
}
]
how can i get the result like:
"buckets": [
{
"key": "value1",
"doc_count": 2
},
{
"key": "value2",
"doc_count": 1
}
]
finally I modify the mapping use nested:
"keywords": {
"type": "nested",
"properties": {
"count": {
"type": "integer"
},
"keyword": {
"type": "keyword"
}
}
},
and query:
GET /my_index/_search
{
"size": 0,
"aggs": {
"keywords": {
"nested": {
"path": "keywords"
},
"aggs": {
"keyword_name": {
"terms": {
"field": "keywords.keyword"
},
"aggs": {
"sums": {
"sum": {
"field": "keywords.count"
}
}
}
}
}
}
}
}
result:
"buckets": [{
"key": "value1",
"doc_count": 495,
"sums": {
"value": 609
}
},
{
"key": "value2",
"doc_count": 440,
"sums": {
"value": 615
}
},
{
"key": "value3",
"doc_count": 319,
"sums": {
"value": 421
}
},
...]
I am a novice in Java and I am looking for a way to flatten json documents.
I have tried Object mapper but without success and I have also tried to do with json node but still get no success .
I found this link but the results is not what I need :https://github.com/wnameless/json-flattener
I have also been helped before but the example was too specific and I cannot do the same things because the documents is too long this is why I am looking for a way to make it generic: Flatten json documents in Java
I need to transform "any" json documents like in the example below :
Here is an example of my documents
Documents recieved:
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 10,
"max_score": 0,
"hits": []
},
"aggregations": {
"groupe": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "a",
"doc_count": 1,
"date": {
"buckets": [
{
"key_as_string": "2017-05-03T00:00:00.000Z",
"key": 1493769600000,
"doc_count": 1,
"value": {
"value": 1
}
},
{
"key_as_string": "2017-05-03T01:00:00.000Z",
"key": 1493776800000,
"doc_count": 1,
"value": {
"value": 3
}
}
]
}
},
{
"key": "b",
"doc_count": 4,
"date": {
"buckets": [
{
"key_as_string": "2017-05-03T00:00:00.000Z",
"key": 1493769600000,
"doc_count": 1,
"value": {
"value": 4
}
},
{
"key_as_string": "2017-05-03T01:00:00.000Z",
"key": 1493773200000,
"doc_count": 1,
"value": {
"value": 3
}
}
]
}
}
]
}
}
}
Document Transformed:
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 10,
"max_score": 0,
"hits": []
},
"aggregations": {
"groupe": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "a",
"doc_count": 1,
"date": {
"buckets": [
{
"key_as_string": "2017-05-03T00:00:00.000Z",
"key": 1493769600000,
"doc_count": 1,
"value": {
"value": 1
}
}
]
}
},
{
"key": "a",
"doc_count": 1,
"date": {
"buckets": [
{
"key_as_string": "2017-05-03T02:00:00.000Z",
"key": 1493776800000,
"doc_count": 1,
"value": {
"value": 3
}
}
]
}
},
{
"key": "b",
"doc_count": 1,
"date": {
"buckets": [
{
"key_as_string": "2017-05-03T02:00:00.000Z",
"key": 1493776800000,
"doc_count": 1,
"value": {
"value": 4
}
}
]
}
},
"key": "b",
"doc_count": 1,
"date": {
"buckets": [
{
"key_as_string": "2017-05-03T02:00:00.000Z",
"key": 1493776800000,
"doc_count": 1,
"value": {
"value": 4
}
}
]
}
}
]
}
}
}
{
"items": [
{
"itemId": 1,
"itemName": "OKRA",
"stock": [
{
"stock": 50,
"salePrice": 150
},
{
"stock": 100,
"salePrice": 75
}
]
},
{
"itemId": 2,
"itemName": "CUCUMBER",
"stock": [
{
"stock": 100,
"salePrice": 10
},
{
"stock": 200,
"salePrice": 5
}
]
}
],
"count": 2,
"total_records": 428,
"current_page": 1,
"per_page": 2,
"total_pages": 215
}
Excepted Output should be
{
"items": [
{
"itemId": 1,
"itemName": "OKRA",
"salePrice": 150,
"stock": 50
},
{
"itemId": 1,
"itemName": "OKRA",
"salePrice": 75,
"stock": 100
},
{
"itemId": 2,
"itemName": "CUCUMBER",
"salePrice": 10,
"stock": 100
},
{
"itemId": 2,
"itemName": "CUCUMBER",
"salePrice": 5,
"stock": 200
}
]
}
My spec file
[
{
"operation": "shift",
"spec": {
"items": {
"*": {
"stock": {
"*": {
"#2": "itemsArray[]",
"#": "stockArray[]"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"itemsArray": {
"*": { // bizArray index
"itemId": "items[&1].itemId",
"itemName": "items[&1].itemName",
"stock": "items[&1].stock[&1].stock",
"salePrice": "[&1].stock[&1].salePrice"
}
}
}
}
]
Anyone help me to get the Excepted output.I am new to JOLT.
[
{
"operation": "shift",
"spec": {
"items": {
"*": {
"stock": {
"*": {
"#2": "itemsArray[]",
"#": "stockArray[]"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"itemsArray": {
"*": {
"itemId": "items[&1].itemId",
"itemName": "items[&1].itemName"
}
},
"stockArray": {
"*": {
"stock": "items[&1].stock",
"salePrice": "items[&1].salePrice"
}
}
}
}
]