The following bson is personaddress collection:
{
"id" : "123456",
"name" : "foo",
"address" : [
{
"local" : "yes",
"location" : [
{
"place" : {
"_id":"VZG",
},
"place_lat" : "18",
"place_lan" : "83",
},
{
"place" : {
"name" : "kerala",
"district" : "palakkad",
"pincode" : "5203689",
},
"place_lat" : "18",
"place_lan" : "83",
}
]
}
]
}
I have an another places collection:
{
"_id":"VZG",
"name" : "vizag",
"district" : "Visakhaptanam,
"pincode" : "568923",
}
Using lookup in mongodb aggregation, I want to embed places collection in personaddress collection
I tried using
Aggregation aggregation = newAggregation(lookup("places", "address.location.place._id", "_id", "myplaces"), unwind("myplaces"));
AggregationResults<OutputDocument> aggResults = mongoTemplate.aggregate(aggregation, PersonAddressDocument.class, OutputDocument.class);
Can anyone help me?
Since you have nested arrays, you need to apply the $unwind operator first in order to denormalise the embedded documents before using the $lookup pipeline (unless you have already flattened them in your aggregation operation):
db.personaddress.aggregate([
{ "$unwind": "$address" },
{ "$unwind": "$address.location" },
{
"$lookup": {
"from": "places",
"localField": "address.location.place._id",
"foreignField": "_id",
"as": "address.location.place",
}
}
])
which can then be implemented as (untested):
LookupOperation lookupOperation = LookupOperation.newLookup()
.from("places")
.localField("address.location.place._id")
.foreignField("_id")
.as("address.location.place");
Aggregation agg = newAggregation(
unwind("address"),
unwind("address.location"),
lookupOperation
);
AggregationResults<OutputDocument> aggResults = mongoTemplate.aggregate(
agg, PersonAddressDocument.class, OutputDocument.class
);
If your Spring Data version does not support this, a workaround is to implement the AggregationOperation interface to take in a DBObject:
public class CustomGroupOperation implements AggregationOperation {
private DBObject operation;
public CustomGroupOperation (DBObject operation) {
this.operation = operation;
}
#Override
public DBObject toDBObject(AggregationOperationContext context) {
return context.getMappedObject(operation);
}
}
Then implement the $lookup operation as a DBObject in the aggregation pipeline:
DBObject lookupOperation = (DBObject)new BasicDBObject(
"$lookup", new BasicDBObject("from", "places")
.append("localField", "address.location.place._id")
.append("foreignField", "_id")
.append("as", "address.location.place")
);
which you can then use as:
Aggregation agg = newAggregation(
unwind("address"),
unwind("address.location"),
lookupOperation
);
AggregationResults<OutputDocument> aggResults = mongoTemplate.aggregate(
agg, PersonAddressDocument.class, OutputDocument.class
);
db.productgroups.aggregate([
// Unwind the source
{ "$unwind": "$products" },
// Do the lookup matching
{ "$lookup": {
"from": "products",
"localField": "products",
"foreignField": "_id",
"as": "productObjects"
}},
// Unwind the result arrays ( likely one or none )
{ "$unwind": "$productObjects" },
// Group back to arrays
{ "$group": {
"_id": "$_id",
"products": { "$push": "$productObjects" },
}}
])
Related
I'm new in spring and I have this situation..
This is my Json.
{
"_id": {
"$oid": "60ba776d3ef89419f8668333"
},
"reference": "20210906164455",
"transactionReference": "999999999999",
"status": "PARTIALLY",
"currency": "BRL",
"amount": {
"$numberDecimal": "99.80"
},
"ucode": "XXXXXXXXXXXXXXXXXXX",
"refunds": [
{
"_id": {
"$oid": "60ba77f03ef89419f8668337"
},
"currency": "BRL",
"amount": {
"$numberDecimal": "1.10"
},
"status": "PARTIALLY",
"createDate": {
"$date": "2021-06-04T18:58:57.145Z"
}
},
{
"_id": {
"$oid": "60ba7b6d3ef89419f8668339"
},
"currency": "BRL",
"amount": {
"$numberDecimal": "10.10"
},
"status": "PARTIALLY",
"createDate": {
"$date": "2021-06-04T19:13:49.229Z"
}
}
],
"confirmed": true,
"createDate": {
"$date": "2021-09-01T00:56:45.235Z"
},
"lastModifiedDate": {
"$date": "2021-09-04T19:15:57.787Z"
},
"amountRefunded": {
"$numberDecimal": "21.30"
}
}
I made this query
db.collection.aggregate([
{
"$addFields": {
"refunds": {
"$concatArrays": [
"$refunds",
[
{
"amount": "$amount",
"createDate": "$createDate"
}
]
]
}
}
},
{
"$unwind": {
"path": "$refunds"
}
},
{
"$replaceRoot": {
"newRoot": {
"$mergeObjects": [
"$$ROOT",
"$refunds"
]
}
}
},
{
"$unset": [
"refunds"
]
},
{
"$sort": {
"createDate": -1
}
},
{
"$limit": 10
}
])
And now I have two objects like I would like. Pratical example
So now I need to transfer this code to Java using Aggregation.
I made this implementation but the problem is... I'm losting the other values and only the values inside of array refunds appears.
AggregationOperation match = Aggregation.match(criteria);
AggregationOperation unwind = Aggregation.unwind("refunds");
AggregationOperation sort = Aggregation.sort(Sort.Direction.DESC, "createDate");
AggregationOperation replaceRoot = Aggregation.replaceRoot("refunds");
AggregationOperation limit = Aggregation.limit(20);
Aggregation aggregation = Aggregation.newAggregation(match, unwind, sort, replaceRoot, limit);
List<Payments> paymentRefunds = mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Payments.class), Payments.class).getMappedResults();
How can I reply the Mongo query to Aggregation?
So... I solved the problem with the code below!
AggregationOperation match = Aggregation.match(criteria);
AggregationOperation unwind = new UnwindOperation(Fields.field(refunds),true);
AggregationOperation sort = Aggregation.sort(Sort.Direction.DESC, createDate);
AggregationOperation limit = Aggregation.limit(request.getRows());
AggregationOperation replaceRoot = ReplaceRootOperation.builder().withDocument(new Document("$mergeObjects", Arrays.asList(Aggregation.ROOT, "$refunds")));
AggregationOperation addFieldAndConcatArrays = aoc -> new Document("$addFields", new Document(refunds, ArrayOperators.ConcatArrays.arrayOf(List.of(new Document(amount, "$amount"))).concat("$refunds").toDocument(aoc)));
AggregationOperation unset = aoc -> new Document("$unset", refunds);
Aggregation aggregation = Aggregation.newAggregation(match, addFieldAndConcatArrays, unwind, replaceRoot, unset, sort, limit);
return mongoTemplate.aggregate(aggregation, Payment.class, Payment.class).getMappedResults();
I'm having an issue implementing a Filter on a Projection that I have working in the Mongo Shell. I've got a Census object that contains a list of Employees.
{
"_id": "ID",
"name": "census1",
"employees": [ {
"eeId": "EE_ID1"
},
{
"eeId": "EE_ID2"
},
{
"eeId": "EE_ID3"
}
}
Realistically this could contain a lot of employees. So I'd like to be able to retrieve the main Census object, and a subset of employees. I've already implemented 'slice', so this is going to be retrieving a set of employees by their eeId.
This works fine:
db.census.aggregate(
[
{
$match: {
"_id": ObjectId("ID1")
}
},
{
$project: {
"censusName": 1,
"employees" : {
$filter : {
input: "$employees",
as: "employees",
cond: { $in: [ "$$employees.eeId", ["EE_ID1", "EE_ID3"]] }
}
}
}
}
]
).toArray()
The problem is, I can't get it implemented in Java. Here 'employeeIds' is a String of the IDs I want.
MatchOperation matchCensusIdStage = Aggregation.match(new Criteria("id").is(censusId));
ProjectionOperation projectStage = Aggregation.project("censusName")
.and(Filter.filter("employees")
.as("employees")
.by(In.arrayOf(employeeIds).containsValue("employees.eeId")))
.as("employees");
Aggregation aggregation = Aggregation.newAggregation(matchCensusIdStage, projectStage);
return mongoTemplate.aggregate(aggregation, Census.class, Census.class).getMappedResults().get(0);
For this, no results are returned. I've also tried implementing it with a BasicDBObject but got stuck there too.
EDIT (workaround):
I did get a solution using aggregation but not with the filter on the project. This is what I did:
db.parCensus.aggregate(
// Pipeline
[
{
$match: {
"_id": ObjectId("ID1")
}
},
{
$project: {
"_id": 0, "employee": "$employees"
}
},
{
$unwind: "$employee"
},
{
$match: {
"employee.eeId": { $in: ["EE_ID1", "EE_ID3"] }
}
}
]
).toArray()
Java Code:
MatchOperation matchCensusIdStage = Aggregation.match(new Criteria("id").is(censusId));
ProjectionOperation projectStage = Aggregation.project("censusName").and("employees").as("employee");
UnwindOperation unwindStage = Aggregation.unwind("employee");
MatchOperation matchEmployeeIdsStage = Aggregation.match(new Criteria("employee.eeId").in(employeeIds));
Aggregation aggregation = Aggregation.newAggregation(matchCensusIdStage, projectStage, unwindStage, matchEmployeeIdsStage);
I know I could add a $group at the end to put it back into one Census object, but I just created a separate CensusEmployee object to store it all.
The aggregation query posted in the question post works fine. The MongoDB Spring Data API for the aggregation ArrayOperators.In syntax is not clear. I couldn't implement a solution based on this aggregation (and no answers related to on the net).
But, the alternative solution is based on the following aggregation query - and it works fine.
db.collection.aggregate( [
{ $unwind: "$employees" },
{ $match: { "employees.eeId": { $in: ["EE_ID1", "EE_ID3"] } } },
{ $group: { _id: "$_id", name: { $first: "$name" }, employees: { $push: "$employees" } } }
] )
The Java code:
List<String> empsToMatch = Arrays.asList("EE_ID1", "EE_ID3");
MongoOperations mongoOps = new MongoTemplate(MongoClients.create(), "test");
Aggregation agg = newAggregation(
unwind("employees"),
match(Criteria.where("employees.eeId").in(empsToMatch )),
group("_id")
.first("name").as("name")
.push("employees").as("employees")
);
AggregationResults<Document> results = mongoOps.aggregate(agg, "collection", Document.class);
I'm using spring to run an agreegation query on mongodb.
i would add some filter for my projection. so i want to write the flow code using spring
db.mycollection.aggregate( [
{$match: {"_id" : { "$binary" : "sU7XGDnFYz53KHVhP+sZlQ==", "$type" : "03" } }},
{$project: {
myarray: {
$filter: {
input: "$myarray",
as: "item",
cond: { $in: [ "$$item._id", ["JshGyMImCsSceiPqqCCinlAtTrIkwvlx", "0000022211"] ] }
}
},
"colomn" :1,
myarray2: {
$filter: {
input: "$myarray2",
as: "item",
cond: { $in: [ "$$item.name", ["name1", "name2"] ] }
}
}
}
}
] );
My attempt writing with Spring Mongo
/*<arrayname,list ids>*/
Map<String, List<Object>> arraysAggregationMap = new HashMap<>();
...
ProjectionOperation projectionOperation = project();
//adding projections
for (String f: fields) {
projectionOperation = projectionOperation.and(f).as(f);
}
// filters
for (Entry<String, List<Object>> entry: arraysAggregationMap.entrySet()
) {
projectionOperation = projectionOperation.and(filter(entry.getKey()).as("item").by(
in("item"+"._id",entry.getValue()).toString())).as(entry.getKey());
}
Aggregation aggregation;
if(fields.size()>0 ||arraysAggregationMap.size() >0) aggregation = newAggregation(match(creteria),projectionOperation);
else aggregation = newAggregation(match(creteria));
AggregationResults results = mongoOperation.aggregate(aggregation,entityClass,Document.class);
But the $filter is not working the same.
I am facing difficulty in retrieving the nested document object of another nested list. Please help me to resolve the same. My mongoDB document is as follows:
{
"_id" : "PT5",
"departmentId" : "DEPT5",
"subDepartmentList" : [
{
"subDepartmentId" : "SUBDEPT19",
"subDepartmentName" : "X-Ray",
"labServiceList" : [
{
"_id" : "123abc",
"subDepartmentId" : "SUBDEPT19",
"labServiceName" : "serviceOne"
},
{
"_id" : "123def",
"subDepartmentId" : "SUBDEPT19",
"labServiceName" : "hello",
}
]
},
{
"subDepartmentId" : "SUBDEPT21",
"subDepartmentName" : "Haemotology",
"labServiceList" : [
{
"_id" : "456abc",
"subDepartmentId" : "SUBDEPT21",
"labServiceName" : "abcd",
}
]
}
]
}
From the above document I want to retrieve only one object of labServiceList by using its _id value(Ex: "_id" : "123abc" in this document). And I don't want to get any other fields apart from the matching nested document. I have tried with the below query:
db.labServiceMasters.aggregate([
{"$project": {
"subDepartmentList": {"$filter": {
"input": '$subDepartmentList.labServiceList',
"as": 'labServiceList',
"cond": {"$eq": ['$$labServiceList._id', '123abc']}
}},
"_id": 0
}}
])
Also I have tried using $map operator, but nothing goes in my way. Please help me to resolve this problem. And also please help me to write the query for the same using mongoTemplate in Java. Any suggestions would be appreciable. Thanks in advance :-)
You actually need to nest a $map inside the $filter and another $filter inside the $map. And use $arrayElemAt to get the single entries:
db.labServiceMasters.aggregate([
{ "$project": {
"subDepartmentList": {
"$arrayElemAt": [
{ "$filter": {
"input": {
"$map": {
"input": "$subDepartmentList",
"as": "sd",
"in": {
"$arrayElemAt": [
{ "$filter": {
"input": "$$sd.labServiceList",
"as": "ls",
"cond": { "$eq": [ "$$ls._id", "123abc" ] }
}},
0
]
}
}
},
"as": "sd",
"cond": { "$ne": [ "$$sd", null ] }
}},
0
]
}
}}
])
Returns:
{
"_id" : "PT5",
"subDepartmentList" : {
"_id" : "123abc",
"subDepartmentId" : "SUBDEPT19",
"labServiceName" : "serviceOne"
}
}
Which for spring-mongodb is:
Aggregation aggregation = newAggregation(
project("subDepartmentList").and(new AggregationExpression() {
#Override
public DBObject toDbObject(AggregationOperationContext context) {
return new BasicDBObject(
"$arrayElemAt", Arrays.asList(
new BasicDBObject("$filter",
new BasicDBObject("input",
new BasicDBObject("$map",
new BasicDBObject("input","$subDepartmentList")
.append("as","sd")
.append("in",new BasicDBObject(
"$arrayElemAt", Arrays.asList(
new BasicDBObject("$filter",
new BasicDBObject("input","$$sd.labServiceList")
.append("as","ls")
.append("cond", new BasicDBObject("$eq", Arrays.asList("$$ls._id","123abc")))
),
0
)
))
)
)
.append("as","sd")
.append("$ne", Arrays.asList("$$sd", null))
),
0
)
);
}
}).as("subDepartmentList")
);
And serializes the same:
{
"aggregate": "labServiceMasters",
"pipeline": [
{
"$project": {
"subDepartmentList": {
"$arrayElemAt": [
{
"$filter": {
"input": {
"$map": {
"input": "$subDepartmentList",
"as": "sd",
"in": {
"$arrayElemAt": [
{
"$filter": {
"input": "$$sd.labServiceList",
"as": "ls",
"cond": {
"$eq": [
"$$ls._id",
"123abc"
]
}
}
},
0.0
]
}
}
},
"as": "sd",
"$ne": [
"$$sd",
null
]
}
},
0.0
]
}
}
}
]
}
How to create the following nested agg query with ES JAVA API
Let's say my query is looking like the following :
GET /agg_vitaly_test_api_2016-11-01/_search
{
"size": 0,
"query": {
"range": {
"time": {
"gte": "1477962000000",
"lte": "1477965600000"
}
}
},
"aggs" : {
"group_by_time" : {
"date_histogram" : {
"field" : "time",
"interval" : "hour"
},
"aggs": {
"sum_player_load": {
"sum": {
"field": "playerload"
}
}
}
}
}
the java code should be somethink like this :
SearchRequestBuilder searchRequestBuilder = FETCH_CLIENT.prepareSearch().setIndices(indexName).setTypes(pixelType.getType()).setSize(0);
// here need to complete the nested aggregation....
AggregationBuilder aggb = addAggregation(groupBy);
searchRequestBuilder.addAggregation(aggb);
// ....
SearchResponse res = searchRequestBuilder.execute().actionGet();
please any suggestions ? :)
BR
Your addAggregation method would be like:
public AggregationBuilder addAggregation(){
return AggregationBuilders.dateHistogram("group_by_time").field("time").interval(DateHistogram.Interval.HOUR)
.subAggregation(AggregationBuilders.sum("sum_player_load").field("playerload"));
}