I'm having an issue implementing a Filter on a Projection that I have working in the Mongo Shell. I've got a Census object that contains a list of Employees.
{
"_id": "ID",
"name": "census1",
"employees": [ {
"eeId": "EE_ID1"
},
{
"eeId": "EE_ID2"
},
{
"eeId": "EE_ID3"
}
}
Realistically this could contain a lot of employees. So I'd like to be able to retrieve the main Census object, and a subset of employees. I've already implemented 'slice', so this is going to be retrieving a set of employees by their eeId.
This works fine:
db.census.aggregate(
[
{
$match: {
"_id": ObjectId("ID1")
}
},
{
$project: {
"censusName": 1,
"employees" : {
$filter : {
input: "$employees",
as: "employees",
cond: { $in: [ "$$employees.eeId", ["EE_ID1", "EE_ID3"]] }
}
}
}
}
]
).toArray()
The problem is, I can't get it implemented in Java. Here 'employeeIds' is a String of the IDs I want.
MatchOperation matchCensusIdStage = Aggregation.match(new Criteria("id").is(censusId));
ProjectionOperation projectStage = Aggregation.project("censusName")
.and(Filter.filter("employees")
.as("employees")
.by(In.arrayOf(employeeIds).containsValue("employees.eeId")))
.as("employees");
Aggregation aggregation = Aggregation.newAggregation(matchCensusIdStage, projectStage);
return mongoTemplate.aggregate(aggregation, Census.class, Census.class).getMappedResults().get(0);
For this, no results are returned. I've also tried implementing it with a BasicDBObject but got stuck there too.
EDIT (workaround):
I did get a solution using aggregation but not with the filter on the project. This is what I did:
db.parCensus.aggregate(
// Pipeline
[
{
$match: {
"_id": ObjectId("ID1")
}
},
{
$project: {
"_id": 0, "employee": "$employees"
}
},
{
$unwind: "$employee"
},
{
$match: {
"employee.eeId": { $in: ["EE_ID1", "EE_ID3"] }
}
}
]
).toArray()
Java Code:
MatchOperation matchCensusIdStage = Aggregation.match(new Criteria("id").is(censusId));
ProjectionOperation projectStage = Aggregation.project("censusName").and("employees").as("employee");
UnwindOperation unwindStage = Aggregation.unwind("employee");
MatchOperation matchEmployeeIdsStage = Aggregation.match(new Criteria("employee.eeId").in(employeeIds));
Aggregation aggregation = Aggregation.newAggregation(matchCensusIdStage, projectStage, unwindStage, matchEmployeeIdsStage);
I know I could add a $group at the end to put it back into one Census object, but I just created a separate CensusEmployee object to store it all.
The aggregation query posted in the question post works fine. The MongoDB Spring Data API for the aggregation ArrayOperators.In syntax is not clear. I couldn't implement a solution based on this aggregation (and no answers related to on the net).
But, the alternative solution is based on the following aggregation query - and it works fine.
db.collection.aggregate( [
{ $unwind: "$employees" },
{ $match: { "employees.eeId": { $in: ["EE_ID1", "EE_ID3"] } } },
{ $group: { _id: "$_id", name: { $first: "$name" }, employees: { $push: "$employees" } } }
] )
The Java code:
List<String> empsToMatch = Arrays.asList("EE_ID1", "EE_ID3");
MongoOperations mongoOps = new MongoTemplate(MongoClients.create(), "test");
Aggregation agg = newAggregation(
unwind("employees"),
match(Criteria.where("employees.eeId").in(empsToMatch )),
group("_id")
.first("name").as("name")
.push("employees").as("employees")
);
AggregationResults<Document> results = mongoOps.aggregate(agg, "collection", Document.class);
Related
I wanna code es aggregation in java. Can I change the query below to java?
... some query
"aggs": {
"ip_address": {
"terms": {
"field": "ip_address"
},
"aggs": {
"dup_docs": {
"top_hits": {
"sort": [
{
"updated_at": {
"order": "desc"
}
}
],
"size": 1
}
}
}
}
}
I think Using the AggregationBuilders provided by elasticsearch, it seems to be, but I'm not sure.Please help me.
Yes, you need to use AggregationBuilders provided in JHLRC and build the queries, you need to use the terms aggregation builder and top tag hits aggregation builder for your aggregation.
I have this query working, I have the expected results using it in kibana.
GET my_index/_search
{
"query": {
"bool" : {
"must" : {
"match_all" : {}
},
"filter" : {
"geo_shape" : {
"SitePoint" : {
"shape": {
"type": "polygon",
"coordinates" : [
[[18.85491,-33.92305],
[18.8604,-33.9319],
[18.85618,-33.9399],
[18.84809,-33.94153],
[18.85491,-33.92305]]
]
},
"relation": "within"
}
}
}
}
}
}
and I'm trying to build something solid using ElasticsearchRepository with Kotlin or Java, I really don't find much about it on the internet or I don't understand the documentation as I should.
val coor = mutableListOf(org.locationtech.jts.geom.Coordinate(18.85491,-33.92305),
org.locationtech.jts.geom.Coordinate(18.8604,-33.9319),
org.locationtech.jts.geom.Coordinate(18.85618,-33.9399),
org.locationtech.jts.geom.Coordinate(18.84809,-33.94153),
org.locationtech.jts.geom.Coordinate(18.85491,-33.92305))
val query = QueryBuilders
.geoShapeQuery("objects", ShapeBuilders.newPolygon(coor))
.relation(ShapeRelation.WITHIN)
just running this to see the query i have a java.lang.ClassNotFoundException: org.locationtech.spatial4j.exception.InvalidShapeException
Can someone help me?
after a bit of research this way I can use Spring's JPA to query
val filter = QueryBuilders.geoShapeQuery("SitePoint", ShapeBuilders.newPolygon(coordinates)).relation(ShapeRelation.WITHIN)
val searchQuery = NativeSearchQueryBuilder()
.withQuery(matchQuery("id", id))
.withFilter(filter)
.build()
return template.queryForList(searchQuery, MyClass::class.java)
I'm using spring to run an agreegation query on mongodb.
i would add some filter for my projection. so i want to write the flow code using spring
db.mycollection.aggregate( [
{$match: {"_id" : { "$binary" : "sU7XGDnFYz53KHVhP+sZlQ==", "$type" : "03" } }},
{$project: {
myarray: {
$filter: {
input: "$myarray",
as: "item",
cond: { $in: [ "$$item._id", ["JshGyMImCsSceiPqqCCinlAtTrIkwvlx", "0000022211"] ] }
}
},
"colomn" :1,
myarray2: {
$filter: {
input: "$myarray2",
as: "item",
cond: { $in: [ "$$item.name", ["name1", "name2"] ] }
}
}
}
}
] );
My attempt writing with Spring Mongo
/*<arrayname,list ids>*/
Map<String, List<Object>> arraysAggregationMap = new HashMap<>();
...
ProjectionOperation projectionOperation = project();
//adding projections
for (String f: fields) {
projectionOperation = projectionOperation.and(f).as(f);
}
// filters
for (Entry<String, List<Object>> entry: arraysAggregationMap.entrySet()
) {
projectionOperation = projectionOperation.and(filter(entry.getKey()).as("item").by(
in("item"+"._id",entry.getValue()).toString())).as(entry.getKey());
}
Aggregation aggregation;
if(fields.size()>0 ||arraysAggregationMap.size() >0) aggregation = newAggregation(match(creteria),projectionOperation);
else aggregation = newAggregation(match(creteria));
AggregationResults results = mongoOperation.aggregate(aggregation,entityClass,Document.class);
But the $filter is not working the same.
The following bson is personaddress collection:
{
"id" : "123456",
"name" : "foo",
"address" : [
{
"local" : "yes",
"location" : [
{
"place" : {
"_id":"VZG",
},
"place_lat" : "18",
"place_lan" : "83",
},
{
"place" : {
"name" : "kerala",
"district" : "palakkad",
"pincode" : "5203689",
},
"place_lat" : "18",
"place_lan" : "83",
}
]
}
]
}
I have an another places collection:
{
"_id":"VZG",
"name" : "vizag",
"district" : "Visakhaptanam,
"pincode" : "568923",
}
Using lookup in mongodb aggregation, I want to embed places collection in personaddress collection
I tried using
Aggregation aggregation = newAggregation(lookup("places", "address.location.place._id", "_id", "myplaces"), unwind("myplaces"));
AggregationResults<OutputDocument> aggResults = mongoTemplate.aggregate(aggregation, PersonAddressDocument.class, OutputDocument.class);
Can anyone help me?
Since you have nested arrays, you need to apply the $unwind operator first in order to denormalise the embedded documents before using the $lookup pipeline (unless you have already flattened them in your aggregation operation):
db.personaddress.aggregate([
{ "$unwind": "$address" },
{ "$unwind": "$address.location" },
{
"$lookup": {
"from": "places",
"localField": "address.location.place._id",
"foreignField": "_id",
"as": "address.location.place",
}
}
])
which can then be implemented as (untested):
LookupOperation lookupOperation = LookupOperation.newLookup()
.from("places")
.localField("address.location.place._id")
.foreignField("_id")
.as("address.location.place");
Aggregation agg = newAggregation(
unwind("address"),
unwind("address.location"),
lookupOperation
);
AggregationResults<OutputDocument> aggResults = mongoTemplate.aggregate(
agg, PersonAddressDocument.class, OutputDocument.class
);
If your Spring Data version does not support this, a workaround is to implement the AggregationOperation interface to take in a DBObject:
public class CustomGroupOperation implements AggregationOperation {
private DBObject operation;
public CustomGroupOperation (DBObject operation) {
this.operation = operation;
}
#Override
public DBObject toDBObject(AggregationOperationContext context) {
return context.getMappedObject(operation);
}
}
Then implement the $lookup operation as a DBObject in the aggregation pipeline:
DBObject lookupOperation = (DBObject)new BasicDBObject(
"$lookup", new BasicDBObject("from", "places")
.append("localField", "address.location.place._id")
.append("foreignField", "_id")
.append("as", "address.location.place")
);
which you can then use as:
Aggregation agg = newAggregation(
unwind("address"),
unwind("address.location"),
lookupOperation
);
AggregationResults<OutputDocument> aggResults = mongoTemplate.aggregate(
agg, PersonAddressDocument.class, OutputDocument.class
);
db.productgroups.aggregate([
// Unwind the source
{ "$unwind": "$products" },
// Do the lookup matching
{ "$lookup": {
"from": "products",
"localField": "products",
"foreignField": "_id",
"as": "productObjects"
}},
// Unwind the result arrays ( likely one or none )
{ "$unwind": "$productObjects" },
// Group back to arrays
{ "$group": {
"_id": "$_id",
"products": { "$push": "$productObjects" },
}}
])
I have the following aggregation working on mongodb, but I not figured out how to replicate it on Spring Data.
db.pedido.aggregate( [
{ $unwind: "$items" },
{ $project: { itemValue: { $multiply: [ "$items.price", "$items.quantity"] } } },
{ $group: { _id: "$_id", orderValue: { $sum: "$itemValue" } } },
{ $match: { orderValue: { $gte: 150 } } }
] )
I've tried something like this,
Aggregation agg = Aggregation.newAggregation(Order.class,
project("items"),
unwind("items"),
project("items.price", "items.quantity").and("items.price").multiply("items.quantity").as("itemValue"),
group("_id").sum("itemValue").as("orderValue"),
match(where("orderValue").gte(150))
);
return operations.aggregate(agg, Order.class, Order.class).getMappedResults();
but I got the java.lang.IllegalArgumentException: Invalid reference 'price'! What's wrong in the Spring Aggregation? There is any way to use the mongodb aggregation with #Query methods, or so other way to do this?
I am using MongoDB 3.2 and spring-data-mongodb 1.9.1.RELEASE