I have the following aggregation working on mongodb, but I not figured out how to replicate it on Spring Data.
db.pedido.aggregate( [
{ $unwind: "$items" },
{ $project: { itemValue: { $multiply: [ "$items.price", "$items.quantity"] } } },
{ $group: { _id: "$_id", orderValue: { $sum: "$itemValue" } } },
{ $match: { orderValue: { $gte: 150 } } }
] )
I've tried something like this,
Aggregation agg = Aggregation.newAggregation(Order.class,
project("items"),
unwind("items"),
project("items.price", "items.quantity").and("items.price").multiply("items.quantity").as("itemValue"),
group("_id").sum("itemValue").as("orderValue"),
match(where("orderValue").gte(150))
);
return operations.aggregate(agg, Order.class, Order.class).getMappedResults();
but I got the java.lang.IllegalArgumentException: Invalid reference 'price'! What's wrong in the Spring Aggregation? There is any way to use the mongodb aggregation with #Query methods, or so other way to do this?
I am using MongoDB 3.2 and spring-data-mongodb 1.9.1.RELEASE
Related
I need to perform an aggregation operation with 2 lookups that looks something like this:
db.coll1.aggregate([
{
$match : { name: "someName" }
},
{
$lookup: {
from: "coll2",
localField: "_id",
foreignField: "coll1Id",
as: "coll2"
}
},
{ $unwind: "$coll2"},
{
$match: {"coll2.Name": "someName2"}
},
{
$lookup:{
from: "coll3",
localField: "coll2._id",
foreignField: "coll2_id",
as: "coll3"
}
},
{
$unwind: "$coll3"
},
{
$match: {"coll3.someField" : "something"}
},
{
"$project": {
"coll1.name": 1,
"coll2.age": 1,
"coll3.something": 1
}
}
]).pretty();
I'm trying to perform this operation in a repository layer with the following code.
#Aggregation(pipeline = {
"{ $match: { name: ?0 }}",
"{ $lookup:{ from: 'features', localField: '_id', foreignField: 'coll1Id',as: 'coll2'}}",
"{ $unwind: $coll2}",
"{ $match: {"coll2.Name": "someName2"}},
// and so on
})
public ResponseModel myRepoMethod(String name, String name2 , String name3);
and was getting a error when doing the 2nd match (basically while trying to access the nested "coll2.Name" field).
status":500,"error":"Internal Server Error","trace":"java.lang.StackOverflowError\r\n\tat java.base/java.lang.StringBuilder.(StringBuilder.java:106)\r\n\tat java.base/java.util.Formatter$FormatSpecifier.print(Formatter.java:3266)\r\n\tat java.base/java.util.Formatter$FormatSpecifier.print(Formatter.java:3261)\r\n\tat java.base/java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2957)\r\n\tat java.base/java.util.Formatter$FormatSpecifier.print(Formatter.java:2918)\r\n\tat java.base/java.util.Formatter.format(Formatter.java:2689)\r\n\tat java.base/java.util.Formatter.format(Formatter.java:2625)\r\n\tat java.base/java.lang.String.format(String.java:4143)\r\n\tat org.bson.json.JsonParseException.(JsonParseException.java:56)\r\n\tat org.springframework.data.mongodb.util.json.JsonScanner.nextToken(JsonScanner.java:118)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingJsonReader.popToken(ParameterBindingJsonReader.java:725)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingJsonReader.readBsonType(ParameterBindingJsonReader.java:164)\r\n\tat org.bson.AbstractBsonReader.verifyBSONType(AbstractBsonReader.java:679)\r\n\tat org.bson.AbstractBsonReader.checkPreconditions(AbstractBsonReader.java:721)\r\n\tat org.bson.AbstractBsonReader.readStartDocument(AbstractBsonReader.java:449)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:235)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:67)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.readValue(ParameterBindingDocumentCodec.java:371)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:248)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:67)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.readValue(ParameterBindingDocumentCodec.java:371)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:248)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:67)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.readValue(ParameterBindingDocumentCodec.java:371)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:248)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:67)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.readValue(ParameterBindingDocumentCodec.java:371)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:248)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.decode(ParameterBindingDocumentCodec.java:67)\r\n\tat org.springframework.data.mongodb.util.json.ParameterBindingDocumentCodec.readValue(ParameterBindingDocumentCodec.java:371)\r\n\tat
I understand this is happening because Spring isn't able to figure out the nested "coll2.Name" field which would normally be understood by the Mongo shell.
Is there a work around for this?
In Mongo Repository when we define custom query so we use #Query annotion. like below
#Query(value = "{$group:{emp_country:$emp_country,emp_city:$emp_city}},$project:{emp_country:1,emp_city:1,_id=0}").
in query I want to use aggregation method like $group,$project so how to achieve this without using aggregation with mongo template, use only mongo repository.
sample Data:
[
{
"emp_country":"country1",
"emp_city":"city1"
},
{
"emp_country":"country1",
"emp_city":"city1"
},
{
"emp_country":"country1",
"emp_city":"city2"
}
]
expected Output
[
{
"emp_country":"country1",
"emp_city":"city1"
},
{
"emp_country":"country1",
"emp_city":"city2"
}
]
The #Aggregation annotation is available since spring-data-mongodb 2.2
#Aggregation(pipeline = {"{ $group: ... }", "{ $project : ... }")
List<Aggregate> aggregationViaAnnotatedRepositoryMethod();
More details can be found in the Reference Documentation.
I wanna code es aggregation in java. Can I change the query below to java?
... some query
"aggs": {
"ip_address": {
"terms": {
"field": "ip_address"
},
"aggs": {
"dup_docs": {
"top_hits": {
"sort": [
{
"updated_at": {
"order": "desc"
}
}
],
"size": 1
}
}
}
}
}
I think Using the AggregationBuilders provided by elasticsearch, it seems to be, but I'm not sure.Please help me.
Yes, you need to use AggregationBuilders provided in JHLRC and build the queries, you need to use the terms aggregation builder and top tag hits aggregation builder for your aggregation.
I'm having an issue implementing a Filter on a Projection that I have working in the Mongo Shell. I've got a Census object that contains a list of Employees.
{
"_id": "ID",
"name": "census1",
"employees": [ {
"eeId": "EE_ID1"
},
{
"eeId": "EE_ID2"
},
{
"eeId": "EE_ID3"
}
}
Realistically this could contain a lot of employees. So I'd like to be able to retrieve the main Census object, and a subset of employees. I've already implemented 'slice', so this is going to be retrieving a set of employees by their eeId.
This works fine:
db.census.aggregate(
[
{
$match: {
"_id": ObjectId("ID1")
}
},
{
$project: {
"censusName": 1,
"employees" : {
$filter : {
input: "$employees",
as: "employees",
cond: { $in: [ "$$employees.eeId", ["EE_ID1", "EE_ID3"]] }
}
}
}
}
]
).toArray()
The problem is, I can't get it implemented in Java. Here 'employeeIds' is a String of the IDs I want.
MatchOperation matchCensusIdStage = Aggregation.match(new Criteria("id").is(censusId));
ProjectionOperation projectStage = Aggregation.project("censusName")
.and(Filter.filter("employees")
.as("employees")
.by(In.arrayOf(employeeIds).containsValue("employees.eeId")))
.as("employees");
Aggregation aggregation = Aggregation.newAggregation(matchCensusIdStage, projectStage);
return mongoTemplate.aggregate(aggregation, Census.class, Census.class).getMappedResults().get(0);
For this, no results are returned. I've also tried implementing it with a BasicDBObject but got stuck there too.
EDIT (workaround):
I did get a solution using aggregation but not with the filter on the project. This is what I did:
db.parCensus.aggregate(
// Pipeline
[
{
$match: {
"_id": ObjectId("ID1")
}
},
{
$project: {
"_id": 0, "employee": "$employees"
}
},
{
$unwind: "$employee"
},
{
$match: {
"employee.eeId": { $in: ["EE_ID1", "EE_ID3"] }
}
}
]
).toArray()
Java Code:
MatchOperation matchCensusIdStage = Aggregation.match(new Criteria("id").is(censusId));
ProjectionOperation projectStage = Aggregation.project("censusName").and("employees").as("employee");
UnwindOperation unwindStage = Aggregation.unwind("employee");
MatchOperation matchEmployeeIdsStage = Aggregation.match(new Criteria("employee.eeId").in(employeeIds));
Aggregation aggregation = Aggregation.newAggregation(matchCensusIdStage, projectStage, unwindStage, matchEmployeeIdsStage);
I know I could add a $group at the end to put it back into one Census object, but I just created a separate CensusEmployee object to store it all.
The aggregation query posted in the question post works fine. The MongoDB Spring Data API for the aggregation ArrayOperators.In syntax is not clear. I couldn't implement a solution based on this aggregation (and no answers related to on the net).
But, the alternative solution is based on the following aggregation query - and it works fine.
db.collection.aggregate( [
{ $unwind: "$employees" },
{ $match: { "employees.eeId": { $in: ["EE_ID1", "EE_ID3"] } } },
{ $group: { _id: "$_id", name: { $first: "$name" }, employees: { $push: "$employees" } } }
] )
The Java code:
List<String> empsToMatch = Arrays.asList("EE_ID1", "EE_ID3");
MongoOperations mongoOps = new MongoTemplate(MongoClients.create(), "test");
Aggregation agg = newAggregation(
unwind("employees"),
match(Criteria.where("employees.eeId").in(empsToMatch )),
group("_id")
.first("name").as("name")
.push("employees").as("employees")
);
AggregationResults<Document> results = mongoOps.aggregate(agg, "collection", Document.class);
I have this query working, I have the expected results using it in kibana.
GET my_index/_search
{
"query": {
"bool" : {
"must" : {
"match_all" : {}
},
"filter" : {
"geo_shape" : {
"SitePoint" : {
"shape": {
"type": "polygon",
"coordinates" : [
[[18.85491,-33.92305],
[18.8604,-33.9319],
[18.85618,-33.9399],
[18.84809,-33.94153],
[18.85491,-33.92305]]
]
},
"relation": "within"
}
}
}
}
}
}
and I'm trying to build something solid using ElasticsearchRepository with Kotlin or Java, I really don't find much about it on the internet or I don't understand the documentation as I should.
val coor = mutableListOf(org.locationtech.jts.geom.Coordinate(18.85491,-33.92305),
org.locationtech.jts.geom.Coordinate(18.8604,-33.9319),
org.locationtech.jts.geom.Coordinate(18.85618,-33.9399),
org.locationtech.jts.geom.Coordinate(18.84809,-33.94153),
org.locationtech.jts.geom.Coordinate(18.85491,-33.92305))
val query = QueryBuilders
.geoShapeQuery("objects", ShapeBuilders.newPolygon(coor))
.relation(ShapeRelation.WITHIN)
just running this to see the query i have a java.lang.ClassNotFoundException: org.locationtech.spatial4j.exception.InvalidShapeException
Can someone help me?
after a bit of research this way I can use Spring's JPA to query
val filter = QueryBuilders.geoShapeQuery("SitePoint", ShapeBuilders.newPolygon(coordinates)).relation(ShapeRelation.WITHIN)
val searchQuery = NativeSearchQueryBuilder()
.withQuery(matchQuery("id", id))
.withFilter(filter)
.build()
return template.queryForList(searchQuery, MyClass::class.java)