I need your help for using MongoDB aggregation framework with java driver.
I don't understand how to write my request, even with this documentation.
I want to get the 200 oldest views from all items in my collection. Here is my mongo query (which works like I want in console mode):
db.myCollection.aggregate(
{$unwind : "$views"},
{$match : {"views.isActive" : true}},
{$sort : {"views.date" : 1}},
{$limit : 200},
{$project : {"_id" : 0, "url" : "$views.url", "date" : "$views.date"}}
)
Items in this collection have one or many views.
My question is not about the request result, I want to know the java syntaxe.
Finally found the solution, I get the same result than with the original request.
Mongo Driver 3 :
Aggregate doc
MongoCollection<Document> collection = database.getCollection("myCollection");
AggregateIterable<Document> output = collection.aggregate(Arrays.asList(
new Document("$unwind", "$views"),
new Document("$match", new Document("views.isActive", true)),
new Document("$sort", new Document("views.date", 1)),
new Document("$limit", 200),
new Document("$project", new Document("_id", 0)
.append("url", "$views.url")
.append("date", "$views.date"))
));
// Print for demo
for (Document dbObject : output)
{
System.out.println(dbObject);
}
You can make it more readable with static import :
import static com.mongodb.client.model.Aggregates.*;.
See koulini answer for complet example.
Mongo Driver 2 :
Aggregate doc
Iterable<DBObject> output = collection.aggregate(Arrays.asList(
(DBObject) new BasicDBObject("$unwind", "$views"),
(DBObject) new BasicDBObject("$match", new BasicDBObject("views.isActive", true)),
(DBObject) new BasicDBObject("$sort", new BasicDBObject("views.date", 1)),
(DBObject) new BasicDBObject("$limit", 200),
(DBObject) new BasicDBObject("$project", new BasicDBObject("_id", 0)
.append("url", "$views.url")
.append("date", "$views.date"))
)).results();
// Print for demo
for (DBObject dbObject : output)
{
System.out.println(dbObject);
}
Query conversion logic :
Thank to this link
It is worth pointing out, that you can greatly improve the code shown by the answers here, by using the Java Aggregation methods for MongoDB.
Let's take as a code example, the OP's answer to his own question.
AggregateIterable<Document> output = collection.aggregate(Arrays.asList(
new Document("$unwind", "$views"),
new Document("$match", new Document("views.isActive", true)),
new Document("$sort", new Document("views.date", 1)),
new Document("$limit", 200),
new Document("$project", new Document("_id", 0)
.append("url", "$views.url")
.append("date", "$views.date"))
));
We can rewrite the above code as follows;
import static com.mongodb.client.model.Aggregates.*;
AggregateIterable output = collection.aggregate(Arrays.asList(
unwind("$views"),
match(new Document("views.isActive",true)),
sort(new Document("views.date",1)),
limit(200),
project(new Document("_id",0)
.append("url","$views.url")
.append("date","$views.date"))
));
Obviously, you will need the corresponding static import but beyond that, the code in the second example is cleaner, safer (as you don't have to type the operators yourself every time), more readable and more beautiful IMO.
Using previous example as a guide, here's how to do it using mongo driver 3 and up:
MongoCollection<Document> collection = database.getCollection("myCollection");
AggregateIterable<Document> output = collection.aggregate(Arrays.asList(
new Document("$unwind", "$views"),
new Document("$match", new Document("views.isActive", true))
));
for (Document doc : output) {
...
}
Here is a simple way to count employee by departmentId..
Details at: Aggregation using Java API
Map<Long, Integer> empCountMap = new HashMap<>();
AggregateIterable<Document> iterable = getMongoCollection().aggregate(Arrays.asList(
new Document("$match",
new Document("active", Boolean.TRUE)
.append("region", "India")),
new Document("$group",
new Document("_id", "$" + "deptId").append("count", new Document("$sum", 1)))));
iterable.forEach(new Block<Document>() {
#Override
public void apply(final Document document) {
empCountMap.put((Long) document.get("_id"), (Integer) document.get("count"));
}
});
Related
I want to perform an aggregation in Java: here's my attempt
Example of dept collection.
{
"_id" : ObjectId("5d4dc8635dd32dbcba4ae0ae"),
"name" : "Sales"
}
Example of employee_dept collection
{
"_id" : ObjectId("5d5411be6cd7524f36a7933f"),
"dept_id" : ObjectId("5d4dc8635dd32dbcba4ae0ae"),
"employee_id" : ObjectId("5d4dc8635dd32dbcba4ae0af")
}
Example of output expected
{
"_id" :"5d4dc8635dd32dbcba4ae0ae",
"name" : "Sales"
}
Java code
DBObject match = new BasicDBObject("$match", new BasicDBObject("employee_id", "5d4dc8635dd32dbcba4ae0af"));
// build the $lookup operations
DBObject lookupFields = new BasicDBObject("from", "dept");
lookupFields.put("localField", "dept_id");
lookupFields.put("foreignField", "_id");
lookupFields.put("as", "dept");
DBObject lookup = new BasicDBObject("$lookup", lookupFields);
// build the $projection operations
DBObject projectFields = new BasicDBObject("name", 1);
projectFields.put("_id", 1);
DBObject project = new BasicDBObject("$project", projectFields);
List<DBObject> pipeline = Arrays.asList(match, lookup, project);
AggregateIterable aggregateIterable = dbCollection.aggregate(pipeline);
for(Object result: aggregateIterable) {
System.out.println(result);
}
Issue: aggregateIterable is not getting output due to some reason
B) if you don't mind adding how to project for $employee_dept._id and employee_id within the following?
Document project = new Document("$project", new BasicDBObject("name", "$dept.name")
.append("e_id", "$employee_department._id")
.append("employee_id", "$employee_department.employee_id")
.append("dept_id", "$dept._id"));
Issues:
The comparison of employee_id of type ObjectId with a string
In projection, the name and _id are inside 'dept' array and not at
the root level
Fixed code:
Document match = new Document("$match", new Document("employee_id", new ObjectId("5d4dc8635dd32dbcba4ae0af")));
// build the $lookup operations
Document lookupFields = new Document("from", "dept");
lookupFields.put("localField", "dept_id");
lookupFields.put("foreignField", "_id");
lookupFields.put("as", "dept");
Document lookup = new Document("$lookup", lookupFields);
// build unwind operation
Document unwind = new Document("$unwind", "$dept");
// build the $projection operations
Document projectFields = new Document("name", "$dept.name");
projectFields.put("_id", new Document("$toString", "$dept._id"));
Document project = new Document("$project", projectFields);
List<Document> pipeline = Arrays.asList(match, lookup, unwind, project);
AggregateIterable<Document> aggregateIterable = groupDAO.database.getCollection("employee_dept")
.aggregate(pipeline);
for (Document result : aggregateIterable) {
System.out.println(result.toJson());
}
Query in shell window is as follows:
db.labServiceMasters.aggregate(
{$unwind: '$subDepartmentList'},
{$unwind: '$subDepartmentList.labServiceList'},
{$match: {'subDepartmentList.labServiceList._id': '123def'}},
{$project: {_id: 1, labServiceList: ['$subDepartmentList.labServiceList']}}
)
I have converted the above query to Java code as shown below, but I am not getting the result inside labServiceList. And also please let me know how to read the result as my POJO object.
DBObject unwind1 = new BasicDBObject("$unwind" , "$subDepartmentList");
DBObject unwind2 = new BasicDBObject("$unwind" , "$subDepartmentList.labServiceList");
DBObject match = new BasicDBObject("$match", new BasicDBObject("subDepartmentList.labServiceList._id", labServiceId));
DBObject project = new BasicDBObject("$project", new BasicDBObject("_id",1).append("subDepartmentList.labServiceList", 1));
AggregationOutput output=mongoTemplate.getCollection(laboratoryMasterCollection).aggregate(unwind1, unwind2, match, project);
List<DBObject> results = (List<DBObject>) output.results();
for(DBObject response: results){
System.out.println("INside for each loop of Results");
Master master=(Master) response;
System.out.println("master:: "+response.getDepartmentId());
}
I am trying to aggregate values based on group by, match and sort. However, my matching field type is ObjectId. I have an input parameter which is a type of ObjectId(ObjectId settingId), however, below code does not return anything.
Can anyone find the problem in my code?
AggregateIterable < Document > iterable = thermalComfortCollection.aggregate(Arrays.asList(
new Document("$group", new Document("_id", "$Timestamp").append("ThermalComfortList", new Document("$push", "$ThermalComfort"))),
new Document("$match", new Document("settingID", settingId)),
new Document("$sort", new Document("_id", 1))));
You doing a group on the first stage, print that result to check if there's a "settingID" field on the top level.
From you $group stage it seems like the output will be:
[
{
_id : value of $Timestamp,
ThermalComfortList : []},
...
]
When posisble do the $match stage before the $group stage. $match is then able to use the (i hope available) index on settingID
You can use the BasicDBObject as follows:
DBObject match = new BasicDBObject("$match", new BasicDBObject("settingID", new ObjectId("")));
DBObject group = new BasicDBObject("$group", new BasicDBObject("_id", "$Timestamp").append("ThermalComfortList", new BasicDBObject("$push", "$ThermalComfort")));
DBObject group = new BasicDBObject("$sort", new BasicDBObject("_id", 1));
List<DBObject> aggregateList = new ArrayList<DBObject>();
aggregateList.add(match11);
aggregateList.add(group11);
aggregateList.add(group11);
AggregationOutput result = collection.aggregate(aggregateList);
I'm trying to perform an aggregation operation using in Java using the mongo-java-driver. I've performed some other find operations, but I'm unable to do the following aggregation correctly in Java:
db.I1.aggregate([
{ "$match": { "ci": 862222} },
{ "$match": { "gi": { "$ne": null } }},
{ "$group": {
"_id": {
"ci": "$ci",
"gi": "$gi",
"gn": "$gn",
"si": "$si"
}
}},
{ "$group": {
"_id": {
"ci": "$_id.ci",
"gi": "$_id.gi",
"gn": "$_id.gn"
},
"sn": { "$sum": 1 }
}},
{ "$sort" : { "_id.gi" : 1}}
])
I've tried several ways and methods to perform that aggregation in Java, but I'm unable to include the group fields "ci", "gi", "gn","si" correctly in the coll.aggregate(asList()) method. What I got so far, is the following:
MongoCollection<Document> coll = mongo.getCollection("I1");
Document matchCourse = new Document("$match",
new Document("ci", Integer.parseInt(courseid)));
Document matchGroupNotNull = new Document("$match",
new Document("gi", new Document("$ne", null)));
List<Object> list1 = new BasicDBList();
list1.add(new BasicDBObject("ci", "$ci"));
list1.add(new BasicDBObject("gi", "$gi"));
list1.add(new BasicDBObject("gn", "$gn"));
list1.add(new BasicDBObject("si", "$si"));
Document group1 = new Document(
"_id", list1).append("count", new Document("$sum", 1));
List<Object> list2 = new BasicDBList();
list2.add(new BasicDBObject("ci", "$_id.ci"));
list2.add(new BasicDBObject("gi", "$_id.gi"));
list2.add(new BasicDBObject("gn", "$_id.gn"));
Document group2 = new Document(
"_id", list2).append("sn", new Document("$sum", 1));
Document sort = new Document("$sort",
new Document("_id.gi", 1));
AggregateIterable<Document> iterable = coll.aggregate(asList(matchCourse,
matchGroupNotNull, group1, group2, sort));
I know it's not correct, but I included it to give you an idea of what I am doing. I've googled this in many different ways and read several pages, but I didn't find any solution. The available documentation for MongoDB-Java(1, 2) is too short for me and doesn't include this case.
How can I perform that query in Java? Any help would be appreciated.
Thank you very much!!
Finally I've come to a solution. There were some errors in the question that I posted, as it was the last attempt after reaching some point of desperation, but finally, here is the final solution:
MongoDatabase mongo = // initialize your connection;
Document matches = new Document("$match",
new Document("gi", new Document("$ne", null))
.append("ci", Integer.parseInt(courseid)));
Document firstGroup = new Document("$group",
new Document("_id",
new Document("ci", "$ci")
.append("gi", "$gi")
.append("gn", "$gn")
.append("si", "$si"))
.append("count", new Document("$sum", 1)));
Document secondGroup = new Document("$group",
new Document("_id",
new Document("ci", "$_id.ci")
.append("gi", "$_id.gi")
.append("gn", "$_id.gn"))
.append("ns", new Document("$sum", 1)));
Document sort = new Document("$sort",
new Document("_id.gi", 1));
List<Document> pipeline = Arrays.asList(matches, firstGroup,
secondGroup, sort);
AggregateIterable<Document> cursor = mongo.getCollection("I1")
.aggregate(pipeline);
for(Document doc : cursor) { // do stuff with doc }
Instead of trying to create lists of key-values, I just appended the elements to the documents. Hope it will be useful for somebody!!
This question is quite old but was the top match on google when I searched so if anyone is looking for a solution to this I managed to do it in the following way
Aggregation.group(Fields.fields()
.and("field1")
.and("field2"))
.first("name")
.`as`("name")
.count().`as`("count")
This will produce the following MDB query:
{ "$group" :
{ "_id" :
{ "field1" : "$field1", "field2" : "$field2"},
"name" : { "$first" : "$name"},
"count" : { "$sum" : 1}
}
I have a query in MySQL that contains the following condition:
WHERE START_TIME < ? AND START_TIME+DURATION >= ?
How should I migrate this to MongoDB using Java driver and aggregation framework?
The first condition will become:
DBObject match = new BasicDBObject("$match", new BasicDBObject("start_time", "{ $lt : "+timestamp+"}") );
But I'm not sure about the second.
Thanks.
EDIT
I've tried to work with Asya Kamsky answer, this is what I got but it's not working:
BasicDBList dbList = new BasicDBList();
dbList.add("$start_time");
dbList.add("$duration");
DBObject matchLT = new BasicDBObject("$match", new BasicDBObject("start_time", new BasicDBObject("$lt",timestamp)));
DBObject project = new BasicDBObject("$project", new BasicDBObject("end_time", new BasicDBObject("$add", dbList)));
DBObject matchGTE = new BasicDBObject("$match", new BasicDBObject("end_time", new BasicDBObject("$gte",timestamp)));
//GROUP CODE GOES HERE
AggregationOutput output = collection.aggregate(matchLT, project, matchGTE, group);
Here's how you do it in Aggregation Framework, I'm sure you can translate this to Java:
db.collection.aggregate([ {$match: {start_time:{$lt:ISODate("xxxx-xx-xx")}}},
{$project:{end_time:{$add:["$start_time","$duration"]}}},
{$match:{end_time:{$gt:ISODate("yyyy-yy-yy")}}}
] );