I want to perform an aggregation in Java: here's my attempt
Example of dept collection.
{
"_id" : ObjectId("5d4dc8635dd32dbcba4ae0ae"),
"name" : "Sales"
}
Example of employee_dept collection
{
"_id" : ObjectId("5d5411be6cd7524f36a7933f"),
"dept_id" : ObjectId("5d4dc8635dd32dbcba4ae0ae"),
"employee_id" : ObjectId("5d4dc8635dd32dbcba4ae0af")
}
Example of output expected
{
"_id" :"5d4dc8635dd32dbcba4ae0ae",
"name" : "Sales"
}
Java code
DBObject match = new BasicDBObject("$match", new BasicDBObject("employee_id", "5d4dc8635dd32dbcba4ae0af"));
// build the $lookup operations
DBObject lookupFields = new BasicDBObject("from", "dept");
lookupFields.put("localField", "dept_id");
lookupFields.put("foreignField", "_id");
lookupFields.put("as", "dept");
DBObject lookup = new BasicDBObject("$lookup", lookupFields);
// build the $projection operations
DBObject projectFields = new BasicDBObject("name", 1);
projectFields.put("_id", 1);
DBObject project = new BasicDBObject("$project", projectFields);
List<DBObject> pipeline = Arrays.asList(match, lookup, project);
AggregateIterable aggregateIterable = dbCollection.aggregate(pipeline);
for(Object result: aggregateIterable) {
System.out.println(result);
}
Issue: aggregateIterable is not getting output due to some reason
B) if you don't mind adding how to project for $employee_dept._id and employee_id within the following?
Document project = new Document("$project", new BasicDBObject("name", "$dept.name")
.append("e_id", "$employee_department._id")
.append("employee_id", "$employee_department.employee_id")
.append("dept_id", "$dept._id"));
Issues:
The comparison of employee_id of type ObjectId with a string
In projection, the name and _id are inside 'dept' array and not at
the root level
Fixed code:
Document match = new Document("$match", new Document("employee_id", new ObjectId("5d4dc8635dd32dbcba4ae0af")));
// build the $lookup operations
Document lookupFields = new Document("from", "dept");
lookupFields.put("localField", "dept_id");
lookupFields.put("foreignField", "_id");
lookupFields.put("as", "dept");
Document lookup = new Document("$lookup", lookupFields);
// build unwind operation
Document unwind = new Document("$unwind", "$dept");
// build the $projection operations
Document projectFields = new Document("name", "$dept.name");
projectFields.put("_id", new Document("$toString", "$dept._id"));
Document project = new Document("$project", projectFields);
List<Document> pipeline = Arrays.asList(match, lookup, unwind, project);
AggregateIterable<Document> aggregateIterable = groupDAO.database.getCollection("employee_dept")
.aggregate(pipeline);
for (Document result : aggregateIterable) {
System.out.println(result.toJson());
}
Related
I am trying to aggregate values based on group by, match and sort. However, my matching field type is ObjectId. I have an input parameter which is a type of ObjectId(ObjectId settingId), however, below code does not return anything.
Can anyone find the problem in my code?
AggregateIterable < Document > iterable = thermalComfortCollection.aggregate(Arrays.asList(
new Document("$group", new Document("_id", "$Timestamp").append("ThermalComfortList", new Document("$push", "$ThermalComfort"))),
new Document("$match", new Document("settingID", settingId)),
new Document("$sort", new Document("_id", 1))));
You doing a group on the first stage, print that result to check if there's a "settingID" field on the top level.
From you $group stage it seems like the output will be:
[
{
_id : value of $Timestamp,
ThermalComfortList : []},
...
]
When posisble do the $match stage before the $group stage. $match is then able to use the (i hope available) index on settingID
You can use the BasicDBObject as follows:
DBObject match = new BasicDBObject("$match", new BasicDBObject("settingID", new ObjectId("")));
DBObject group = new BasicDBObject("$group", new BasicDBObject("_id", "$Timestamp").append("ThermalComfortList", new BasicDBObject("$push", "$ThermalComfort")));
DBObject group = new BasicDBObject("$sort", new BasicDBObject("_id", 1));
List<DBObject> aggregateList = new ArrayList<DBObject>();
aggregateList.add(match11);
aggregateList.add(group11);
aggregateList.add(group11);
AggregationOutput result = collection.aggregate(aggregateList);
I have the following working MongoDB aggregation shell command:
db.followrequests.aggregate([{
$match: {
_id: ObjectId("551e78c6de5150da91c78ab9")
}
}, {
$unwind: "$requests"
}, {
$group: {
_id: "$_id",
count: {
$sum: 1
}
}
}]);
Which returns:
{ "_id" : ObjectId("551e78c6de5150da91c78ab9"), "count" : 7 }
I need to implement this in Java, I am trying the following:
List<DBObject> aggregationInput = new ArrayList<DBObject>();
BasicDBObject match = new BasicDBObject();
match.put("$match", new BasicDBObject().put("_id",new ObjectId(clientId)));
aggregationInput.add(match);
BasicDBObject unwind = new BasicDBObject();
unwind.put("$unwind", "$requests");
aggregationInput.add(unwind);
BasicDBObject groupVal = new BasicDBObject();
groupVal.put("_id", "$_id");
groupVal.put("count", new BasicDBObject().put("$sum", 1));
BasicDBObject group = new BasicDBObject();
group.put("$group", groupVal);
aggregationInput.add(group);
AggregationOutput output = followRequestsCol.aggregate(aggregationInput);
for (DBObject result : output.results()) {
System.out.println(result);
}
I am getting an exception:
mongodb the match filter must be an expression in an object.
Can you please help me identify the error in the above code. Thanks!
Try to print the value of aggregationInput and you will realise that .put() does not return a BasicDBObject but just the previous value associated to the key you update. Therefore, when you do:
match.put("$match", new BasicDBObject().put("_id",new ObjectId(clientId)));
You are actually setting $match to null, as new BasicDBObject().put("_id",new ObjectId(clientId)) returns null.
Update you code to something like:
List <DBObject> aggregationInput = new ArrayList <DBObject> ();
BasicDBObject match = new BasicDBObject();
BasicDBObject matchQuery = new BasicDBObject();
matchQuery.put("_id", new ObjectId());
match.put("$match", matchQuery);
aggregationInput.add(match);
BasicDBObject unwind = new BasicDBObject();
unwind.put("$unwind", "$requests");
aggregationInput.add(unwind);
BasicDBObject groupVal = new BasicDBObject();
groupVal.put("_id", "$_id");
groupVal.put("count", new BasicDBObject().put("$sum", 1));
BasicDBObject group = new BasicDBObject();
group.put("$group", groupVal);
aggregationInput.add(group);
AggregationOutput output = followRequestsCol.aggregate(aggregationInput);
for (DBObject result : output.results()) {
System.out.println(result);
}
Or, slightly more readable, use the fluent BasicDBObjectBuilder:
final DBObject match = BasicDBObjectBuilder.start()
.push("$match")
.add("_id", new ObjectId())
.get();
aggregationInput.add(match);
And it should work fine.
Each {} must be new DBObject. Use also .append(key,value) method to make more elegant.
Try this:
List<DBObject> pipeline = new ArrayList<DBObject>(Arrays.asList(
new BasicDBObject("$match", new BasicDBObject("_id",
new ObjectId("551e78c6de5150da91c78ab9"))),
new BasicDBObject("$unwind", "$requests"),
new BasicDBObject("$group",
new BasicDBObject("_id","$_id").append("count", new BasicDBObject("$sum", 1)))));
AggregationOutput output = followRequestsCol.aggregate(pipeline);
for (DBObject result : output.results()) {
System.out.println(result);
}
This is the final working version, based on the above suggestions
// Use mongodb aggregation framework to determine the count of followers
Integer returnCount = 0;
List aggregationInput = new ArrayList();
BasicDBObject match = new BasicDBObject();
BasicDBObject matchQuery = new BasicDBObject();
matchQuery.put("_id", new ObjectId(clientId));
match.put("$match", matchQuery);
aggregationInput.add(match);
BasicDBObject unwind = new BasicDBObject();
unwind.put("$unwind", "$requests");
aggregationInput.add(unwind);
BasicDBObject groupVal = new BasicDBObject();
groupVal.put("_id", null);
BasicDBObject sum = new BasicDBObject();
sum.put("$sum", 1);
groupVal.put("count", sum);
BasicDBObject group = new BasicDBObject();
group.put("$group", groupVal);
aggregationInput.add(group);
AggregationOutput output = followRequestsCol.aggregate(aggregationInput);
for (DBObject result : output.results()) {
returnCount = (Integer) result.get("count");
break;
}
return returnCount;
I need your help for using MongoDB aggregation framework with java driver.
I don't understand how to write my request, even with this documentation.
I want to get the 200 oldest views from all items in my collection. Here is my mongo query (which works like I want in console mode):
db.myCollection.aggregate(
{$unwind : "$views"},
{$match : {"views.isActive" : true}},
{$sort : {"views.date" : 1}},
{$limit : 200},
{$project : {"_id" : 0, "url" : "$views.url", "date" : "$views.date"}}
)
Items in this collection have one or many views.
My question is not about the request result, I want to know the java syntaxe.
Finally found the solution, I get the same result than with the original request.
Mongo Driver 3 :
Aggregate doc
MongoCollection<Document> collection = database.getCollection("myCollection");
AggregateIterable<Document> output = collection.aggregate(Arrays.asList(
new Document("$unwind", "$views"),
new Document("$match", new Document("views.isActive", true)),
new Document("$sort", new Document("views.date", 1)),
new Document("$limit", 200),
new Document("$project", new Document("_id", 0)
.append("url", "$views.url")
.append("date", "$views.date"))
));
// Print for demo
for (Document dbObject : output)
{
System.out.println(dbObject);
}
You can make it more readable with static import :
import static com.mongodb.client.model.Aggregates.*;.
See koulini answer for complet example.
Mongo Driver 2 :
Aggregate doc
Iterable<DBObject> output = collection.aggregate(Arrays.asList(
(DBObject) new BasicDBObject("$unwind", "$views"),
(DBObject) new BasicDBObject("$match", new BasicDBObject("views.isActive", true)),
(DBObject) new BasicDBObject("$sort", new BasicDBObject("views.date", 1)),
(DBObject) new BasicDBObject("$limit", 200),
(DBObject) new BasicDBObject("$project", new BasicDBObject("_id", 0)
.append("url", "$views.url")
.append("date", "$views.date"))
)).results();
// Print for demo
for (DBObject dbObject : output)
{
System.out.println(dbObject);
}
Query conversion logic :
Thank to this link
It is worth pointing out, that you can greatly improve the code shown by the answers here, by using the Java Aggregation methods for MongoDB.
Let's take as a code example, the OP's answer to his own question.
AggregateIterable<Document> output = collection.aggregate(Arrays.asList(
new Document("$unwind", "$views"),
new Document("$match", new Document("views.isActive", true)),
new Document("$sort", new Document("views.date", 1)),
new Document("$limit", 200),
new Document("$project", new Document("_id", 0)
.append("url", "$views.url")
.append("date", "$views.date"))
));
We can rewrite the above code as follows;
import static com.mongodb.client.model.Aggregates.*;
AggregateIterable output = collection.aggregate(Arrays.asList(
unwind("$views"),
match(new Document("views.isActive",true)),
sort(new Document("views.date",1)),
limit(200),
project(new Document("_id",0)
.append("url","$views.url")
.append("date","$views.date"))
));
Obviously, you will need the corresponding static import but beyond that, the code in the second example is cleaner, safer (as you don't have to type the operators yourself every time), more readable and more beautiful IMO.
Using previous example as a guide, here's how to do it using mongo driver 3 and up:
MongoCollection<Document> collection = database.getCollection("myCollection");
AggregateIterable<Document> output = collection.aggregate(Arrays.asList(
new Document("$unwind", "$views"),
new Document("$match", new Document("views.isActive", true))
));
for (Document doc : output) {
...
}
Here is a simple way to count employee by departmentId..
Details at: Aggregation using Java API
Map<Long, Integer> empCountMap = new HashMap<>();
AggregateIterable<Document> iterable = getMongoCollection().aggregate(Arrays.asList(
new Document("$match",
new Document("active", Boolean.TRUE)
.append("region", "India")),
new Document("$group",
new Document("_id", "$" + "deptId").append("count", new Document("$sum", 1)))));
iterable.forEach(new Block<Document>() {
#Override
public void apply(final Document document) {
empCountMap.put((Long) document.get("_id"), (Integer) document.get("count"));
}
});
Need help in creation aggregation query. At this time i have next code:
MongoClient client = new MongoClient();
DB base = client.getDB("school");
DBCollection collection = base.getCollection("students");
DBObject match = new BasicDBObject("$match", new BasicDBObject("scores.type", "homework"));
DBObject unwind = new BasicDBObject("$unwind", "scores");
DBObject sortFields = new BasicDBObject("_id", 1);
sortFields.put("scores.score", -1);
DBObject sort = new BasicDBObject("$sort", sortFields);
List<DBObject> pipeline = Arrays.asList(match, unwind, match, sort);
AggregationOutput output = collection.aggregate(pipeline);
for (DBObject result : output.results()) {
System.out.println(result);
}
But after run console said:
> Command failed with error 15982: 'exception: field path references
> must be prefixed with a '$' ('scores'' on server 127.0.0.1:27017. The full response is { "errmsg" : "exception: field path references must be prefixed with a '$' ('scores'", "code" : 15982, "ok" : 0.0 }
For this I used mongo aggreagtion query as below :
db.students.aggregate([
{ $match: { "scores.type": "homework" }},
{'$unwind' : '$scores' },
{ $match : {"scores.type" : "homework"}},
{$sort : {_id:1, "scores.score":1}}
])
What I'm doing wrong?
Thx yogesh!
The answer is
DBObject unwind = new BasicDBObject("$unwind", "$scores");
I want to use mongo aggregate in Java, but I cannot define $group code as:
In javascript:
$group = {
'_id':null,
'money_bank':{
'$sum':{
'$cond':[{'$eq'=>{'$type':'bank'}},'$amount',0]
}
}
In java:
BasicDBList eqList = new BasicDBList();
eqList.add("$type");
eqList.add("bank");
DBObject eqObject = BasicDBObjectBuilder.start().add("$eq", eqList).get();
BasicDBList condList = new BasicDBList();
condList.add(eqObject);
condList.add("$amount");
condList.add(0);
DBObject conObj = BasicDBObjectBuilder.start().add("$cond", condList).get();
DBObject sumObj = BasicDBObjectBuilder.start().add("$sum", conObj).get();
DBObject moneyObj = BasicDBObjectBuilder.start().add("money_bank", sumObj).get();
DBObject idObj = BasicDBObjectBuilder.start().add("_id", null).get();
BasicDBList groupList = new BasicDBList();
groupList.add(idObj);
groupList.add(moneyObj);
DBObject group = new BasicDBObject("$group", groupList);
But when I executed code, the error:
"errmsg" : "exception: a group's fields must be specified in an object"
Please help me in Java.
You've slightly misunderstood the correct way to turn this into a Java call, two of the places you've used a BasicDBList you actually needed to use a BasicDBObject instead - that's what that error means, it means you need to use an object not a list.
I've made the change to your code, but I haven't tested it against MongoDB, I leave that up to you.
The minimal changes needed to give you the equivalent of the Javascript is:
BasicDBObject eqArgs = new BasicDBObject("$type", "bank");
DBObject eqObject = BasicDBObjectBuilder.start().add("$eq", eqArgs).get();
BasicDBList condList = new BasicDBList();
condList.add(eqObject);
condList.add("$amount");
condList.add(0);
DBObject conObj = BasicDBObjectBuilder.start().add("$cond", condList).get();
DBObject sumObj = BasicDBObjectBuilder.start().add("$sum", conObj).get();
BasicDBObject groupArgs = new BasicDBObject();
groupArgs.append("_id", null);
groupArgs.append("money_bank", sumObj);
DBObject group = new BasicDBObject("$group", groupArgs);
// Using an assert to check it's correct, remove in production code
Assert.assertThat(group.toString(), is("{ \"$group\" : { \"_id\" : null , " +
"\"money_bank\" : { " +
"\"$sum\" : { " +
"\"$cond\" : [ { \"$eq\" : { \"$type\" : \"bank\"}} , " +
"\"$amount\" , 0]" +
"}" +
"}" +
"}" +
"}"));
However you can simplify it even more (although I admit the Java is still uglier than the JavaScript):
BasicDBList condList = new BasicDBList();
condList.add(new BasicDBObject("$eq", new BasicDBObject("$type", "bank")));
condList.add("$amount");
condList.add(0);
DBObject sumObj = new BasicDBObject("$sum", new BasicDBObject("$cond", condList));
DBObject group = new BasicDBObject("$group", new BasicDBObject("_id", null).append("money_bank", sumObj));
For some reason the BasicDBObjectBuilder is more verbose than using BasicDBObject.