Develop Reference

Group aggregation using spring data mongodb

I tried to write a group aggregation query using the year value from a date object as a key, but for some reason I'm getting this exception.
org.springframework.data.mapping.PropertyReferenceException: No property year(invoiceDate)
Here is the mongo query which I'm trying to replicate:
db.collection.aggregate([
{
$match:
{
"status": "Active"
}
},
{
$group:
{
"_id":{$year:"$invoiceDate"}
}
},
{
$sort:
{
"_id" : -1
}
}
])
And this is my Java implementation:
Aggregation aggregation = Aggregation.newAggregation(
match(new Criteria().andOperator(criteria())),
Aggregation.group("year(invoiceDate)")
).withOptions(newAggregationOptions().allowDiskUse(true).build());
I also didn't find a way how I can apply the sorting on the results from the grouping.

You're basically looking for extractYear() which maps to the $year operator with MongoDB:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(new Criteria().andOperator(criteria())),
Aggregation.project().and("invoiceDate").extractYear().as("_id"),
Aggregation.group("_id"),
Aggregation.sort(Sort.Direction.DESC, "_id)
)
This generally needs to go into a $project in order to make the helpers happy.
If you really want the expression within the $group then you can add a custom operation expression:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(new Criteria().andOperator(criteria())),
new AggregationOperation() {
#Override
public Document toDocument(AggregationOperationContext aggregationOperationContext) {
return new Document("$group",
new Document("_id", new Document("$year","$invoiceDate") )
);
}
},
Aggregation.sort(Sort.Direction.DESC, "_id)
)

MongoDB compare document fields in aggregation pipeline

I have a collection of documents like the following:
{ name : "John" ,
age : 25.0 ,
bornIn : "Milan" ,
city : [ {
name : "Roma" ,
state : "IT" ,
mayor : "John"
}]
}
{ name : "Jim" ,
age : 35.0 ,
bornIn : "Madrid" ,
city : [ {
name : "Madrid" ,
state : "ESP" ,
mayor : "Jim"
}]
}
I want to retrieve all the documents that have the field $bornIn equal to the field $city.name. I need to do this as an intermediate stage of a pipeline, so I can't use the $where operator.
I searched online and I found a suggestion to implement something like this:
{ $project:
{ matches:
{ $eq:[ '$bornIn', '$city.name' ] }
}
},
{ $match:
{ matches:true }
} )
But it didn't work neither via shell nor via Java driver as it marks the fields as different.
For the sake of completeness I report my code:
final DBObject eq = new BasicDBObject();
LinkedList eqFields = new LinkedList();
eqFields.add("$bornIn");
eqFields.add("$city.name");
eq.put("$eq", eqFields);
projectFields.put("matches", eq);
final DBObject proj = new BasicDBObject("$project", projectFields);
LinkedList agg = new LinkedList();
agg.add(proj);
final AggregationOutput aggregate = table.aggregate( agg);
Do you have any suggestion? I'm using MongoDB 3.2, and I need to do this via Java Driver.
Thanks!!
PS. It is not relevant but actually the documents above are the output of a $lookup stage among collections "cities" and "persons", with join on $name/$mayor.. it is super cool!! :D :D

I'm a little rusty on how Mongo deals with deep equality searching arrays of objects, but this is definitely doable with $unwind
db.foo.aggregate([
{$unwind: "$city"},
{ $project:
{ matches:
{ $eq:[ '$bornIn', '$city.name' ] }
}
},
{ $match:
{ matches:true }
}
]);
I'm not on a computer with Mongo right now, so my syntax might be off a bit.

Aggregation Framework - Group Operation throw NumberFormatException

I'm using Spring-Data MongoDB aggregation framework.
Here is a code sample:
Aggregation agg = Aggregation.newAggregation(
match(Criteria.where("type").is("PROMO")),
group("locale")//.count().as("counts")
);
AggregationResults<Message> results = mongoTemplate.aggregate(agg, "message", Message.class);
return results.getMappedResults();
Throw:
org.springframework.core.convert.ConversionFailedException: Failed to convert from type java.lang.String to type java.math.BigDecimal for value 'CL'; nested exception is java.lang.NumberFormatException
CL is a value on locale field, but i dont understand why throw that exception.
Im using a similar example from the documentation.
SOLVED:
Aggregation agg = Aggregation.newAggregation(
match(Criteria.where("type").is("PROMO")),
group("created", "text").addToSet("locale").as("countries").addToSet("device.deviceType").as("platforms").count().as("count")
);
I try a simple example on mongo console. After that, map the operations to the builder.
I dont undesrtand why dont work before. If someone can clear the problem will be great.
The model "message":
{ "_id" : "90.0", "device" : { "_id" : "5faf92fd-37f2-4d42-a01a-dd1abce0c1af", "deviceType" : "iPhone", "countryId" : "AR" }, "text" : "Text", "created" : ISODate("2014-01-03T15:56:27.096Z"), "status" : "SENT", "type" : "PROMO" }

My simple answer is avoid using the MongoDB Spring Data classes directly for aggregation and use the standard MongoDB Java objects e.g. DBObject / AggregationOutput. The reason for that is I have lost several hours trying to get anything but basic aggregation queries working in MongoDB Spring data (and that is using the latest which as of today is spring-data-mongodb 1.5.0.RELEASE).
However, constructing aggregation queries using the standard MongoDB Java objects can be a pain (especially if nested / complex) as you end up creating countless DBObject groupFields = new BasicDBObject("_id", null); and the code looks a mess.
I recommend adding the following 3 wrapper methods to your code.
protected DBObject dbObj (String key, Object value) {
return new BasicDBObject (key, value);
}
protected DBObject dbObj (Object ... objs) {
DBObject dbObj = new BasicDBObject();
if (objs.length % 2 == 0) {
for (int i = 0; i < objs.length; i+=2) {
dbObj.put((String)objs[i], objs[i+1]);
}
}
return dbObj;
}
protected DBObject dbList (Object ... objs) {
BasicDBList dbList = new BasicDBList();
for (Object obj : objs) {
dbList.add(obj);
}
return (DBObject)dbList;
}
This enables easy translation between your JSON based queries and your Java code. e.g. if you had the following complex query (taken from http://docs.mongodb.org/manual/tutorial/aggregation-zip-code-data-set/)
db.zipcodes.aggregate(
{
$group: {
_id: { state: "$state", city: "$city" },
pop: { $sum: "$pop" }
}
},{
$sort: { pop: 1 }
},{
$group: {
_id: "$_id.state",
biggestCity: { $last: "$_id.city" },
biggestPop: { $last: "$pop" },
smallestCity: { $first: "$_id.city" },
smallestPop: { $first: "$pop" }
}
},{
$project: {
_id: 0,
state: "$_id",
biggestCity: {
name: "$biggestCity",
pop: "$biggestPop"
},
smallestCity: {
name: "$smallestCity",
pop: "$smallestPop"
}
}
});
... then your Java code would look something like this ...
List<DBObject> aggregation = Arrays.asList (
dbObj ("$group", dbObj (
"_id", dbObj ("state", "$state", "city", "$city"),
"pop", dbObj ("$sum", "$post")
)),
dbObj ("$sort", dbObj ("pop", 1)),
dbObj ("$group", dbObj (
"_id", "$_id.state",
"biggestCity", dbObj ("$last", "$_id.city"),
"biggestPop", dbObj ("$last", "$pop"),
"smallestCity", dbObj ("$first", "$_id.city"),
"smallestPop", dbObj ("$first", "$pop")
)),
dbObj ("$project", dbObj (
"_id", 0,
"state", "$_id",
"biggestCity", dbObj ("name", "$biggestCity", "pop", "$biggestPop"),
"smallestCity", dbObj ("name", "$smallestCity", "pop", "$smallestPop")
))
);
// Run aggregation query
DBCollection collection = mongoTemplate.getCollection(COLLECTION_NAME);
AggregationOutput output = collection.aggregate (aggregation);
Doing it this way, if it works in your editor (e.g. RoboMongo) it will work in your Java code although you will have to manually convert the objects from the result, which isn't too painful i.e.
List<MyResultClass> results = new ArrayList<MyResultClass>();
Iterator<DBObject> it = output.results().iterator();
while (it.hasNext()) {
DBObject obj = it.next();
MyResultClass result = mongoTemplate.getConverter().read(MyResultClass.class, obj);
results.add(result);
}
However, you may find the Spring Data Aggregation stuff does work OK for you. I love Spring and I do use Mongo Spring Data in various parts of my code, it is the aggregation support that lets it down e.g. doing a "$push" inside a "$group" with multiple items doesn't seem to work. I'm sure it will improve with time (and better documentation). Other people have echoed these thoughts e.g. http://movingfulcrum.tumblr.com/post/61693014502/spring-data-and-mongodb-a-mismatch-made-in-hell - see section 4.
Happy coding!

Updating inner doc from mongo

This is my sample doc.
{
"_id" : ObjectId("51f20148a85e39af87510305"),
"group_name" : "sai",
"privileges" : [
"Notification",
"News Letter"
],
"users" : [
{
"full_name" : "sumit",
"user_name" : "sumitdesh",
"password" : "magicmoments",
"status" : "Active"
},
{
"full_name" : "ad",
"user_name" : "asd",
"password" : "asdf",
"status" : "Active"
}
]
}
I want to replace inner doc from users array with a new doc.
This is my java code:
BasicDBObject g1=new BasicDBObject();
g1.put("full_name", "ram");
g1.put("user_name", "ram123");
g1.put("password", "pass$123");
g1.put("status", "Inactive");
BasicDBObject doc=new BasicDBObject();
doc.put("users",g1);
BasicDBObject q=new BasicDBObject("users.user_name","asd");
con.update(q,doc);
Any help is appreciated
Expected output is as follows
I want to replace inner doc with these values
{
"_id" : ObjectId("51f20148a85e39af87510305"),
"group_name" : "sai",
"privileges" : [
"Notification",
"News Letter"
],
"users" : [
{
"full_name" : "sumit",
"user_name" : "sumitdesh",
"password" : "magicmoments",
"status" : "Active"
},
{
"full_name" : "ram",
"user_name" : "ram123",
"password" : "pass$123",
"status" : "Inactive"
}
]
}

I must combine $set and $ operators, then you can update an specific item of array.
BasicDBObject g1 = new BasicDBObject();
g1.put("users.$.full_name", "ram");
g1.put("users.$.user_name", "ram123");
g1.put("users.$.password", "pass$123");
g1.put("users.$.status", "Inactive");
BasicDBObject doc = new BasicDBObject();
doc.put("$set", g1);
BasicDBObject q = new BasicDBObject("users.user_name","asd");
con.update(q,doc);

Your code will create a new document consisting only of the new user. To add a new element to an array in an existing document, use the $push operator
BasicDBObject where = new BasicDBObject("_id", new ObjectId("51f20148a85e39af87510305");
BasicDBObject doc = //... your new user object
BasicDBObject push = new BasicDBObject("$push", doc);
con.update(where, push);
To modify a field of an existing document, you can use the set-operator combined with the $-placeholder. This changes the user_name from "foo" to "bar"
BasicDBObject where = new BasicDBObject("users.user_name", "foo");
BasicDBObject value = new BasicDBObject("users.$.user_name", "bar");
BasicDBObject set = new BasicDBObject("$set", value);
con.update(where, set);
But the least headache-inducing way is to just keep the whole DBObject around when you retrieve an object from the database, mirror all modifications in the DBObject, and then call
con.save(dbObject);
An ORM wrapper library can do this for you. But while it is the easiest way, it isn't the most efficient way, because the whole document will be sent to the database. This is a problem which can be easily filed under "premature optimization" when writes are infrequent and the documents are small, but when you save often and have huge documents, it can become an issue.

This is actually fairly simple, if you follow the documentation.
The first difficulty is finding the document to update. You're looking for the user whose user_name field is 'asd', which is done, rather neatly, through the following query:
{'users.user_name': 'asd'}
The field name needs to be escaped in the mongo shell (it's a compound name) but you need not worry about that in Java.
Now that you've found your user, you need to change it. MongoDB magically stores the matched array index as $, which allows you to write the update criteria as:
{
$set: {
'users.$': {
full_name: 'ram',
user_name: 'ram123',
password: 'pass$123',
status: 'inactive'
}
}
}
You obviously know your way around the Java connector, I'll leave the transformation of my JSON object to instances of BasicDBObject to you.

How aggregate in mongoDB by _id consists of two element(in Java)?

MongoDB looks like:
{ "_id" : ObjectId("503d5024ff9038cdbfcc9da4"),
"employee" : 61,
"department" : "Sales",
"amount" : 77,
"type" : "airfare"
}
In mongoDB console I can aggregate by two params (departments and type):
db.workers.aggregate(]{$group:{_id:{department:"$department",type:"$type"},amount_sum:{$sum:"$amount"}}}])
How do it in java?

You have this working in the shell, the question is how to turn this into Java.
db.workers.aggregate([{$group:{_id:{department:"$department",type:"$type"},
amount_sum:{$sum:"$amount"}}}])
This is very similar to the example in Java tutorial for MongoDB.
The only difference is that they use a simple DBObject for _id part of $group and you need to make a document to use as your _id. Replace the line:
DBObject groupFields = new BasicDBObject( "_id", "$department");
with:
DBObject docFields = new BasicDBObject("department", "$department");
docFields.put("type", "$type");
DBObject groupFields = new BasicDBObject( "_id", docFields);
and you should be all set.