Looking at com.mongodb.reactivestreams.client.MongoCollection interface we see that aggregation can be invoked using list of Bson elements.
public interface MongoCollection<TDocument> {
...
AggregatePublisher<TDocument> aggregate(List<? extends Bson> list);
}
It is clear how to use it when aggregation steps are JSONs (see Example with simple JSONs)
Unfortunately, when any aggregation step contains a function (which is allowed by native MongoDB query), for instance, $accumulator the same approach cannot be applied due to it causes violation of Bson format (org.bson.json.JsonParseException) (see Example with functions)
What is the best way to convert a native MongoDB aggregation query into a result in Java?
(suppose that queries are complex and it is not expedient to rewrite them with Mongo aggregation builders in Java)
Example with simple JSONs:
ReactiveMongoOperations mongo = /* ... */;
var match = BasicDBObject.parse("{ $match: {name: \"Jack\"} }");
var project = BasicDBObject.parse("{ $project: {_id: 0, age: 1, name: 1} }";
var queryParts = List.of(match, project);
Flux<PersonInfo> infoFlux = mongo
.getCollection("person")
.flatMapMany(person -> person.aggregate(queryParts).toFlux())
.map(it -> objectMapper.readValue(it.toJson(), PersonInfo.class))
.collectList()
Example with functions:
// here for conciseness it is just a counting accumulator; generally functions are more complex
var match = BasicDBObject.parse("""
{
$group: {
_id: "$token",
count: {
$accumulator: {
init: function() {
return {owned: 0, total: 0}
},
accumulate: function(state, owner) {
return {
total: state.total + 1
}
},
accumulateArgs: ["$owner"],
merge: function(a, b) {
return {
total: a.total + b.total
}
},
lang: "js"
}
},
minPriceEth: {$min: "$priceEth"}
}
}
""");
Related
So I came up with a query that I'm not really sure how to translate into Spring Data. This is the query:
db.collection.aggregate([{
{
$group: {
_id: "$field",
count: {
$sum: "$count"
},
data: {
"$addToSet": "$$ROOT"
}
}
},
{
$sort: {
count: -1
}
},
{
$limit: 10
},
{
$unwind: "$data"
}
])
The problematic part is unwind. In this raw query I'm putting the original document("$$ROOT") that is being processed into data, so after the set is being processed I can just unwind it and end up with just the documents that I really want. This works fine from the shell. The problem is that I'm not seeing how to do the same operation in Java/Kotlin, I can just add to the set without having a way to reference that data later. Could someone help me out and write how this would look in Java/Kotlin with Spring Data?
You can try this
public List<Object> test() {
Aggregation aggregation = Aggregation.newAggregation(
group("field")
.sum("count").as("count")
.addToSet("$$ROOT").as("data"),
sort(Sort.Direction.DESC, "count"),
limit(10),
unwind("data")
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_COLLECTION.class), Object.class).getMappedResults();
}
I have the following MongoDB query and I dont know how to convert in Spring Data java code, the group and replaceRoot operations.
db.getCollection('operationData').aggregate([
{ $match: type: "OPERATION_CHEAP", amount: {$gte: 1000},
createdAt: {$gte: ISODate("2020-01-24T23:00:00.000Z")}
},
{ $project: { amount: 1, operationId: 1 } },
{ $sort: { amount: -1 } },
{ $group: { _id: '$operationId', g: { $first: {data: '$$ROOT'} }} }, <----
{ $replaceRoot: { newRoot: '$g.data' }}, <------
{ $sort: { amount: 1 } }
])
This is the code for the match operation:
Criteria criterias = new Criteria()
.andOperator(Criteria.where(Operation.AMOUNT)
.gte(minAmount)
.and(Operation.TYPE).is(OperationTypeEnum.CHEAP_OPERATION)
.and("createdAt").gte(startDate).lte(endDate));
MatchOperation matchOperation = Aggregation.match(criterias);
This is the code for the project operation:
ProjectionOperation projectionOperation = Aggregation.project("amount", "operationId");
And this is the Aggregation operation:
Aggregation aggregation = Aggregation.newAggregation(matchOperation, projectionOperation,
sort(direction, "amount"));
AggregationResults<OperationDataVO> aggregate = mongoTemplate.aggregate(aggregation,
COLLECTION, OperationDataVO.class);
I cant find out how to create and integrate the GroupOperation
Try this way:
Aggregation.group("operationId").first("$$ROOT").as("g");
Aggregation.replaceRoot("g");
I tried to write a group aggregation query using the year value from a date object as a key, but for some reason I'm getting this exception.
org.springframework.data.mapping.PropertyReferenceException: No property year(invoiceDate)
Here is the mongo query which I'm trying to replicate:
db.collection.aggregate([
{
$match:
{
"status": "Active"
}
},
{
$group:
{
"_id":{$year:"$invoiceDate"}
}
},
{
$sort:
{
"_id" : -1
}
}
])
And this is my Java implementation:
Aggregation aggregation = Aggregation.newAggregation(
match(new Criteria().andOperator(criteria())),
Aggregation.group("year(invoiceDate)")
).withOptions(newAggregationOptions().allowDiskUse(true).build());
I also didn't find a way how I can apply the sorting on the results from the grouping.
You're basically looking for extractYear() which maps to the $year operator with MongoDB:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(new Criteria().andOperator(criteria())),
Aggregation.project().and("invoiceDate").extractYear().as("_id"),
Aggregation.group("_id"),
Aggregation.sort(Sort.Direction.DESC, "_id)
)
This generally needs to go into a $project in order to make the helpers happy.
If you really want the expression within the $group then you can add a custom operation expression:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(new Criteria().andOperator(criteria())),
new AggregationOperation() {
#Override
public Document toDocument(AggregationOperationContext aggregationOperationContext) {
return new Document("$group",
new Document("_id", new Document("$year","$invoiceDate") )
);
}
},
Aggregation.sort(Sort.Direction.DESC, "_id)
)
I have a query like this (simplified):
db.collection.aggregate([
{ $match: { main_id: ObjectId("58f0f67f50c6af16709fd2c7") } },
{
$group: {
_id: "$name",
count: { $sum: 1 },
sum: { $sum: { $add: ["$P31", "$P32"] } }
}
}
])
I do this query from Java, and I want to map it on my class, but I don't want _id to be mapped on name field. Because if I do something like this:
#JsonProperty("_id")
private String name;
then when I save this data back to mongo (after some modification) the data is saved with name as _id while I want a real Id to be generated.
So, how can I rename _id after $group operation?
You can achieve this by adding a $project stage at the end of your pipeline like this :
{ $project: {
_id: 0,
name: "$_id",
count: 1,
sum: 1
}
}
try it online: mongoplayground.net/p/QpVyh-0I-bP
From mongo v3.4 you could use $addFields in conjunction with $project to avoid to write all the fields in $project that could be very tedious.
This happen in $project because if you include specifically a field, the other fields will be automatically excluded.
Example:
{
$addFields: { my_new_id_name: "$_id" }
},
{
$project: { _id: 0 }
}
db.report.aggregate(
{
$group: {_id: '$name'}
},
{
$project:{
name:"$_id",
_id:false} }
)
Starting in Mongo 4.2, you can use a combination of $set / $unset stages:
// { x: 1, z: "a" }
// { x: 2, z: "b" }
db.collection.aggregate([
{ $set: { y: "$x" } },
{ $unset: "x" }
])
// { y: 1, z: "a" }
// { y: 2, z: "b" }
The $set stage adds the new field to documents and the $unset stage removes/excludes the field to be renamed from documents.
if you are using find method you can't do this, but if you using aggregation it is very easy like this:
db.collectionName.aggregate([
{
$project: {
newName: "$existingKeyName"
}
}
]);
As all of the answers are written the solution in MongoDB query despite the question seeks the solution in Java, posting my approach using Java for posterities.
After the grouping, we can rename the _id fieldname using
Projections.computed("<expected field name>", "$_id")))
To Transform the core part of the query mentioned in the question to Java
Bson mainIdMatch = match(eq("main_id", new ObjectId("58f0f67f50c6af16709fd2c7")));
Bson group = Aggregates.group("$name", Accumulators.sum("count", 1L));
Bson project = Aggregates.project(Projections.fields(Projections.excludeId(),
Projections.computed("name", "$_id")));
reportMongoCollection.aggregate(Arrays.asList(mainIdMatch, group, project))
.into(new ArrayList<>());
To answer specifically, I have added an excerpt from the above code snippet, where I am renaming _id field value as name using Projections.computed("name", "$_id") which map the values of _id which we got as a result of grouping to the field called name. Also, we should exclude the id using Projections.excludeId().
Aggregates.project(Projections.fields(Projections.excludeId(),
Projections.computed("name", "$_id")))
I'm using mongodb to store data for my java program and I have a collection with an array field that has a lot of things in it but i want only to get the length, without all the other data.
Now i'm using this to get it:
((UUID[])document.get("customers")).length
How can I make this not to download all the array?
A possible answer is to create an int that counts the pushes and the pulls of the array but it's not the cleanest method.
You are looking for aggregation framework where you can use the $size operator in your pipeline, this counts and returns the total the number of items in an array:
db.collection.aggregate([
{
"$project": {
"_id": 0, "customer_count": { "$size": "$customers" }
}
}
]);
where the Java equivalent:
DBObject projectFields = new BasicDBObject("_id", 0);
projectFields.put("customer_count", new BasicDBObject( "$size", "$customers" ));
DBObject project = new BasicDBObject("$project", projectFields);
AggregationOutput output = db.getCollection("collectionName").aggregate(project);
System.out.println("\n" + output);
You can use MongoDB's Aggregation Framework to get the size of the array. For example, given the following document structure:
> db.macross.findOne()
{
"_id" : "SDF1",
"crew" : [
"Rick",
"Minmay",
"Roy",
"Max",
"Misa",
"Milia"
]
}
get the size of the array
> db.macross.aggregate(
{ $match: { _id: "SDF1" } },
{ $unwind: "$crew" },
{ $group: { _id: "", count: { $sum: 1 } } },
{ $project: { _id: 0, count: 1 } }
)
{ "count" : 6 }
More detailed and interesting examples are available in the docs.