I'm using mongodb to store data for my java program and I have a collection with an array field that has a lot of things in it but i want only to get the length, without all the other data.
Now i'm using this to get it:
((UUID[])document.get("customers")).length
How can I make this not to download all the array?
A possible answer is to create an int that counts the pushes and the pulls of the array but it's not the cleanest method.
You are looking for aggregation framework where you can use the $size operator in your pipeline, this counts and returns the total the number of items in an array:
db.collection.aggregate([
{
"$project": {
"_id": 0, "customer_count": { "$size": "$customers" }
}
}
]);
where the Java equivalent:
DBObject projectFields = new BasicDBObject("_id", 0);
projectFields.put("customer_count", new BasicDBObject( "$size", "$customers" ));
DBObject project = new BasicDBObject("$project", projectFields);
AggregationOutput output = db.getCollection("collectionName").aggregate(project);
System.out.println("\n" + output);
You can use MongoDB's Aggregation Framework to get the size of the array. For example, given the following document structure:
> db.macross.findOne()
{
"_id" : "SDF1",
"crew" : [
"Rick",
"Minmay",
"Roy",
"Max",
"Misa",
"Milia"
]
}
get the size of the array
> db.macross.aggregate(
{ $match: { _id: "SDF1" } },
{ $unwind: "$crew" },
{ $group: { _id: "", count: { $sum: 1 } } },
{ $project: { _id: 0, count: 1 } }
)
{ "count" : 6 }
More detailed and interesting examples are available in the docs.
Related
I am new to mongodb and aggregation framework.
We have a class UserMetaData and a list of UserMetaData. I need to fetch data according to the userMetaDataList that is passed to the method solve().
Currently I am iterating the list and one by one fetching the corresponding collection from the monogdb. Since the db calls are made for each element in the list, this becomes a highly expensive operation.
Is there any way to fetch all the required data from mongodb in one shot(more like a bulk fetch operation).
mongodb - perform batch query the solution provided in this does not fulfill the requirements of the current scenario.
Please help!!
This is how I am doing currently.
class UserMetaData{
String userId;
String vehicleId;
String vehicleColour;
String orderId;
}
public List<String> getOrderIds(List<UserMetaData> userMetaDataList) {
List<String> orderIds = new ArrayList<>();
for (UserMetaData userMetadata : userMetaDataList) {
try {
BasicDBObject matchDBObject = new BasicDBObject("user_id", new BasicDBObject("$eq", userMetadata.getUserId()));
matchDBObject.append("vehicle_id", new BasicDBObject("$eq", userMetadata.getVehicleID()));
matchDBObject.append("vehicle_colour", new BasicDBObject("$in", ImmutableSet.of("WHITE", "BLACK")));
Document document = eventCollection.find(matchDBObject)
.projection(new BasicDBObject("order_id", "1"))
.first();
orderIds.add(document.get("order_id").toString());
} catch (Exception e) {
log.info("Exception occurred while fetching order id for user_id: {} asset_id:{} - {}", metadata.getUserId(), metadata.getAssetID(), e);
}
}
return ordersIds;
}
I want to fetch all the corresponding data in a single query.
Requesting help.
You can join all filters with $OR condition and fetch the full list at once ...
I want to fetch all the corresponding data in a single query.
You can use this approach and perform the query as a single operation (avoids the for-loop).
Consider sample documents in the collection test:
{ "_id" : ObjectId("621762e2cda7c6394d557f37"), "userid" : 1, "name" : "ijk", "orderid" : "11" }
{ "_id" : ObjectId("621762efcda7c6394d557f38"), "userid" : 12, "name" : "abc", "orderid" : "99" }
{ "_id" : ObjectId("621762fccda7c6394d557f39"), "userid" : 13, "name" : "xyz", "orderid" : "100" }
The array of objects to filter:
var DOCS = [
{ userid: 12, name: "abc" },
{ userid: 13, name: "xyz" }
]
The query to filter by DOCS:
db.test.find(
{
$expr: {
$in: [ { userid: "$userid", name: "$name" }, DOCS ]
}
},
{
orderid: 1
}
)
The output has documents with userids 12 and 13.
[ EDIT - ADD ]
This aggregation an improvement over the find:
db.test.aggregate([
// This matches the 'userid' and 'name' fields with the input list 'DOCS'
{
$match: {
$expr: {
$in: [ { userid: "$userid", name: "$name" }, DOCS ]
}
}
},
// The grouping will select only the first matching for the 'userid' and 'name'
// (this is as per the question post's code: `.first()`)
{
$group: {
_id: {
userid: "$userid",
name: "$name"
},
orderid: {
$first: "$orderid"
}
}
},
// Remove the '_id' field
// Now the result has just the 'orderid' field only
{
$unset: "_id"
}
])
Trying to convert the following mongo query but cant find a solution to trim and toLower using Spring Aggregation queries
db.qabr.aggregate([
{ $match: {name: { "$ne": '' } }},
{ $group: {
_id: { 'name': { input: { $trim: { input: { $toLower : '$name' }}}}, 'qabrNumber': '$qabrNumber', 'qabristaan': '$qabristaan' }, // can be grouped on multiple properties
count: { "$sum": 1 }
}},
{ $match: {
count: { "$gt": 1 } // Duplicates considered as count greater than one
}
}
]
)
If I do the following:
AddFieldsOperation trimAndLowerName = Aggregation.addFields()
.addFieldWithValue("name", StringOperators.Trim.valueOf(
StringOperators.ToLower.lowerValueOf("name")
))
.build();
I keep getting 168 (InvalidPipelineOperator): 'Unrecognized expression '$trim'' on server
However running just trim or just toLower works:
AddFieldsOperation trimName = Aggregation.addFields()
.addFieldWithValue("name", StringOperators.ToLower.lowerValueOf("name"))
.build();
I am moving a Mongodb query from Javascript to Java. The object format is as follows:
{
"record": {
"unknownName1": {
"count": 5,
"domain": "domain1"
}, {
...
}, {
"unknownNameN": {
"count": 3,
"domain": "domainN"
}
}
}
The Javascript query has the following portion:
[
{
$project: {
record: {
$objectToArray: "$record"
}
}
}, {
$unwind: { "$record"
}, {
$group: {
device: "$record.k"
},
count: {
$sum: "$record.v.count"
},
domain: {
$min: "$record.v.domain"
}
}
]
I have translated the above to use the Mongodb Java Driver 3 api and have the following:
List<Bson> query = Arrays.asList(
project(include("record")),
unwind("$record"),
group(computed("device", "$record.k"),
sum("count", "$record.v.count"),
min("domain", "$record.v.domain"))
);
The issue I am having is I can't seem to find an equivalent to $objectToArray using the Mongodb Java Driver and the subsequent sum and min operations depend on dot operating the k and v values generated from using $objectToArray.
Is there an equivalent way to map an object with unknown key names into the k and v format used by $objectToArray using the Mongodb Java Driver, preferrably version 3+?
Below will do.
project(computed("record", eq("$objectToArray", "$record")))
I have a query like this (simplified):
db.collection.aggregate([
{ $match: { main_id: ObjectId("58f0f67f50c6af16709fd2c7") } },
{
$group: {
_id: "$name",
count: { $sum: 1 },
sum: { $sum: { $add: ["$P31", "$P32"] } }
}
}
])
I do this query from Java, and I want to map it on my class, but I don't want _id to be mapped on name field. Because if I do something like this:
#JsonProperty("_id")
private String name;
then when I save this data back to mongo (after some modification) the data is saved with name as _id while I want a real Id to be generated.
So, how can I rename _id after $group operation?
You can achieve this by adding a $project stage at the end of your pipeline like this :
{ $project: {
_id: 0,
name: "$_id",
count: 1,
sum: 1
}
}
try it online: mongoplayground.net/p/QpVyh-0I-bP
From mongo v3.4 you could use $addFields in conjunction with $project to avoid to write all the fields in $project that could be very tedious.
This happen in $project because if you include specifically a field, the other fields will be automatically excluded.
Example:
{
$addFields: { my_new_id_name: "$_id" }
},
{
$project: { _id: 0 }
}
db.report.aggregate(
{
$group: {_id: '$name'}
},
{
$project:{
name:"$_id",
_id:false} }
)
Starting in Mongo 4.2, you can use a combination of $set / $unset stages:
// { x: 1, z: "a" }
// { x: 2, z: "b" }
db.collection.aggregate([
{ $set: { y: "$x" } },
{ $unset: "x" }
])
// { y: 1, z: "a" }
// { y: 2, z: "b" }
The $set stage adds the new field to documents and the $unset stage removes/excludes the field to be renamed from documents.
if you are using find method you can't do this, but if you using aggregation it is very easy like this:
db.collectionName.aggregate([
{
$project: {
newName: "$existingKeyName"
}
}
]);
As all of the answers are written the solution in MongoDB query despite the question seeks the solution in Java, posting my approach using Java for posterities.
After the grouping, we can rename the _id fieldname using
Projections.computed("<expected field name>", "$_id")))
To Transform the core part of the query mentioned in the question to Java
Bson mainIdMatch = match(eq("main_id", new ObjectId("58f0f67f50c6af16709fd2c7")));
Bson group = Aggregates.group("$name", Accumulators.sum("count", 1L));
Bson project = Aggregates.project(Projections.fields(Projections.excludeId(),
Projections.computed("name", "$_id")));
reportMongoCollection.aggregate(Arrays.asList(mainIdMatch, group, project))
.into(new ArrayList<>());
To answer specifically, I have added an excerpt from the above code snippet, where I am renaming _id field value as name using Projections.computed("name", "$_id") which map the values of _id which we got as a result of grouping to the field called name. Also, we should exclude the id using Projections.excludeId().
Aggregates.project(Projections.fields(Projections.excludeId(),
Projections.computed("name", "$_id")))
The database MongoDB I have stored documents in the format:
{
"achievement": [
{
"userFromId":"max",
"userToId":"peter",
"date":"2016-01-25",
"pointCount":1,
"description":"good work",
"type":"THANKS"
}
]
}
How to get the number of records in the database (if any) for the a certain date, in which people are thanking the other people.
I created a query to retrieve data:
DBObject clause1 = new BasicDBObject("userFromId", userFromId);
DBObject clause2 = new BasicDBObject("userToId", userToId);
DBObject clause3 = new BasicDBObject("sendDate", localDate);
DBObject clause4 = new BasicDBObject("type", Thanks);
BasicDBList or = new BasicDBList();
or.add(clause1);
or.add(clause2);
or.add(clause3);
or.add(clause4);
DBObject query = new BasicDBObject("$or", or);
But I do not know how to get the number of records and how can rewrite the query using aggregation?
For example:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.group("userFromId")
.first("userFromId").as("userFromId")
.sum("pointCount").as("pointCount"));
I do not know how to add a few more parameters.
What the return request if the data to the database does not exist?
Thanks for any help
You can use something like this. This will count all the number of documents matching the below criteria.
Regular Query
db.collection.count({ $or: [ { "userFromId": userFromId }, { "userToId": userToId } ] });
Using Aggregation
db.collection.aggregate( [
{ $match: { $or: [ { "userFromId": userFromId }, { "userToId": userToId } ] } },
{ $group: { _id: null, count: { $sum: 1 } } }
] );