Develop Reference

Spring Java Mongodb Aggregation check userId in array

I have a MongoDB database with users and questions.
My Question document:
{
"_id" : "b943d57a-f4c3-4394-86f7-dd1d1b5e2563",
"_class" : "org.company.app.model.Question",
"statement" : "First question for testing purposes",
"userMail" : "themail#mail.com",
"responses" : [
{
"responseId" : "6b60e900-0fec-47d2-8853-e1ea3508abe6",
"responseType" : "TEXT",
"mail" : "themail2#mail.com",
"responseTime" : ISODate("2017-11-22T11:23:10.848Z"),
"data" : "New Response 1",
"likes" : [
"themail3#mail.com",
"themail5#mail.com"
],
"dislikes" : [],
"state" : "PUBLISHED",
"stateTime" : ISODate("2017-11-22T11:23:10.848Z")
}
],
"categories" : [
"Category1",
"Category2",
"Category3"
],
"creationTimestamp" : ISODate("2017-10-26T14:50:12.717Z"),
"lastUpdateTimestamp" : ISODate("2017-11-02T15:35:20.818Z"),
"deleted" : false,
"active" : true,
"state" : "PUBLISHED"
}
and the user document:
{
"_id" : "1f4b5091-c755-4083-880e-e1c4696f7236",
"_class" : "org.company.app.model.User",
"allowsComms" : true,
"registrationConfirmed" : false,
"placeOfWork" : "Urban records",
"name" : "John",
"surname" : "Doe",
"mail" : "themail3#mail.com",
"phone" : "[ \"1111111111\" , \"2222222222222\"]",
"birthDate" : ISODate("1971-12-10T23:00:00.000Z"),
"gender" : "MALE",
"nationality" : "US",
"language" : "en",
"socialNetworks" : {
"INSTAGRAM" : "#instagram",
"TWITTER" : "#twitter"
},
"specialty" : "alternative rock",
"subspecialty" : "sub_3",
"creationTimestamp" : ISODate("2017-10-31T10:13:12.131Z"),
"lastUpdateTimestamp" : ISODate("2017-11-13T10:23:37.637Z"),
"deleted" : false,
"active" : true,
"profileImage" : "image"
}
From that point, I'm creating a new aggregation to join both collections, get user information and count likes and dislikes instead of return all the user ids at that array.
public List<JoinedResponse> getResponses(String userId, String questionId, long itemsPerPage, long requestedPage) {
Assert.hasLength(questionId, "questionId cannot be null or empty");
MatchOperation questionIdMatch = Aggregation.match(new Criteria("_id").is(questionId));
MatchOperation responsesStateMatch = Aggregation.match(Criteria.where("responses.state")
.in(QuestionState.PUBLISHED.toString(), QuestionState.APPROVED.toString()));
LookupOperation userInfoLookUpOperation = LookupOperation.newLookup().from("User").localField("$responses.mail").foreignField("mail")
.as("userInfo");
ProjectionOperation fieldsProjectionOperation = Aggregation.project(Fields.from(
Fields.field("responseId", "$responses.responseId"),
Fields.field("responseType", "$responses.responseType"),
Fields.field("mail", "$responses.mail"),
Fields.field("responseTime", "$responses.responseTime"),
Fields.field("data", "$responses.data"),
Fields.field("state", "$responses.state"),
Fields.field("userName", "$userInfo.name"),
Fields.field("userSurname", "$userInfo.surname"),
Fields.field("profileImage", "$userInfo.profileImage")))
.and("$responses.likes").size().as("likes")
.and("$responses.dislikes").size().as("dislikes");
// Creates the Aggregation: THE ORDER IS IMPORTANT!!!
Aggregation ag = Aggregation.newAggregation(
questionIdMatch,
Aggregation.unwind("responses"),
responsesStateMatch,
userInfoLookUpOperation,
Aggregation.unwind("userInfo"),
fieldsProjectionOperation,
Aggregation.project("responseId", "userName", "userSurname", "mail", "responseType", "responseTime",
"data", "likes", "dislikes", "profileImage", "state"/*, "isLiked"*/),
Aggregation.skip((long) ((requestedPage - 1) * itemsPerPage)),
Aggregation.limit(itemsPerPage));
logger.debug(ag.toString());
AggregationResults< JoinedResponse > output = mongoTemplate.aggregate(ag, "Question",
JoinedResponse);
Well, at this point, I need to know if the userId is in the liked or disliked list and I have spend a lot of hours investigating with no result. Of course, I would like to do it with aggregation.
Any help will be appreciated.
Thank you!

How to project more than the first sub-document when using $elemMatch

I have a collection with documents of the following form:
{
"_id" : { "$oid" : "67bg............"},
"ID" : "xxxxxxxx",
"senses" : [
{
"word" : "hello",
"lang" : "EN",
"source" : "EN_DICTIONARY"
},
{
"word" : "coche",
"lang" : "ES",
"source" : "ES_DICTIONARY"
},
{
"word" : "bye",
"lang" : "EN",
"source" : "EN_DICTIONARY"
}
]
}
I want to find all documents that match at least one sense with lang=X and source=Y and return the matched Documents with only those senses which match lang=X and source=Y.
I tried this:
DBObject sensesQuery = new BasicDBObject();
sensesQuery.put("lang", "EN");
sensesQuery.put("source", "EN_DICTIONARY");
DBObject matchQuery = new BasicDBObject("$elemMatch",sensesQuery);
DBObject fields = new BasicDBOject();
fields.put("senses",matchQuery);
DBObject projection = new BasicDBObject();
projection.put("ID",1)
projection.put("senses",matchQuery);
DBCursor cursor = collection.find(fields,projection)
while(cursor.hasNext()) {
...
}
My query works for matching documents, but not for the projection. Taking the above document as an example, if I run my query I get this result:
{
"_id" : { "$oid" : "67bg............"},
"ID" : "xxxxxxxx",
"senses" : [
{
"word" : "hello",
"lang" : "EN",
"source" : "EN_DICTIONARY"
}
]
}
But I want this :
{
"_id" : { "$oid" : "67bg............"},
"ID" : "xxxxxxxx",
"senses" : [
{
"word" : "hello",
"lang" : "EN",
"source" : "EN_DICTIONARY"
},
{
"word" : "bye",
"lang" : "EN",
"source" : "EN_DICTIONARY"
}
]
}
I read about aggregation but I did not understand how to use it in the MongoDB Java driver.
Thanks

You are using the $elemMatch operator on the projection aswell as on the filter.
From the docs
The $elemMatch operator limits the contents of an field from the query results to contain only the first element matching the $elemMatch condition.
So, the behaviour you are seeing is the expected behaviour for elemMatch-in-a-projection.
If you want to project all sub documents in the senses array within documents which match the filter condition then you could use this:
projection.put("senses", 1);
But, if you want to project only those sub documents which match your filter condition then $elemMatch will not work for you since it only ever returns the first element matching the $elemMatch condition. Your alternative is to use the aggregation framework, for example:
db.collection.aggregate([
// matches documents with a senses sub document having the given lang and source values
{$match: {'senses.lang': 'EN', 'senses.source': 'EN_DICTIONARY'}},
// projects on the senses sub document and filters the output to only return sub
// documents having the given lang and source values
{$project: {
senses: {
$filter: {
input: "$senses",
as: "sense",
cond: { $eq: [ "$$sense.lang", 'EN' ], $eq: [ "$$sense.source", 'EN_DICTIONARY' ] }
}
}
}
}
])
Here's that aggregation call using the MongoDB Java driver:
Document filter = new Document("senses.lang", "EN").append("senses.source", "EN_DICTIONARY");
DBObject filterExpression = new BasicDBObject();
filterExpression.put("input", "$senses");
filterExpression.put("as", "sense");
filterExpression.put("cond", new BasicDBObject("$and", Arrays.<Object>asList(
new BasicDBObject("$eq", Arrays.<Object>asList("$$sense.lang", "EN")),
new BasicDBObject("$eq", Arrays.<Object>asList("$$sense.source", "EN_DICTIONARY")))
));
BasicDBObject projectionFilter = new BasicDBObject("$filter", filterExpression);
AggregateIterable<Document> documents = collection.aggregate(Arrays.asList(
new Document("$match", filter),
new Document("$project", new Document("senses", projectionFilter))));
for (Document document : documents) {
logger.info("{}", document.toJson());
}
The resulting output is:
2017-10-01 17:15:39 [main] INFO c.s.mongo.MongoClientTest - { "_id" : { "$oid" : "59d10cdfc26584cd8b7a0d3b" }, "senses" : [{ "word" : "hello", "lang" : "EN", "source" : "EN_DICTIONARY" }, { "word" : "bye", "lang" : "EN", "source" : "EN_DICTIONARY" }] }
Update 1: following this comment:
After a long period of testing, trying to understand why the query was slow, I noticed that the "$match" parameter does not work, the query should select only records that have at least one sense with source = Y AND lang = X and project them , but the query also returns me documents with senses = []
This filter: new Document("senses.lang", "EN").append("senses.source", "EN_DICTIONARY") will not match documents which have no senses attribute nor will it match documents which have an empty senses attribute. To verify this I added the following documents to my own collection:
{
"_id" : ObjectId("59d72a24c26584cd8b7b70a5"),
"ID" : "yyyyyyyy"
}
{
"_id" : ObjectId("59d72a3ac26584cd8b7b70ae"),
"ID" : "zzzzzzzzz",
"senses" : []
}
And re ran the above code and I still get the desired result.
I suspect your statment that the above code does not work is either a false negative or the documents you are querying are different to the sample I have been working with.
To help you diagnose this issue for yourself you could ...
Play around with other operators e.g. the $match stage behaves the same with and without an $exists operator:
new Document("senses", new BasicDBObject("$exists", true))
.append("senses.lang", new BasicDBObject("$eq", "EN"))
.append("senses.source", new BasicDBObject("$eq", "EN_DICTIONARY"))
Remove the $project stage to see exactly what the $match stage produces.

Java query for mongodb to find documents where list size is less than n

I'm trying to find all of the documents in my db where the the size of my "states" list only contains one state but I'm struggling with the syntax of the java code.
My db looks like this:
{ "_id" : 13218 , "country" : { "MY" : 11 , "US" : 4} , "state" : { "WA" : 4 }}
{ "_id" : 95529 , "country" : { "US" : 6 } , "state" : { "MI" : 6 }}
{ "_id" : 22897 , "country" : { "US" : 4 } , "state" : { "CA" : 2 , "TX" : 1 , "WY" : 1 }}
What I want to do is print out every "_id" found from the US that only has a single state. So, the only "_id" that'd be returned here is 95529.
here is the relevant portion of code:
DBObject query = new BasicDBObject("country.US", new BasicDBObject("$gt", 4));
//query.put("state.2", new BasicDBObject("$exists", true));
//This is my attempt at checking the list length but it doesn't work
DBCursor dbCursor = dBcollection.find(query);
while (dbCursor.hasNext()){
DBObject record = dbCursor.next();
Object _id= record.get("_id");
Object state= record.get("state");
System.out.println(_id + "," + state);
}
current output looks like this:
95529, { "MI" : 6 }
22897, { "CA" : 2 , "TX" : 1 , "WY" : 1 }

The essential problem you have here is that your data is not in fact a "list". As a "hash" or "map" which is what it really is there is no concept of "length" in a MongoDB sense.
You would be better off changing your data to use actual "arrays" which is what a list actually is:
{
"_id" : 13218 ,
"country" : [
{ "code": "MY", "value" : 11 },
{ "code": "US", "value" : 4 },
],
"state" : [{ "code": "WA", "value" : 4 }]
},
{
"_id" : 95529 ,
"country" : [{ "code": "US", "value" : 6 }],
"state" : [{ "code": "MI", "value" : 6 }]
},
{
"_id" : 22897 ,
"country" : [{ "code": "US", "value" : 4 }],
"state" : [
{ "code": "CA", "value" : 2 },
{ "code": "TX", "value" : 1 },
{ "code": "WY", "value" : 1 }
]
}
Then getting those documents that only have a single state is a simple matter of using the $size operator.
DObject query = new BasicDBObject("country",
new BasicDBObject( "$elemMatch", new BasicDBObject(
"code", "US").put( "value", new BasicDBObject( "$gt", 4 )
)
);
query.put( "state": new BasicDBObject( "$size", 1 ) );
This ultimately gives you a lot more flexibilty in issuing queries as you don't need to specify the explicit "key" in each query. Also as noted, there is a concept of length here that does not otherwise exist.
If you keep your data in it's current form then the only way to do this is with the JavaScript evaluation of the $where operator. That is not very efficient as the interpreted code needs to be run for each document in order to determine if the condition matches:
DBObject query = new BasicDBObject("country.US", new BasicDBObject("$gt", 4));
query.put("state", new BasicDBObject( "$type", 3 ));
query.put("$where","return Object.keys( this.state ).length === 1");
Also using the $type operator in order to make sure that "state" is actually present and an "Object" that is expected. So possible, but not a really great idea do to performance.
Try to change your document structure as it will make other sorts of queries possible without using JavaScript evaluation.

Select elements of an array inside an array

This is the object in the database
{
"_id" : { "$oid" : "53a9ce071e24a7a0a4bef03a"} ,
"name" : "name4" ,
"sections" : [
{
"id" : "sectionId1" ,
"subs" : [
{ "name" : "name1" , "enable" : true} ,
{ "name" : "name2" , "enable" : false} ,
{ "name" : "name3" , "enable" : true}
]
},
{
"id" : "sectionId2",
"subs" : [
{ "name" : "name1" , "enable" : true} ,
{ "name" : "name5" , "enable" : false} ,
{ "name" : "name6" , "enable" : true}
]
},
{ "id" : "sectionId3"}
]
}
and this is my code :
BasicDBObject query = new BasicDBObject();
query.append("name", "name4");
query.append("sections", new BasicDBObject(
"$elemMatch", new BasicDBObject("id", "sectionId2")
));
query.append("sections.subs", new BasicDBObject(
"$elemMatch", new BasicDBObject("name", "name1")
));
I am trying to access the 'name1' in 'subs' of 'sectionId2'. But my query returns the sub in 'sectionId1'. I am having this problem only for 'name1'. I can access 'name2', 'name3' etc without any errors because they are unique.
Thanks in advance!

It may be because of the $elemMatch that you are using, the $elemMatch projection returns only the first matching element from the array. see it here

MongoDB Query in Array of Arrays and add if not present

I have
{
"Districts" :
[{ "name" : "Krishna"
, "Locations" : [{ "name" : "Vijayawada"}
,{ "name" : "Machilipatnam"}]}
, { "name" : "Guntur"
, "Locations" : [{ "name" : "Satenpalli"}]}
]
, "_id" : 1
, "name" : "Andhra Pradesh"
}
I am trying to create one more Location "Achampet" if District name is "Guntur" so the result should be this below. The result should be the same even if I try to add Achampet more than once.
{
"Districts" :
[{ "name" : "Krishna"
, "Locations" : [{ "name" : "Vijayawada"}
,{ "name" : "Machilipatnam"}]}
, { "name" : "Guntur"
, "Locations" : [{ "name" : "Satenpalli"}
,{ "name" : "Achampet"}]}
]
, "_id" : 1
, "name" : "Andhra Pradesh"
}
But my java code doesn't work
DBObject newLoc = new BasicDBObject("Districts", new BasicDBObject("name", distName).append("Locations", new BasicDBObject("name", locName)));
if (statesColl.findOne(newLoc) == null) {
DBObject updateLoc = new BasicDBObject("$push", newLoc);
statesColl.update(queryDist, updateLoc);
}
It is creating a new District everytime I try to add a location. How can I fix this?

This is how you can do it using the $ positional operator in Java:
...
DBObject selectQuery = new BasicDBObject("_id", 1); // Matches the document
selectQuery.append("Districts.name", distName); // Matches the element in the array where District name = Guntur
BasicDBObject updateFields = new BasicDBObject();
updateFields.put("Districts.$.Locations", new BasicDBObject("name":"Achampet"));
DBObject updateQuery = new BasicDBObject("$addToSet", updateFields);
statesColl.update(selectQuery, updateQuery);
...