Develop Reference

Issue when trying to query a MongoDB data

I have a document in a collection that has the following attributes:
nodeid : long
type: string
bagid: long
So, nodes can be on a bag, and be of different types.
I need to find,
all nodes of type A, or nodes of type B in a given list of nodes, or, nodes of type C in a given bag.
How can I design that query in MongoDB? I had all IN clauses but it is the works way to go performance wise. Could you please point me into the right direction? I could not find an aggregation or reduce that would help me make this simpler.
I tried also doing a text search, using the three elements, but the or in the text search, for instance "type: A \"type: B node:X\" \"type: B node: Y\" and so on, does not work.
Thanks
Edit, adding samples:
{ "_id" : BinData(3,"NJUuYHEBAAAdCda3V+kXvg=="),
"type" : "question", "bagid" : NumberLong(1067),
"topics" : [ NumberLong(33), NumberLong(67), NumberLong(203), NumberLong(217) ],
"nodeid" : NumberLong(15855),
"creationDate" : ISODate("2020-04-09T18:23:17.812Z"),
"_class" : "com.test.NodeEvent" }
{ "_id" : BinData(3,"NJUuYHEBAAAdCda3V+kXvg=="),
"type" : "comment", "bagid" : NumberLong(1067),
"topics" : [ NumberLong(33), NumberLong(67), NumberLong(203), NumberLong(217) ],
"nodeid" : NumberLong(15857),
"creationDate" : ISODate("2020-04-09T18:23:17.812Z"),
"_class" : "com.test.NodeEvent" }
{ "_id" : BinData(3,"NJUuYHEBAAAdCda3V+kXvg=="),
"type" : "question", "bagid" : NumberLong(1069),
"topics" : [ NumberLong(33), NumberLong(67) ],
"nodeid" : NumberLong(15859), "creationDate" : ISODate("2020-04-09T18:23:17.812Z"),
"_class" : "com.test.NodeEvent" }

You can build your queries using the MongoDB Query Language. The query is written using the db.collection.find method. It uses various query operators like, $or, $in, and $and.
I need to find, all nodes of type A, or nodes of type B in a given
list of nodes, or, nodes of type C in a given bag. How can I design
that query in mongo?
db.collection.find( { $or: [
{ type: "A" },
{ type: "B", node: { $in: [ "A", "C" ] },
{ type: "C", bag: 130 }
]
} );
In the above query the condition { type: "C", bag: 130 } is equivalent to { $and: [ { type: "C" }, { bag: 130 } ] } }. This is also the case with the condition { type: "B", node: { $in: [ "A", "C" ] }. But, using the $and is optional, in this case (see the documentation for details).
The query would be something like, given bag 1067 give me all nodes
with topics: 33 or 203
db.collection.find( { bag: 1067, topics: { $in: [ 33, 203 ] } } )
The output depends upon the data in the collection. The find method returns a cursor, and you can apply various cursor methods on the retrned documents (for example, you can sort them by a field).

mongo java for finding and upserting a nested array element

I have a mongo collection element with the following structure
{
"_id" : ObjectId("568eaba5e4b0fa3cdf9aaf85"),
"type" : "X",
"textId" : "568eaba5e4b0fa3cdf9aaf84",
"createDate" : "2016-01-07 18:17:09",
"likes" : [
{
"creatorName" : "ABCD",
"creationDate" : "2016-01-10 19:48:37",
"likeId" : "56932615569aae9d1459eb9b"
}
]
}
My need is as follows:
When a new like is posted by a creatorName for a particular element identified by textId, I should
a) check if the like is already present for the creatorName for the specific textId object. If present, do nothing
b) if the like is not present, I should insert a new element into the likes array
How to do this effectively in java? I see that we have an update method and a findAndModify method. Which one to use and how to handle this in the most effective manner?

Unique indexes
You can create a unique index as follows:
db.collection.createIndex( { "likes.creatorName": 1 }, { unique: true } )
As official documentation says:
A unique index causes MongoDB to reject all documents that contain a
duplicate value for the indexed field.
if the creatorName already exists, write operation will fail as follows:
WriteResult({
"nInserted" : 0,
"writeError" : {
"code" : 11000,
"errmsg" : "E11000 duplicate key error index: test.collection.$likes.creatorName_1 dup key: { : 1.0 }"
}
})
Then, you don't need to worry about if creatorName already exists or not.
On the other hand, if you need to know if the creatorName already exists, you need to have a look to the WriteResult returned value (0 or 1).

You may insert likes by update function with query condition:
db.collection.update({
"textId" : "568eaba5e4b0fa3cdf9aaf84",
"likes.likeId": {"$nin": ["56932615569aae9d1459eb10"]},
"likes.creatorName" : {"$nin": ["ABCE"]}
},
{
"$push" : {
"likes" : {
"creatorName" : "ABCE",
"creationDate" : "2016-01-10 21:48:37",
"likeId" : "56932615569aae9d1459eb10"
}
}
}
)
Updated 1 existing record(s)
If you try to add new like for the same creatorName, it will not update.
db.collection.update({
"textId" : "568eaba5e4b0fa3cdf9aaf84",
"likes.likeId": {"$nin": ["56932615569aae9d1459eb11"]},
"likes.creatorName" : {"$nin": ["ABCD"]}
},
{
"$push" : {
"likes" : {
"creatorName" : "ABCD",
"creationDate" : "2016-01-11 19:48:37",
"likeId" : "56932615569aae9d1459eb11"
}
}
}
)
Updated 0 record(s)
Java driver does not give any performance advantage.
If you add index depending on query condition, it will improve the performance. (Needs only once)
db.colecction.createIndex({"textId":1, "likes.likeId":1, "likes.creatorName":1})

Effective use of Array field in Mongo Collection

We are using Mongo DB in our application and in our collection we are storing array as field. eg:
{
"_id" : ObjectId("54ef67573848ec32b156b053"),
"articleId" : "46384262",
"host" : "example.com",
"url" : "http://example.com/articleshow/46384262.cms",
"publishTime" : NumberLong("1424954100000"),
"tags" : [
"wind power",
"mytrah",
"make in india",
"government",
"andhra pradesh"
],
"catIds" : [
"2147477890",
"13352306",
"13358350",
"13358361"
]
}
Now my situation is need to create index on tags and catIds array as they are search field.
But creating an index on array field increases the size of indexes tremendously.
Could you please suggest a better way of achiving this.

You can restructure your collection in this way. Now you will have 3 collections:
Coll1, documents look like this:
{
"_id" : ObjectId("54ef67573848ec32b156b053"),
"articleId" : "46384262",
... your other stuff
}
Tags, documents look like this:
{
'_id': 1,
'name': 'wind power'
}
{
'_id': 2,
'name': 'mytrash'
}
....
and a collection that links coll1 to tags:
{
"collID" : ObjectId("54ef67573848ec32b156b053"),
"tagID": 1
}
{
"collID" : ObjectId("54ef67573848ec32b156b053"),
"tagID": 2
}
Mongo does not have joins, so you need to do joins on the application layer. And it will take you 3 mongo queries. The size of indexes should be smaller, but test it before making significant changes.

MongoDB search in embedded array

I am trying to store all my written java files in a MongoDB and so far I've applied a schema like this (incomplete entry):
{
"_id" : ObjectId("52b861c230044fd08d6c27c4"),
"interfaces" : [
{
"methodInterfaces" : [
{
"name" : "add",
"name_lc" : "add",
"returnType" : "Integer",
"returnType_lc" : "integer",
"parameterTypes" : [
"Integer",
"Integer"
],
"parameterTypes_lc" : [
"integer",
"integer"
]
},
{
"name" : "isValid",
"name_lc" : "isvalid",
"returnType" : "Boolean",
"returnType_lc" : "boolean",
"parameterTypes" : [
"Integer",
"Double"
],
"parameterTypes_lc" : [
"integer",
"double"
]
}
],
"name" : "Calculator",
"name_lc" : "calculator",
"filename" : "Calculator.java",
"filename_lc" : "calculator.java"
}
],
"name" : "Calculator",
"name_lc" : "calculator",
"filename" : "Calculator",
"filename_lc" : "calculator",
"path" : "/xyz/Calculator.java",
"md5" : "6dec7e62c5e4f9060c7612c252cd741",
"lastModification" : ""
}
So far I am able to query a class that contains a method name, but I am not able to query a class with a certain name (let interfaces.name_lc="calculator") that must contain two methods with particular names (let's say "add" and "divide") which themselves should have two integer, resp. an integer and a double as parameters and both return an integer (don't question whether this is reasonable or not -- just for illustration purposes).
This is just one example; it can be more complex, of course.
I don't know how I can query for a particular class with method and specified parameters. I need to describe it sharp and want sharp results.
I am not able to construct a query, that would only return files like Calculator ( add(integer,integer):integer; divide(integer,double):integer; ). I get, e.g., OtherClass ( add():void; method(integer):integer; ), which is not what I want. I am trying this for days now, and maybe one can enlighten me, how to solve this in MongoDB. Thanks a lot in advance!

I'm not sure you'll be able to do this in MongoDB with your document structure. The issue I ran into is around the parameters - I'm assuming you care about the order of the parameters (i.e. doSomething(String, int) is not the same as doSomething(int, String)), and the query operators to check all the values in an array treat the array as a set, so a) order agnostic and b) eliminates duplicates (so doSomething(String, String) matches doSomething(String)) (this is because I was using the $all keyword, see the documentation, especially the note at the bottom).
I managed to get a large part of the query you wanted, however, which might point you in the right direction.
{ "$and" : [ //putting these in an "and" query means all parts have to match
{ "interfaces.methodInterfaces" :
{ "$elemMatch" : { "name" : "add"}} //this bit finds documents where the method name is "add"
} ,
{ "interfaces.methodInterfaces" :
{ "$elemMatch" : { "returnType" : "Integer"}} // This bit matches the return type
} ,
{ "interfaces.methodInterfaces.parameterTypes" :
{ "$all" : [ "Integer" , "Integer"]} //This *should* find you all documents where the parameter types matches this array. But it doesn't, as it treats it as a set
}
]}
If you're querying via the Java driver, this looks like:
BasicDBObject findMethodByName = new BasicDBObject("interfaces.methodInterfaces",
new BasicDBObject("$elemMatch", new BasicDBObject("name", "add")));
BasicDBObject findMethodByReturn = new BasicDBObject("interfaces.methodInterfaces",
new BasicDBObject("$elemMatch", new BasicDBObject("returnType", "Integer")));
BasicDBObject findMethodByParams = new BasicDBObject("interfaces.methodInterfaces.parameterTypes",
new BasicDBObject("$all", asList("Integer", "Integer")));
BasicDBObject query = new BasicDBObject("$and", asList(findMethodByName, findMethodByReturn, findMethodByParams));
DBCursor found = collection.find(query);
I haven't included matching the class name, as this didn't seem to be the tricky part - just build up another simple query for that an add it into the "$and".
Since the arrays to store parameter types are not giving what you want, I suggest you think about something a little more structured, although it's a bit unwieldy. Instead of
"parameterTypes" : [
"Integer",
"Integer"
]
Consider something like
"parameterTypes" : {
"param1" : "Integer",
"param2" : "Integer"
}
Then you won't be doing set operations, you can query each parameter individually. This means you'll also get them in the correct order.

Morphia Query on Array of Subdocuments using elem

I have the following document structure in mongodb collection "Contact". There is an array of subdocuments called "numbers":
{
"name" : "Bill",
"numbers" : [
{
"type" : "home",
"number" : "01234",
},
{
"type" : "business",
"number" : "99099"
},
{
"type" : "fax",
"number" : "77777"
}
]
}
When I want to query only for "home" and "business" numbers, I can do something like this in mongodb-shell:
db.Contact.find({ numbers: { $elemMatch: {
type : { $in : ["home", "business"]},
number: { $regex : "^012" }
}}});
But how to do this in morphia? Is there any way?
I understand "$elemMatch" is supported in morphia. So I could do something like:
query.filter("numbers elem", ???);
But how exactly do I add a combined query for the subdocument?

It is too late, but maybe others can find it handy.
I found that solution https://groups.google.com/forum/#!topic/morphia/FlEjBoSqkhg
query.filter("numbers elem",
BasicDBObjectBuilder.start()
.push("type").add("$in", new String[]{"home", "business"}).pop()
.push("number").add("$regex", "^012").pop().get());

Instead of using morphia, consider using jongo. It lets you query MongoDB as you were using MongoDB shell. Furthermore, it will give you more freedom when mapping your array elements. Here is how your example will look with jongo:
contacts_collection.find("{numbers : {$elemMatch: {
type: {$in :#},
number: {$regex: #}
}
}
}",
new String[]{"home", "business"}, "^012")
.as(Contact.class);
Note that, if you only need a single number object (or multiple) from the array you can use a custom result mapper/handler. You just have to substitute .as(Contact.class) with :
.map(new ResultHandler<Number>() {...})
For a full example take a look at my blog post or at my GitHub repository