I have the following JSON:
{"pid":"b00l16vp","categories":{"category1":["factual", "arts culture and the media", "history"]}}
{"pid":"b0079mpp","categories":{"category2":["childrens", "entertainment and comedy", "animation"],"category1":["signed"]}}
{“pid":"b00htbn3"}
{“pid":"b00gdhqw","categories":{"category2":["factual"],"category3":["scotland"],"category4":["lifestyle and leisure", "food and drink"],"category1":["entertainment", "games and quizzes"]}}
My intention is to query the categories object using an array of String by combining all of the arrays in a single array.
I have the following code:
String [] cats = ["childrens", "signed"]
BasicDBObject theProjections = new BasicDBObject()
for (int i = 1; i <= 5; i++) {
String identifier = "categories.category" + i
String cleanIdentifier = "\$" + identifier
//If the category does not exist, put in a blank category
def temp = [cleanIdentifier, []]
theMegaArray.add(new BasicDBObject('$ifNull', temp))
}
//The megaArray is the array created in the above loop which combines all arrays
BasicDBObject theData = new BasicDBObject('$setUnion', theMegaArray)
BasicDBObject theFilter = new BasicDBObject('input', theData)
theFilter.put("as", "megaArray")
//all of the values found in cats should match the megaArray
theFilter.put("cond", new BasicDBObject('$all', ["\$\$megaArray", cats]))
theProjections.put('$filter', theFilter)
FindIterable iterable = collection.find(criteria).projection(theProjections)
I have used this question to come to write this code so far. The $setUnion expects all of the fields to appear however in my JSON, there are a varied number of categrories arrays hence I have used the $ifNull to populate empty categories with []. The $filter has been used to query the cats array on the megaArray.
When running this I'm getting the following error:
Caused by: com.mongodb.MongoQueryException: Query failed with error code 2 and error message '>1 field in obj: { input: { $setUnion: [ { $ifNull: [ "$categories.category1", [] ] }, { $ifNull: [ "$categories.category2", [] ] }, { $ifNull: [ "$categories.category3", [] ] }, { $ifNull: [ "$categories.category4", [] ] }, { $ifNull: [ "$categories.category5", [] ] } ] }, as: "megaArray", cond: { $all: [ "$$megaArray", [ "factual" ] ] } }'
I'm not entirely sure what that means as the looks right. I should also note that the category object does not always exist but I'm not sure if this matters.
You can achieve the same result using the aggregation framework
To do this you can first project the data to create the 'megaArray' and then match against the new array.
String [] cats = new String[] {"childrens", "signed"};
List<DBObject> theMegaArray = new ArrayList<>();
BasicDBObject theProjections = new BasicDBObject();
for (int i = 1; i <= 5; i++) {
String identifier = "categories.category" + i;
String cleanIdentifier = "$" + identifier;
//If the category does not exist, put in a blank category
Object[] temp = new Object[] {cleanIdentifier, new Object[]{}};
theMegaArray.add(new BasicDBObject("$ifNull", temp));
}
theProjections.put("_id", 1);
theProjections.put("pid", 1);
theProjections.put("categories",1);
theProjections.put("allCategories", new BasicDBObject("$setUnion", theMegaArray));
BasicDBObject theFilter = new BasicDBObject("allCategories", new BasicDBObject("$all", cats));
List<BasicDBObject> pipeline = new ArrayList<>();
pipeline.add(new BasicDBObject("$project", theProjections));
pipeline.add(new BasicDBObject("$match", theFilter));
AggregateIterable iterable = collection.aggregate(pipeline);
The code sample above adds a new array called "allCategories" in the project stage and then matches against this new document.
Another projection stage could be added to remove the allCategories array from the final output
You can use the below aggregation pipeline.
Shell Query for reference:
db.collection.aggregate([{
"$project": {
"pid": 1,
"categories": 1,
"filter": {
"$eq": [{
"$setUnion": [{
"$ifNull": ["$categories.category1", []]
}, {
"$ifNull": ["$categories.category2", []]
}, {
"$ifNull": ["$categories.category3", []]
}, {
"$ifNull": ["$categories.category4", []]
}, {
"$ifNull": ["$categories.category5", []]
}]
},
["childrens", "signed"]
]
}
}
}, {
"$match": {
"filter": true
}
}])
Java Code:
String[] cats = {"childrens", "signed"};
// Combining the optional categories arrays
BasicDBList theMegaArray = new BasicDBList();
for (int i = 1; i <= 5; i++) {
String identifier = "categories.category" + i;
String cleanIdentifier = "$" + identifier;
theMegaArray.add(new BasicDBObject("$ifNull", Arrays.asList(cleanIdentifier, Collections.EMPTY_LIST)));
}
BasicDBObject theData = new BasicDBObject("$setUnion", theMegaArray);
// Add equals filter - Compare the arrays and output boolean filter field
BasicDBObject theFilter = new BasicDBObject("$eq", Arrays.asList(theData, cats));
// Add projections to keep the output fields
BasicDBObject theProjections = new BasicDBObject();
theProjections.put("filter", theFilter);
theProjections.put("pid", 1);
theProjections.put("categories", 1);
// Add $project stage
BasicDBObject theProject = new BasicDBObject("$project", theProjections);
// Add $match stage to compare the boolean filter field to true to keep matching documents
BasicDBObject theMatch = new BasicDBObject("$match", new BasicDBObject("filter", true));
// Add stages to piepline
BasicDBList pipeline = new BasicDBList();
pipeline.add(theProject);
pipeline.add(theMatch);
// Run aggregation
AggregateIterable iterable = collection.aggregate(pipeline);
Related
I have a problem to query the mongoDB data. Here is the document:
{
productId:1,
warehouses:[
warehouseId:1,
productBatch:[
{
unitPrice:15.5,
quantity:1000,
expireSearchTimestamp:14145555545,
currentQuantity:50
},{
unitPrice:17.5,
quantity:1000,
expireSearchTimestamp:14145555545,
currentQuantity:50
}
]
]
}
and by code is
public List<ProductSearchResult> findCustomSearch(List<Integer> medicines,
List<Integer> warehousesIds, int quantity)
{
UnwindOperation unwind1 = Aggregation.unwind("warehouses");
UnwindOperation unwind2 = Aggregation.unwind("warehouses.productBatch");
ProjectionOperation project = Aggregation.project("warehouses.productBatch");
MatchOperation match = Aggregation.match(Criteria.where("productId").in(productIds)
.and("warehouses.warehouseId").in(warehousesIds)
.and("warehouses.productBatch.currentQuantity").gte(quantity));
SortOperation sort = Aggregation.sort(Direction.ASC, "productBatch.unitPrice");
LimitOperation limit = Aggregation.limit(3);
Aggregation aggregation = Aggregation
.newAggregation(unwind1, unwind2, project, sort, limit)
.withOptions(Aggregation.newAggregationOptions().allowDiskUse(true).explain(true)
.cursor(new BasicDBObject()).build());
AggregationResults<ProductSearchResult> results = mongoTemplate.aggregate(aggregation,
"medicine", ProductSearchResult.class);
List<ProductSearchResult> mappedResults = results.getMappedResults();
return mappedResults;
}
this is the function output
[
{
"$unwind": "$warehouses"
},
{
"$unwind": "$warehouses.productBatch"
},
{
"$match": {
"productId": {
"$in": [
20,
21
]
},
"warehouses.warehouse_id": {
"$in": [
1,
2,
3,
4,
5
]
},
"warehouses.productBatch.currentQuantity": {
"$gte": 10
}
}
},
{
"$project": {
"medicine_search": "$warehouses.productBatch"
}
},
{
"$sort": {
"productBatch.unitPrice": 1
}
},
{
"$limit": 3
}
]
when I run this function I got empty list but according to mongo compass I got 3 element.
thanks
I don't know why this block of code not work but the following one is satisfy for me
public void finalTry(List<Integer> productIds,
List<Integer> warehousesIds, int q, int limitN)
{
DBCollection collection = mongoTemplate.getCollection("product");
DBObject limit = new BasicDBObject("$limit", limitN);
DBObject sort = new BasicDBObject("$sort",
new BasicDBObject("productBatch.unitPrice", 1));
DBObject unwind1 = new BasicDBObject("$unwind", "$warehouses");
DBObject unwind2 = new BasicDBObject("$unwind", "$warehouses.productBatch");
DBObject project = new BasicDBObject("$project",
new BasicDBObject("productBatch", "$warehouses.productBatch"));
DBObject match1 = new BasicDBObject("$match",
new BasicDBObject("productId", new BasicDBObject("$in", productIds)));
DBObject match2 = new BasicDBObject("$match", new BasicDBObject("warehouses.warehouseId",
new BasicDBObject("$in", warehousesIds)));
DBObject match3 = new BasicDBObject("$match", new BasicDBObject(
"warehouses.productBatch.currentQuantity", new BasicDBObject("$gte", q)));
List<DBObject> pipeline = Arrays.asList(match1, unwind1, match2, unwind2, match3, project,
sort, limit);
AggregationOutput output = collection.aggregate(pipeline);
for (DBObject d : output.results())
{
System.out.println(d)
}
}
Thanks
I have the following database:
{ stream :{ "name": "name1",
"error1": 1,
"error2": 1,
"error3": 1 }
}
,
{ stream : {"name": "name1",
"error1": 2,
"error2": 1,
"error3": 1 }
}
,
{ stream : {"name": "name2",
"error1": 1,
"error2": 1,
"error3": 1 }
}
I would like to group it by name and sum every time some different combination of errors.
this is what I did in mongo, I need to create the following query dynamically in java
db.collection.aggregate([{$group: {_id: "$stream.name",error1: {$sum:"$stream.error1" },error2: {$sum: "$stream.error2" }} ])
the thing is that every time I need different combinations of the errors:error1 with error2, only error 1 etc..
this is what I did: (the arguments in the "if" are some boolean values that I am getting)
List<String> totalError = new ArrayList<String>();
BasicDBObject group = new BasicDBObject( "$group", new BasicDBObject("_id","$stream.name" ));
if (error1)
{
group.append("error1",new BasicDBObject ("$sum", "$stream.error1" ));
}
if (error2)
{
group.append("error2",new BasicDBObject ("$sum", "$stream.error2" ));
}
if (error3)
{
group.append("error3",new BasicDBObject ("$sum", "$stream.error3" ));
}
the problem is that I am getting:
{ "$group" : { "_id" : "$stream.name"} , "error1" : { "$sum: "$stream.error1"} , "error2" : { "$sum" : "$stream.error2"}
},
instead of:
{ "$group" : { "_id" : "$stream.name", "error1" : { "$sum: "$stream.error1"} , "error2" : { "$sum" : "$stream.error2"}}
if I knew what error combination I need I could use append in the constructor of group dbobject.. but I don't know the combination and I need to use the "ifs"
Try
BasicDBObject fields = new BasicDBObject("_id","$stream.name" );
if (error1)
fields.append("error1",new BasicDBObject ("$sum","$stream.error1"));
if (error2)
fields.append("error2",new BasicDBObject ("$sum","$stream.error2"));
if (error3)
fields.append("error3",new BasicDBObject ("$sum","$stream.error3"));
BasicDBObject group = new BasicDBObject( "$group", fields);
You should use helper functions when possible.
List<BsonField> fieldAccumulators = new ArrayList<>();
if (error1)
fieldAccumulators.add(Accumulators.sum("error1","$stream.error1"));
if (error2)
fieldAccumulators.add(Accumulators.sum("error2","$stream.error2"));
if (error3)
fieldAccumulators.add(Accumulators.sum("error3","$stream.error3"));
collection.aggregate(Arrays.asList(Aggregates.group("$stream.name", fieldAccumulators)));
I am trying to insert pairs in a mongoDB Document. This how i am doing it so far:
private static MongoCollection<Document> getCollectionn(String link, String coll){
...
}
public static void main(String[] args) {
MongoCollection<Document> fb_users;
fb_users = getCollectionn("mongodb://...", "fb_users2");
Document myDoc;
BasicDBObject query = new BasicDBObject();
query.put("fbid", "1");
myDoc = fb_users.find(query).first();
int postId=5;
int rating=3;
fb_users.updateOne(myDoc,
new Document("$push", new Document("ratings", java.util.Arrays.asList(rating, postId))));
Object a = myDoc.get("ratings");
ArrayList<Document> b = (ArrayList<Document>) a;
System.out.println(b.get(0);//prints [3, 5]
}
This is how the array document is inserted in the collection after two runs:
(the document already exists)
{
"_id": {
"$oid": "56f173b5e4b04eaac6531030"
},
"fbid": "1",
"ratings": [
[
3,
5
],
[
3,
5
]
]
}
In println i get the result [3, 5]
My question is this:
How can i recieve the numder 3 and the number 5 seperatly? Should i insert the documents with another way? Iam using mongo java driver 3.2.0.
Ok i found the solution. Here it is:
Document myDoc;
BasicDBObject query = new BasicDBObject();
query.put("fbid", "1");
myDoc = fb_users.find(query).first();
int postId=5;
int rating=3;
Document listItem = new Document("ratings", new Document(String.valueOf(rating), String.valueOf(postId)));
Document updateQuery = new Document("$push", listItem);
fb_users.updateOne(myDoc, updateQuery);
MongoCursor<Document> curs = fb_users.find(query).iterator();
ArrayList<Document> list = (ArrayList<Document>) myDoc.get("ratings");
Iterator<Document> iterateList = list .iterator();
while(iterateList .hasNext()){
Document pair = iterateList .next();
for(String element : pair.keySet()){
System.out.println(element + " " + pair.get(element));
}
}
and the json format is and should be like this:
{
"_id": {
"$oid": "56f173b5e4b04eaac6531030"
},
"fbid": "1",
"ratings": [
{
"3": "5"
},
{
"3": "5"
}
]
}
I am struggling to build up a query for one of the criteria, I have a documents like this:
[
{
"status": "a"
},
{
"status": "b"
},
{
"status": "foo"
},
{
"status": "a",
"flightDetails": [
{
"test": "xyz"
}
]
}
]
My criteria to select like this:
Get all data where status is either a, b or c, or
Get all data where status is either a, b or c and flightDetails, which size is > 0
Where not all documents contains flightDetails key.
I tried my java code like this:
BasicDBObject inQuery = new BasicDBObject();
inQuery.append("status", new BasicDBObject("$nin", Arrays.asList("a", "b", "c")));
BasicDBObject inQuery2 = new BasicDBObject();
inQuery2.append("$where", "return Object.keys( this.flightDetails").length > 0");
BasicDBList criteria = new BasicDBList();
criteria.add(inQuery);
criteria.add(inQuery2);
BasicDBObject finalQuery = new BasicDBObject("$or",criteria);
But it returns all the data, seems my query is not correct. I searched lot and based on some SO answers, I tried to build query. Can someone kindly help me in this? Since I am beginner in NoSQL and MongoDB and started on this few days back.
Using the dot notation for the "0" element of the array along with $exists means that it has some content, and of course exists:
BasicDBObject query = BasicDBObject("status",
new BasicDBObject("$in", Arrays.asList("a", "b", "c"))
)
.append("flightDetails.0",
new BasicDBObject("$exists",true)
);
Which is:
{
"status": { "$in": [ "a","b","c" ] },
"flightDetails.0": { "$exists": true }
}
For those used to the JSON notation.
To incorporate "no array at all" just contruct the $or condition:
{
"status": { "$in": [ "a","b","c" ] },
"$or": [
{ "flightDetails.0": { "$exists": true } },
{ "flightDetails": { "$exists": false } }
]
}
Or for Java:
BasicDBObject query = BasicDBObject("status",
new BasicDBObject("$in", Arrays.asList("a", "b", "c"))
)
.append("$or",Arrays.<Object>asList(
new BasicDBObject("flightDetails.0",
new BasicDBObject("$exists",true)
),
new BasicDBObject("flightDetails",
new BasicDBObject("$exists",false)
)
));
My mongo collections contains following documents
{
"syslog": [
{
"alertId": "N/A",
"time": "06:46:57",
"device": "10.10.20.1"
},
{
"alertId": "N/A",
"time": "06:47:30",
"device": "10.10.20.2"
}
]
}
now I want to read only values of given device.
Suppose if I set device="10.10.20.2" it will shows only "alertId": "N/A",
"time": "06:47:30",
"device": "10.10.20.2" for this I write java code as below
BasicDBObject criteria = new BasicDBObject();
criteria.put("syslog.device","10.10.20.2");
DBCursor cur = coll.find(criteria);
while(cur.hasNext() && !isStopped()) {
String json = cur.next().toString();
}//end of while
When I print json it shows me whole json values. How I should find only selected values of given device?
You should use $elemMatch operator to project array fields. You should update your query as follows :
db.collection.find({"syslog.device" : "10.10.20.2"},{syslog : {$elemMatch : {device : "10.10.20.2"}}})
In Java :
BasicDBObject criteria = new BasicDBObject();
criteria.put("syslog.device","10.10.20.2");
BasicDBObject elemMatchObj = new BasicDBObject();
elemMatchObj.put("$elemMatch", new BasicDBObject("device", "10.10.20.2"));
DBCursor cur = coll.find(criteria, new BasicDBObject("syslog", elemMatchObj));
while(cur.hasNext() && !isStopped()) {
String json = cur.next().toString();
}//end of while