I am trying to query a mongodb collection with two 'or' clauses via Java API. The following query works fine when run through any mongo client:
{
$and : [{
$or : [ { “field1” : “value1”}, {“field1” : “value2”}]
},{
$or : [ { “field2” : “value3”} , { “field2” : “value4”}]
}]
}
But when I try to run this using Java API, I get the following error:
org.springframework.data.mongodb.InvalidMongoDbApiUsageException: Due to limitations of the com.mongodb.BasicDBObject, you can't add a second '$or' expression specified as '$or : [ { “field1” : “value1”}, {“field1” : “value2”}]'. Criteria already contains '$or : [ { “field2” : “value3”} , { “field2” : “value4”}]'.
at org.springframework.data.mongodb.core.query.Criteria.setValue(Criteria.java:640)
at org.springframework.data.mongodb.core.query.Criteria.getCriteriaObject(Criteria.java:573)
at org.springframework.data.mongodb.core.query.Criteria.createCriteriaList(Criteria.java:630)
at org.springframework.data.mongodb.core.query.Criteria.andOperator(Criteria.java:539)
You can do it through Java API by
1) preparing a list of your 'or' criterias.
2) create new criteria object.
3) 'feed' the list created in (1) into the object created in (2),
using the criteria object's 'andOperator' method.
Something like this ->
Query query = new Query();
Map<String, String> params; // let's say this arrived with some values
List<Criteria> criterias = new ArrayList<>();
for (String k : params.keySet()) {
String paramVal = params.get(k);
switch (k) {
case "someVal1":
criterias.add(new Criteria().orOperator(Criteria.where("metadata.field1").is(paramVal),
Criteria.where("metadata.field2").is(paramVal)));
break;
case "someVal2":
criterias.add(new Criteria().orOperator(Criteria.where("metadata.arrayField3").in(paramVal),
Criteria.where("metadata.arrayField4").in(paramVal),
Criteria.where("metadata.arrayField5").in(paramVal)));
break;
default:
break;
}
}
if (criterias.size() > 0){
query.addCriteria(new Criteria().andOperator(criterias.toArray(new Criteria[criterias.size()])));
}
Try splitting that into an $in operator. Let's break down the first $or expression:
{
"$or": [
{ "filed1" : "value1" },
{ "filed1" : "value2" }
]
}
This can be converted into an $in operator as
{ "field1": { "$in": ["value1", "value2"] } }
Similarly
{
"$or": [
{ "filed2" : "value3" },
{ "filed2" : "value4" }
]
}
can be expressed as
{ "field2": { "$in": ["value3", "value4"] } }
Combining the two expressions into one implicit AND query yields:
db.collection.find({
"field1": { "$in": ["value1", "value2"] },
"field2": { "$in": ["value3", "value4"] }
})
If possible, upgrade to 3.x driver and stop using the old and obsolete handcrafted bson documents.
This works under 3.x:
public static void main(String[] args) throws Exception {
try (
MongoClient mongoClient = new MongoClient();
) {
MongoCollection<Document> coll = mongoClient.getDatabase("test").getCollection("mycoll");
Bson f = Filters.and(
Filters.or(
Filters.eq("field_a", "Test 1"),
Filters.eq("field_a", "Test 2")
),
Filters.or(
Filters.eq("field_b", 1999),
Filters.eq("field_b", 2000)
)
);
coll.find(f).forEach((Consumer<Document>) d -> System.out.println(d));
}
}
This worked for me,,
Query query = new Query(new Criteria().orOperator(
Criteria.where("field1").regex(searchStr),
Criteria.where("field2").regex(searchStr),
Criteria.where("field3").regex(searchStr)));
List<XXX> result = mongoTemplate.find(query, XXX.class);
Related
I have this:
A field which is a map where the keys are UUIDs and the value another object which is not relevant.
A list of UUIDs that should be passed as parameter.
I want to:
delete from the collection all documents where all keys of the map are included in the list of UUIDs
The object:
#Document
public class MyClass
{
private Map<UUID, anotherObject> myMap;
}
With derived queries I am not able to reach the UUID because has no name -> deleteByMyMap...
And with a query I know that there is a way to convert the map into an array ($expr and $objectToArray) but I do not know if it makes sense.
Is there any way to do this?
How can I access just the key of the map?
This is one way of doing it, use an aggregation pipeline to get _id of all documents matching your criteria:
db.collection.aggregate([
{
"$addFields": {
keysOfMap: {
"$map": {
"input": {
"$objectToArray": "$myMap"
},
"as": "item",
"in": "$$item.k"
}
},
}
},
{
"$addFields": {
"difference": {
"$setDifference": [
"$keysOfMap",
[
"63f62530-89b1-439e-bcb3-2c7ab614ecda",
"dcbb1469-3ca0-4547-b7d1-296ba2f0a01d"
]
]
}
}
},
{
"$match": {
difference: []
}
},
{
"$project": {
_id: 1
}
}
])
How it works:
At first, the map the converted into an array, and then that array is mapped to the keys.
Then the difference, between the keys array and the list of ids is calculated.
Then all documents having empty differences are picked up and their _id is projected.
Using these ids, you can simply do:
db.collection.remove({_id: {$in: [// the list here]}});
Playground for the aggregation.
try this it might help:
Get keys in a single document
You can also use aggregation to get keys in a single document:
db.activities.aggregate([ {"$project":{"arrayofkeyvalue":{"$objectToArray":"$$ROOT"}}}, {"$project":{"keys":"$arrayofkeyvalue.k"}} ])
to delete:
db['name1.name2.name3.Properties'].remove([ { "key" : "name_key1" }, { "key" : "name_key2" }, { "key" : "name_key3" } )]
This could be also an answer:
db.collection.aggregate([
{
"$project": {
"mapAsArray": {
"$objectToArray": "$map"
}
}
},
{
"$match": {
"mapAsArray": {
"$not": {
"$elemMatch": {
"k": {
$nin: [
"3478956c-3a01-404f-84e7-2a076e165215",
"1e1d1efb-5bf9-48ac-93ca-4a2df5a9f7eb"
]
}
}
}
}
}
}
])
Here the mongoplayground
The map to spring-data-mongodb:
ProjectionOperation projectionOperation = project()
.and(ObjectOperators.valueOf(myMap).toArray()).as(myMapToArray);
MatchOperation matchOperation = match(where(myMapToArray)
.not()
.elemMatch(where("k").nin(myList)));
AggregationResults<myObject> aggregate = mongoTemplate.aggregate(newAggregation(projectionOperation, matchOperation),
myObject.class, myObject.class);
I have a requirement of fetching data from mongodb and being done using Java Spring Reactive Mongodb library. I am using the following code for this:
Criteria criteria = Criteria.where(QUERYFIELD1).in(listOfIds)
.andOperator(Criteria.where(QUERYFIELD2).gte(ChildDate));
Query query = Query.query(criteria).noCursorTimeout();
reactiveMongoTemplate.find(query, Document.class, COLLECTIONNAME);
Where QUERYFIELD1 is "ChildId" and QUERYFIELD2 is a "ChildDate". Following is the structure of my document:
{
"_id": {
"$oid": "6296968fa63757a93e1cd123"
},
"Level1": {
"Level2": [
{
"ChildId": "1234",
"ChildDate": {
"$date": "2021-04-01T04:00:00.000Z"
}
},
{
"ChildId": "5678",
"ChildDate": {
"$date": "2017-05-16T04:00:00.000Z"
}
},
{
"ChildId": "3456",
"ChildDate": {
"$date": "2008-09-16T04:00:00.000Z"
}
},
{
"ChildDate": {
"$date": "2022-06-01T04:00:00.000Z"
},
"ChildId": "7891"
}
]
}
}
I am trying to find a document which should match the criteria within the Objects under Level2. For e.g. if My criteria has ChildId as "3456" and ChildDate as "2022-06-01T04:00:00.000Z" then I should get empty results as ChildId is matching with Object3 and ChildDate is matching with Object4. But when I use below query, I get 1 record as the match:
{ "Level1.Level2.ChildId" : "3456", "Level1.Level2.ChildDate" : { $gt: new Date("2022-01-01T05:00:00.000+00:00")}}
I am trying to achieve this using Spring Reactive MongoDB. Please help.
You can use $elemMatch for finding the documents that their array includes an item which matches the conditions:
db.collection.find({
"Level1.Level2": {
$elemMatch: {
ChildId: "3456",
ChildDate: {$gt: new Date("2008-09-16T05:00:00.000Z")}
}
}
})
See how it works on the playground example
I am new to mongodb and aggregation framework.
We have a class UserMetaData and a list of UserMetaData. I need to fetch data according to the userMetaDataList that is passed to the method solve().
Currently I am iterating the list and one by one fetching the corresponding collection from the monogdb. Since the db calls are made for each element in the list, this becomes a highly expensive operation.
Is there any way to fetch all the required data from mongodb in one shot(more like a bulk fetch operation).
mongodb - perform batch query the solution provided in this does not fulfill the requirements of the current scenario.
Please help!!
This is how I am doing currently.
class UserMetaData{
String userId;
String vehicleId;
String vehicleColour;
String orderId;
}
public List<String> getOrderIds(List<UserMetaData> userMetaDataList) {
List<String> orderIds = new ArrayList<>();
for (UserMetaData userMetadata : userMetaDataList) {
try {
BasicDBObject matchDBObject = new BasicDBObject("user_id", new BasicDBObject("$eq", userMetadata.getUserId()));
matchDBObject.append("vehicle_id", new BasicDBObject("$eq", userMetadata.getVehicleID()));
matchDBObject.append("vehicle_colour", new BasicDBObject("$in", ImmutableSet.of("WHITE", "BLACK")));
Document document = eventCollection.find(matchDBObject)
.projection(new BasicDBObject("order_id", "1"))
.first();
orderIds.add(document.get("order_id").toString());
} catch (Exception e) {
log.info("Exception occurred while fetching order id for user_id: {} asset_id:{} - {}", metadata.getUserId(), metadata.getAssetID(), e);
}
}
return ordersIds;
}
I want to fetch all the corresponding data in a single query.
Requesting help.
You can join all filters with $OR condition and fetch the full list at once ...
I want to fetch all the corresponding data in a single query.
You can use this approach and perform the query as a single operation (avoids the for-loop).
Consider sample documents in the collection test:
{ "_id" : ObjectId("621762e2cda7c6394d557f37"), "userid" : 1, "name" : "ijk", "orderid" : "11" }
{ "_id" : ObjectId("621762efcda7c6394d557f38"), "userid" : 12, "name" : "abc", "orderid" : "99" }
{ "_id" : ObjectId("621762fccda7c6394d557f39"), "userid" : 13, "name" : "xyz", "orderid" : "100" }
The array of objects to filter:
var DOCS = [
{ userid: 12, name: "abc" },
{ userid: 13, name: "xyz" }
]
The query to filter by DOCS:
db.test.find(
{
$expr: {
$in: [ { userid: "$userid", name: "$name" }, DOCS ]
}
},
{
orderid: 1
}
)
The output has documents with userids 12 and 13.
[ EDIT - ADD ]
This aggregation an improvement over the find:
db.test.aggregate([
// This matches the 'userid' and 'name' fields with the input list 'DOCS'
{
$match: {
$expr: {
$in: [ { userid: "$userid", name: "$name" }, DOCS ]
}
}
},
// The grouping will select only the first matching for the 'userid' and 'name'
// (this is as per the question post's code: `.first()`)
{
$group: {
_id: {
userid: "$userid",
name: "$name"
},
orderid: {
$first: "$orderid"
}
}
},
// Remove the '_id' field
// Now the result has just the 'orderid' field only
{
$unset: "_id"
}
])
I have the following database:
{ stream :{ "name": "name1",
"error1": 1,
"error2": 1,
"error3": 1 }
}
,
{ stream : {"name": "name1",
"error1": 2,
"error2": 1,
"error3": 1 }
}
,
{ stream : {"name": "name2",
"error1": 1,
"error2": 1,
"error3": 1 }
}
I would like to group it by name and sum every time some different combination of errors.
this is what I did in mongo, I need to create the following query dynamically in java
db.collection.aggregate([{$group: {_id: "$stream.name",error1: {$sum:"$stream.error1" },error2: {$sum: "$stream.error2" }} ])
the thing is that every time I need different combinations of the errors:error1 with error2, only error 1 etc..
this is what I did: (the arguments in the "if" are some boolean values that I am getting)
List<String> totalError = new ArrayList<String>();
BasicDBObject group = new BasicDBObject( "$group", new BasicDBObject("_id","$stream.name" ));
if (error1)
{
group.append("error1",new BasicDBObject ("$sum", "$stream.error1" ));
}
if (error2)
{
group.append("error2",new BasicDBObject ("$sum", "$stream.error2" ));
}
if (error3)
{
group.append("error3",new BasicDBObject ("$sum", "$stream.error3" ));
}
the problem is that I am getting:
{ "$group" : { "_id" : "$stream.name"} , "error1" : { "$sum: "$stream.error1"} , "error2" : { "$sum" : "$stream.error2"}
},
instead of:
{ "$group" : { "_id" : "$stream.name", "error1" : { "$sum: "$stream.error1"} , "error2" : { "$sum" : "$stream.error2"}}
if I knew what error combination I need I could use append in the constructor of group dbobject.. but I don't know the combination and I need to use the "ifs"
Try
BasicDBObject fields = new BasicDBObject("_id","$stream.name" );
if (error1)
fields.append("error1",new BasicDBObject ("$sum","$stream.error1"));
if (error2)
fields.append("error2",new BasicDBObject ("$sum","$stream.error2"));
if (error3)
fields.append("error3",new BasicDBObject ("$sum","$stream.error3"));
BasicDBObject group = new BasicDBObject( "$group", fields);
You should use helper functions when possible.
List<BsonField> fieldAccumulators = new ArrayList<>();
if (error1)
fieldAccumulators.add(Accumulators.sum("error1","$stream.error1"));
if (error2)
fieldAccumulators.add(Accumulators.sum("error2","$stream.error2"));
if (error3)
fieldAccumulators.add(Accumulators.sum("error3","$stream.error3"));
collection.aggregate(Arrays.asList(Aggregates.group("$stream.name", fieldAccumulators)));
I am new to MongoDB, I want to remove an element from array structure as shown below:
{
"Data" : [
{
"url" : "www.adf.com"
"type":7
},
{
"url" : "www.pqr.com"
"type":2
}
{
"url" : "www.adf.com"
"type":3
},
{
"url" : "www.pqr.com"
"type":5
}
],
}
I want to remove url=www.adf.com which has type as lowest values i.e in this document my query should remove type=3 and return document as below:
{
"Data" : [
{
"url" : "www.adf.com"
"type":7
},
{
"url" : "www.pqr.com"
"type":2
}
{
"url" : "www.pqr.com"
"type":5
}
],
}
The query shown by #shakthydoss can be described in java as follows:
MongoClient mongoClient = new MongoClient("SERVER", 27017);
DB db = mongoClient.getDB("DB_NAME");
DBCollection coll1 = db.getCollection("COLLECTION_NAME");
DBObject eleMatch = new BasicDBObject();
eleMatch.put("url", "www.pqr.com");
eleMatch.put("type", new BasicDBObject("$lte", 50));
BasicDBObject up = new BasicDBObject();
up.put("$elemMatch", eleMatch);
BasicDBList basicDBList = new BasicDBList();
basicDBList.add(up);
DBObject query = new BasicDBObject("Data", new BasicDBObject(" $all", basicDBList));
coll1.find(query);
Use $all with $elemMatch
If the field contains an array of documents, you can use the $all with the $elemMatch operator.
db.inventory.find( {
Data: { $all: [
{ "$elemMatch" : { url : "www.pqr.com": "M", type: { $lte: 50} } },
] }
} )