I got a JSON string that looks something like this:
String tmp = "[
{
"ID":"12",
"Date":"2018-02-02",
"ObjData":[
{
"Name":"AAA",
"Order":"12345",
"Extra1":{
"Temp":"3"
},
"Extra2":{
"Temp":"5"
}
},
{
"Name":"BBB",
"Order":"54321",
"Extra1":{
"Temp":"3"
},
"Extra2":{
"Temp":"5"
}
}
]
}
]"
I would like to remove for example the the document where ´Order´ equals "54321" from ´ObjData´. I got the following code:
Document doc = new Document();
doc = Document.parse(tmp);
Document fields = new Document("ID", "12")
.append("ObjData", Arrays.asList(new Document("Order", "54321")));
Document update = new Document("$pull", fields);
coll.updateOne(doc, update);
I am trying to use the ´pull´ method to remove the entire document from the array where the ´Order´ equals 54321 but for some reason it's not working, I am probably doing something wrong. Could someone point out the issue please?
Also, what would be the best way to keep count of the documents within the array so that once all documents are pulled the entire document is deleted from the database? Would it be good to add some kind of ´size´ attribute and keep track of the size and decrease it after each pull?
To remove document with Order=54321 from internal array from any document (if you don't know ID) you can use empty filter like:
Document filter = new Document();
Document update = new Document("$pull", new Document("ObjData", new Document("Order", "54321")));
coll.updateOne(filter, update);
Updating records to remove values from ObjData array
The first parameter to the updateOne method is a query to find the document you want to update, not the full document.
So for your code, assuming ID is a unique value and that there's an item in your collection with an ID of "12":
// { ID: "12" }
Document query = new Document("ID", "12");
// { ObjData: { $pull: { Order: "54321" } } }
Document update = new Document("ObjData",
new Document("$pull",
new Document("Order", "54321")
)
);
coll.updateOne(query, update);
Alternatively, if you want to remove the order from all documents in the database, just replace query with an empty Document, i.e.:
// { <empty> }
Document query = new Document();
Deleting records with empty ObjData array
As for removing records when the size reaches zero, you can use a filter with $size:
db.myColl.deleteMany({ ObjData: { $size: 0 } })
This is also doable using the Java driver:
// { ObjData: { $size: 0 } }
Document query = new Document("ObjData",
new Document("$size", 0)
);
coll.deleteMany(query);
Note that for large collections (i.e. where myColl is large, not the ObjData array), this may not perform very well. If this is the case, then you may want to track the size separately (as you hinted at in your question) and index it to make it faster to search on since you can't create an index on array size in MongoDB.
References
updateOne documentation for updating documents using the Java driver
deleteOne documentation for deleting documents using the Java driver
$pull documentation for removing documents from an array
$size documentation for filtering documents based on the size of an array
Related
Currently I am on the latest version of MongoDB and coding in Java. I am trying to update multiple documents at a time, but I need to find the corresponding document first after reading from a json file. I can do this one at a time, but I have about 6million documents that I need to search and update so wanted to do this properly.
This is how the Database looks like. I am searching by lifeId
{
"_id" : "wwwww",
"mId" : "xxxxx",
"lifeId" : yyyyy
}
Below is my json file:
{
"Items":
[
{"gseid":xxxxx,"mlifeno":xxxx,"firstname":"xxx","lastname":"xxx","emailaddress1":"xxxx#hotmail.com","dateofbirth":"06-AUG-60 12.00.00.000000000 AM","phonenumber":"xxxxxx","phonetype":"Home","street1":"xxxxx","city":"xxxxx","postalcode":"xxxx","preferred":1,"addresstype":"Home"}
,{"gseid":xxxx,"mlifeno":xxx,"firstname":"xxx","lastname":"xxx","emailaddress1":"xxxx#msn.com","dateofbirth":"07-AUG-48 12.00.00.000000000 AM","phonenumber":"xxxx","phonetype":"Mobile","street1":"xxxxx","city":"xxx","postalcode":"xxx","preferred":1,"addresstype":"Home"}
,{"gseid":xxxx,"mlifeno":xxx,"firstname":"xxx","lastname":"xxx","emailaddress1":"xxx#yahoo.com","dateofbirth":"06-MAR-71 04.00.00.000000000 PM","phonenumber":"xxxx","phonetype":"Home","street1":"xxxxx","street2":"xxxxx","city":"Bolingbrook","postalcode":"xxxx","preferred":1,"addresstype":"Home"}
]
}
I loop through the json file and can insert one at a time, but what I want to know is if it's possible to insert multiple (maybe 1000 at a time). I am searching by "mlifeno" in the json file and matching it to "lifeId" in the DB.
The below code iterates through the json file items
JSONArray itemsArr = (JSONArray) itemsObj.get("Items");
for(Object temp : itemsArr){
JSONObject d = (JSONObject) parser.parse(String.valueOf((JSONObject)temp));
Long mlifeno = Long.parseLong(String.valueOf(d.get("mlifeno"))); //mlifeno
// findAndAddCol(mlifeno,database,d); //find a document by mlifeno and update the columns
}
}
The below is where I find the database document and insert, one by one (which is what I don't want)
public static void findAndAddCol(Long mlife, MongoDatabase database, JSONObject temp){
MongoCollection<Document> collection = database.getCollection("CrosswalkColl");
JsonWriterSettings prettyPrint = JsonWriterSettings.builder().indent(true).build();
Bson filter = eq("lifeId",mlife); //find document by life number
//Update profile object after all inserted into object
BasicDBObject updateFields = new BasicDBObject();
JSONObject jsonPro = new JSONObject();
JSONArray arrPro = new JSONArray();
JSONObject last = new JSONObject();
temp.forEach((key,value) -> {
//Don't add life number or gseid to profile object
if(key.toString().equals("mlifeno")){
}else if(key.toString().equals("gseid")){
//add gseid
// Bson updateOperation = set(key.toString(),value);
updateFields.append(key.toString(),value);
}else{
jsonPro.put(key.toString(),value);
}
});
updateFields.append("Profile",jsonPro);
BasicDBObject setQuery = new BasicDBObject();
setQuery.append("$set", updateFields);
//List with Batch Operation
UpdateResult updateResult = collection.updateOne(filter, setQuery); //this will search for the correct document and update
The code above works, but what I want to do is loop through maybe a 1000 at a time and bulkwrite them instead of having to do this one by one.
I am a little confused as to how to add an element to an array in an exisiting mongodb document, or why my results are not showing properly and how I expect.
There is only one document in the collection and will only ever be one. The mongo document looks like when I do a db.collection-name.find.pretty() command in a mongo session on the command line:
{
"_id" : ObjectID("1234567890"),
"details" : {
...
},
"calculations" : [
{
"count" : 1,
"total" : 10,
"mean" : 2.5
},
{
"count" : 2,
"total" : 20,
"mean" : 6.4
}
]
}
I want to add another object to the calculations list.
The Java code I am running is based upon THIS example:
// Get the database and collection
MongoDatabase database = mongo.getDatabase(dataBaseName);
MongoCollection<Document> collection = database.getCollection(collectionName);
Document document = collection.find().first(); // will only ever be one document
// The object comes in as a Map
Map<String, Object> incomingMap = new HashMap<>();
incomingMap.put("count", 3);
incomingMap.put("total", 4);
incomingMap.put("mean", 7.9);
// convert to a Document
Document newDocument = new Document();
incomingMap.forEach((k, v) -> {
newDocument.append(k, v);
});
// append this to the collection - this is where I am confused
// for this example just hardcoding the _id value for simplicity
collection.updateOne(new Document("_id", "1234567890"), Updates.push("calculations", newDocument));
However when I do a System.out.println(collection.find().first()) in the code after this or db.collection-name.find.pretty() in a mongo session the new document has not been added. There are no errors thrown and completes fine.
What I am wondering is
Is the line collection.updateOne(new Document("_id", "1234567890"), Updates.push("calculations", newDocument)); correct?
Has it been added but not been saved - if so how do I save?
Can I do this at a document level, for example document.update(new Documen(Updates.push("calculations", newDocument)); or similar?
I have also tried collection.findAndUpdateOne(new Document("_id", "1234567890"), Updates.push("calculations", newDocument)); with the same result
Is how I am getting/hardcoding the document ID incorrect?
You have filter condition issue (your _id is ObjectId type)
new Document("_id", ObjectId("1234567890"))`
Always make sure your documents updated correctly. Look code fagment:
UpdateResult result = collection.updateOne(filter, update);
log.info("Update with date Status : " + result.wasAcknowledged());
log.info("Nº of Record Modified : " + result.getModifiedCount());
https://api.mongodb.com/java/3.1/com/mongodb/client/result/UpdateResult.html
I trying to perform a query in my mongo database using java code. I want to filter the result of the query using cursor. Basically I want to filter the cursor results two times. My query return some documents, and I want to filter them based on a field of document and a sub-field of the first field. For example:
DBCursor cursor = coll.find(query);
while(cursor.hasNext()) {
BasicDBObject obj = (BasicDBObject) cursor.next();
System.out.println(obj.getString("images"));
}
Return the image field from all quered documents. What should I do if I want to return the field "link" which is a subfield of field "images"? I tried obj.getString("images").getString("link"), however it doesn't work. Images is an array with three fields the first one is the filed "link". When the above return is the following: [ { "link" : "http://distilleryimage1.ak.instagram.com/fc7c5_7.jpg" , "phash" : "01000010101000101010111101" , "persons" : 1}] . I want to return just the first field link.
Just get images as ArrayList:
ArrayList<BasicDBObject> images = (ArrayList<BasicDBObject>)obj.get("images");
for(BasicDBObject image: images)
{
String link = image.getString("link");
.......
}
I am doing lucene search on my resources.I have a case where I search for a particular product and I need to do it on a grouping search via 'keywords' field.I can get to know the total number of products grouped by keywords associated with it.How can I get all the documents related to this search, so that I can retrieve other needed fields from it. I tried using AbstractAllGroupHeadsCollector but couldnt find and got confused with its usage.Here is my code.
Thanks in advance.
Integer totalGroupCount = null;
IndexReader ir = DirectoryReader.open(indexLocation);
IndexSearcher is = new IndexSearcher(ir);
GroupingSearch groupingSearch = new GroupingSearch("keywords");
groupingSearch.setGroupSort(Sort.RELEVANCE);
groupingSearch.setFillSortFields(true);
groupingSearch.setCachingInMB(4.0, true);
groupingSearch.setAllGroups(true);
//TermQuery query = new TermQuery(new Term("products", "wfa packages"));
TopGroups<BytesRef> result = groupingSearch.search(is, query, 0, 10);
// Render groupsResult...
totalGroupCount = result.totalGroupCount; // The group count
GroupDocs<BytesRef>[] d=result.groups;
System.out.println("total groups="+result.totalGroupedHitCount);
You have your GroupDocs array, that's most of the way there already. You can then get the scoreDocs from each GroupDocs, and lookup the document with the doc id, from ScoreDoc.doc, like:
for (GroupDocs<BytesRef> group : d) {
for (ScoreDoc scoredoc : group.scoreDocs) {
Document doc = is.doc(scoredoc.doc);
//Do stuff
}
}
How can I upsert data into mongodb collection with java-driver?
I try (with empty collection):
db.getCollection(collection).update(new BasicDBObject("_id", "12"), dbobject, true, false);
But document was created with _id == ObjectID(...). Not with "12" value.
This code (js) add document with _id = "12" as expected
db.metaclass.update(
{ _id:12},
{
$set: {b:1}
},
{ upsert: true }
)
mongo-java-driver-2.11.2
If you are using mongo-java driver 3, following .updateOne() method with {upsert, true} flag works.
void setLastIndex(MongoClient mongo, Long id, Long lastIndexValue) {
Bson filter = Filters.eq("_id", id);
Bson update = new Document("$set",
new Document()
.append("lastIndex", lastIndexValue)
.append("created", new Date()));
UpdateOptions options = new UpdateOptions().upsert(true);
mongo.getDatabase(EventStreamApp.EVENTS_DB)
.getCollection(EventCursor.name)
.updateOne(filter, update, options);
}
You cannot set _id if dbobject is just a document and does not contain an update operator eg: $set, $setOnInsert.
Just passing a document will replace the whole document meaning it doesn't set an _id a falls back to ObjectId
So your example works if you use an update operator eg:
db.getCollection(collection).update(
new BasicDBObject("_id", "12"),
new BasicDBObject("$set", new BasicDBObject("Hi", "world")), true, false)
You can use the replaceOne method and specify the ReplaceOptions (since 3.7) :
private static final ReplaceOptions REPLACE_OPTIONS
= ReplaceOptions.createReplaceOptions(new UpdateOptions().upsert(true));
db.getCollection(collection).replaceOne(new BasicDBObject("_id", "12"), dbobject, REPLACE_OPTIONS);
For older versions you can directly pass the UpdateOptions to the replaceOne method :
private static final UpdateOptions UPDATE_POLICY = new UpdateOptions().upsert(true);
db.getCollection(collection).replaceOne(new BasicDBObject("_id", "12"), dbobject, UPDATE_POLICY);
As mentioned in the documentation :
replaceOne() replaces the first matching document in the collection
that matches the filter, using the replacement document.
If upsert: true and no documents match the filter, replaceOne()
creates a new document based on the replacement document.
This is to upsert with scala driver which i couldnot find in web
con.updateOne(
equal("vendor_id", vendorId),
inc("views_count", f.views),
UpdateOptions().upsert(true))
to do so import the following
import org.mongodb.scala.model.UpdateOptions