Develop Reference

Java driver: how to get the objectId of an updated object with Mongodb's updateFirst method

I'm trying to get the objectId of an object that I have updated - this is my java code using the java driver:
Query query = new Query();
query.addCriteria(Criteria.where("color").is("pink"));
Update update = new Update();
update.set("name", name);
WriteResult writeResult = mongoTemplate.updateFirst(query, update, Colors.class);
Log.e("object id", writeResult.getUpsertedId().toString());
The log message returns null. I'm using a mongo server 3.0 on mongolab as I'm on the free tier so it shouldn't return null. My mongo shell is also:
MongoDB shell version: 3.0.7
Is there an easy way to return the object ID for the doc that I have just updated? What is the point of the method getUpsertedId() if I cannot return the upsertedId?
To do what I want, I currently have to issue two queries which is highly cumbersome:
//1st query - updating the object first
Query query = new Query();
query.addCriteria(Criteria.where("color").is("pink"));
Update update = new Update();
update.set("name", name);
WriteResult writeResult = mongoTemplate.updateFirst(query, update, Colors.class);
//2nd query - find the object so that I can get its objectid
Query queryColor = new Query();
queryColor.addCriteria(Criteria.where("color").is("pink"));
queryColor.addCriteria(Criteria.where("name").is(name));
Color color = mongoTemplate.findOne(queryColor, Color.class);
Log.e("ColorId", color.getId());
As per David's answer, I even tried his suggestion to rather use upsert on the template, so I changed the code to the below and it still does not work:
Query query = new Query();
query.addCriteria(Criteria.where("color").is("pink"));
Update update = new Update();
update.set("name", name);
WriteResult writeResult = mongoTemplate.upsert(query, update, Colors.class);
Log.e("object id", writeResult.getUpsertedId().toString());

Simon, I think its possible to achieve in one query. What you need is a different method called findAndModify().
In java driver for mongoDB, it has a method called findOneAndUpdate(filter, update, options).
This method returns the document that was updated. Depending on the options you specified for the method, this will either be the document as it was before the update or as it is after the update. If no documents matched the query filter, then null will be returned. Its not required to pass options, in that case it will return the document that was updated before update operation was applied.
A quick look at the mongoTemplate java driver docs here: http://docs.spring.io/spring-data/mongodb/docs/current/api/org/springframework/data/mongodb/core/FindAndModifyOptions.html tells me that you can use the method call:
public <T> T findAndModify(Query query,
Update update,
FindAndModifyOptions options,
Class<T> entityClass)
You can also change the FindAndModifyOptions class to take on an 'upsert' if the item was not found in the query.If it is found, the object will just be modified.

Upsert only applies if both
The update options had upsert on
A new document was actually created.
Your query neither has upsert enabled, nor creates a new document. Therefore it makes perfect sense that the getUpsertedId() returns null here.
Unfortunately it is not possible to get what you want in a single call with the current API; you need to split it into two calls. This is further indicated by the Mongo shell API for WriteResults:
The _id of the document inserted by an upsert. Returned only if an
upsert results in an insert.

This is an example to do this with findOneAndUpdate(filter,update,options) in Scala:
val findOneAndUpdateOptions = new FindOneAndUpdateOptions
findOneAndUpdateOptions.returnDocument(ReturnDocument.AFTER)
val filter = Document.parse("{\"idContrato\":\"12345\"}")
val update = Document.parse("{ $set: {\"descripcion\": \"New Description\" } }")
val response = collection.findOneAndUpdate(filter,update,findOneAndUpdateOptions)
println(response)

String Mongo delete all files from GridFs

Using Spring Mongo is there a way of deleting multiple files from Mongo(stored with GridFS) all at once using only query or something similar? For example if I have in collection .files a field Language I want to be able to delete all entries from .files and also from .chunks in a single query. Not sure if this is possible in Mongo (outside Spring).
Tried to use GridFsTemplate but the delete method calls GridFS.remove method (code from Spring-Mongodb)
public void delete(Query query) {
getGridFs().remove(getMappedQuery(query));
}
And this method gets all files in memory and then deletes them one by one:
public void remove( DBObject query ){
for ( GridFSDBFile f : find( query ) ){
f.remove();
}
}
void remove(){
_fs._filesCollection.remove( new BasicDBObject( "_id" , _id ) );
_fs._chunkCollection.remove( new BasicDBObject( "files_id" , _id ) );
}
UPDATE: from comments:
Yes, you can't perform such kind of queries in MongoDB. Also, there is no direct way of deleting bunch of files in GridFS using both mongofiles command line tool and mongo java driver.

To remove an entire collection at once, you can perform one of these operations:
If your collection is a MongoDB collection, you can delete it with
MongoDatabase db = client.getDatabase("db");
db.getCollection("MyCollectionName").drop();
If you are not certain collection named "MyCollectionName" exists, you should check that first, as described here.
If your collection is a GridFS collection, you can delete it with
MongoDatabase db = client.getDatabase("db");
db.getCollection("MyCollectionName.files").drop();
db.getCollection("MyCollectionName.chunks").drop();
This works, since GridFS collections consist of a .files and a .chunks collection behind the screens, and those are accessible (and removable) in the 'classic' way.

DBObject query;
//write appropriate query
List<GridFSDBFile> fileList = gfs.find(query);
for (GridFSDBFile f : fileList)
{
gfs.remove(f.getFilename());
// if you have not set fileName and
// your _id is of ObjectId type, then you can use
//gfs.remove((ObjectId) file.getId());
}

CouchBase Editing documents from code

IN order to test couchbase, I am needing to create servlet that edit in 1.000 JSON documents row '"flag": false' to '"flag":true'. How i can do this?
My view, that finds documents with row '"flag": false':
function (doc, meta) {
if (meta.type == "json" && doc.flag == false) {
emit(doc.flag, null);
}
}
My servlet, that print results:
doGet(....
View view = client.getView("des1", "flag");
Query query = new Query();
query.setIncludeDocs(true);
ViewResponse result = client.query(view, query);
for(ViewRow row : result) {
resp.getWriter().println(row.getId());
}
Sorry for my bad English)

You cannot edit the fields in the JSON document directly. What you must do is to retrieve the documents you want to update (you already get them from the view), convert them to a Java object, edit the "flag" property, serialize the Java object back to JSON and replace the document with the new one.
You can use GSON to take care of the conversion between a java object and json.

Update collection in MongoDb via Apache Spark using Mongo-Hadoop connector

I would like to update a specific collection in MongoDb via Spark in Java.
I am using the MongoDB Connector for Hadoop to retrieve and save information from Apache Spark to MongoDb in Java.
After following Sampo Niskanen's excellent post regarding retrieving and saving collections to MongoDb via Spark, I got stuck with updating collections.
MongoOutputFormat.java includes a constructor taking String[] updateKeys, which I am guessing is referring to a possible list of keys to compare on existing collections and perform an update. However, using Spark's saveAsNewApiHadoopFile() method with parameter MongoOutputFormat.class, I am wondering how to use that update constructor.
save.saveAsNewAPIHadoopFile("file:///bogus", Object.class, Object.class, MongoOutputFormat.class, config);
Prior to this, MongoUpdateWritable.java was being used to perform collection updates. From examples I've seen on Hadoop, this is normally set on mongo.job.output.value, maybe like this in Spark:
save.saveAsNewAPIHadoopFile("file:///bogus", Object.class, MongoUpdateWritable.class, MongoOutputFormat.class, config);
However, I'm still wondering how to specify the update keys in MongoUpdateWritable.java.
Admittedly, as a hacky way, I've set the "_id" of the object as my document's KeyValue so that when a save is performed, the collection will overwrite the documents having the same KeyValue as _id.
JavaPairRDD<BSONObject,?> analyticsResult; //JavaPairRdd of (mongoObject,result)
JavaPairRDD<Object, BSONObject> save = analyticsResult.mapToPair(s -> {
BSONObject o = (BSONObject) s._1;
//for all keys, set _id to key:value_
String id = "";
for (String key : o.keySet()){
id += key + ":" + (String) o.get(key) + "_";
}
o.put("_id", id);
o.put("result", s._2);
return new Tuple2<>(null, o);
});
save.saveAsNewAPIHadoopFile("file:///bogus", Object.class, Object.class, MongoOutputFormat.class, config);
I would like to perform the mongodb collection update via Spark using MongoOutputFormat or MongoUpdateWritable or Configuration, ideally using the saveAsNewAPIHadoopFile() method. Is it possible? If not, is there any other way that does not involve specifically setting the _id to the key values I want to update on?

I tried several combination of config.set("mongo.job.output.value","....") and several combination of
.saveAsNewAPIHadoopFile(
"file:///bogus",
classOf[Any],
classOf[Any],
classOf[com.mongodb.hadoop.MongoOutputFormat[Any, Any]],
mongo_config
)
and none of them worked.
I made it to work by using MongoUpdateWritable class as output of my map method:
items.map(row => {
val mongo_id = new ObjectId(row("id").toString)
val query = new BasicBSONObject()
query.append("_id", mongo_id)
val update = new BasicBSONObject()
update.append("$set", new BasicBSONObject().append("field_name", row("new_value")))
val muw = new MongoUpdateWritable(query,update,false,true)
(null, muw)
})
.saveAsNewAPIHadoopFile(
"file:///bogus",
classOf[Any],
classOf[Any],
classOf[com.mongodb.hadoop.MongoOutputFormat[Any, Any]],
mongo_config
)
The raw query executed in mongo is then something like this:
2014-11-09T13:32:11.609-0800 [conn438] update db.users query: { _id: ObjectId('5436edd3e4b051de6a505af9') } update: { $set: { value: 10 } } nMatched:1 nModified:0 keyUpdates:0 numYields:0 locks(micros) w:24 3ms

Spring Data MongoDB: BigInteger to ObjectId conversion

I have a problem with update query using Spring Data MongoDB. I retrieve some object's _id as BigInteger value. Then I want to make following query:
Query query = new Query(Criteria.where("_id").is(id));
Update update = new Update();
update.set("version",version);
mongoOperations.updateFirst(query, update, Audit.class);
Query part fails to match any documents since id value passed to is() somehow must be converted to ObjectId. I can't find any documentation on this kind of conversion. Will appreciate any help.
p.s.: SpringData Mongodb version 1.2

You can convert it also manually:
ObjectId convertedId = new ObjectId(bigInteger.toString(16));
Query query = new Query(Criteria.where("_id").is(convertedId));

You probably want to write a custom Spring converter BigInteger => ObjectId and ObjectId => BigInteger.
See the doc part here:
http://static.springsource.org/spring-data/data-document/docs/current/reference/html/#d0e2670
------UPDATE------
It seems that this kind of converter already exists in the Spring-Data-MongoDB library:
http://static.springsource.org/spring-data/data-document/docs/1.0.0.M1/api/org/springframework/data/document/mongodb/SimpleMongoConverter.ObjectIdToBigIntegerConverter.html
So you just have to specify it in your Spring configuration.

Alternatively you can add an 'id' field to your collection classes or potentially a base class and annotate it with org.springframework.data.annotation.Id, as below:
import org.springframework.data.annotation.Id;
public abstract class BaseDocument {
#Id
protected long id;
This will allow you to perform the queries of the form:
public boolean doesDocumentExist(Class clazz, long documentId) {
Query queryCriteria = new Query(Criteria.where("id").is(documentId));
return mongoTemplate.count(queryCriteria, clazz) == 1;
}
Annotating your own id field with '#Id' will store your id as the mongo objectId, therefore saving you from doing the conversion yourself.

//get the converter from the mongoTemplate
MappingMongoConverter converter = (MappingMongoConverter)mongoTemplate.getConverter();
//get the conversion service from the mongo converter
ConversionService conversionService = converter.getConversionService();
//iterate the status list and get the each id to add the arraylist
for(Status status: statusList){
ObjectId objectIdVal = conversionService.convert(status.getId(), ObjectId.class);
**//here status.getId() returns the BigInteger**
statusArrayList.add(objectIdVal);
}
//get the users list whose status is active and cancel
query.addCriteria(new Criteria().where("status.$id").in(statusArrayList));
List<User> usersList = mongoTemplate.find(query, User.class);

You can convert a BigIngeter to ObjectId using the hex representation of the BigInteger. However, an ObjectId is supposed to be exactly 24 characters long, and parsing a shorter string will fail in Java. Thus it's better to ensure that the hex representation is 0-padded appropriately:
String hexString24 = StringUtils.leftPad(bigInteger.toString(16), 24, "0");
ObjectId convertedId = new ObjectId(hexString24);
Query query = new Query(Criteria.where("_id").is(convertedId));