I'm currently having some problems with the usage of java, aggregations and the mongodb.
I have 2 collections in a mongodb.
example collection: person
{
id: 1
name: "Oliver"
companyId: 5
}
example collection: company
{
id: 5
name: asdf
}
Now I want to join those collections by companyId/id (lookup aggregation?) and want to iterate over the result. I dont want to load the whole resultset in to the memory, rather iterate 1 by 1. I think i need some kind of cursor (mongoCursor?).
Im working with Java and Spring. So I have the possibilities to use the Java Mongo Driver (version: 3.7.1) or the Options which provides the Springframework (version 5.0.6).
edit:
In the following example Cursor.hasNext() is always false.
DBObject match = new BasicDBObject("$match",
new BasicDBObject("companyId", "id"));
DBObject lookupFields = new BasicDBObject("from", "company");
lookupFields.put("localField", "companyId");
lookupFields.put("foreignField", "id");
lookupFields.put("as", "personWithCompany");
DBObject lookup = new BasicDBObject("$lookup", lookupFields);
DBObject projectFields = new BasicDBObject("id", 1);
projectFields.put("name", 1);
projectFields.put("companyName", "$company.name);
List<DBObject> pipeline = Arrays.asList(match, lookup, project);
Cursor cursor = mongoTemplate.getCollection("person").aggregate(pipeline, AggregationOptions.builder().allowDiskUse(true).build());
while (cursor.hasNext()) {
DBObject dbObject = cursor.next();
}
person.aggregate([
"$lookup": {
"from": "company",
"localField": "companyId",
"foreignField": "id",
"as": "PersonDetails"
},
{$unwind: "$PersonDetails"}
]);
joining with Person Collection with company
Related
Suppose we have the following documents in a MongoDB collection:
{
"_id":ObjectId("562e7c594c12942f08fe4192"),
"shapes":[
{
"shape":"square",
"color":"blue"
},
{
"shape":"circle",
"color":"red"
}
]
},
{
"_id":ObjectId("562e7c594c12942f08fe4193"),
"shapes":[
{
"shape":"square",
"color":"black"
},
{
"shape":"circle",
"color":"green"
}
]
}
And the MongoDB query is
db.test.find({"shapes.color": "red"}, {_id: 0, 'shapes.$': 1});
Can someone tell me how to write it in Java?
I am using:
List<BasicDBObject> obj = new ArrayList<>();
obj1.add(new BasicDBObject("shapes.color", "red"));
List<BasicDBObject> obj1 = new ArrayList<>();
obj2.add(new BasicDBObject("shapes.$", "1"));
BasicDBObject parameters1 = new BasicDBObject();
parameters1.put("$and", obj1);
DBCursor cursor = table.find(parameters1,obj2).limit(500);
and I am not getting anything.
The syntax of the Mongo Shell find function is:
db.collection.find(query, projection)
query document Optional. Specifies selection filter using query operators. To return all documents in a collection, omit this parameter or pass an empty document ({}).
projection document Optional. Specifies the fields to return in the documents that match the query filter.
When translating this for execution by the Mongo Java driver you need to construct separate BasicDBObject instances for;
the query
the projection
Here's an example:
MongoCollection<Document> table = ...;
// {"shapes.color": "red"}
BasicDBObject query = new BasicDBObject("shapes.color", "red");
// {_id: 0, 'shapes.$': 1}
BasicDBObject projection = new BasicDBObject("shapes.$", "1").append("_id", 0);
FindIterable<Document> documents = table
// assign the query
.find(query)
// assign the projection
.projection(projection);
System.out.println(documents.first().toJson());
Given the sample documents included in your question the above code will print out:
{
"shapes": [
{
"shape": "circle",
"color": "red"
}
]
}
This is identical to the output from db.test.find({"shapes.color": "red"}, {_id: 0, 'shapes.$': 1});.
I would like to retrieve the following information:
select names from database where address like 'colombo' and age>20;
but for MongoDB in Java. Essentially, it should return all names that contain the word colombo ang age greater than 20 in them. I know that there is the $in operator in MongoDB, but how do I do the same in Java, using the Java driver? I've been trying to look for it everywhere but am getting nothing. I've tried:
query = new BasicDBObject("names", new BasicDBObject("$in", "colombo"), new BasicDBObject("age", "$gt20"));
But it didn't worked :( Please help!
Try this
BasicDBObject query = new BasicDBObject("names", new BasicDBObject("$in", Arrays.asList("colombo")));
query.append("age", new BasicDBObject("$gt", 20));
FindIterable<Document> find = collection.find(query);
MongoCursor<Document> iterator = find.iterator();
Document doc = null;
while (iterator.hasNext()) {
doc = iterator.next();
System.out.println(doc);
}
The $in operator will not be suitable for such as you can only use it to match values that are in an array or to search for documents where the value of a field equals any value in a specified array.
In your case you need a $regex operator to fulfil the query by performing a SQL LIKE operation:
db.collection.find({
"names": { "$regex": /colombo/, "$options": "i" },
"age": { "$gt": 20 }
})
or
db.collection.find({
"names": /colombo/i },
"age": { "$gt": 20 }
})
which can be implemented in Java as
Pattern pattern = Pattern.compile("colombo", Pattern.CASE_INSENSITIVE);
BasicDBObject query = new BasicDBObject("names", pattern)
.append("$age", new BasicDBObject("$gt", 20));
DBCursor result = coll.find(query);
If using the 3.0.x and newer drivers:
Document regx = new Document();
regx.append("$regex", "(?)" + Pattern.quote("colombo"));
regx.append("$options", "i");
Document query = new Document("names", regx).append("$age", new Document("$gt", 20));
FindIterable<Document> iterable = db.getCollection("coll").find(query);
I'm rather new to MongoDB and I'm trying to create a query which I though would be pretty trivial (well, alteast with SQL it would) but I can't get it done.
So have a collection patients in this collections a single patient is identified using the id property. (NOT mongodbs _id!!) There can be multiple version of a single patient, his version is determined by the meta.versionId field.
In order to query for all "current versions of patients" I need to get for every patient with a specific id the one with the maximum versionId.
So far I've got this:
AggregateIterable<Document> allPatients = db.getCollection("patients").aggregate(Arrays.asList(
new Document("$group", new Document("_id", "$id")
.append("max", new Document("$max", "$meta.versionId")))));
allPatients.forEach(new Block<Document>() {
#Override
public void apply(final Document document) {
System.out.println(document.toJson());
}
});
Which results in the following output (using my very limited test data):
{ "_id" : "2.25.260185450267055504591276882440338245053", "max" : "5" }
{ "_id" : "2.25.260185450267055504591276882441338245099", "max" : "0" }
Seems to work so far, but I need to get the whole patients collection.
Now I only know that for the id : 2.25.260185450267055504591276882440338245053 the max version is "5" and so on. Of course I could now create an own query for every single entry and sequentially get each patient document for a specific id/versionId-combo from mongodb but this seems like a terrible solution! Is there any other way to get it done?
If you know the columns that you want to retrieve , say patient name , address, etc I guess you can append those columns to the document with value 1.
AggregateIterable<Document> allPatients = db.getCollection("patients").aggregate(Arrays.asList(
new Document("$group", new Document("_id", "$id")
.append("max", new Document("$max", "$meta.versionId")).append("name",1).append("address",1))));
An approach that could work for you would be to first order the documents getting in the pipeline by the meta.versionId field using the $sort pipeline operator. However, be aware that the $sort stage has a limit of 100 megabytes of RAM. By default, if it exceeds this limit, $sort will produce an error.
To allow for the handling of large datasets, set the allowDiskUse option to true to enable $sort operations to write to temporary files. See the allowDiskUse option in aggregate() method for details.
After sorting, you can then group the ordered documents, carry out the aggregation using the $first or $last operators (depending on the previous sort direction) to get the other fields.
Consider running the following mongo shell pipeline operation as a way of
demonstrating this concept:
Mongo shell
pipeline = [
{ "$sort": {"meta.versionId": -1}}, // order the documents by the versionId field descending
{
"$group": {
"_id": "$id",
"max": { "$first": "$meta.versionId" }, // get the maximum versionId
"active": { "$first": "$active" }, // Whether this patient's record is in active use
"name": { "$first": "$name" }, // A name associated with the patient
"telecom": { "$first": "$telecom" }, // A contact detail for the individual
"gender": { "$first": "$gender" }, // male | female | other | unknown
"birthDate": { "$first": "$birthDate" } // The date of birth for the individual
/*
And many other fields
*/
}
}
]
db.patients.aggregate(pipeline);
Java test implementation
public class JavaAggregation {
public static void main(String args[]) throws UnknownHostException {
MongoClient mongo = new MongoClient();
DB db = mongo.getDB("test");
DBCollection coll = db.getCollection("patients");
// create the pipeline operations, first with the $sort
DBObject sort = new BasicDBObject("$sort",
new BasicDBObject("meta.versionId", -1)
);
// build the $group operations
DBObject groupFields = new BasicDBObject( "_id", "$id");
groupFields.put("max", new BasicDBObject( "$first", "$meta.versionId"));
groupFields.put("active", new BasicDBObject( "$first", "$active"));
groupFields.put("name", new BasicDBObject( "$first", "$name"));
groupFields.put("telecom", new BasicDBObject( "$first", "$telecom"));
groupFields.put("gender", new BasicDBObject( "$first", "$gender"));
groupFields.put("birthDate", new BasicDBObject( "$first", "$birthDate"));
// append any other necessary fields
DBObject group = new BasicDBObject("$group", groupFields);
List<DBObject> pipeline = Arrays.asList(sort, group);
AggregationOutput output = coll.aggregate(pipeline);
for (DBObject result : output.results()) {
System.out.println(result);
}
}
}
I am using MongoDB in my application and was needed to insert multiple documents inside a MongoDB collection .
The version I am using is of 1.6
I saw an example here
http://docs.mongodb.org/manual/core/create/
in the
Bulk Insert Multiple Documents Section
Where the author was passing an array to do this .
When I tried the same , but why it isn't allowing , and please tell me how can I insert multiple documents at once ??
package com;
import java.util.Date;
import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.MongoClient;
public class App {
public static void main(String[] args) {
try {
MongoClient mongo = new MongoClient("localhost", 27017);
DB db = mongo.getDB("at");
DBCollection collection = db.getCollection("people");
/*
* BasicDBObject document = new BasicDBObject();
* document.put("name", "mkyong"); document.put("age", 30);
* document.put("createdDate", new Date()); table.insert(document);
*/
String[] myStringArray = new String[] { "a", "b", "c" };
collection.insert(myStringArray); // Compilation error at this line saying that "The method insert(DBObject...) in the type DBCollection is not applicable for the arguments (String[])"
} catch (Exception e) {
e.printStackTrace();
}
}
}
Please let me know what is the way so that I can insert multiple documents at once through java .
DBCollection.insert accepts a parameter of type DBObject, List<DBObject> or an array of DBObjects for inserting multiple documents at once. You are passing in a string array.
You must manually populate documents(DBObjects), insert them to a List<DBObject> or an array of DBObjects and eventually insert them.
DBObject document1 = new BasicDBObject();
document1.put("name", "Kiran");
document1.put("age", 20);
DBObject document2 = new BasicDBObject();
document2.put("name", "John");
List<DBObject> documents = new ArrayList<>();
documents.add(document1);
documents.add(document2);
collection.insert(documents);
The above snippet is essentially the same as the command you would issue in the MongoDB shell:
db.people.insert( [ {name: "Kiran", age: 20}, {name: "John"} ]);
Before 3.0, you can use below code in Java
DB db = mongoClient.getDB("yourDB");
DBCollection coll = db.getCollection("yourCollection");
BulkWriteOperation builder = coll.initializeUnorderedBulkOperation();
for(DBObject doc :yourList)
{
builder.insert(doc);
}
BulkWriteResult result = builder.execute();
return result.isAcknowledged();
If you are using mongodb version 3.0 , you can use
MongoDatabase database = mongoClient.getDatabase("yourDB");
MongoCollection<Document> collection = database.getCollection("yourCollection");
collection.insertMany(yourDocumentList);
As of MongoDB 2.6 and 2.12 version of the driver you can also now do a bulk insert operation. In Java you could use the BulkWriteOperation. An example use of this could be:
DBCollection coll = db.getCollection("user");
BulkWriteOperation bulk = coll.initializeUnorderedBulkOperation();
bulk.find(new BasicDBObject("z", 1)).upsert().update(new BasicDBObject("$inc", new BasicDBObject("y", -1)));
bulk.find(new BasicDBObject("z", 1)).upsert().update(new BasicDBObject("$inc", new BasicDBObject("y", -1)));
bulk.execute();
Creating Documents
There're two principal commands for creating documents in MongoDB:
insertOne()
insertMany()
There're other ways as well such as Update commands. We call these operations, upserts. Upserts occurs when there're no documents that match the selector used to identify documents.
Although MongoDB inserts ID by it's own, We can manually insert custom IDs as well by specifying _id parameter in the insert...() functions.
To insert multiple documents we can use insertMany() - which takes an array of documents as parameter. When executed, it returns multiple ids for each document in the array. To drop the collection, use drop() command. Sometimes, when doing bulk inserts - we may insert duplicate values. Specifically, if we try to insert duplicate _ids, we'll get the duplicate key error:
db.startup.insertMany(
[
{_id:"id1", name:"Uber"},
{_id:"id2", name:"Airbnb"},
{_id:"id1", name:"Uber"},
]
);
MongoDB stops inserting operation, if it encounters an error, to supress that - we can supply ordered:false parameter. Ex:
db.startup.insertMany(
[
{_id:"id1", name:"Uber"},
{_id:"id2", name:"Airbnb"},
{_id:"id1", name:"Airbnb"},
],
{ordered: false}
);
Your insert record format like in MongoDB that query retire from any source
EG.
{
"_id" : 1,
"name" : a
}
{
"_id" : 2,
"name" : b,
}
it is mongodb 3.0
FindIterable<Document> resulutlist = collection.find(query);
List docList = new ArrayList();
for (Document document : resulutlist) {
docList.add(document);
}
if(!docList.isEmpty()){
collectionCube.insertMany(docList);
}
I'm using mongo 2.2.3 and the java driver.
My dilemma, I have to $push a field and value into an array, but I cant seem to figure out how to do this. A sample of my data:
"_id" : 1,
"scores" : [
{
"type" : "homework",
"score" : 78.97979
},
{
"type" : "homework",
"score" : 6.99
},
{
"type" : "quiz",
"score" : 99
}
]
I can $push in the shell:
db.collection.update({_id:1},{$push:{scores:{type:"quiz", score:99}}})
but it's when I translate this into java I confuse my self and chuck my keyboard at a wall.
my java code (incomplete and wrong) so far:
DBObject find = new BasicDBObject("_id", 1);
DBObject push = new BasicDBObject("$push", new BasicDBObject(
"scores", new BasicDBObject()));
DBObject listItem = new BasicDBObject("scores", new BasicDBObject("type","quiz").append("score",99));
DBObject updateQuery = new BasicDBObject("$push", listItem);
myCol.update(findQuery, updateQuery);
Since mongodb-driver 3.1. there is a builder class com.mongodb.client.model.Updates with appropriate methods for each update case. In this case this would be:
Document score = new Document().append("type", "quiz")
.append("score",99);
collection.updateOne(eq("_id", "1"),Updates.addToSet("scores", score));
If you're more comforable with the query format of the shell, you may find it's easier to use JSON.parse to contstruct your DBObject for the $push:
import com.mongodb.util.JSON;
String json = "{$push:{scores:{type:'quiz', score:99}}}";
DBObject push = (DBObject) JSON.parse(json);
Using Jongo, you can do as in the shell:
db.collection.update({_id:1},{$push:{scores:{type:"quiz", score:99}}})
Becomes in Java:
collection.update("{_id:1}").with("{$push:{scores:{type:#, score:#}}}", "quiz", 99);
No fancy DBObject needed ;-)
MongoDB Java driver can simplify this. Use $each instead of $push.
$each mongodb reference document
Java sample -
BasicDBObject addressSpec = new BasicDBObject();
addressSpec.put("id", new ObjectId().toString());
addressSpec.put("name", "one");
BasicDBObject addressSpec2 = new BasicDBObject();
addressSpec2.put("id", new ObjectId().toString());
addressSpec2.put("name", "two");
List<BasicDBObject> list = new ArrayList<>();
list.add(addressSpec); list.add(addressSpec2);
UpdateResult updateOne = individualCollection.updateOne(Filters.eq("_id", "5b7c6b612612242a6d34ebb6"),
Updates.pushEach("subCategories", list));