Develop Reference

MongoDB: How to get all objects from one array inside one Collection field

At the moment, I have the problem that I can't find a document by its ID and then get the Grades.
Grades consist of multiple elements, and each of them has multiple fields.
So, an arrays with multiple objects.
I basically just want to get the grades corresponding to the ID of the document.
My Collection
{
"_id" : ObjectId("5c6eba356fd07954a489067c"),
"StudentName" : "David",
"StudentAge" : 28
"Grades": [{"subject", "grade"}, ...]
}
{
same
}
My code so far. Unfortunately I can't get any further,
since I haven't found anything remotely helpful on the interent for hours
Document findByIdQuery = new Document(ID, new ObjectId(id));
Filterabel<StudentEntity> grades = mongoCollection().find(findByIdQuery);
also tried
Filterabel<StudentEntity> grades =
mongoCollection().find(findByIdQuery).projection(
Projections.fields(
Projections.computed("grades", lt("grades.qty", 20) )))

Try this code :
MongoCollection<Document> customCollection = mongoTemplate.getCollection("<YOUR_COLLECTION_NAME>");
BasicDBObject criteria = new BasicDBObject("_id", new ObjectId("<YOUR_OBECT_ID>"));
FindIterable<Document> cursor = customCollection.find(criteria);
MongoCursor<Document> iterator = cursor.iterator();
while (iterator.hasNext()) {
Document doc = iterator.next();
List<Document> grades = doc.get("Grades", List.class);
System.out.println(grades);
for(Document grade : grades){
String subject= grade.get("subject", String.class);
String grade= grade.get("grade", String.class);
}
}
iterator.close();

Remove document from array in MongoDB Java

I got a JSON string that looks something like this:
String tmp = "[
{
"ID":"12",
"Date":"2018-02-02",
"ObjData":[
{
"Name":"AAA",
"Order":"12345",
"Extra1":{
"Temp":"3"
},
"Extra2":{
"Temp":"5"
}
},
{
"Name":"BBB",
"Order":"54321",
"Extra1":{
"Temp":"3"
},
"Extra2":{
"Temp":"5"
}
}
]
}
]"
I would like to remove for example the the document where ´Order´ equals "54321" from ´ObjData´. I got the following code:
Document doc = new Document();
doc = Document.parse(tmp);
Document fields = new Document("ID", "12")
.append("ObjData", Arrays.asList(new Document("Order", "54321")));
Document update = new Document("$pull", fields);
coll.updateOne(doc, update);
I am trying to use the ´pull´ method to remove the entire document from the array where the ´Order´ equals 54321 but for some reason it's not working, I am probably doing something wrong. Could someone point out the issue please?
Also, what would be the best way to keep count of the documents within the array so that once all documents are pulled the entire document is deleted from the database? Would it be good to add some kind of ´size´ attribute and keep track of the size and decrease it after each pull?

To remove document with Order=54321 from internal array from any document (if you don't know ID) you can use empty filter like:
Document filter = new Document();
Document update = new Document("$pull", new Document("ObjData", new Document("Order", "54321")));
coll.updateOne(filter, update);

Updating records to remove values from ObjData array
The first parameter to the updateOne method is a query to find the document you want to update, not the full document.
So for your code, assuming ID is a unique value and that there's an item in your collection with an ID of "12":
// { ID: "12" }
Document query = new Document("ID", "12");
// { ObjData: { $pull: { Order: "54321" } } }
Document update = new Document("ObjData",
new Document("$pull",
new Document("Order", "54321")
)
);
coll.updateOne(query, update);
Alternatively, if you want to remove the order from all documents in the database, just replace query with an empty Document, i.e.:
// { <empty> }
Document query = new Document();
Deleting records with empty ObjData array
As for removing records when the size reaches zero, you can use a filter with $size:
db.myColl.deleteMany({ ObjData: { $size: 0 } })
This is also doable using the Java driver:
// { ObjData: { $size: 0 } }
Document query = new Document("ObjData",
new Document("$size", 0)
);
coll.deleteMany(query);
Note that for large collections (i.e. where myColl is large, not the ObjData array), this may not perform very well. If this is the case, then you may want to track the size separately (as you hinted at in your question) and index it to make it faster to search on since you can't create an index on array size in MongoDB.
References
updateOne documentation for updating documents using the Java driver
deleteOne documentation for deleting documents using the Java driver
$pull documentation for removing documents from an array
$size documentation for filtering documents based on the size of an array

Library to create JSON file from parsed text document

I parsed a timetable from a text document and reached the step where I have to create a JSON file from the raw data like in the sample below. Is there any library which could help me create a JSON formatted file from a raw text document?
I appreciate any help.
Sample how it could look like:
{"route": 1
"info": [
{"direction": "Surrey Quays"},
{"stops": [{"stops_name": "Lancaster Place"},
{"arrival_time":{
"mon-fri": ["04:41", "05:41", "06:09"],
"sat": [ "05:38", "06:07","06:37"]
}
}
]
}
Some sample from the text document
Surrey Quays
1
Lancaster Place
mon-fri 04:41 05:41 06:09
sat 04:41 05:41 06:09
Edit:
for (Entry<String, List<String>> entry : map.entrySet()) {
String key = entry.getKey();
List<String> timeEntries = entry.getValue();
JSONObject timeTable = new JSONObject();
timeTable.put("route", route);
JSONObject info = new JSONObject();
info.put("direction", direction);
JSONObject stops = new JSONObject();
stops.put("stops_name", key);
JSONObject arrivals = new JSONObject();
JSONArray arrivalMoFr = new JSONArray();
JSONArray someArray = new JSONArray(timeEntries);
arrivalMoFr.put( someArray);
arrivals.put("mon-fri", arrivalMoFr);
stops.put("arrival_time", arrivals);
info.put("stops", stops);
timeTable.put("info", info);
System.out.println(timeTable.toString(3));
}
**Some of the result **
"arrival_time": {"mon-fri": [[
"05:04",
"05:39",
"19:11",
"19:41",
"20:11"
]]},

You could use this JSON library from json.org . But it is just one example of all the libraries out there.
This is how you could use it:
Let's say you already have your parser (meaning you already have a method that can read the text document and that knows what to do with the data)
JSONObject timeTable = new JSONObject(); // first create the "main" JSON Object
timeTable.put("route", 1); // this first entry is just a number
JSONObject info = new JSONObject(); // now you need a second JSON OBject to put all the info in
info.put("direction", "Surrey Quays"); // first info, the direction
JSONObject stops = new JSONObject(); // now you need another Object for all the stops and stop related info
stops.put("stops_name", "Tension Way"); // like the name for the stop
JSONObject arrivals = new JSONObject(); // now all the arrivals
JSONArray arrivalMoFr = new JSONArray(); // this is the first time where an array makes sense here
arrivalMoFr.put("05:38"); // arrival time 1
arrivalMoFr.put("06:07"); // arrival time 2
arrivalMoFr.put("06:37"); // ...
arrivals.put("mon-fri",arrivalMoFr); // add the arrival times array as mon-fri to the arraivals object, do the same with all other arrival times (Saturday, Sunday...)
stops.put("arrival_time", arrivals); // now add the arrival time object to your stops
info.put("stops", stops); // and put the stops to your info
timeTable.put("info", info); // once you added all your infos you can put the info into your timeTable
System.out.println(timeTable.toString(3)); // this is just for testing. The number 3 tells you how many whitespaces you want for text-identaion
And this is the output I am getting:
{
"route": 1,
"info": {
"stops": {
"arrival_time": {"mon-fri": [
"05:38",
"06:07",
"06:37"
]},
"stops_name": "Tension Way"
},
"direction": "Surrey Quays"
}
}
I realize this is not exactly the same as in your example. But I think you get the idea. Remember, the order of the elements in a JSON Object is irrelevant. When reading the JSON, the program will read it like json.get("stops_name"); and it does not care if the stops name are before or after the arrival times.
ADDITION
I saw that you put the "stops_name" and the "arrival_time"-array in separate JSON Objects. Well if you want to put them all in an object called "stops", which suggests, that you will list more than one stop, I suggest to put them together. Because the data in the JSON object has no particular order.
"stops": [
{
"stop_name" : "Lanvester Place",
"arrival" : {...}
},{
"stop_name" : "Some other Place",
"arrival" : {...}
}
...
]
To your EDIT:
your are getting double braces [[ because you are adding an array to your arrivals Object where the first entry is again an array.
so instead of doing this:
JSONArray arrivalMoFr = new JSONArray();
JSONArray someArray = new JSONArray(timeEntries);
arrivalMoFr.put( someArray);
arrivals.put("mon-fri", arrivalMoFr);
you should do this:
JSONArray arrivalMoFr = new JSONArray(timeEntries); // put your list of time entries directly into the arrivalMoFr array
arrivals.put("mon-fri", arrivalMoFr); // then add the arrivalMoFr array to the arrivals object

You need to parse the text file first. I sugest to create dedicated Java classes for your data. Make sure the classes reflect the structure of the desired JSON.
Once you have that you can feed youf Java object to a JSON library which turns it into JSON for free. I'm using GSON for that. I'm sure you will find some more candidates on SO or Google.

Insert JSON Array into mongodb

I'm trying to insert a string that represents a JSON array into a mongodb collection with this,
String str = "[{\"id\":1,\"data\":\"data1\"},{\"id\":2,\"data\":\"data2\"},{\"id\":3,\"data\":\"data3\"}]";
DBObject dbObject = (DBObject) JSON.parse(str);
collection.insert(dbObject);
But I get the exception,
Exception in thread "main" java.lang.IllegalArgumentException: BasicBSONList can only work with numeric keys, not: [_id]
Can anyone show me the correct way to do this?

String json = "[{\"id\":1,\"data\":\"data1\"},{\"id\":2,\"data\":\"data2\"},{\"id\":3,\"data\":\"data3\"}]";
MongoCredential credential = MongoCredential.createCredential("root", "sample", "root".toCharArray());
MongoClient mongoClient = new MongoClient(new ServerAddress("localhost"), Arrays.asList(credential));
MongoDatabase db = mongoClient.getDatabase("sample");
MongoCollection<Document> collection = db.getCollection("loginTracking");
List<Document> jsonList = new ArrayList<Document>();
net.sf.json.JSONArray array = net.sf.json.JSONArray.fromObject(json);
for (Object object : array) {
net.sf.json.JSONObject jsonStr = (net.sf.json.JSONObject) JSONSerializer.toJSON(object);
Document jsnObject = Document.parse(jsonStr.toString());
jsonList.add(jsnObject);
}
collection.insertMany(jsonList);
mongoClient.close();

as per java doc the insert() can accept either single DBObject or an array or List of them.
So, in order to save, you need to convert your JSON array into an array/List of DBObjects, or save each array's item

I found a good way for achieve that:
(ArrayList<Document>) JSON.parse("[String json array]");
I had a problem with this, because i need append to this document a property that is a Json Array:
Document objAddendumVersion = new Document();
objAddendumVersion.append("_id", new ObjectId());
objAddendumVersion.append("Array", My Array here!);
But the problem is that Document.parse() doesn't work with Arrays, so i could solve it using the above line. So the final code is:
Document objAddendumVersion = new Document();
objAddendumVersion.append("_id", new ObjectId());
objAddendumVersion.append("Array", (ArrayList<Document>) JSON.parse("[String json array]"));
And it works perfect. Yes i know that exist more better ways for do that, but for the moment i'm using this.
I wait that be useful for someone with the same trouble.

Upgrading mongo search from runCommand to find using Java driver

I am using a Java driver to run some mongo text searches.
An example of my previous code is (where values is a String passed in):
DBCollection coll = db.getCollection("testCollection");
//create the basic object
DBObject searchCmd = new BasicDBObject();
//create the search cmd
searchCmd.put("text", "testCollection"); // the name of the collection (string)
// define the search word
searchCmd.put("search", value); // the term to search for (string)
// define the return values
searchCmd.put("project", new BasicDBObject("score", 1).append("name", 1).append("path", 0).append("_id", 0));
// get the results
BasicDBObject commandResult = db.command(searchCmd);
// Just out the results key
BasicDBList results = (BasicDBList) commandResult.get("results");
then I loop over the "results" and I get for each it score by
// Get the number ii
BasicDBObject element = (BasicDBObject) results.get(ii);
// Now get the score
double score = (double) element.get("score");
I want to upgrade to use find since that seems the way 2.6 and later prefers it. So far I have:
DBCollection coll = db.getCollection("testCollection");
BasicDBObject query = new BasicDBObject();
query.append("$text", new BasicDBObject("$search", value));
DBCursor cursor = coll.find(query);
However, I am not sure how to get the score.
I tried doing something like:
query.append("score", new BasicDBObject("$meta", "textScore"));
But this does not work. I would like to be able to get the name and the score so that I can then insert them into a new collection that will also hold the score.
I can get the name easily by:
while (cursor.hasNext())
{
DBObject next = cursor.next();
String name = next.get("name").toString();
}
But how do I get the score?

I found this interesting page: http://api.mongodb.org/java/current/
it appears that find can take a second DBObject which has the fields.
I created a new object:
BasicDBObject fields = new BasicDBObject();
fields.append("score", new BasicDBObject("$meta", "textScore"));
and I am calling find using:
DBCursor cursor = coll.find(query, fields);
and now I can get the score the same way I can get the name.