I am facing an issue with using Mongo DB on a Java servlet.
My servlet has many methods(~20) of accessing the database for retrieving and adding data. A very brief example of one :
public static String getSomething(String s) {
String json = "[]";
JSONArray jsonArray = new JSONArray();
DBCollection table;
try {
Mongo mongo = new Mongo("localhost", 27017);
DB db = mongo.getDB( "myDb" );
BasicDBObject quoteQuery = new BasicDBObject("abc", abc);
DBCursor cursor = table.find(quoteQuery);
try {
while(cursor.hasNext()) {
jsonArray.put(cursor.next());
}
} finally {
cursor.close();
}
// ...
Now the problem is when this Java servlet is deployed in the linux server, it works fine for 10 days or so.
After that it crashes.
When I go to mongodb.log in my var/log directory I get the following repetitive output:
"connection refused because too many open connections"
I am not sure on where to edit things now or how to deal with this. I have tried to grow the limit of open connections in the server but still have the same results.
Any suggestions?
from the API doc : http://api.mongodb.org/java/2.11.3/
public class Mongo extends Object
A database connection with internal connection pooling. For most applications, you should have one Mongo instance for the entire JVM.
You should create Mongo objects very sparingly, ideally even only one per classloader at any time. To reduce the number of Mongo objects you could create it in the servlet's init method and re-use that instance on every call.
EDIT: just had a look at our code, we manage the Mongo instance using a classic singleton class (and always fetch a Mongo using that class's getInstance() method) because if you have multiple servlets / entrypoints in your app just using init() will still generate one instance per servlet, and still won't satisfy the manual section cited by #FredClose
You may create the mongo object for once instead of creating it on each getSomething call.
public SomeClass{
static Mongo mongo = new Mongo("localhost", 27017);
static DB db = mongo.getDB( "myDb" );
public static String getSomething(String s) {
String json = "[]";
JSONArray jsonArray = new JSONArray();
DBCollection table;
try {
BasicDBObject quoteQuery = new BasicDBObject("abc", abc);
DBCursor cursor = table.find(quoteQuery);
while(cursor.hasNext()) {
jsonArray.put(cursor.next());
}
}
Actually the ideal case is not using static access at all and inject DB object from a central controller.
Your are creating connections in MongoDB, but you are not closing connections. For any database, it is very very important to close a connection, otherwise it will reach to its maximum limit and you wont be able to execute your program properly. Following code will be helpful i hope:
public static String getSomething(String s) {
String json = "[]";
JSONArray jsonArray = new JSONArray();
try {
MongoClient mongoClient = new MongoClient("localhost", 27017);
DB db = mongoClient.getDB("myDb");
DBCollection collection = db.getCollection("NAME OF YOUR COLLECTION");
BasicDBObject quoteQuery = new BasicDBObject("abc", "VARIABLE THAT YOU WANT TO FIND");
DBCursor cursor = collection.find(quoteQuery);
try {
while (cursor.hasNext()) {
jsonArray.put(cursor.next());
}
} finally {
cursor.close();
}
mongoClient.close();
} catch (Exception e) {
}
return jsonArray.toString();
}
In this code, 'MongoClient' is closed after its purpose is over.
Arun Gupta #arungupta
New sample shows how to use Mongo within a #JavaEE7 app: New sample to show basic usage of Mongo in a Java EE application
As per above mentioned issue, its like you are creating the Mongo Object for every request.I will suggest to use the single Object through out your application.FOr this you can find the "MongoClient and connection pooling".
MongoClient will handle connection pooling for you automatically.
mongoClient = new MongoClient(URI, connectionOptions);
Here the mongoClient object holds your connection pool, and will give your app connections as needed. You should strive to create this object once as your application initializes and re-use this object throughout your application to talk to your database. The most common connection pooling problem we see results from applications that create a MongoClient object way too often, sometimes on each database request. If you do this you will not be using your connection pool as each MongoClient object maintains a separate pool that is not being reused by your application.
Related
I am trying to find how to use mongo Atlas search indexes, from java application, which is using spring-data-mongodb to query the data, can anyone share an example for it
what i found was as code as below, but that is used for MongoDB Text search, though it is working, but not sure whether it is using Atlas search defined index.
TextQuery textQuery = TextQuery.queryText(new TextCriteria().matchingAny(text)).sortByScore();
textQuery.fields().include("cast").include("title").include("id");
List<Movies> movies = mongoOperations
.find(textQuery, Movies.class);
I want smaple java code using spring-data-mongodb for below query:
[
{
$search: {
index: 'cast-fullplot',
text: {
query: 'sandeep',
path: {
'wildcard': '*'
}
}
}
}
]
It will be helpful if anyone can explain how MongoDB Text Search is different from Mongo Atlas Search and correct way of using Atalas Search with the help of java spring-data-mongodb.
How to code below with spring-data-mongodb:
Arrays.asList(new Document("$search",
new Document("index", "cast-fullplot")
.append("text",
new Document("query", "sandeep")
.append("path",
new Document("wildcard", "*")))),
new Document())
Yes, spring-data-mongo supports the aggregation pipeline, which you'll use to execute your query.
You need to define a document list, with the steps defined in your query, in the correct order. Atlas Search must be the first step in the pipeline, as it stands. You can translate your query to the aggregation pipeline using the Mongo Atlas interface, they have an option to export the pipeline array in the language of your choosing. Then, you just need to execute the query and map the list of responses to your entity class.
You can see an example below:
public class SearchRepositoryImpl implements SearchRepositoryCustom {
private final MongoClient mongoClient;
public SearchRepositoryImpl(MongoClient mongoClient) {
this.mongoClient = mongoClient;
}
#Override
public List<SearchEntity> searchByFilter(String text) {
// You can add codec configuration in your database object. This might be needed to map
// your object to the mongodb data
MongoDatabase database = mongoClient.getDatabase("aggregation");
MongoCollection<Document> collection = database.getCollection("restaurants");
List<Document> pipeline = List.of(new Document("$search", new Document("index", "default2")
.append("text", new Document("query", "Many people").append("path", new Document("wildcard", "*")))));
List<SearchEntity> searchEntityList = new ArrayList<>();
collection.aggregate(pipeline, SearchEntity.class).forEach(searchEntityList::add);
return searchEntityList;
}
}
I wonder how many roundtrips that are made to the server when using transactions MongoDB? For example if the Java driver is used like this:
ClientSession clientSession = client.startSession();
TransactionOptions txnOptions = TransactionOptions.builder()
.readPreference(ReadPreference.primary())
.readConcern(ReadConcern.LOCAL)
.writeConcern(WriteConcern.MAJORITY)
.build();
TransactionBody txnBody = new TransactionBody<String>() {
public String execute() {
MongoCollection<Document> coll1 = client.getDatabase("mydb1").getCollection("foo");
MongoCollection<Document> coll2 = client.getDatabase("mydb2").getCollection("bar");
coll1.insertOne(clientSession, new Document("abc", 1));
coll2.insertOne(clientSession, new Document("xyz", 999));
return "Inserted into collections in different databases";
}
};
try {
clientSession.withTransaction(txnBody, txnOptions);
} catch (RuntimeException e) {
// some error handling
} finally {
clientSession.close();
}
In this case two documents are stored in a transaction:
coll1.insertOne(clientSession, new Document("abc", 1));
coll2.insertOne(clientSession, new Document("xyz", 999));
Are the "insert operations" stacked up and sent to the server in one roundtrip or are two calls (or more?) actually made to the server?
Each insert is sent separately. You can use use bulk writes to batch write operations together.
The commit at the end is a separate operation also.
im new to MongoDB and completely confused by the queries. I simply need to update a document in a mongodb database by adding a string value (example: Temperature) to a list of strings. From research I know that I have to use the $push method for that. I think the code has to look somehow like this:
BasicDBObject newDocument = new BasicDBObject().append("$set",
new BasicDBObject().append("Subscribed Topics", topic));
collection.update(new BasicDBObject().append("Sensor Type", sensorType), newDocument);
new BasicDBObject("$push",
new BasicDBObject("Subscribed Topics", topic));
The field with the array is called "Subscribed Topics", "topic" is a String (Temperature). Then I want to update the document in the collection with the corresponding "Sensor Type". However, I do not really know how to call the $push part correctly. I hope someone can help me sort this part of the code.
Best regards.
Update, I tried to implemented as suggested in the duplicate question but still got error. Very unsure if thats the right way anyway.
DBObject listItem = new BasicDBObject("Subscribed Topics", "Light");
DBObject updateQuery = new BasicDBObject("$push", listItem);
collection.update(query, updateQuery);`
I create a new Object with the value Light in for Key Subscribed Topics (the array). Why do I push it to a new Object then?
My goodness! This question got me descending into the long forgotten world of Java again - after all these years... ;) Anyhoo, here's a complete working example that might give you a clue of what's going on. You can run the code several times and see how the number of elements in the "Subscribed Topics" array increases.
I used the following driver: https://oss.sonatype.org/content/repositories/releases/org/mongodb/mongo-java-driver/3.3.0/mongo-java-driver-3.3.0.jar
import com.mongodb.MongoClient;
import com.mongodb.client.MongoDatabase;
import com.mongodb.client.MongoCollection;
import org.bson.Document;
import org.bson.conversions.Bson;
import static com.mongodb.client.model.Filters.*;
import static com.mongodb.client.model.Updates.*;
public class MongoDbPush {
public static void main(String[] args)
{
MongoClient mongoClient = new MongoClient();
MongoDatabase database = mongoClient.getDatabase("pushExampleDb");
MongoCollection<Document> collection = database.getCollection("pushExampleCollection");
String sensorType = "Temperature";
// try to load existing document from MongoDB
Document document = collection.find(eq("Sensor Type", sensorType)).first();
if(document == null)
{
// no test document, let's create one!
document = new Document("Sensor Type", sensorType);
// insert it into MongoDB
collection.insertOne(document);
// read it back from MongoDB
document = collection.find(eq("Sensor Type", sensorType)).first();
}
// see what it looks like in JSON (on the first run you will notice that it has got an "_id" but no "Subscribed Topics" array yet)
System.out.println(document.toJson());
// update the document by adding an entry to the "Subscribed Topics" array
Bson filter = eq("Sensor Type", sensorType);
Bson change = push("Subscribed Topics", "Some Topic");
collection.updateOne(filter, change);
// read one more time from MongoDB
document = collection.find(eq("Sensor Type", sensorType)).first();
// see what the document looks like in JSON (on the first run you will notice that the "Subscribed Topics" array has been created and has got one element in it)
System.out.println(document.toJson());
mongoClient.close();
}
}
The above method still works, however, with updated Mongo Driver the below is also a viable mechanism.
The below works for Mongo Driver 3.6 onward (in this case using 3.12.4)
MongoClient mongoClient = new MongoClient();
MongoDatabase database = mongoClient.getDatabase("pushExampleDb");
MongoCollection<Document> collection = database.getCollection("pushExampleCollection");
collection.findOneAndUpdate(Filters.eq("Sensor Type",<theSensorTypeNameComesHere>),
Updates.pushEach("Subscribed Topics",<listContainingTheValuesComeHere>));
Refer: $push and $each from MongoDB Manual
Not necessarily specific to dropwizard, but for the life of me I can't figure out how to easily create a healthcheck for mongodb. This is in java, using version 3.3.0 of mongodb's own java driver.
I was hoping there would be a method that doesn't change the state of the database if it succeeds, but also throws an Exception when the query (or connection, or whatever) fails in order to return a health or unhealthy state. Ideally I'd perform a find, but this doesn't throw an Exception as far as I can tell.
I would just list all collections in database like:
MongoClient client = new MongoClient(addr, opts);
MongoDatabase db = client.getDatabase(database);
try {
MongoIterable<String> allCollections = db.listCollectionNames();
for (String collection : allCollections) {
System.out.println("MongoDB collection: " + collection);
}
} catch (Exception me) {
// problems with mongodb
}
this question is very similar to another post
I basically want to use the mongodb version of the sql "like" '%m%' operator
but in my situation i'm using the java api for mongodb, while the other post is using mongodb shell
i tried what was posted in the other thread and it worked fine
db.users.find({"name": /m/})
but in java, i'm using the put method on the BasicDBObject and passing it into the find() method on a DBCollections object
BasicDBObject q = new BasicDBOBject();
q.put("name", "/"+m+"/");
dbc.find(q);
but this doesn't seem to be working.
anyone has any ideas?
You need to pass an instance of a Java RegEx (java.util.regex.Pattern):
BasicDBObject q = new BasicDBObject();
q.put("name", java.util.regex.Pattern.compile(m));
dbc.find(q);
This will be converted to a MongoDB regex when sent to the server, as well as any RegEx flags.
You must first quote your text and then use the compile to get a regex expression:
q.put("name", Pattern.compile(Pattern.quote(m)));
Without using java.util.Pattern.quote() some characters are not escaped.
e.g. using ? as the m parameter will throw an exception.
To make it case insensitive:
Document doc = new Document("name", Pattern.compile(keyword, Pattern.CASE_INSENSITIVE));
collection.find(doc);
In spring data mongodb, this can be done as:
Query query = new Query();
query.limit(10);
query.addCriteria(Criteria.where("tagName").regex(tagName));
mongoOperation.find(query, Tags.class);
Document doc = new Document("name", Pattern.compile(keyword));
collection.find(doc);
Might not be the actual answer, ( Executing the terminal query directly )
public void displayDetails() {
try {
// DB db = roleDao.returnDB();
MongoClient mongoClient = new MongoClient("localhost", 5000);
DB db = mongoClient.getDB("test");
db.eval("db.test.update({'id':{'$not':{'$in':[/su/]}}},{$set:{'test':'test3'}},true,true)", new Object[] {});
System.out.println("inserted ");
} catch (Exception e) {
System.out.println(e);
}
}
if(searchType.equals("employeeId"))
{
query.addCriteria(Criteria.where(searchType).regex(java.util.regex.Pattern.compile(searchValue)));
employees = mongoOperations.find(query, Employee.class, "OfficialInformation");
}