Develop Reference

Mongo Connection is created multiple times in RESTful API and never released

I have written a RESTful API using Apache Jersey. I am using MongoDB as my backend. I used Morphia (v.1.3.4) to map and persist POJO to database. I tried to follow "1 application 1 connection" in my API as recommended everywhere but I am not sure I am successful. I run my API in Tomcat 8. I also ran Mongostat to see the details and connection. At start, Mongostat showed 1 connection to MongoDB server. I tested my API using Postman and it was working fine. I then created a load test in SoapUI where I simulated 100 users per second. I saw the update in Mongostat. I saw there were 103 connections. Here is the gif which shows this behaviour.
I am not sure why there are so many connections. The interesting fact is that number of mongo connection are directly proportional to number of users I create on SoapUI. Why is that? I found other similar questions but I think I have implemented there suggestions.
Mongo connection leak with morphia
Spring data mongodb not closing mongodb connections
My code looks like this.
DatabaseConnection.java
// Some imports
public class DatabaseConnection {
private static volatile MongoClient instance;
private static String cloudhost="localhost";
private DatabaseConnection() { }
public synchronized static MongoClient getMongoClient() {
if (instance == null ) {
synchronized (DatabaseConnection.class) {
if (instance == null) {
ServerAddress addr = new ServerAddress(cloudhost, 27017);
List<MongoCredential> credentialsList = new ArrayList<MongoCredential>();
MongoCredential credentia = MongoCredential.createCredential(
"test", "test", "test".toCharArray());
credentialsList.add(credentia);
instance = new MongoClient(addr, credentialsList);
}
}
}
return instance;
}
}
PourService.java
#Secured
#Path("pours")
public class PourService {
final static Logger logger = Logger.getLogger(Pour.class);
private static final int POUR_SIZE = 30;
#POST
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public Response createPour(String request)
{
WebApiResponse response = new WebApiResponse();
Gson gson = new GsonBuilder().setDateFormat("dd/MM/yyyy HH:mm:ss").create();
String message = "Pour was not created.";
HashMap<String, Object> data = null;
try
{
Pour pour = gson.fromJson(request, Pour.class);
// Storing the pour to
PourRepository pourRepository = new PourRepository();
String id = pourRepository.createPour(pour);
data = new HashMap<String, Object>();
if ("" != id && null != id)
{
data.put("id", id);
message = "Pour was created successfully.";
logger.debug(message);
return response.build(true, message, data, 200);
}
logger.debug(message);
return response.build(false, message, data, 500);
}
catch (Exception e)
{
message = "Error while creating Pour.";
logger.error(message, e);
return response.build(false, message, new Object(),500);
}
}
PourDao.java
public class PourDao extends BasicDAO<Pour, String>{
public PourDao(Class<Pour> entityClass, Datastore ds) {
super(entityClass, ds);
}
}
PourRepository.java
public class PourRepository {
private PourDao pourDao;
final static Logger logger = Logger.getLogger(PourRepository.class);
public PourRepository ()
{
try
{
MongoClient mongoClient = DatabaseConnection.getMongoClient();
Datastore ds = new Morphia().map(Pour.class)
.createDatastore(mongoClient, "tilt45");
pourDao = new PourDao(Pour.class,ds);
}
catch (Exception e)
{
logger.error("Error while creating PourDao", e);
}
}
public String createPour (Pour pour)
{
try
{
return pourDao.save(pour).getId().toString();
}
catch (Exception e)
{
logger.error("Error while creating Pour.", e);
return null;
}
}
}

When I work with Mongo+Morphia I get better results using a Factory pattern for the Datastore and not for the MongoClient, for instance, check the following class:
public DatastoreFactory(String dbHost, int dbPort, String dbName) {
final Morphia morphia = new Morphia();
MongoClientOptions.Builder options = MongoClientOptions.builder().socketKeepAlive(true);
morphia.getMapper().getOptions().setStoreEmpties(true);
final Datastore store = morphia.createDatastore(new MongoClient(new ServerAddress(dbHost, dbPort), options.build()), dbName);
store.ensureIndexes();
this.datastore = store;
}
With that approach, everytime you need a datastore you can use the one provided by the factory. Of course, this can implemented better if you use a framework/library that support factory pattern (e.g.: HK2 with org.glassfish.hk2.api.Factory), and also singleton binding.
Besides, you can check the documentation of MongoClientOptions's builder method, perhaps you can find a better connection control there.

Can't find a codec for class com.mongodb.BasicDBObject error in Java using mondodb

I am using mongodb's async driver 3.0.2 (http://mongodb.github.io/mongo-java-driver/3.0/driver-async/) with Java.
I am trying to find the 10 closest documents to a place. The following query I would use in a mongodb shell to accomplish this:
db.locations.find( { loc :
{ $geoWithin :
{ $centerSphere :
[ [ 40 , -40 ] , 10 / 3963.2 ]
} } } ).limit(10);
I need to though run this in java so created the query below, but when I run it I get this exception:
org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class com.mongodb.BasicDBObject.
CODE:
BasicDBObject geometery = new BasicDBObject("$centerSphere", asList(
asList(40, -40), 10 / 3963.2));
BasicDBObject operator = new BasicDBObject("$geoWithin", geometery);
BasicDBObject query = new BasicDBObject("loc", operator);
Block<Document> postsBlock = new Block<Document>() {
#Override
public void apply(final Document document) {
System.out.println(document.toJson());
}
};
SingleResultCallback<Void> postsCallback = new SingleResultCallback<Void>() {
#Override
public void onResult(final Void result, final Throwable t) {
System.out.println("Operation Finished!");
}
};
try {
collection.find(query).limit(10).forEach(postsBlock, postsCallback);
} catch (Exception exc) {
exc.printStackTrace();
}

In your connection you need to specify the codec registry com.mongodb.MongoClient.getDefaultCodecRegistry() should do fine
For the async driver
MongoClientSettings settings = MongoClientSettings.builder().readPreference(readPreference)
.codecRegistry(com.mongodb.MongoClient.getDefaultCodecRegistry()).socketSettings(sockSettings)
.connectionPoolSettings(connPoolSettings).credentialList(credentials))
.clusterSettings(clusterSettings).build();
LOG.info("MongoClientSettings: {}, {}, {}, {}", sockSettings, connPoolSettings, clusterSettings, credentials);
MongoClient mgc = MongoClients.create(settings);
for the normal driver
MongoClientOptions settings = MongoClientOptions.builder().readPreference(readPreference)
.codecRegistry(com.mongodb.MongoClient.getDefaultCodecRegistry()).build();
MongoClient mgc= new MongoClient(servers,credentials,settings);

How to check if a Mongo repository is up and running [duplicate]

I wonder, if there is a way to check if mongoDB server is running from java driver for mongoDB?
According to the tutorial, I can do
Mongo m = new Mongo();
// or
Mongo m = new Mongo( "localhost" , 27017 );
// and
DB db = m.getDB( "mydb" );
But how to check that I can use these Mongo and DB? I see no isConnected() method in the API.
db.getConnector().isOpen()
returns true
The only way I found is call db.getDatabaseNames() and catch MongoException.
If there some more civilized approach?

You can run a ping command
Mongo mongo = new Mongo();
DBObject ping = new BasicDBObject("ping", "1");
try {
mongo.getDB("dbname").command(ping);
} catch (MongoException e) {
...
}

I've found this to be more direct than the ping command:
Mongo mongo = new Mongo();
try {
mongo.getConnector().getDBPortPool(mongo.getAddress()).get().ensureOpen();
} catch (Exception e) {
...
}

if there is a way to check if mongoDB server is running from java driver for MongoDB?
So if you can do the following:
Mongo m = new Mongo( "localhost" , 27017 );
DB db = m.getDB( "mydb" );
Then you are connected to the database, otherwise that m.getDB() would be throwing an exception. If you can connect to the database, then the MongoDB server is running.
The only way I found is call db.getDatabaseNames() and catch MongoException.
If there some more civilized approach?
Is there something specifically wrong with this approach?
The driver basically runs in a sandbox where it can or cannot connect. You're asking the driver to know something specific about the server (is process X running?), but that's not the driver's job. It can either connect or it can't, it's not responsible for operating the service/process, just for connecting to it.
To know that the process is actually running, you need administrative functions on that server that allow you to check that mongod is indeed running with the correct parameters.

public boolean keepAlive(Mongo mongo) {
return mongo.getAddress() != null;
}
This will return null for address if mongo is down. You can look within the implementation of getAddress() to see why it is a good way to check the mongo's status.
I assume you've initialized the mongo parameter properly.

I haven't tested this thoroughly (only using a localhost mongo) but it appears to work so far:
public boolean mongoRunningAt(String uri) {
try {
Mongo mongo = new Mongo(new MongoURI(uri));
try {
Socket socket = mongo.getMongoOptions().socketFactory.createSocket();
socket.connect(mongo.getAddress().getSocketAddress());
socket.close();
} catch (IOException ex) {
return false;
}
mongo.close();
return true;
} catch (UnknownHostException e) {
return false;
}
}
And the tests I've used:
#Test
public void whenMongoNotAvailableAtSpecificURLThenTheLoaderKnows() {
assertThat(mongoRunningAt("mongodb://127.0.0.1:12345"), is(false));
}
#Test
public void whenMongoAvailableAtSpecificURLThenTheLoaderKnows() {
assertThat(mongoRunningAt("mongodb://127.0.0.1:27017"), is(true));
}
It's not exactly using a well defined public API so use at your own risk.

How do I change my java code to use replset? with Spring Data and MongoDB

I would like to know how to change my java code to support replset with spring-data and MongoDB.
I have 3 MongoDB servers running.. example:
./mongod --dbpath=/home/jsmith/tmp/db1 --replSet=spring --port=27017
./mongod --dbpath=/home/jsmith/tmp/db2 --replSet=spring --port=27027
./mongod --dbpath=/home/jsmith/tmp/db3 --replSet=spring --port=27037
if I do rs.status() I can see that if the db on 27017 goes down then one of the others become primary so I know that mongoDB is working right but in my java code if I try to run it I get the following error:
Exception in thread "main" org.springframework.dao.DataAccessResourceFailureException: can't call something : /127.0.0.1:27017/demo
Its looking only on port 27017
here is my mongodbconfig:
#Configuration
#EnableMongoRepositories
#ComponentScan(basePackageClasses = {MongoDBApp.class})
#PropertySource("classpath:application.properties")
public class MongoConfiguration extends AbstractMongoConfiguration {
#Override
protected String getDatabaseName() {
return "demo";
}
#Override
public Mongo mongo() throws Exception {
return new Mongo(new ArrayList<ServerAddress>() {{ add(new ServerAddress("127.0.0.1", 27017)); add(new ServerAddress("127.0.0.1", 27027)); add(new ServerAddress("127.0.0.1", 27037)); }});
}
#Override
protected String getMappingBasePackage() {
return "com.xxxx.mongodb.example.domain";
}
}
how do I change it to support replset? but if its reading and one of the servers goes down I get a error.. anyway to make in reconnect?

The URI method should work, or there's a clearer way to initialise the replica set using a list of servers:
final List<ServerAddress> seeds = Arrays.asList(new ServerAddress("127.0.0.1", 27017),
new ServerAddress("127.0.0.1", 27027),
new ServerAddress("127.0.0.1", 27037));
final Mongo mongo = new Mongo(seeds);

This is how I do it:
String mongoURI="mongodb://myUsrName:pass#mongoServer-001.company.com:27017,mongoServer-002.company.com:27017,mongoServer-003.company.com:27017/myDBname?waitqueuemultiple=1500&w=1&maxpoolsize=40&safe=true";
MongoURI uri = new MongoURI(mongoURI);
Mongo mongo = new Mongo(uri);
I specify the 3 servers in the URI (along with extra parameters like max pool size).
The third server (mongoServer-003) is the arbiter and it doesn't store any info. The arbiter helps in the election of the primary server when the current primary goes down. Take a look at this article.
With this configuration, the app can keep working even if the primary server goes down.

You can also do it the CustomEditorConfigurer and a class that implements PropertyEditorRegistrar.
So you your configuration class needs this:
#Bean
public static CustomEditorConfigurer customEditorConfigurer(){
CustomEditorConfigurer configurer = new CustomEditorConfigurer();
configurer.setPropertyEditorRegistrars(
new PropertyEditorRegistrar[]{new ServerAddressPropertyEditorRegistrar()});
return configurer;
}
#Override
protected String getDatabaseName() {
return authenticationDb;
}
#Override
#Bean
public MongoClient mongoClient() {
MongoClient mongoClient = new MongoClient(Arrays.asList(new ServerAddress(host, port)), mongoCredentials(), mongoClientOptions());
return mongoClient;
}
public final class ServerAddressPropertyEditorRegistrar implements PropertyEditorRegistrar {
#Override
public void registerCustomEditors(PropertyEditorRegistry registry) {
registry.registerCustomEditor(ServerAddress[].class, new ServerAddressPropertyEditor());
}
}

how to check from a driver, if mongoDB server is running

I wonder, if there is a way to check if mongoDB server is running from java driver for mongoDB?
According to the tutorial, I can do
Mongo m = new Mongo();
// or
Mongo m = new Mongo( "localhost" , 27017 );
// and
DB db = m.getDB( "mydb" );
But how to check that I can use these Mongo and DB? I see no isConnected() method in the API.
db.getConnector().isOpen()
returns true
The only way I found is call db.getDatabaseNames() and catch MongoException.
If there some more civilized approach?

You can run a ping command
Mongo mongo = new Mongo();
DBObject ping = new BasicDBObject("ping", "1");
try {
mongo.getDB("dbname").command(ping);
} catch (MongoException e) {
...
}

I've found this to be more direct than the ping command:
Mongo mongo = new Mongo();
try {
mongo.getConnector().getDBPortPool(mongo.getAddress()).get().ensureOpen();
} catch (Exception e) {
...
}

if there is a way to check if mongoDB server is running from java driver for MongoDB?
So if you can do the following:
Mongo m = new Mongo( "localhost" , 27017 );
DB db = m.getDB( "mydb" );
Then you are connected to the database, otherwise that m.getDB() would be throwing an exception. If you can connect to the database, then the MongoDB server is running.
The only way I found is call db.getDatabaseNames() and catch MongoException.
If there some more civilized approach?
Is there something specifically wrong with this approach?
The driver basically runs in a sandbox where it can or cannot connect. You're asking the driver to know something specific about the server (is process X running?), but that's not the driver's job. It can either connect or it can't, it's not responsible for operating the service/process, just for connecting to it.
To know that the process is actually running, you need administrative functions on that server that allow you to check that mongod is indeed running with the correct parameters.

public boolean keepAlive(Mongo mongo) {
return mongo.getAddress() != null;
}
This will return null for address if mongo is down. You can look within the implementation of getAddress() to see why it is a good way to check the mongo's status.
I assume you've initialized the mongo parameter properly.

I haven't tested this thoroughly (only using a localhost mongo) but it appears to work so far:
public boolean mongoRunningAt(String uri) {
try {
Mongo mongo = new Mongo(new MongoURI(uri));
try {
Socket socket = mongo.getMongoOptions().socketFactory.createSocket();
socket.connect(mongo.getAddress().getSocketAddress());
socket.close();
} catch (IOException ex) {
return false;
}
mongo.close();
return true;
} catch (UnknownHostException e) {
return false;
}
}
And the tests I've used:
#Test
public void whenMongoNotAvailableAtSpecificURLThenTheLoaderKnows() {
assertThat(mongoRunningAt("mongodb://127.0.0.1:12345"), is(false));
}
#Test
public void whenMongoAvailableAtSpecificURLThenTheLoaderKnows() {
assertThat(mongoRunningAt("mongodb://127.0.0.1:27017"), is(true));
}
It's not exactly using a well defined public API so use at your own risk.