I've got a MongoDB cartridge installed on openshift and i'm having troubles with connecting to it from java code. Ip address, port and credentials are taken from openshift's RockMongo cartridge. The following method invocation:
public Document insert(String audio, String username) {
Document document = new Document();
document.put("username", username);
document.put("audio", audio);
document.put("timestamp", new Date());
collection.insertOne(document);
return document;
}
and this mongo client configuration:
private static MongoClient build() throws UnknownHostException {
if (mongoClient == null) {
mongoClient = new MongoClient(
new MongoClientURI( "mongodb://admin:password#X.X.X.X:27017/dbName"));
}
return mongoClient;
}
public static MongoCollection<Document> getCollection(String collectionName) {
try {
build();
} catch (UnknownHostException e) {
}
MongoDatabase db = mongoClient.getDatabase(dbName);
MongoCollection<Document> collection = db.getCollection(collectionName);
return collection;
}
results in INFO: No server chosen by PrimaryServerSelector from cluster description ClusterDescription, and exception: Timed out after 30000 ms while waiting for a server that matches PrimaryServerSelector.
EDIT: I can't connect with mongoDB service on openshift via mongo terminal application either: "exception: connect failed", so I think it's openshift configuration issue. Port forwarding and the service itself are started
I suppose you have not correctly configure cluster (message in logs told about this problem), I'm not sure how OpenShift Cratridge works, but I recommend you to check if it has mondo-db correctly started. Check it via ssh client and run mongo-db command to check its status and if started. Take a look on this question: Java MongoClient cannot connect to primary, I suppose it give you some idea how to check where you have problem.
Related
I am using datastax java driver 3.1.0 to connect to cassandra cluster and my cassandra cluster version is 2.0.10.
Below is the singleton class I am using to connect to cassandra cluster.
public class CassUtil {
private static final Logger LOGGER = Logger.getInstance(CassUtil.class);
private Session session;
private Cluster cluster;
private static class Holder {
private static final CassUtil INSTANCE = new CassUtil();
}
public static CassUtil getInstance() {
return Holder.INSTANCE;
}
private CassUtil() {
List<String> servers = TestUtils.HOSTNAMES;
String username =
TestUtils.loadCredentialFile().getProperty(TestUtils.USERNAME);
String password =
TestUtils.loadCredentialFile().getProperty(TestUtils.PASSWORD);
// is this right setting?
PoolingOptions poolingOptions = new PoolingOptions();
poolingOptions.setConnectionsPerHost(HostDistance.LOCAL, 4, 10).setConnectionsPerHost(
HostDistance.REMOTE, 2, 4);
Builder builder = Cluster.builder();
cluster =
builder
.addContactPoints(servers.toArray(new String[servers.size()]))
.withRetryPolicy(DowngradingConsistencyRetryPolicy.INSTANCE)
.withPoolingOptions(poolingOptions)
.withReconnectionPolicy(new ConstantReconnectionPolicy(100L))
.withLoadBalancingPolicy(
DCAwareRoundRobinPolicy
.builder()
.withLocalDc(
!TestUtils.isProduction() ? "DC2" : TestUtils.getCurrentLocation()
.get().name().toLowerCase()).build())
.withCredentials(username, password).build();
try {
session = cluster.connect("testkeyspace");
StringBuilder sb = new StringBuilder();
Set<Host> allHosts = cluster.getMetadata().getAllHosts();
for (Host host : allHosts) {
sb.append("[");
sb.append(host.getDatacenter());
sb.append(host.getRack());
sb.append(host.getAddress());
sb.append("]");
}
LOGGER.logInfo("connected: " + sb.toString());
} catch (NoHostAvailableException ex) {
LOGGER.logError("error= ", ExceptionUtils.getStackTrace(ex));
} catch (Exception ex) {
LOGGER.logError("error= " + ExceptionUtils.getStackTrace(ex));
}
}
public void shutdown() {
LOGGER.logInfo("Shutting down the whole cassandra cluster");
if (null != session) {
session.close();
}
if (null != cluster) {
cluster.close();
}
}
public Session getSession() {
if (session == null) {
throw new IllegalStateException("No connection initialized");
}
return session;
}
public Cluster getCluster() {
return cluster;
}
}
What is the settings I need to use to connect to local cassandra nodes first and if they are down, then only talk to remote nodes. Also my pooling configuration options is right here which I am using in the above code?
By default the datastax drivers will only connect to nodes in the local DC. If you do not use withLocalDc it will attempt to discern the local datacenter from the DC of the contact point it is able to connect to.
If you want the driver to fail over to host in remote data center(s) you should use withUsedHostsPerRemoteDc, i.e.:
cluster.builder()
.withLoadBalancingPolicy(DCAwareRoundRobinPolicy.builder()
.withLocalDc("DC1")
.withUsedHostsPerRemoteDc(3).build())
With this configuration, the driver will establish connections to 3 hosts in each remote DC, and only send queries to them if all hosts in the local datacenter is down.
There are other strategies for failover to remote data centers. For example, you could run your application clients in each same physical data center as your C* data centers, and then when a physical data center fails, you can fail over at a higher level (like your load balancer).
Also my pooling configuration options is right here which I am using in the above code?
I think what you have is fine. The defaults are fine too.
i am trying to use cassandra as database for an app i am working on. The app is a Netbeans platform app.
In order to start the cassandra server on my localhost i issue Runtime.getRuntime().exec(command)
where command is the string to start the cassandra server and then i connect to the cassandra sever with the datastax driver. However i get the error:
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042 (com.datastax.driver.core.TransportException: [/127.0.0.1:9042] Cannot connect))
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:199)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:80)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1154)
at com.datastax.driver.core.Cluster.getMetadata(Cluster.java:318)
at org.dhviz.boot.DatabaseClient.connect(DatabaseClient.java:43)
at org.dhviz.boot.Installer.restored(Installer.java:67)
....
i figure it out that the server requires some time to start so i have added the line Thread.sleep(MAX_DELAY_SERVER) which seem to resolve the problem.
Is there any more elegant way to sort this issue?
Thanks.
Code is below.
public class Installer extends ModuleInstall {
private final int MAX_DELAY_SERVER = 12000;
//private static final String pathSrc = "/org/dhviz/resources";
#Override
public void restored() {
/*
-*-*-*-*-*DESCRIPTION*-*-*-*-*-*
IMPLEMENT THE CASSANDRA DATABASE
*********************************
*/
DatabaseClient d = new DatabaseClient();
// launch an instance of the cassandra server
d.loadDatabaseServer();
/*wait for MAX_DELAY_SERVER milliseconds before launching the other instructions.
*/
try {
Thread.sleep(MAX_DELAY_SERVER);
Logger.getLogger(Installer.class.getName()).log(Level.INFO, "wait for MAX_DELAY_SERVER milliseconds before the connect database");
} catch (InterruptedException ex) {
Exceptions.printStackTrace(ex);
Logger.getLogger(Installer.class.getName()).log(Level.INFO, "exeption in thread sleep");
}
d.connect("127.0.0.1");
}
}
public class DatabaseClient {
private Cluster cluster;
private Session session;
private ShellCommand shellCommand;
private final String defaultKeyspace = "dhviz";
final private String LOAD_CASSANDRA = "launchctl load /usr/local/Cellar/cassandra/2.1.2/homebrew.mxcl.cassandra.plist";
final private String UNLOAD_CASSANDRA = "launchctl unload /usr/local/Cellar/cassandra/2.1.2/homebrew.mxcl.cassandra.plist";
public DatabaseClient() {
shellCommand = new ShellCommand();
}
public void connect(String node) {
//this connect to the cassandra database
cluster = Cluster.builder()
.addContactPoint(node).build();
// cluster.getConfiguration().getSocketOptions().setConnectTimeoutMillis(12000);
Metadata metadata = cluster.getMetadata();
System.out.printf("Connected to cluster: %s\n",
metadata.getClusterName());
for (Host host
: metadata.getAllHosts()) {
System.out.printf("Datatacenter: %s; Host: %s; Rack: %s\n",
host.getDatacenter(), host.getAddress(), host.getRack());
}
session = cluster.connect();
Logger.getLogger(DatabaseClient.class.getName()).log(Level.INFO, "connected to server");
}
public void loadDatabaseServer() {
if (shellCommand == null) {
shellCommand = new ShellCommand();
}
shellCommand.executeCommand(LOAD_CASSANDRA);
Logger.getLogger(DatabaseClient.class.getName()).log(Level.INFO, "database cassandra loaded");
}
public void unloadDatabaseServer() {
if (shellCommand == null) {
shellCommand = new ShellCommand();
}
shellCommand.executeCommand(UNLOAD_CASSANDRA);
Logger.getLogger(DatabaseClient.class.getName()).log(Level.INFO, "database cassandra unloaded");
}
}
If you are calling cassandra without any parameters in Runtime.getRuntime().exec(command) it's likely that this is spawning cassandra as a background process and returning before the cassandra node has fully started and is listening.
I'm not sure why you are attempting to embed cassandra in your app, but you may find using cassandra-unit useful for providing a mechanism to embed cassandra in your app. It's primarily used for running tests that require a cassandra instance, but it may also meet your use case.
The wiki provides a helpful example on how to start an embedded cassandra instance using cassandra-unit:
EmbeddedCassandraServerHelper.startEmbeddedCassandra();
In my experience cassandra-unit will wait until the server is up and listening before returning. You could also write a method that waits until a socket is in use, using logic opposite of this answer.
I have changed the code to the following taking inspiration from the answers below. Thanks for your help!
cluster = Cluster.builder()
.addContactPoint(node).build();
cluster.getConfiguration().getSocketOptions().setConnectTimeoutMillis(50000);
boolean serverConnected = false;
while (serverConnected == false) {
try {
try {
Thread.sleep(MAX_DELAY_SERVER);
} catch (InterruptedException ex) {
Exceptions.printStackTrace(ex);
}
cluster = Cluster.builder()
.addContactPoint(node).build();
cluster.getConfiguration().getSocketOptions().setConnectTimeoutMillis(50000);
session = cluster.connect();
serverConnected = true;
} catch (NoHostAvailableException ex) {
Logger.getLogger(DatabaseClient.class.getName()).log(Level.INFO, "trying connection to cassandra server...");
serverConnected = false;
}
}
I wonder, if there is a way to check if mongoDB server is running from java driver for mongoDB?
According to the tutorial, I can do
Mongo m = new Mongo();
// or
Mongo m = new Mongo( "localhost" , 27017 );
// and
DB db = m.getDB( "mydb" );
But how to check that I can use these Mongo and DB? I see no isConnected() method in the API.
db.getConnector().isOpen()
returns true
The only way I found is call db.getDatabaseNames() and catch MongoException.
If there some more civilized approach?
You can run a ping command
Mongo mongo = new Mongo();
DBObject ping = new BasicDBObject("ping", "1");
try {
mongo.getDB("dbname").command(ping);
} catch (MongoException e) {
...
}
I've found this to be more direct than the ping command:
Mongo mongo = new Mongo();
try {
mongo.getConnector().getDBPortPool(mongo.getAddress()).get().ensureOpen();
} catch (Exception e) {
...
}
if there is a way to check if mongoDB server is running from java driver for MongoDB?
So if you can do the following:
Mongo m = new Mongo( "localhost" , 27017 );
DB db = m.getDB( "mydb" );
Then you are connected to the database, otherwise that m.getDB() would be throwing an exception. If you can connect to the database, then the MongoDB server is running.
The only way I found is call db.getDatabaseNames() and catch MongoException.
If there some more civilized approach?
Is there something specifically wrong with this approach?
The driver basically runs in a sandbox where it can or cannot connect. You're asking the driver to know something specific about the server (is process X running?), but that's not the driver's job. It can either connect or it can't, it's not responsible for operating the service/process, just for connecting to it.
To know that the process is actually running, you need administrative functions on that server that allow you to check that mongod is indeed running with the correct parameters.
public boolean keepAlive(Mongo mongo) {
return mongo.getAddress() != null;
}
This will return null for address if mongo is down. You can look within the implementation of getAddress() to see why it is a good way to check the mongo's status.
I assume you've initialized the mongo parameter properly.
I haven't tested this thoroughly (only using a localhost mongo) but it appears to work so far:
public boolean mongoRunningAt(String uri) {
try {
Mongo mongo = new Mongo(new MongoURI(uri));
try {
Socket socket = mongo.getMongoOptions().socketFactory.createSocket();
socket.connect(mongo.getAddress().getSocketAddress());
socket.close();
} catch (IOException ex) {
return false;
}
mongo.close();
return true;
} catch (UnknownHostException e) {
return false;
}
}
And the tests I've used:
#Test
public void whenMongoNotAvailableAtSpecificURLThenTheLoaderKnows() {
assertThat(mongoRunningAt("mongodb://127.0.0.1:12345"), is(false));
}
#Test
public void whenMongoAvailableAtSpecificURLThenTheLoaderKnows() {
assertThat(mongoRunningAt("mongodb://127.0.0.1:27017"), is(true));
}
It's not exactly using a well defined public API so use at your own risk.
I have the following log in my mongo console:
Tue Jul 23 17:20:01.301 [initandlisten] waiting for connections on port 27017
Tue Jul 23 17:20:01.401 [websvr] admin web console waiting for connections on port 28017
Tue Jul 23 17:20:01.569 [initandlisten] connection accepted from 127.0.0.1:58090 #1 (1 connection now open)
Tue Jul 23 17:20:01.570 [initandlisten] connection accepted from 127.0.0.1:58089 #2 (2 connections now open)
Tue Jul 23 17:20:21.799 [initandlisten] connection accepted from 127.0.0.1:58113 #3 (3 connections now open)
....
....
....
likewise the log goes on and now it is in 112. Each time when i start mongo server this happens. I only have a singleton connection in my code. What can be the issue here:
public static DB getConnection(String databaseName) throws AppConnectionException {
if (null != db) {
Logger.debug("Returning existing db connection...!");
return db;
}
Logger.debug("Creating new db connection...!");
final String connStr = PropertyRetreiver.getPropertyFromConfigurationFile("rawdata.url");
try {
final MongoClientURI uri = new MongoClientURI(connStr);
final MongoClient client = new MongoClient(uri);
db = client.getDB(databaseName);
} catch (UnknownHostException e) {
throw new AppConnectionException(
"Unable to connect to the given host / port.");
}
return db;
}
MongoClient has internal connection pool. Maximum number of connections can be configured (default is 100). You can set it by using MongoClientOptions like this:
MongoClientOptions options = MongoClientOptions.builder()
.connectionsPerHost(100)
.autoConnectRetry(true)
.build();
And then give these options to MongoClient (checked it in Mongo Java API v2.11.1).
Connections in pool are maintained open (opening and closing connection is usually an expensive operation) so that they can be later reused.
I would also refactor your MongoDB client singleton using enum for example to avoid putting synchronized on this method.
Here is a sketch of what I mean:
public enum MongoDB {
INSTANCE;
private static final String MONGO_DB_HOST = "some.mongohost.com";
private Mongo mongo;
private DB someDB;
MongoDB() {
MongoClientOptions options = MongoClientOptions.builder()
.connectionsPerHost(100)
.autoConnectRetry(true)
.readPreference(ReadPreference.secondaryPreferred())
.build();
try {
mongo = new MongoClient(MONGO_DB_HOST, options);
} catch (UnknownHostException e) {
e.printStackTrace();
}
someDB = mongo.getDB("someDB");
//authenticate if needed
//boolean auth = someDB.authenticate("username", "password".toCharArray());
//if(!auth){
// System.out.println("Error Connecting To DB");
//}
}
public DB getSomeDB() {
return someDB;
}
//call it on your shutdown hook for example
public void close(){
mongo.close();
}
}
Then, you can access your database via
MongoDB.INSTANCE.getSomeDB().getCollection("someCollection").count();
I wonder, if there is a way to check if mongoDB server is running from java driver for mongoDB?
According to the tutorial, I can do
Mongo m = new Mongo();
// or
Mongo m = new Mongo( "localhost" , 27017 );
// and
DB db = m.getDB( "mydb" );
But how to check that I can use these Mongo and DB? I see no isConnected() method in the API.
db.getConnector().isOpen()
returns true
The only way I found is call db.getDatabaseNames() and catch MongoException.
If there some more civilized approach?
You can run a ping command
Mongo mongo = new Mongo();
DBObject ping = new BasicDBObject("ping", "1");
try {
mongo.getDB("dbname").command(ping);
} catch (MongoException e) {
...
}
I've found this to be more direct than the ping command:
Mongo mongo = new Mongo();
try {
mongo.getConnector().getDBPortPool(mongo.getAddress()).get().ensureOpen();
} catch (Exception e) {
...
}
if there is a way to check if mongoDB server is running from java driver for MongoDB?
So if you can do the following:
Mongo m = new Mongo( "localhost" , 27017 );
DB db = m.getDB( "mydb" );
Then you are connected to the database, otherwise that m.getDB() would be throwing an exception. If you can connect to the database, then the MongoDB server is running.
The only way I found is call db.getDatabaseNames() and catch MongoException.
If there some more civilized approach?
Is there something specifically wrong with this approach?
The driver basically runs in a sandbox where it can or cannot connect. You're asking the driver to know something specific about the server (is process X running?), but that's not the driver's job. It can either connect or it can't, it's not responsible for operating the service/process, just for connecting to it.
To know that the process is actually running, you need administrative functions on that server that allow you to check that mongod is indeed running with the correct parameters.
public boolean keepAlive(Mongo mongo) {
return mongo.getAddress() != null;
}
This will return null for address if mongo is down. You can look within the implementation of getAddress() to see why it is a good way to check the mongo's status.
I assume you've initialized the mongo parameter properly.
I haven't tested this thoroughly (only using a localhost mongo) but it appears to work so far:
public boolean mongoRunningAt(String uri) {
try {
Mongo mongo = new Mongo(new MongoURI(uri));
try {
Socket socket = mongo.getMongoOptions().socketFactory.createSocket();
socket.connect(mongo.getAddress().getSocketAddress());
socket.close();
} catch (IOException ex) {
return false;
}
mongo.close();
return true;
} catch (UnknownHostException e) {
return false;
}
}
And the tests I've used:
#Test
public void whenMongoNotAvailableAtSpecificURLThenTheLoaderKnows() {
assertThat(mongoRunningAt("mongodb://127.0.0.1:12345"), is(false));
}
#Test
public void whenMongoAvailableAtSpecificURLThenTheLoaderKnows() {
assertThat(mongoRunningAt("mongodb://127.0.0.1:27017"), is(true));
}
It's not exactly using a well defined public API so use at your own risk.