Develop Reference

Creating Connection Pool In Standalone Java Application

I need to use a connection pool in a standalone (as in non-web) Java application. Where I work, we are not allowed to use APIs without going through layers of security, and the job needs to be completed soon. Below is my attempt at creating this connection pool.
I have unit tested this code and tested it within the context of the overall application a hundred times and in all cases the tests passed with zero errors, and in addition the performance of each run is just under three thousand times faster than a simple connect, retrieve data, disconnect in serial approach; however, I still have nagging concerns that there could be issues with this approach that I simply haven't unearthed yet. I would appreciate any advice anyone has concerning the below code. This is my first post on this site; please let me know if I've made any errors in etiquette. I did search this site about this problem before posting. Please see below the code for an invocation example. Thanks. --JR
package mypackage;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.Map;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.TimeUnit;
/**
* Note: This class is only instantiated once per application run.
* Multiple instantiations, as specified in the release notes,
* are not supported.
*/
public class ConnectionManager {
// Use a blocking queue to store the database connections.
// The application will only be called once, by a single user,
// but within the application many threads will require
// a connection.
private BlockingQueue<Connection> connectionQueue = null;
// Load the connection queue with a user-defined number of connections.
// Params contains a map of all non hard-coded variables in the
// application.
public ConnectionManager(int howMany, Map<String, Object> params) {
Database database = new Database();
connectionQueue = new ArrayBlockingQueue<Connection>(howMany);
for(int i = 0; i < howMany; i++) {
connectionQueue.add(database.getConn(params));
}
}
// Return a connection from the queue, waiting up to 15 minutes to do so.
// 15 minutes is hard-coded because it is the standard time-out for all
// processes at our agency. This application must complete in less
// than fifteen minutes (is currently completing in thirty five seconds).
public Connection getConnection() {
Connection conn = null;
try {
conn = connectionQueue.poll(15, TimeUnit.MINUTES);
}
catch(InterruptedException e) {
e.printStackTrace();
}
catch(SQLException e) {
e.printStackTrace();
}
return conn;
}
// Returns a connection to the connection queue.
public void returnConnectionToManager(Connection conn) {
connectionQueue.add(conn);
}
// Called on the last line of the application program's dispatcher.
// Closes all active connections (which will only exist if there
// was a failure within one of the worker threads).
public void closeAllConnections() {
for(Connection conn : connectionQueue) {
try {
conn.close();
}
catch(SQLException e) {
e.printStackTrace();
}
}
}
}
Invocation example:
...
private ConnectionManager cm;
...
public Table(Map<String, Object> params, String method) {
...
cm = (ConnectionManager) params.get("cm");
}
// Execute a chunk of SQL code without requiring processing of a
// result set. Acquires connection from pool via cm.getConnection
// and releases connection via cm.returnConnectionToManager.
// (Database is just a helper class with simple methods for
// closing prepared statement, result sets, etc.)
private void execute(String sql) {
PreparedStatement ps = null;
Connection conn = null;
try {
conn = cm.getConnection();
ps = conn.prepareStatement(sql);
ps.execute();
}
catch (SQLException e) {
e.printStackTrace();
}
finally {
database.closePreparedStatement(ps);
cm.returnConnectionToManager(conn);
}
}

Your code looks good, but there is one serious problem, that clients of your API needs to take care of getting and releasing connection, one of them forget, and memory/resource leak is ready.
Make a one place in which you posts your queries to execute, in this place take connection, execute query and return the connection to the pool. It will secure you that the connections are returned. If you need to invoke multiple queries one after another in a single connection make the method accept an array or list of SQL queries to execute in order. The idea is to encapsulate each request to the db, so you manage all connections. It could be donethat you write an interface that has en execute(Connection conn) which you need to implement, and you could have then some Service that takes such object gives it a connection and then releases the resources back to connection pool.
Something like:
interface SqlWork {
execute(Connection conn);
}
SqlWork myWork = new SqlWork () {
execute(Connection conn) {
// do you work with the conn here
}
}
class SqlExecutionService {
ConnectionManager cm = ...;
public void execute(SqlWork sqlWork) {
Connection conn = null;
try {
conn = cm.getConnection();
sqlWork.execute(conn);
} catch (Your exceptions here) {
//serve or rethrow them
}
finally
{
if (conn!=null) {
cm.returnConnectionToManager(conn);
}
}
}
}
Example of use:
SqlExecutionService sqlExecService = ...;
sqlExecService.execute(myWork);

OrientDB - database object not syncronized with current thread

I have a serious problem with OrientDB ODatabaseDocument object in Java.
To prevent the desyncronization between ODatabaseDocument object (templateDb) and current thread, before close the connection, I force with activateOnCurrentThread the syncronization, but when I close the connection, I always get the following error:
java.lang.IllegalStateException: Current database instance (com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx#302a2a53) is not active on current thread (Thread[btpool0-3,5,main]). Current active database is: com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx#2040c7d9
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.checkIfActive(ODatabaseDocumentTx.java:3138)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.command(ODatabaseDocumentTx.java:667)
My code:
finally {
if (!templateDb.isActiveOnCurrentThread()) {
templateDb.activateOnCurrentThread();
}
templateDb.close();
}
EDIT
String connectionUrl = customer.getDbName();
if (!connectionUrl.startsWith("remote:")) {
connectionUrl = storageConnectionPrefix + connectionUrl;
}
try {
ODatabaseDocumentTx g = new ODatabaseDocumentTx(connectionUrl);
g.activateOnCurrentThread();
g.open(username, password);
g.begin();
return g;
} catch (Exception e) {
e.printStackTrace();
}

In orientDB version 2.1 you must explicit the activateOnCurrentThread related to your DB object. Try as follow:
templateDb.activateOnCurrentThread();

cassandra single node connection error

i am trying to use cassandra as database for an app i am working on. The app is a Netbeans platform app.
In order to start the cassandra server on my localhost i issue Runtime.getRuntime().exec(command)
where command is the string to start the cassandra server and then i connect to the cassandra sever with the datastax driver. However i get the error:
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042 (com.datastax.driver.core.TransportException: [/127.0.0.1:9042] Cannot connect))
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:199)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:80)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1154)
at com.datastax.driver.core.Cluster.getMetadata(Cluster.java:318)
at org.dhviz.boot.DatabaseClient.connect(DatabaseClient.java:43)
at org.dhviz.boot.Installer.restored(Installer.java:67)
....
i figure it out that the server requires some time to start so i have added the line Thread.sleep(MAX_DELAY_SERVER) which seem to resolve the problem.
Is there any more elegant way to sort this issue?
Thanks.
Code is below.
public class Installer extends ModuleInstall {
private final int MAX_DELAY_SERVER = 12000;
//private static final String pathSrc = "/org/dhviz/resources";
#Override
public void restored() {
/*
-*-*-*-*-*DESCRIPTION*-*-*-*-*-*
IMPLEMENT THE CASSANDRA DATABASE
*********************************
*/
DatabaseClient d = new DatabaseClient();
// launch an instance of the cassandra server
d.loadDatabaseServer();
/*wait for MAX_DELAY_SERVER milliseconds before launching the other instructions.
*/
try {
Thread.sleep(MAX_DELAY_SERVER);
Logger.getLogger(Installer.class.getName()).log(Level.INFO, "wait for MAX_DELAY_SERVER milliseconds before the connect database");
} catch (InterruptedException ex) {
Exceptions.printStackTrace(ex);
Logger.getLogger(Installer.class.getName()).log(Level.INFO, "exeption in thread sleep");
}
d.connect("127.0.0.1");
}
}
public class DatabaseClient {
private Cluster cluster;
private Session session;
private ShellCommand shellCommand;
private final String defaultKeyspace = "dhviz";
final private String LOAD_CASSANDRA = "launchctl load /usr/local/Cellar/cassandra/2.1.2/homebrew.mxcl.cassandra.plist";
final private String UNLOAD_CASSANDRA = "launchctl unload /usr/local/Cellar/cassandra/2.1.2/homebrew.mxcl.cassandra.plist";
public DatabaseClient() {
shellCommand = new ShellCommand();
}
public void connect(String node) {
//this connect to the cassandra database
cluster = Cluster.builder()
.addContactPoint(node).build();
// cluster.getConfiguration().getSocketOptions().setConnectTimeoutMillis(12000);
Metadata metadata = cluster.getMetadata();
System.out.printf("Connected to cluster: %s\n",
metadata.getClusterName());
for (Host host
: metadata.getAllHosts()) {
System.out.printf("Datatacenter: %s; Host: %s; Rack: %s\n",
host.getDatacenter(), host.getAddress(), host.getRack());
}
session = cluster.connect();
Logger.getLogger(DatabaseClient.class.getName()).log(Level.INFO, "connected to server");
}
public void loadDatabaseServer() {
if (shellCommand == null) {
shellCommand = new ShellCommand();
}
shellCommand.executeCommand(LOAD_CASSANDRA);
Logger.getLogger(DatabaseClient.class.getName()).log(Level.INFO, "database cassandra loaded");
}
public void unloadDatabaseServer() {
if (shellCommand == null) {
shellCommand = new ShellCommand();
}
shellCommand.executeCommand(UNLOAD_CASSANDRA);
Logger.getLogger(DatabaseClient.class.getName()).log(Level.INFO, "database cassandra unloaded");
}
}

If you are calling cassandra without any parameters in Runtime.getRuntime().exec(command) it's likely that this is spawning cassandra as a background process and returning before the cassandra node has fully started and is listening.
I'm not sure why you are attempting to embed cassandra in your app, but you may find using cassandra-unit useful for providing a mechanism to embed cassandra in your app. It's primarily used for running tests that require a cassandra instance, but it may also meet your use case.
The wiki provides a helpful example on how to start an embedded cassandra instance using cassandra-unit:
EmbeddedCassandraServerHelper.startEmbeddedCassandra();
In my experience cassandra-unit will wait until the server is up and listening before returning. You could also write a method that waits until a socket is in use, using logic opposite of this answer.

I have changed the code to the following taking inspiration from the answers below. Thanks for your help!
cluster = Cluster.builder()
.addContactPoint(node).build();
cluster.getConfiguration().getSocketOptions().setConnectTimeoutMillis(50000);
boolean serverConnected = false;
while (serverConnected == false) {
try {
try {
Thread.sleep(MAX_DELAY_SERVER);
} catch (InterruptedException ex) {
Exceptions.printStackTrace(ex);
}
cluster = Cluster.builder()
.addContactPoint(node).build();
cluster.getConfiguration().getSocketOptions().setConnectTimeoutMillis(50000);
session = cluster.connect();
serverConnected = true;
} catch (NoHostAvailableException ex) {
Logger.getLogger(DatabaseClient.class.getName()).log(Level.INFO, "trying connection to cassandra server...");
serverConnected = false;
}
}

Lock wait timeout exceeded with Hibernate and MySQL (using play framework)

In my web application I'm using Stateless sessions with Hibernate to have better performances on my inserts and updates.
It was working fine with H2 database (the one used in play framework in dev mode).
But when I test it with MySQL I get the following exception :
ERROR ~ Lock wait timeout exceeded; try restarting transaction
ERROR ~ HHH000315: Exception executing batch [Lock wait timeout exceeded; try restarting transaction]
Here is the code :
public static void update() {
Session session = (Session) JPA.em().getDelegate();
StatelessSession stateless = this.session.getSessionFactory().openStatelessSession();
try {
stateless.beginTransaction();
// Fetch all products
{
List<ProductType> list = ProductType.retrieveAllWithHistory();
for (ProductType pt : list) {
updatePrice(pt, stateless);
}
}
// Fetch all raw materials
{
List<RawMaterialType> list = RawMaterialType.retrieveAllWithHistory();
for (RawMaterialType rm : list) {
updatePrice(rm, stateless);
}
}
} catch (Exception ex) {
play.Logger.error(ex.getMessage());
ExceptionLog.log(ex, Thread.currentThread());
} finally {
stateless.getTransaction().commit();
stateless.close();
}
}
private static void updatePrice(ProductType pt, StatelessSession stateless) {
pt.priceDelta = computeDelta();
pt.unitPrice = computePrice();
stateless.update(pt);
PriceHistory ph = new PriceHistory(pt, price);
stateless.insert(ph);
}
private static void updatePrice(RawMaterialType rm, StatelessSession stateless) {
rm.priceDelta = computeDelta();
rm.unitPrice = computePrice();
stateless.update(rm);
PriceHistory ph = new GoodPriceHistory(rm, price);
stateless.insert(ph);
}
In this example I have 3 simple Entities (ProductType, RawMaterialType and PriceHistory).
computeDelta and computePrice are just algorithm functions with no DB stuff.
retrieveAllWithHistory functions are functions that fetch some data from the database using Play framework model functions.
So, this code retrieves some data, edit some, create new one and finally save everything.
Why have I a lock exception with MySQL and no exception with H2 ?

I'm not sure why you have a commit in a finally block. Give this structure a try:
try {
factory.getCurrentSession().beginTransaction();
factory.getCurrentSession().getTransaction().commit();
} catch (RuntimeException e) {
factory.getCurrentSession().getTransaction().rollback();
throw e; // or display error message
}
Also, it might be helpful for you to check this documentation.

H2 on Tomcat shutdown hanging, memory leak when using Tomcat Datasource

This is related to these 2 posts:
What is the proper way to close H2?
Tomcat doesn't stop. How can I debug this?
Basically H2 keeps a lock on the database, even when all connections are closed, and so when stopping Tomcat it hangs waiting on a thread, the process is still running.
The only way I managed to get H2 to not lock the database is by issueing the statement SHUTDOWN IMMEDIATELY command (the vanilla or compact did not release the lock).
This is performed in my ServletContextListener class in the contextDestroyed like this (I have omitted comments and log lines):
ServletContext ctx = servletContextEvent.getServletContext();
DataSource closeDS = databaseConnection.getDatasource();
Connection closeConn = null;
PreparedStatement closePS = null;
try {
closeConn = closeDS.getConnection();
closePS = closeConn.prepareStatement("SHUTDOWN IMMEDIATELY");
closePS.execute();
} catch (Exception ex) {
} finally {
if (closePS != null) {
try { closePS.close(); } catch (SQLException ex) {}
}
if (closeConn != null) {
try { closeConn.close(); } catch (SQLException ex) {}
}
}
try {
databaseConnection.close();
databaseConnection = null;
ctx.setAttribute("databaseConnection", null);
} catch(Exception e) {
}
Enumeration<Driver> drivers = DriverManager.getDrivers();
while (drivers.hasMoreElements()) {
Driver driver = drivers.nextElement();
try {
DriverManager.deregisterDriver(driver);
} catch (Exception e) {
}
}
Now the lock is released Tomcat stops (although I still get the severe memory leak messages in the logs) but now I receive also a number of error stacks in the logs thus:
INFO: Illegal access: this web application instance has been stopped already.
Could not load java.lang.ThreadGroup.
The eventual following stack trace is caused by an error thrown
for debugging purposes as well as to attempt to terminate
the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1531)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1491)
at org.h2.engine.DatabaseCloser.reset(DatabaseCloser.java:43)
at org.h2.engine.Database.close(Database.java:1155)
at org.h2.engine.DatabaseCloser.run(DatabaseCloser.java:80)
10-sep-2013 13:31:41 org.apache.catalina.loader.WebappClassLoader loadClass
The question is: how can I shut down the database without causing illegal state exceptions. Is there something wrong in my code to call the shutdown command?
Why is this such an issue with H2? I do not have this issue with JBoss or Websphere where the application also runs using datasources provided by the container.