Develop Reference

JDBC connection pool design

I have developed a JDBC connection pool using synchronized methods like getConnection and returnConnection. This works well and it is fast enough for my purposes. Now the problem happens when this connection pool has to be shared in other packages of our application and so other developers will make use it as well. I feel it is a bit confusing as they always need to perform a returnConnection and I am afraid they may forget to do so.
Thinking about it I came up with the idea to expose only only method in my connection pool and force the other developers to encapsulate their code and so I handle the getConnection / returnConnection inside the connection pool.
It would be something like this:
public MyConnectionPool {
private Connection getConnection() {
//return connection
}
private void returnConnection(Connection connection) {
//add connection to list
}
public void executeDBTask(DBTaskIF task) {
Connection connection = getConnection();
task.execute(connection);
returnConnection(connection);
}
}
where:
public interface DBTaskIF {
public execute(Connection connection);
}
with an example of this DBTaskIF:
connectionPool.executeDBTask( new DBTaskIF() {
public void execute(Connection connection) {
PreparedStatement preStmt = null;
try {
preStmt = connection.prepareStatement(Queries.A_QUERY);
preStmt.setString(1, baseName);
preStmt.executeUpdate();
} finally {
if(preStmt!=null) {
try {
preStmt.close();
} catch (SQLException e) {
log.error(e.getStackTrace());
}
}
}}});
I hope you can get the idea. What I want to know is your opinion about this approach. I want to propose this to the development team and I worry some one comes up saying that this is not standard or OOP or something else...
Any comments are much appreciated.

I feel it is a bit confusing as they always need to perform a returnConnection and I am afraid they may forget to do so.
Thinking about it I came up with the idea to expose only only method in my connection pool and force the other developers to encapsulate their code and so I handle the getConnection returnConnection inside the connection pool.
I'm concerned with this statement. APIs should not (never?) assume that whoever uses them will do so in some way that is not enforced contractually by whichever method it exposes.
And java.sql.Connection is a widely used interface so you'll be making enemies by telling people how to use it with your pool.
Instead, you should assume that your Connection instances will be used correctly, i.e., that they will be closed (connection.close() in a finally block) once their use is over (see, for instance, Connecting with DataSource Objects):
Connection con;
PreparedStatement stmt;
try {
con = pool.getConnection();
con.setAutoCommit(false);
stmt = con.prepareStatement(...);
stmt.setFloat(1, ...);
stmt.setString(2, ...);
stmt.executeUpdate();
con.commit();
stmt.close();
} catch (SQLException e) {
con.rollback();
} finally {
try {
if(con!=null)
con.close();
if(stmt!=null) {
stmt.close();
} catch (SQLException e) {
...
} finally {
}
}
And the Connection implementation of your pool should be recycled when closed.
I second #lreeder's comment that you're really reinventing the wheel here and that most connection pools already available are definitely fast enough for most purposes, and underwent many fine tweakings over time. This also applies to embedded databases.

Disclaimer; this is just my opinion, but I have written custom connection pools before.
I find Java code where you have to create inner class impls a little clunky. However in Java8 lambda or Scala anonymous functions this would be a clean design. I probably would just expose returnConnection() as a public method and allow callers to use it directly.

Third option: use a utility class that takes care of most of the administration.
Not only forgetting to close a Connection can cause trouble, but also forgetting to close a Statement or a Resultset can cause trouble. This is similar to using various IO streams in a method: at some point you make an extra utility class in which you register all opened IO streams so that if an error occurs, you can call close in the utility class and be sure that all opened IO streams are closed.
Such a utility class will not cover all use cases but there is always the option to write another one for other (more complex) use cases. As long as they keep the same kind of contract, using them should just make things easier (and will not feel forced).
Wrapping or proxying a Connection to change the behavior of close to return the Connection to the pool is in general how connection pools prevent connections from actually being closed. But if a connection pool is not used, the application is usually written in a different manner: a connection (or two) is created (at startup) and used wherever a query is executed and the connection is only closed when it is known that a connection is not needed for a while (at shutdown). In contrast, when a pool is used, the connection is "closed" as soon as possible so that other processes can re-use the connection. This together with the option to use a utility class, made me decide to NOT wrap or proxy a connection, but instead let the utility class actually return the connection to the pool if a pool was used (i.e. not call connection.close() but call pool.release(connection)). Usage example of such a utility class is here, the utlity class itself is here.
Proxying causes small delays which is why for example BoneCP decided to wrap Connection and Datasource (wrapping causes very little overhead). The Datasource interface changes with each Java version (at least from 1.6 to 1.7) which means the code will not compile with older/newer versions of Java. This made me decide to proxy the Datasource because it is easier to maintain, but it is not easy to setup (see the various proxy helper classes here). Proxying also has the drawback of making stack-traces harder to read (which makes debugging harder) and sometimes makes exceptions disappear (I have seen this happen in JBoss where the underlying object threw a runtime exception from the constructor).
tl;dr If you make your own specialized pool, also deliver a utility class which makes it easy to use the pool and takes care of most of the administration that is required (like closing used resources) so that it is unlikely to be forgotten. If a utility class is not an option, wrapping or proxying is the standard way to go.

Find statements related to a Connection

I am developing on a big application which uses a Connection Pool and has lots of classes using its connections.
Recently we had some issues because some classes were not closing the statements before invoking the connection.close() method, wrongly believing that when the connection is closed any related statement is also closed.
At the moment I am refactoring some code and developing an abstract class which will manage the connections (get them from and put them back into datasource) and leave the statement details to be implemented into subclasses.
In order to avoid future errors related to statements not being closed, I thought about implementing some check before returing the conneciton into the Pool and if I find some opened statements either close them or log a Warning.
The resulting class would look like this:
public abstract class AbstractDatabaseLoader {
private DataSource dataSource;
public final DatabaseValues load(DatabaseParams params) {
DatabaseValues result = null;
Connection connection = null;
try {
connection = dataSource.getConnection();
result = load(connection, params);
} catch (Exception ex) {
// some logging;
} finally {
if (connection != null) {
try {
if (validateStatements(connection)){
logger.warn("The Connection is being returned into the POOL with opened Statments!");
}
connection.close();
} catch (SQLException e) {
// some logging.
}
}
}
return result;
}
protected abstract DatabaseValues load(Connection connection, DatabaseParams params);
private boolean validateStatements(Connection connection){
// Do Something here to validate if statements were properly closed.
}
}
However, I found no way to recover the statements from the connection interface.
So:
Is this even possible without having to depend on specific implementations?
Would you recommend this approach?
Many thanks,
Carles

Connection.close API says
Releases this Connection object's database and JDBC resources immediately instead of waiting for them to be automatically released.
if it is even a Connection Pool where the connection is not actually closed, it is the Connection Pool provider duty to close all Statements and ResultSets created by this Connection, and this is what good providers do in practice, see commons-dbcp http://commons.apache.org/proper/commons-dbcp/apidocs/org/apache/commons/dbcp/DelegatingConnection.html#close() it says Closes the underlying connection, and close any Statements that were not explicitly closed.

How to get jdbc connection from hibernate session? [duplicate]

This question already has answers here:
session.connection() deprecated on Hibernate?
(13 answers)
Closed 9 years ago.
I want to get jdbc connection from hibernate session.There is method inside hibernate session
i.e session.connection(); but it has been deprecated. i know this works still but i dont want to use deprecated method as i am sure they must have provide some alternative for this?
At http://docs.jboss.org/hibernate/orm/3.5/api/org/hibernate/Session.html connection method api says using org.hibernate.jdbc.Work for this purpose but i dont find any example for that?

Here is how you can use it:
session.doWork(new Work() {
#Override
public void execute(Connection connection) throws SQLException {
//connection, finally!
}
});

Try this:
((SessionImpl)getSession()).connection()

I had a similar Problem and I used the ConnectionProvider class to get the connection. See my solution:
Session session = entityManager.unwrap(Session.class);
SessionFactoryImplementor sessionFactoryImplementation = (SessionFactoryImplementor) session.getSessionFactory();
ConnectionProvider connectionProvider = sessionFactoryImplementation.getConnectionProvider();
try {
connection = connectionProvider.getConnection();
...
}

Java thread safe database connections

I'm writing a servlet that handles each request by accessing and modifying some table(s) in the database. I want the connections to the database to be thread safe. I don't want to use already existing libraries/frameworks for this (spring, hibernate, etc.).
I know I can use java's ThreadLocal for this in the following way :
public class DatabaseRegistry { //assume it's a singleton
private Properties prop = new Properties();
public static final ThreadLocal<Connection> threadConnection = new ThreadLocal<Connection>();
private Connection connect() throws SQLException {
try {
// This will load the MySQL driver, each DB has its own driver
Class.forName("com.mysql.jdbc.Driver");
// Setup the connection with the DB
Connection connection = DriverManager
.getConnection("jdbc:mysql://" + prop.getProperty("hostname") + "/" + prop.getProperty("database") + "?"
+ "user=" + prop.getProperty("username") + "&password=" + prop.getProperty("password"));
return connection;
} catch (SQLException e) {
throw e;
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
return null;
}
public Connection getConnection() throws SQLException {
if (threadConnection.get() == null) {
Connection connection = connect();
threadConnection.set(connection);
return threadConnection.get();
} else {
return threadConnection.get();
}
}
private void freeConnection(Connection connection) throws SQLException {
connection.close();
threadConnection.remove();
}
}
Each time you call getConnection(), the new connection is added to the ThreadLocal object and then removed when you free the connection.
Is this the proper way of doing this or should the DatabaseRegistry itself extend the ThreadLocal<Connection> class? Or is there an even better way to do this to make all connections thread safe?

I don't think that making database connections thread-safe is a common practice. Usually what you want is either:
Serialize the access to some part of your servlet, so that there is no more than one servlet executing code at a time (e.g. implementing the SingleThreadModel interface).
Locking a particular table / table page / row so you can operate on some particular tuple (by changing the database isolation level).
Using optimistic locking to detect modified rows in a table (using some reference attribute of the table to check if the current version is the same that the one in the table).
AFAIK, the typical use of ThreadLocal<Connection> is to store a unique database connection per thread, so that the same connection can be used in different methods in your business logic without the need of passing it as a parameter each time. Because the common servlet container implementation uses a thread to fulfill an HTTP request, then two different requests are guaranteed to use two different database connections.

I know you said you don't want to use libraries to do this, but you're going to be way better off if you do. Pick a standard connection pool (C3P0, DBCP, or something) and you'll be way happier than if you bake your own. Why can't you use a library to do this?

I am not sure why you want your DB connections to be thread safe. Most of the time establishing connection to the database is the longest part of the transaction. Typically connections are reused between requests and pools of open connections are managed (via frameworks or more typically application servers).
If you are worried about concurrent modifications to the same tables you might want to look at synchronized methods: http://docs.oracle.com/javase/tutorial/essential/concurrency/syncmeth.html

Java: connecting to db to fetch data

I have a question regarding Java when fetching data from, lets say MySQL database. As of now I need to write quite a lot of redundant code when fetching data. And I wonder if there is a better way to do that.
E.g. I have an method which fetch data from a table A. The method for that will look something like this then
public void readDataBase() throws Exception {
try {
Class.forName("com.mysql.jdbc.Driver");
connect = DriverManager
.getConnection("jdbc:mysql://localhost/feedback?"
+ "user=sqluser&password=sqluserpw");
statement = connect.createStatement();
resultSet = statement
.executeQuery("select * from FEEDBACK.COMMENTS");
writeResultSet(resultSet);
} catch (Exception e) {
throw e;
} finally {
close();
}
}
I wonder if there's a better way to write a method such as this one. Because it gets quite ugly when you have to write code such as this, namely that you have to write those line to getConnection all the time in every method that fetch data from the database.

Use Spring, with MyBatis or spring-jdbc for data access instead of raw JDBC.
spring-jdbc is a library wrapping basic JDBC code where you can provide callbacks to specify how you want resultsets mapped to objects and such. Mybatis is a little higher-level, you specify your queries in an xml file.
With Spring the big win is you get declarative transactions so you have no code starting and committing transactions, you also get templates for data access objects, and setting up a connection pool is easy. And there are plenty of examples for how to put the pieces together.

At some point you're better off writing your own DAO which handles all the plumbing for you.

I would do different things depending on whether I am in a single threaded batch job or inside a container.
For single threaded batch job:
create just one connection and reuse it
create just one prepared statement and reuse it
optimize commit calls
For J2EE
use the container managed data source pool

Most of the times when you write a program working with a database you do not open a connection every time you want to do something with it. Instead you open a connection at the beggining of the program and then use it every time when accessing a database.
Take a look at this example (pseudocode!):
class Database {
private Connection conn;
public Database() {
connect();
}
private void connect() {
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection(dbUrl);
}
public void close() {
conn.close();
}
private ResultSet readSth() {
statement = conn.createStatement();
return statement.executeQuery("select * from FEEDBACK.COMMENTS");
}
private void doSth() {
// do sth else with conn
}
}

You can create a class having static method there, that returns you an instance of connection like below:
import java.sql.*;
import java.util.Properties;
public class Getconnection{
private static final String dbClassName = "com.mysql.jdbc.Driver";
private static final String CONNECTION ="jdbc:mysql://127.0.0.1/dbUserData";
private static Properties p = new Properties();
public static Connection getConnection(){
p.put("user","root");
p.put("password","library");
try{
Class.forName(dbClassName);
Connection con=DriverManager.getConnection(CONNECTION,p);
return con;
}
catch(Exception ie){
ie.printStackTrace();
return null;
}
}
}
So that you do not need to create different instances of connection, and change only at one place if you want to...

I think you are suffering from basic code organization. For example, you should only create the connection once and then pass it around to whatever methods need it. Typically, people use connection pools so that only a certain number of connections ever exist to the db (because they are expensive) and a connection pool manager will loan them out as needed and keep track of their states. There are a few connection pool libraries in Java (Apache has one) but I have had good luck using: http://sourceforge.net/projects/c3p0/
I dont really like heavy ORMs such as Hibernate but I do I like MyBatis. It lets me wright SQL and doesnt inject itself all over my domain models: http://www.mybatis.org/java.html I think you would benefit greatly by having a Data Access Object layer (DAO). This layer abstracts out communication with your data so the layers above can just fetch collections of data without worrying about the underlying SQL it took to generate that list.

I've moved away from direct JDBC access since I started using ORM (eg. Hibernate).
Or why dont you have some publicly available static method to read from the db:
public ResultSet readDataBase(String query) throws Exception {
try {
Class.forName("com.mysql.jdbc.Driver");
connect = DriverManager
.getConnection("jdbc:mysql://localhost/feedback?"
+ "user=sqluser&password=sqluserpw");
statement = connect.createStatement();
return statement.executeQuery(query);
} catch (Exception e) {
throw e;
} finally {
close();
}
}
and you can have similar methods when you do update/delete, etc