I am using an API to connect to some hardware terminals and networks. The API allows me to connect to the servers, disconnect and interrogate for data, pretty similar to what JDBC connection allows you to do. Anyway, since this is not using the JDBC Connection interface, I cannot use a connection pooling already existing.
I would like to avoid writing one myself if I can use one already existing, or maybe just build a small adapter on top of that. Anyone knows of any framework/library that would allow me to enable connection pooling, that can handle my connections, can ensure that the connection is alive all the time etc?
I have looked at Commons Pool, but that only gives you a few classes to put/get your connections ... it doesn't do any of the maintenance tasks etc (check if connection is invalid from time to time, reconnect etc). I can add on top of that the mechanism of connection checking and reconnecting if any issues etc in case there is nothing out there that does this already.
Cheers, Stef.
Apache Commons Pool actually supports creating, destroying and checking objects for validity before handing them out with the PoolableObjectFactory class, which you use with the actual pool implementation by passing it as a parameter:
final PoolableObjectFactory objectFactory = new MyPoolableObjectFactoryImpl();
final ObjectPool pool = new GenericObjectPool(objectFactory);
You can look into dbcp http://commons.apache.org/dbcp/
it's BasicDataSource provide methods such as maxActive, maxIdle, maxWait etc check documentation for more
http://commons.apache.org/dbcp/apidocs/org/apache/commons/dbcp/BasicDataSource.html
interesting one if you are going for new implementation try tomcat 7 jdbc connection pool
http://people.apache.org/~fhanik/jdbc-pool/jdbc-pool.html
[edit] useless for the scenario -hGx so upvoting Henning's post
Related
How to reset a JDBC Connection object (that is, a java.sql.Connection object)?
I have explored the concept of connection pooling. When a connection pool is used, a Connection object can be recycled. But how can a connection pool recycle a Connection object? I think that a connection needs to be "reset" (for example, if it is in a transaction, then perhaps rollback it, but there may be more things to reset) before it can be reused. But I cannot find such a "reset" method in the Java documentation about the class java.sql.Connection.
If you are talking about what you as a user of a connection should do, then that is simple: call close() on the connection. Closing the logical connection will signal to the connection pool that the connection is available for reuse, and the connection pooling manager is then responsible for performing the necessary reset, invalidation of statement handles, etc.
If you are talking about what you as the implementer of a connection pooling manager should do, that is where things become complicated.
Historically, JDBC provides no way to 'reset' a java.sql.Connection, other than by your own code (or your third-party connection pool) remembering the initial configuration, and restoring it after use, and keeping track of objects like statements and result sets and closing them when the connection is returned to the pool.
Originally, the intended way to reset connections in JDBC was for an application server to use a driver's javax.sql.ConnectionPoolDataSource as a factory for javax.sql.PooledConnection objects. These PooledConnection objects serve as handles for physical connections to be held in a connection pool (to be clear, ConnectionPoolDataSource is not a connection pool, it is a data source for a connection pool). The application server would then expose a javax.sql.DataSource handing out logical Connection objects, where on Connection.close(), the driver specific implementation of PooledConnection would take care of any necessary reset of a connection (though JDBC underspecifies what is 'necessary').
However, in practice this route is hardly ever used because support in drivers was (and often still is) spotty, inconsistent or downright incorrect, and JDBC wasn't clear enough on exactly what needed to be done when the logical connection was closed. The world has also shifted to using third-party connection pool libraries that do not use ConnectionPoolDataSource.
JDBC 4.3 (Java 9 and higher) introduced the methods Connection.beginRequest() and Connection.endRequest() to be called by connection pooling managers, which would seem to fit such pattern, but unfortunately JDBC 4.3 doesn't actually specify what kind of things an implementation should or should not do in response to beginRequest and endRequest.
In short, there is no real general way to reset a connection in JDBC.
Do you try to close() method and reinitialize Connection object again.
close()
Releases this Connection object's database and JDBC resources immediately instead of waiting for them to be automatically released
I have a severe problem with my database connection in my web application. Since I use a single database connection for the whole application from singleton Database class, if i try concurrent db operations (two users) the database rollsback the transactions.
This is my static method used:
All threads/servlets call static Database.doSomething(...) methods, which in turn call the the below method.
private static /* synchronized*/ Connection getConnection(final boolean autoCommit) throws SQLException {
if (con == null) {
con = new MyRegistrationBean().getConnection();
}
con.setAutoCommit(true); //TODO
return con;
}
What's the recommended way to manage this db connection/s I have, so that I don't incurr in the same problem.
Keeping a Connection open forever is a very bad idea. It doesn't have an endless lifetime, your application may crash whenever the DB times out the connection and closes it. Best practice is to acquire and close Connection, Statement and ResultSet in the shortest possible scope to avoid resource leaks and potential application crashes caused by the leaks and timeouts.
Since connecting the DB is an expensive task, you should consider using a connection pool to improve connecting performance. A decent applicationserver/servletcontainer usually already provides a connection pool feature in flavor of a JNDI DataSource. Consult its documentation for details how to create it. In case of for example Tomcat you can find it here.
Even when using a connection pool, you still have to write proper JDBC code: acquire and close all the resources in the shortest possible scope. The connection pool will on its turn worry about actually closing the connection or just releasing it back to pool for further reuse.
You may get some more insights out of this article how to do the JDBC basics the proper way. As a completely different alternative, learn EJB and JPA. It will abstract away all the JDBC boilerplate for you into oneliners.
Hope this helps.
See also:
Is it safe to use a static java.sql.Connection instance in a multithreaded system?
Am I Using JDBC Connection Pooling?
How should I connect to JDBC database / datasource in a servlet based application?
When is it necessary or convenient to use Spring or EJB3 or all of them together?
I've not much experience with PostgreSql, but all the web applications I've worked on have used a single connection per set of actions on a page, closing it and disposing it when finished.
This allows the server to pool connections and stops problems such as the one that you are experiencing.
Singleton should be the JNDI pool connection itself; Database class with getConnection(), query methods et al should NOT be singleton, but can be static if you prefer.
In this way the pool exists indefinitely, available to all users, while query blocks use dataSource.getConnection() to draw a connection from the pool; exec the query, and then close statement, result set, and connection (to return it to the pool).
Also, JNDI lookup is quite expensive, so it makes sense to use a singleton in this case.
I've been researching all around the web the most efficient way to design a connection pool and tried to analyze into details the available libraries (HikariCP, BoneCP, etc.).
Our application is a heavy-load consumer webapp and most of the time the users are working on similar business objects (thus the underlying SQL queries executed are the often the same, but still there are numerous).
It is designed to work with different DBMS (Oracle and MS SQL Server especially).
So a simplified use case would be :
User goes on a particular JSP page (e.g. Enterprise).
A corresponding Bean is created.
Each time it realizes an action (e.g. getEmployees(), computeTurnover()), the Bean asks the pool for a connection and returns it back when done.
If we want to take advantage of the Prepared Statement caching of the underlying JDBC driver (as PStatements are attached to a connection - jTDS doc.), from what I understand an optimal way of doing it would be :
Analyze what kind of SQL query a particular Bean want to execute before providing it an available connection from the pool.
Find a connection where the same prepared statement has already been executed if possible.
Serve the connection accordingly (and use the benefits of the cache/precompiled statement).
Return the connection to the pool and start over.
Am I missing an important point here (like JDBC drivers capable of reusing cached statements regardless of the connection) or is my analysis correct ?
The different sources I found state it is not possible, but why ?
For your scheme to work, you'd need to be able to get the connection that already has that statement prepared.
This falls foul on two points:
In JDBC you obtain the connection first,
Cached prepared statements (if a driver or connection pool even supports that) aren't exposed in a standardized way (if at all) nor would you be able to introspect them.
The performance overhead of finding the right connection (and the subsequent contention on the few connections that already have it prepared) would probably undo any benefit of reusing the prepared statement.
Also note that some database systems also have a serverside cache for prepared statements (meaning that it already has the plan etc available), limiting the overhead from a new prepare from the client.
If you really think the performance benefit is big enough, you should consider using a data source specific for this functionality (so it is almost guaranteed that the connection will have the statement in its cache).
A solution could be for a connection pool implementation to delay retrieving the connection from the pool until the Connection.prepareStatement() is called. At that time a connection pool would look up available connections by the SQL statement text and then play forward all the calls made before Connection.prepareStatement(). This way it would be possible to get a connection with a ready PreparedStatement without the issues other guys suggested.
In other words, when you request a connection from the pool, it would return a wrapper that logs everything until the first operation requiring DB access (such as prepareStatement() is requested.
You'd need to ask a vendor of your connection pool functionality to add this feature.
I've logged this request with C3P0:
https://github.com/swaldman/c3p0/issues/55
Hope this helps.
In our application we have separate connection pools configured for each module. One module requires use of some PreparedStatement to be reused often. For this need, I would like to hold the connection forever, so that I need not create new PreparedStatement. Is it safe to hold the Connection like this?
You should probably consider the usage of a connection pool implementation -- like UCP(Oracle), bonecp, dbcp, cp030, etc. This pool pool will maintain the connection management, and you don't have to worry.
Connection pools typically keep connection objects for a very long time (could be months). There is nothing wrong in that.
I am trying to use HikariCP connection pool. I was able to get it to work and get a connection that I could use. I am not sure what is the best approach for returning the connection to the pool.
I have the following questions:
Should I close the connection when I am done, rely on idleTimeout
and maxLifetime settings or is there another call that I can use so
as not to hog the connections from the pool?
If I close the connections (instead of returning to the pool), would
that not result in additional connection objects being created
to meet the requirements of the connection pool size?
Looking for helpful suggestions.
As with most connection pools, Hikari doesn't give you an actual JDBC Connection when you ask for one. What it does instead is give you a proxy that implements the Connection interface. In the case of Hikari - it's a ConnectionProxy object.
This proxy serves a few purposes, the main of which is - take the control of opening/closing connections and statements away from you and into the connection pool. This happens automagically and you should be using your connections as usual. This includes closing them after use.
If you look at the source code for Hikari, at the ConnectionProxy class in particular, you will see that the close() method is very different from the standard one. The code reads as:
Mark the connection as closed, do cleanup, reset underlying connection state and params.
Hence, simply calling close() will just clean and return the connection to the pool.