Develop Reference

JDBC Connections not released till end of thread

I have a websphere application server with an MSSQL database connection with 10 in the connection pool. When my worker thread runs I have a process that runs the following:
Psuedo code
for(List<String> items){
Connection connection = null;
Statement statement = null;
try{
connection = getConnection();
statement = connection.createStatement();
String sql = "exec someStoredProcedure '" + item + "';";
statement.execute(sql);
}finally{
//real code includes null checks and try catch for errors
statement.close();
connection.close();
}
}
My problem is that if my loop is larger than 10 it will hang till it hits the timeout and error "ConnectionWaitTimeoutException: J2CA1010E: Connection not available; timed out waiting for 180 seconds".
I would think that closing the statements and connections would allow me to reuse the connection from the pool. The connections do get closed/collected after the thread finishes. I've updated my code to reuse the same connection through the for loop and I can run the process rapid fire over and over without problems because each thread cleans up after itself it seems. Any idea what's going on or how to resolve? I'm worried in the future I'll have a process that needs more than 10 threads over the course of running.

The behavior that you describe sounds inconsistent with how the connection pool is supposed to work. However, there are a number of important details that impact connection reuse which are not apparent from your writeup.
You are either using sharable or unsharable connections. You can find that out by looking at the resource reference that you used for the DataSource lookup. For example, #Resource with shareable=false or a deployment descriptor resource reference with <res-sharing-scope>Unshareable</res-sharing-scope> is unsharable, whereas if you set true for the former or Shareable for the latter or omit the attribute, then the connection is sharable. If you didn't use a resource reference, connections will typically be sharable, unless you explicitly overrode that with advanced configuration.
If you are using an unsharable connection and the entire code block is not encompassed within a global transaction, then every connection.close will return the connection to the pool to become immediately reusable. If you are in a global transaction, then each connection will remain unavailable after close because there is outstanding work on it that still needs to be committed or rolled back.
If you are using a sharable connection and each of the connection requests match (same user/password or absence thereof for all), then you should keep getting the same underlying connection back and should not run out of connections. If however, the connection requests do not match, then the connections are kept around for the duration of the scope (could be a transaction or a request scope like servlet boundary) and you will keep getting a new connection with each request until you run out.
If the above isn't enough to figure out the issue, add more detail to your scenario indicating the sharability, how you request each connections, and the placement of transaction boundaries and request boundaries, so that a more detailed answer for your specific scenario can be given.

Multiple DB connections using a connection pool vs Single connection with multiple statements

I am developing a server working with MySQL, and I have been trying to understand advantage of working with a connection pool vs a single connection that is kept open, and being passed down to the different methods through out the application.
The idea of working with a connection pool is understood, however there could be scenarios that this could create a bottleneck, that wouldn't be in case of working without the pool.
Better explain my meaning using code:
Lets say the following method is called simultaneously connectionPoolSize + 1 (e.g. 10) times, meaning that we have exhausted our connections from the connection pool, the last query attempt will fail since no connections available:
public void getData(con) {
Connection con = null;
Statement s = null;
ResultSet rs = null;
try {
con = connectionPool.getConnection();
s = con.createStatement();
rs = s.executeQuery("SELECT * FROM MY_TABLE;");
// Some long process that takes a while....
catch(Exception e) {
throw new Exception(e.getMessage())
} finally {
s.close();
rs.close();
con.close();
}
}
However if we are using a single connection, that is kept open, and all methods can use it, there is no need for any of the methods to wait for the connection to be sent back to pool (which as we saw above, could take some time).
e.g. call this method also 10 times, this would work
public void getData(con) {
Statement s = null;
ResultSet rs = null;
try {
s = con.createStatement();
rs = s.executeQuery("SELECT * FROM MY_TABLE;");
// Some long process that takes a while....
// But this time we don't care that this will take time,
// since nobody is waiting for us to release the connection
catch(Exception e) {
throw new Exception(e.getMessage())
} finally {
s.close();
rs.close();
}
}
Obviously the statements and result sets will still be kept open until the method is finished, but this doesn't affect the connection itself, so it doesn't hold back any other attempts to use this connection.
I assume there is some further insight that I am missing, I understand the standard is working with connection pools, so how do you handle these issues?

Depends on your use case. Suppose you are building a web application that would be used by multiple users simultaneously. Now if you have a single connection, all the queries from multiple user threads will be queued. And single db connection will process them one by one. So in a multi-user system(mostly all normal cases), single db connection will be a bottleneck & won't work. Additionally, you need to take care of thread safety in case you are writing & committing data to db.
If you need truly simultaneous query execution in db, then you should go ahead with connection pool. Then different user threads can use different connections & can execute queries in parallel.

Connection pools are used to keep a number of opened connections ready for use and to eliminate the need to open a new connection each time it is required.
If your application is single threaded then you probably don’t need a pool and can use a single connection instead.
Even though sharing a connection between multiple threads is permitted there are some pitfalls of this approach. Here is a description for Java DB: https://docs.oracle.com/javadb/10.8.3.0/devguide/cdevconcepts89498.html. You should check if this is also the case for MySQL.
In many cases it is easier to have an individual connection for each thread.

Use session bean to hold datasource connection [duplicate]

I'm running a web application on Tomcat. I have a class that handles all DB queries.
This class contains the Connection object and methods that returns query results.
This is the connection object:
private static Connection conn = null;
It has only one instance (singleton).
In addition, I have methods that execute queries, such as search for a user in the db:
public static ResultSet searchUser(String user, String pass) throws SQLException
This method uses the static Connection object. My question is, is my use in static Connection object thread safe? Or can it cause problems when a lot of users will call the searchUser method?

is my use in static Connection object thread safe?
Absolutely not!
This way the connection going to be shared among all requests sent by all users and thus all queries will interfere with each other. But threadsafety is not your only problem, resource leaking is also your other problem. You're keeping a single connection open during the entire application's lifetime. The average database will reclaim the connection whenever it's been open for too long which is usually between 30 minutes and 8 hours, depending on DB's configuration. So if your web application runs longer than that, the connection is lost and you won't be able to execute queries anymore.
This problem also applies when those resources are held as a non-static instance variable of a class instance which is reused multiple times.
You should always acquire and close the connection, statement and resultset in the shortest possible scope, preferably inside the very same try-with-resources block as where you're executing the query according the following JDBC idiom:
public User find(String username, String password) throws SQLException {
User user = null;
try (
Connection connection = dataSource.getConnection();
PreparedStatement statement = connection.prepareStatement("SELECT id, username, email FROM user WHERE username=? AND password=md5(?)");
) {
statement.setString(1, username);
statement.setString(2, password);
try (ResultSet resultSet = statement.executeQuery()) {
if (resultSet.next()) {
user = new User();
user.setId(resultSet.getLong("id"));
user.setUsername(resultSet.getString("username"));
user.setEmail(resultSet.getString("email"));
}
}
}
return user;
}
Note that you should not return a ResultSet here. You should immediately read it and map it to a non-JDBC class and then return it, so that the ResultSet can safely be closed.
If you're not on Java 7 yet, then use a try-finally block wherein you manually close the closeable resources in the reverse order as you've acquired them. You can find an example here: How often should Connection, Statement and ResultSet be closed in JDBC?
If you worry about connecting performance, then you should be using connection pooling instead. This is built-in into many Java EE application servers and even barebones servletcontainers like Tomcat supports it. Just create a JNDI datasource in the server itself and let your webapp grab it as DataSource. It's transparently already a connection pool. You can find an example in the first link of the list below.
See also:
How should I connect to JDBC database / datasource in a servlet based application?
When my app loses connection, how should I recover it?
Am I Using JDBC Connection Pooling?
Show JDBC ResultSet in HTML in JSP page using MVC and DAO pattern
DAO tutorial with JDBC

If you are only running Select queries (searchUser sounds like only selecting data) there will be no issues, apart from thread contention.
As far as I know, a Connection can only handle one query at a time, so by using a single instance you will essentially serialize database access. But this does not necessarily mean, it is always safe to access a database like this in a multi threaded environment. There might still be issues if concurrent accesses are interleaving.

How to properly keep a DB connection from a Connection Pool opened in JBoss

I'm using JBoss AS 7.1 as a server and I have my DataSource configured with pooling. I'm quite new to this so please excuse any rookie mistakes... after all I'm here to learn.
When a client logs-in it gets a connection to the database and I need to keep that connection(from the pool) open until the user logs-out or the HttpSession expires. This is an absolute requirement coming from our DB Admin. who says that he needs the DB session variables. I am using a servlet for all this.
Playing with the possibilities I have encountered 2 major problems:
As far as I see JBoss automatically closes unused connections => my opened connection returns to the pool. So this might not be the right path.
If I try to store/recall the Connection object like this:
private Hashtable<String, Connection> connections = new Hashtable<String, Connection>();
try {
String strDSName1 = "java:/OracleDSJNDI";
ctx = new InitialContext();
ds1 = (javax.sql.DataSource) ctx.lookup(strDSName1);
System.out.println("Got 1'st ds.");
} catch (Exception e) {
System.out.println("ERROR getting 1'st DS : " + e);
}
connection = ds1.getConnection();
connections.put(session.getId(), connection);
conn = (Connection) connections.get(sessionID);
it throws this exception:
java.sql.SQLException: Connection is not associated with a managed
connection.org.jboss.jca.adapters.jdbc.jdk6.WrappedConnectionJDK6#dee1f37
My question is: How do I properly keep my connection opened?
Thanks

How do I properly keep my connection opened?
You must not do that, let the connection pool handle this.
Behind the scenes, the connection pool will keep a bunch of database connections to the database engine (MySQL, Oracle, SQL Server... depends how you configure it) in SLEEPING state. When you execute this code:
//avoiding all the particular exceptions just for code simplicity purposes...
//in real world applications, you must handle each of these exceptions
public Connection getConnection() throws Exception {
ctx = new InitialContext();
ds1 = (javax.sql.DataSource) ctx.lookup(strDSName1);
return ds1.getConnection();
}
You're asking to the connection pool to retrieve one of these connections available. The connection pool will give you a database connection (if available) and let you use it as long as you want. Then you use it wherever you want/need and close it:
public void foo() throws Exception {
Connection connection = getConnection();
//do what you want/need...
//in the end, you close the connection
//this is A MUST!
connection.close();
}
When executing connection.close() from a connection retrieved by the connection pool, you're not closing the physical database connection but notifying the connection pool this specific database connection must return to the SLEEPING state.
Some advices from the explanation:
You must not try to keep the connection alive, that's connection pool's job.
You must not try to store the connections in any cache-like structure, that's connection pool's job.
You must retrieve a java.sql.Connection in the shortest scope you will need it. Once you have used it, close it.

Your DBA is basically requiring you to avoid connection pooling by making the database connection equivalent to the user's session.
So one option is to not use the connection pool, and instead roll your own functionality that opens/closes the database connection around the user's session. That seems complicated and unusual though.
Another option is to examine the DBA's requirement. The DBA may have to adapt to the idea that he'll need to track state in a different way, e.g. by using a key related to the session to store the state he needs in a table, instead of storing state in the connection layer.
Generally speaking storing state in some component's session handling is adding indirect complexity, because you start having to care about how the component handles expiry and uniqueness, as you're finding here where the HTTP session state handles this differently from the database session.

ResultSet Object is closed - jtds

I am using JTDS to connect to MS-SQL 2005. I am using c3p0 as the DB Connection pool, configured with Spring.
I am randomly getting an SQLException: Invalid state, the ResultSet object is closed in a Groovy script in which I have passed a reference to the connection pool. The script is executed by a timer every so often. By random, I mean that the script works perfectly 99% of the time, but when it fails, it will do so a couple of times, then go back to working correctly again, picking up where it left off. All of the critical work is done in a transaction, pulling off of a Message Queue.
Logic below:
//passed into the groovy context
DataSource source = source;
Connection conn = source.getConnection();
...
//Have to omit proprietary DB stuff... sorry...
PreparedStatement fooStatement = conn.prepareStatement("INSERT INTO foo (x,y,z) VALUES (?,?,?) select SCOPE_IDENTITY();");
ResultSet identRes = fooStatement.executeQuery();
//This is where the execption is thrown.
identRes.next();
...
try{
log.info("Returning SQL connection.");
conn.close();
}catch(Exception ex){}
There is a separate timer thread that runs a similar groovy script, in which we have not seen this issue. That script uses similar calls to get the connection, and close it.
Originally, we thought that the second script may have been grabbing the same connection off the pool, finishing first, then closing the connection. But c3p0's documentation says that calling conn.close() should simply return it to the pool.
Has anyone else seen this, or am I missing something big here?
Thanks.

We solved this... C3P0 was configured to drop connections that were checked out longer than 30 seconds, we did this to prevent dead-lock in the database (we don't control the tuning). One of the transactions was taking horridly long to complete, and C3P0 was dropping the connection, resulting in the ResultSet Closed error. Surprisingly, however, C3P0 was not logging the incident, so we didnt see this in the application's logs.