Develop Reference

How to reset a session in a sql connection pool?

i have a connection pool using javax.sql.DataSource and PostgreSQL JDBC. The process is simple, I'm taking a virtual connection from connection pool, process it, then close the virtual connection (release the connection to connection pool). Everything works fine.
But when I get the same virtual connection second time the session have the same data from the previous execution. Mainly temp tables that created in the previous execution. How can I reset the session after each execution? Or can you suggest any work around to solve the issue without using creating temp table if not exists query syntax.
sample code
import java.sql.Connection;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.sql.DataSource;
public class Base {
public Connection getConnection() {
Context ctx = (Context) new InitialContext().lookup("java:/comp/env");
DataSource ds = (DataSource) ctx.lookup("database");
Connection con = ds.getConnection();
return con;
}
public Connection closeConnection( Connection con){
if (con != null || !con.isClosed()) {
con.close();
}
}
context.xml =>
<Resource
accessToUnderlyingConnectionAllowed="true"
auth="Container"
closeMethod="close"
driverClassName="org.postgresql.Driver"
maxTotal="50"
maxWaitMillis="60000"
removeAbandonedOnBorrow="true"
maxIdle="20"
name="Database"
password="p*******"
type="javax.sql.DataSource"
url="jdbc:postgresql://dbUrl/dbName"
username="dbUser"
validationQuery="select version();"
validationQueryTimeout="60"
/>
// use case
con = getconnection()
CallableStatement st = con.prepareCall("{ doSomething()}");
st.execute()
//processing
st.close()
con.close()
in doSomething() plpgsql function i have a temp table creation (with name t_table), data processing and an array output. when i repeatedly call this from java first execution is success. But from the second execution message saying that t_table already exist is thrown. I thought that when using connection pool, the connections in the pool are session independent IE. every new connections from pool will have a new session. PostgreSQL solution is to drop that particular temp table . But there is no other solution for this from the tomcat side ?

The discard temporary command can be used to clean up a session.
You can configure that as a validation query to be run when the connection is returned to the pool.
With the Tomcat JDBC pool, this would be something like this:
validationQuery="discard temporary"
testWhileIdle="false"
testOnBorrow="false"
testOnReturn="true"
If you want to clean up the session even more, you can consider using discard all instead, but that requires autocommit to be enabled, not sure if that would work for you.
If such a statement isn't allowed, you can wrap it into a function:
create function session_cleanup()
returns boolean
as
$$
discard temporary;
select true;
$$
language sql;
validationQuery="select session_cleanup()"
testWhileIdle="false"
testOnBorrow="false"
testOnReturn="true"

Tomcat JDBC connection pool (releasing connection)

Referring to Tomcat JBDC connection pool, I see in the standalone java example given there, one gets the connection using datasource.getConnection()which is cool. But in the finally block, it says con.close().
Question: When I implement this, it seems obvious that the con I get from datasource will be closed every time in the finally. When this is closed, will the connection pooling mechanism acquire a new connection and adds it to the pool?
I presume there should be a method call like releaseConnection() that will let the pool take its own decision whether to close it or let it be open for some other use.
I've also tried doing this ConnectionPool aPool = datasource.createPool();
But I see there is nothing like release connection on this aPool.
I think I'm missing something here?
Appreciate your help.
Code snippet from Tomcat JBDC connection pool:
DataSource datasource = new DataSource();
datasource.setPoolProperties(p);
Connection con = null;
try {
con = datasource.getConnection();
Statement st = con.createStatement();
ResultSet rs = st.executeQuery("select * from user");
int cnt = 1;
while (rs.next()) {
System.out.println((cnt++)+". Host:" +rs.getString("Host")+
" User:"+rs.getString("User")+" Password:"+rs.getString("Password"));
}
rs.close();
st.close();
} finally {
if (con!=null) try {con.close();}catch (Exception ignore) {}
}

Since you call the close() on a method obtained by the pool it is up to the pool what to do inside this method call. It does not neccessarily have to close the pooled database connection - it may do some cleanup and then add the connetion back to the pool.
This is already answered in Closing JDBC Connections in Pool

OK, my bad, that I did not see the implementation of DataSource.
It extends DataSourceProxy that internally creates a pool before returning a Connectionbased on the PoolProperties
I understand, its upto this DataSource to handle the connections, even though I close the con in finally, DataSource may take necessary action.
Do add a comment/reply if anybody thinks otherwise.

That example only shows how to create and use a data source. For connection pool on Tomcat you may configure JNDI.
// Sample
public static Connection getConnectionFromPool() {
Context initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/TestDB");
return ds.getConnection();
...
Quote from How connection pooling works in Java and JDBC:
A connection pool operates by performing the work of creating
connections ahead of time, In the case of a JDBC connection pool, a
pool of Connection objects is created at the time the application
server (or some other server) starts. These objects are then managed
by a pool manager that disperses connections as they are requested by
clients and returns them to the pool when it determines the client is
finished with the Connection object. A great deal of housekeeping is
involved in managing these connections.
When the connection pool server starts, it creates a predetermined
number of Connection objects. A client application would then perform
a JNDI lookup to retrieve a reference to a DataSource object that
implements the ConnectionPoolDataSource interface. The client
application would not need make any special provisions to use the
pooled data source; the code would be no different from code written
for a nonpooled DataSource.

Trying to use connection pooling outside servlet engine

I have a series of methods running within a servlet engine (Tomcat in this case), using connection pooling to access the database written in this way:
// Gets an RSS_Feed.
public static RSS_Feed get(int rssFeedNo) {
ConnectionPool_DB pool = ConnectionPool_DB.getInstance();
Connection connection = pool.getConnection();
PreparedStatement ps = null;
ResultSet rs = null;
String query = ("SELECT * " +
"FROM RSS_Feed " +
"WHERE RSSFeedNo = ?;");
try {
ps = connection.prepareStatement(query);
ps.setInt(1, rssFeedNo);
rs = ps.executeQuery();
if (rs.next()) {
return mapRSSFeed(rs);
}
else {
return null;
}
}
catch(Exception ex) {
logger.error("Error getting RSS_Feed " + rssFeedNo + "\n", ex);
return null;
}
finally {
Database_Utils.closeResultSet(rs);
Database_Utils.closeStatement(ps);
pool.freeConnection(connection);
}
}
Is it possible to call such a method outside of the servlet engine at all? I would like to do this in a batch process executed from the command line instead of within the servlet engine. I know I could simply rewrite the query without connection pooling but this is one of many queries involved in the process.
The connection pooling is implemented via Apache Common DBCP.
ConnectionPool_DB.getInstance(); reads:
private ConnectionPool_DB() {
try {
InitialContext ic = new InitialContext();
dataSource = (DataSource) ic.lookup(PropertiesFile.getProperty("myApp", "DATASOURCE"));
// dataSource = (DataSource) ic.lookup("java:/comp/env/jdbc/myApp");
}
catch(Exception ex) {
logger.error("Error getting a connection pool's datasource\n", ex);
}
}

I have something like this in a project:
Context ctx = new InitialContext();
DataSource ds = (DataSource) ctx.lookup("DbConnection");
ConnectionPool connectionPool = new ConnectionPool(ds)
And inside context xml I define the resouce like this
<Resource name="DbConnection"
auth="SERVLET"
type="javax.sql.DataSource"
scope="Shareable"
driverClassName="**driverClassName**"
url="**url**"
username="**username**"
password="**password**"
maxActive="10"
maxIdle="10"
maxWait="1000"
/>
So I assume you have something similar
If so you need to write code to create the DataSource yourself.
This should help you with that
http://docs.oracle.com/javase/tutorial/jdbc/basics/sqldatasources.html

Do you mean that you want to share a connection pool between your servlet engine and a batch job? Or that you want to use connection pooling within a batch job?
As to sharing a pool between Tomcat and a batch job: Hmm, I don't see how you'd do it. Tomcat and the batch job would each have their own instance of the Java Virtual Machine. They're not sharing memory, classes, etc, so I don't know where such a common pool would live.
If you mean within a batch job: Sure. I think such a thing is rarely necessary. In batch jobs I normally open a connection at the start of the program and close it at the end. There's not much value to creating a connection pool. Desktop apps are a little trickier. I often create a connection when the app starts and close it when they exit, but arguably this ties up a connection when the user is just staring blindly at the screen (like I often do for the hour or so before lunch), so other times I open a connection every time the user clicks a key that causes something to happen, then release it before going back to "wait" mode. Again, there's little point pooling because in a desktop app, there are no other users to share the pool with.
But can it be done? Sure. I've done it in desktop apps where many things could happen at various times and so it was awkward to pass a single connection around.

Sure, it could be used may be with slightly modification of the JNDI connectivity. But the Tomcat should run.

Why the Database connection is closed after every operation

I am using Connection pooling in TOmcat 6 and i have configued this way inside context.xml file
<Resource name="jdbc/myoracle" auth="Container"
type="javax.sql.DataSource" driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:#127.0.0.1:1521:ORCLE"
username="scott" password="tiger" maxActive="20" maxIdle="10"
maxWait="-1"/>
And this is my Factory class to obtain the Connection using DataSource
public class ConnPoolFactory {
private static DataSource dataSource;
private static Connection connection;
private ConnPoolFactory() {
}
public static synchronized Connection getConnection() throws SQLException {
try {
if (connection == null) {
Context initContext = new InitialContext();
Context envContext = (Context) initContext
.lookup("java:/comp/env");
dataSource = (DataSource) envContext.lookup("jdbc/myoracle");
connection = dataSource.getConnection();
} else {
return connection;
}
} catch (NamingException e) {
e.printStackTrace();
}
return connection;
}
}
And from my servlet inside finally block , i am closing it this way
try {
connection = ConnPoolFactory.getConnection();
finally
{
if(conn!=null)
con.close();
}
From my User Interface i can give different commands (Button press) like Insert , Update , Delete , Select --.
The issue i am facing is that , the application runs only for one command , that is for example if i clicked Insert Button , it is inserting the record fine and after that , if once again i give any command there is a Exception on the Server console saying The Connection is closed .
If i remove that finally block code inside my servlet , then the application runs fine for any number of commands
Could anybody please let me know whats wrong with that finnaly block ??

The Connection variable in ConnPoolFactory should be method-local, not static. The variable you should be testing for null is not connection but dataSource. Once you have got a non-null value of that, you then return dataSource.getConnection(). The caller should then close that connection when he's finished with it.

Your factory is not aware that the connection is closed, and keeps handing it out. I'm assuming that's what you had in mind designing it.
You should either have your servlet turn back the connection after use, or the factory create a new connection every time.
Edit: trying to be more explicit:
Your servlet code will be called once per request. That's what servlet do. It seems to me that you close the connection after any request. Fine. However, your connection factory uses a static to store the connection it created. So on the second call, it will hand out a connection which has already been closed (in effect, it is a connection pool with a single connection rather than a factory).
BTW, you might encounter another nastier bug if you don't close your connection: you'll be sharing a connection between servlet threads if two requests come simultaneously, which might or might not work depending which DB operations you perform.

Java ConnectionPool connection not closing, stuck in 'sleep'

I have a webapp that uses JNDI lookups to get a connection to the database.
The connection works fine and returns the query no problems. The issue us that the connection does not close properly and is stuck in the 'sleep' mode (according to mysql administrator). This means that they become unusable nad then I run out of connections.
Can someone give me a few pointers as to what I can do to make the connection return to the pool successfully.
public class DatabaseBean {
private static final Logger logger = Logger.getLogger(DatabaseBean.class);
private Connection conn;
private PreparedStatement prepStmt;
/**
* Zero argument constructor
* Setup generic databse connection in here to avoid redundancy
* The connection details are in /META-INF/context.xml
*/
public DatabaseBean() {
try {
InitialContext initContext = new InitialContext();
DataSource ds = (DataSource) initContext.lookup("java:/comp/env/jdbc/mysite");
conn = ds.getConnection();
}
catch (SQLException SQLEx) {
logger.fatal("There was a problem with the database connection.");
logger.fatal(SQLEx);
logger.fatal(SQLEx.getCause());
}
catch (NamingException nameEx) {
logger.fatal("There was a naming exception");
logger.fatal(nameEx);
logger.fatal(nameEx.getCause());
}
}
/**
* Execute a query. Do not use for statements (update delete insert etc).
*
* #return A ResultSet of the execute query. A set of size zero if no results were returned. It is never null.
* #see #executeUpdate() for running update, insert delete etc.
*/
public ResultSet executeQuery() {
ResultSet result = null;
try {
result = prepStmt.executeQuery();
logger.debug(prepStmt.toString());
}
catch (SQLException SQLEx) {
logger.fatal("There was an error running a query");
logger.fatal(SQLEx);
}
return result;
}
SNIP
public void close() {
try {
prepStmt.close();
prepStmt = null;
conn.close();
conn = null;
} catch (SQLException SQLEx) {
logger.warn("There was an error closing the database connection.");
}
}
}
This is inside a javabean that uses the database connection.
public LinkedList<ImportantNoticeBean> getImportantNotices() {
DatabaseBean noticesDBBean = new DatabaseBean();
LinkedList<ImportantNoticeBean> listOfNotices = new LinkedList<ImportantNoticeBean>();
try {
PreparedStatement preStmt = noticesDBBean.getConn().prepareStatement("SELECT pseudonym, message, date_to, date_from " +
"FROM importantnotices, users " +
"WHERE importantnotices.username = users.username " +
"AND NOW() >= date_from AND NOW() <= date_to;");
noticesDBBean.setPrepStmt(preStmt);
ResultSet result = noticesDBBean.executeQuery();
while (result.next()) {
ImportantNoticeBean noticeBean = new ImportantNoticeBean();
noticeBean.setAuthor(result.getString("pseudonym"));
noticeBean.setMessage(result.getString("message"));
noticeBean.setDateTo(result.getDate("date_to"));
noticeBean.setDateFrom(result.getDate("date_from"));
listOfNotices.add(noticeBean);
}
result.close();
} catch (SQLException SQLEx) {
logger.error("There was an error in ImportantNoticesBean.getImportantNotices()");
logger.error(SQLEx);
} finally {
noticesDBBean.close();
}
return listOfNotices;
}
<Context reloadable="true">
<Resource name="jdbc/mysite"
auth="Container"
type="javax.sql.DataSource"
username="user"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/mysite"
maxActive="10"
maxIdle="5"
maxWait="6000"
removeAbandoned="true"
logAbandoned="false"
removeAbandonedTimeout="20"
/>
</Context>

You seem to be closing the connection properly - except for the case where prepStmt.close() throws a SQLException, I can't find a connection leak.
What pool implementation are you using? When you close a connection, the pool need not close the underlying MySQL connection immediately - after all that is the point of a connection pool! So from MySQL side, the connections would look alive, although your app is not using any; they might simply be held by the TC connection pool.
You might want to experiment with the settings of the connection pool.Ask it to shrink the pool when the system is idle. Or, ask it to refresh all connections periodically. Or, have a strict upper bound on the number of concurrent connections it ever gets from MySQL etc.
One way to check if your code has a connection leak is to force the ds.getConnection() to always open a new physical connection and conn.close() to release the connection (if your connection pool has settings for those). Then if you watch the connections on MySQL side, you might be able to figure out if the code really has a connection leak or not.

This is a similar question - Connection Pool Settings for Tomcat
This is my response to that question and it fixed the problem for the other guy. It may help you out too.
Tomcat Documentation
DBCP uses the Jakarta-Commons Database Connection Pool. It relies on number of Jakarta-Commons components:
* Jakarta-Commons DBCP
* Jakarta-Commons Collections
* Jakarta-Commons Pool
I'm using the same connection pooling stuff and I'm setting these properties to prevent the same thing it's just not configured through tomcat.
But if the first thing doesn't work try these.
testWhileIdle=true
timeBetweenEvictionRunsMillis=300000

Ok I might have this sorted. I have changed the database config resource to the following:
*SNIP*
maxActive="10"
maxIdle="5"
maxWait="7000"
removeAbandoned="true"
logAbandoned="false"
removeAbandonedTimeout="3"
*SNIP*
This works well enough for now. What is happening, afaik, is that once I reach the ten connections then Tomcat is checking for abandoned connections (idle time > 3). It does this in a batch job each time that max connections is reached. The potential issue with this is if i need more than 10 queries run at the same time (not unique to me). The important thing is that removeAbandonedTimeout is less than maxWait.
Is this what should be happening? ie Is this the way that the pool should operate? If it is is seems, at least to me, that you would wait until something (the connection) is broken before fixing rather than not letting it 'break' in the first place. Maybe I am still not getting it.

The issue us that the connection does not close properly and is stuck in the 'sleep' mode
This was actually only half right.
The problem I ran into was actually that each app was defining a new connection to the database sever. So each time I closed all the connections App A would make a bunch of new connections as per it's WEB.xml config file and run happily. App B would do the same. The problem is that they are independent pools which try to grab up to the server defined limit. It is a kind of race condition I guess. So when App A has finished with the connections it sits waiting to to use them again until the timeout has passed while App B who needs the connection now is denied the resources even though App A has finished with the and should be back in the pool. Once the timeout has passed, the connection is freed up and B (or C etc) can get at it again.
e.g. if the limit is 10 (mySQL profile limit) and each app has been configured to use a max of 10 the there will be 20 attempts at connections. Obviously this is a bad situation.
The solution is to RTFM and put the connection details in the right place. This does make shared posting a pain but there are ways around it (such as linking to other xml files from the context).
Just to be explicit: I put the connection details in the WEB.xml for each app and the had a fight about it.

One thing that #binil missed, you are not closing the result set in the case of an exception. Depending on the driver implementation this may cause the connection to stay open. Move the result.close() call to the finally block.

I am using the same configuration as you are. If the connection in mysql administrator(windows) shows that it is in sleep mode it only means that is pooled but not in use. I checked this running a test program program with multiple threads making random queries to Mysql. if it helps here is my configuration:
defaultAutoCommit="false"
defaultTransactionIsolation="REPEATABLE_READ"
auth="Container"
type="javax.sql.DataSource"
logAbandoned="true"
removeAbandoned="true"
removeAbandonedTimeout="300"
maxActive="-1"
initialSize="15"
maxIdle="10"
maxWait="10000"
username="youruser"
password="youruserpassword"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://yourhost/yourdatabase"/>