I am using Connection pooling in TOmcat 6 and i have configued this way inside context.xml file
<Resource name="jdbc/myoracle" auth="Container"
type="javax.sql.DataSource" driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:#127.0.0.1:1521:ORCLE"
username="scott" password="tiger" maxActive="20" maxIdle="10"
maxWait="-1"/>
And this is my Factory class to obtain the Connection using DataSource
public class ConnPoolFactory {
private static DataSource dataSource;
private static Connection connection;
private ConnPoolFactory() {
}
public static synchronized Connection getConnection() throws SQLException {
try {
if (connection == null) {
Context initContext = new InitialContext();
Context envContext = (Context) initContext
.lookup("java:/comp/env");
dataSource = (DataSource) envContext.lookup("jdbc/myoracle");
connection = dataSource.getConnection();
} else {
return connection;
}
} catch (NamingException e) {
e.printStackTrace();
}
return connection;
}
}
And from my servlet inside finally block , i am closing it this way
try {
connection = ConnPoolFactory.getConnection();
finally
{
if(conn!=null)
con.close();
}
From my User Interface i can give different commands (Button press) like Insert , Update , Delete , Select --.
The issue i am facing is that , the application runs only for one command , that is for example if i clicked Insert Button , it is inserting the record fine and after that , if once again i give any command there is a Exception on the Server console saying The Connection is closed .
If i remove that finally block code inside my servlet , then the application runs fine for any number of commands
Could anybody please let me know whats wrong with that finnaly block ??
The Connection variable in ConnPoolFactory should be method-local, not static. The variable you should be testing for null is not connection but dataSource. Once you have got a non-null value of that, you then return dataSource.getConnection(). The caller should then close that connection when he's finished with it.
Your factory is not aware that the connection is closed, and keeps handing it out. I'm assuming that's what you had in mind designing it.
You should either have your servlet turn back the connection after use, or the factory create a new connection every time.
Edit: trying to be more explicit:
Your servlet code will be called once per request. That's what servlet do. It seems to me that you close the connection after any request. Fine. However, your connection factory uses a static to store the connection it created. So on the second call, it will hand out a connection which has already been closed (in effect, it is a connection pool with a single connection rather than a factory).
BTW, you might encounter another nastier bug if you don't close your connection: you'll be sharing a connection between servlet threads if two requests come simultaneously, which might or might not work depending which DB operations you perform.
Related
I'm using JBoss AS 7.1 as a server and I have my DataSource configured with pooling. I'm quite new to this so please excuse any rookie mistakes... after all I'm here to learn.
When a client logs-in it gets a connection to the database and I need to keep that connection(from the pool) open until the user logs-out or the HttpSession expires. This is an absolute requirement coming from our DB Admin. who says that he needs the DB session variables. I am using a servlet for all this.
Playing with the possibilities I have encountered 2 major problems:
As far as I see JBoss automatically closes unused connections => my opened connection returns to the pool. So this might not be the right path.
If I try to store/recall the Connection object like this:
private Hashtable<String, Connection> connections = new Hashtable<String, Connection>();
try {
String strDSName1 = "java:/OracleDSJNDI";
ctx = new InitialContext();
ds1 = (javax.sql.DataSource) ctx.lookup(strDSName1);
System.out.println("Got 1'st ds.");
} catch (Exception e) {
System.out.println("ERROR getting 1'st DS : " + e);
}
connection = ds1.getConnection();
connections.put(session.getId(), connection);
conn = (Connection) connections.get(sessionID);
it throws this exception:
java.sql.SQLException: Connection is not associated with a managed
connection.org.jboss.jca.adapters.jdbc.jdk6.WrappedConnectionJDK6#dee1f37
My question is: How do I properly keep my connection opened?
Thanks
How do I properly keep my connection opened?
You must not do that, let the connection pool handle this.
Behind the scenes, the connection pool will keep a bunch of database connections to the database engine (MySQL, Oracle, SQL Server... depends how you configure it) in SLEEPING state. When you execute this code:
//avoiding all the particular exceptions just for code simplicity purposes...
//in real world applications, you must handle each of these exceptions
public Connection getConnection() throws Exception {
ctx = new InitialContext();
ds1 = (javax.sql.DataSource) ctx.lookup(strDSName1);
return ds1.getConnection();
}
You're asking to the connection pool to retrieve one of these connections available. The connection pool will give you a database connection (if available) and let you use it as long as you want. Then you use it wherever you want/need and close it:
public void foo() throws Exception {
Connection connection = getConnection();
//do what you want/need...
//in the end, you close the connection
//this is A MUST!
connection.close();
}
When executing connection.close() from a connection retrieved by the connection pool, you're not closing the physical database connection but notifying the connection pool this specific database connection must return to the SLEEPING state.
Some advices from the explanation:
You must not try to keep the connection alive, that's connection pool's job.
You must not try to store the connections in any cache-like structure, that's connection pool's job.
You must retrieve a java.sql.Connection in the shortest scope you will need it. Once you have used it, close it.
Your DBA is basically requiring you to avoid connection pooling by making the database connection equivalent to the user's session.
So one option is to not use the connection pool, and instead roll your own functionality that opens/closes the database connection around the user's session. That seems complicated and unusual though.
Another option is to examine the DBA's requirement. The DBA may have to adapt to the idea that he'll need to track state in a different way, e.g. by using a key related to the session to store the state he needs in a table, instead of storing state in the connection layer.
Generally speaking storing state in some component's session handling is adding indirect complexity, because you start having to care about how the component handles expiry and uniqueness, as you're finding here where the HTTP session state handles this differently from the database session.
Referring to Tomcat JBDC connection pool, I see in the standalone java example given there, one gets the connection using datasource.getConnection()which is cool. But in the finally block, it says con.close().
Question: When I implement this, it seems obvious that the con I get from datasource will be closed every time in the finally. When this is closed, will the connection pooling mechanism acquire a new connection and adds it to the pool?
I presume there should be a method call like releaseConnection() that will let the pool take its own decision whether to close it or let it be open for some other use.
I've also tried doing this ConnectionPool aPool = datasource.createPool();
But I see there is nothing like release connection on this aPool.
I think I'm missing something here?
Appreciate your help.
Code snippet from Tomcat JBDC connection pool:
DataSource datasource = new DataSource();
datasource.setPoolProperties(p);
Connection con = null;
try {
con = datasource.getConnection();
Statement st = con.createStatement();
ResultSet rs = st.executeQuery("select * from user");
int cnt = 1;
while (rs.next()) {
System.out.println((cnt++)+". Host:" +rs.getString("Host")+
" User:"+rs.getString("User")+" Password:"+rs.getString("Password"));
}
rs.close();
st.close();
} finally {
if (con!=null) try {con.close();}catch (Exception ignore) {}
}
Since you call the close() on a method obtained by the pool it is up to the pool what to do inside this method call. It does not neccessarily have to close the pooled database connection - it may do some cleanup and then add the connetion back to the pool.
This is already answered in Closing JDBC Connections in Pool
OK, my bad, that I did not see the implementation of DataSource.
It extends DataSourceProxy that internally creates a pool before returning a Connectionbased on the PoolProperties
I understand, its upto this DataSource to handle the connections, even though I close the con in finally, DataSource may take necessary action.
Do add a comment/reply if anybody thinks otherwise.
That example only shows how to create and use a data source. For connection pool on Tomcat you may configure JNDI.
// Sample
public static Connection getConnectionFromPool() {
Context initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/TestDB");
return ds.getConnection();
...
Quote from How connection pooling works in Java and JDBC:
A connection pool operates by performing the work of creating
connections ahead of time, In the case of a JDBC connection pool, a
pool of Connection objects is created at the time the application
server (or some other server) starts. These objects are then managed
by a pool manager that disperses connections as they are requested by
clients and returns them to the pool when it determines the client is
finished with the Connection object. A great deal of housekeeping is
involved in managing these connections.
When the connection pool server starts, it creates a predetermined
number of Connection objects. A client application would then perform
a JNDI lookup to retrieve a reference to a DataSource object that
implements the ConnectionPoolDataSource interface. The client
application would not need make any special provisions to use the
pooled data source; the code would be no different from code written
for a nonpooled DataSource.
I have a series of methods running within a servlet engine (Tomcat in this case), using connection pooling to access the database written in this way:
// Gets an RSS_Feed.
public static RSS_Feed get(int rssFeedNo) {
ConnectionPool_DB pool = ConnectionPool_DB.getInstance();
Connection connection = pool.getConnection();
PreparedStatement ps = null;
ResultSet rs = null;
String query = ("SELECT * " +
"FROM RSS_Feed " +
"WHERE RSSFeedNo = ?;");
try {
ps = connection.prepareStatement(query);
ps.setInt(1, rssFeedNo);
rs = ps.executeQuery();
if (rs.next()) {
return mapRSSFeed(rs);
}
else {
return null;
}
}
catch(Exception ex) {
logger.error("Error getting RSS_Feed " + rssFeedNo + "\n", ex);
return null;
}
finally {
Database_Utils.closeResultSet(rs);
Database_Utils.closeStatement(ps);
pool.freeConnection(connection);
}
}
Is it possible to call such a method outside of the servlet engine at all? I would like to do this in a batch process executed from the command line instead of within the servlet engine. I know I could simply rewrite the query without connection pooling but this is one of many queries involved in the process.
The connection pooling is implemented via Apache Common DBCP.
ConnectionPool_DB.getInstance(); reads:
private ConnectionPool_DB() {
try {
InitialContext ic = new InitialContext();
dataSource = (DataSource) ic.lookup(PropertiesFile.getProperty("myApp", "DATASOURCE"));
// dataSource = (DataSource) ic.lookup("java:/comp/env/jdbc/myApp");
}
catch(Exception ex) {
logger.error("Error getting a connection pool's datasource\n", ex);
}
}
I have something like this in a project:
Context ctx = new InitialContext();
DataSource ds = (DataSource) ctx.lookup("DbConnection");
ConnectionPool connectionPool = new ConnectionPool(ds)
And inside context xml I define the resouce like this
<Resource name="DbConnection"
auth="SERVLET"
type="javax.sql.DataSource"
scope="Shareable"
driverClassName="**driverClassName**"
url="**url**"
username="**username**"
password="**password**"
maxActive="10"
maxIdle="10"
maxWait="1000"
/>
So I assume you have something similar
If so you need to write code to create the DataSource yourself.
This should help you with that
http://docs.oracle.com/javase/tutorial/jdbc/basics/sqldatasources.html
Do you mean that you want to share a connection pool between your servlet engine and a batch job? Or that you want to use connection pooling within a batch job?
As to sharing a pool between Tomcat and a batch job: Hmm, I don't see how you'd do it. Tomcat and the batch job would each have their own instance of the Java Virtual Machine. They're not sharing memory, classes, etc, so I don't know where such a common pool would live.
If you mean within a batch job: Sure. I think such a thing is rarely necessary. In batch jobs I normally open a connection at the start of the program and close it at the end. There's not much value to creating a connection pool. Desktop apps are a little trickier. I often create a connection when the app starts and close it when they exit, but arguably this ties up a connection when the user is just staring blindly at the screen (like I often do for the hour or so before lunch), so other times I open a connection every time the user clicks a key that causes something to happen, then release it before going back to "wait" mode. Again, there's little point pooling because in a desktop app, there are no other users to share the pool with.
But can it be done? Sure. I've done it in desktop apps where many things could happen at various times and so it was awkward to pass a single connection around.
Sure, it could be used may be with slightly modification of the JNDI connectivity. But the Tomcat should run.
I'm using Axis2 with Apache Tomcat. I need the clients of the web service to be able to query a database, so I found on the internet that, by adding the following resource to the Tomcat's context, it will automatically create a connection pool:
Resource name="jdbc/TestDB" auth="Container"
type="javax.sql.DataSource"
removeAbandoned="true"
removeAbandonedTimeout="30"
maxActive="80"
maxIdle="30"
maxWait="10000"
username="user"
password=""
driverClassName="org.postgresql.Driver"
url = "jdbc:postgresql://localhost:5432/mydb"
useUnicode="true"
characterEncoding="utf-8"
characterSetResults="utf8"
validationQuery="/* ping */ SELECT 1"
It seems to be working, but now what I want to do is reuse the same PreparedStatement, so it won't be parsed every time a client makes a request. So, I made the PreparedStatement static for all client connections and when I create it, i invoke statement.setPoolable(true), which, from what I understand, is redundant (a PreparedStatement is already poolable). I hoped that this way, the PreparedStatement won't be tied to a single connection. Still, I get the error:
java.sql.SQLException: org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement with address * is closed.
The weird thing is, if i call statement.isClosed() before i make any set, it returns false. Then, I set something and it throws that exception.
The code is:
try {
Connection conn;
Context envCtx = (Context) new InitialContext().lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/TestDB");
conn = ds.getConnection();
if(statement == null){
//statement is static, so it won't be null at the second call
statement = conn.prepareStatement(query);
statement.setPoolable(true);
}
if(statement.isClosed()){
statement = conn.prepareStatement(query);
//it never gets here
}
else{
Logger.getLogger(HelloAxisWorld.class.getName()).log(Level.INFO, "STATEMENT IS NOT CLOSED", new Object());
//it always gets here
}
statement.setString(1, par1); //here, the second call throws an exception
ResultSet rs = statement.executeQuery();
while (rs.next()) {
cur_value = rs.getInt("cur_value");
}
rs.close();
conn.close();
}catch (Exception ex) {
Logger.getLogger(HelloAxisWorld.class.getName()).log(Level.SEVERE, null, ex);
}
I don't understand why does statement.isClosed return false, but then the exception says it is closed. Maybe this is not the way to reuse a preparedStatement, but then how can i do it? I read that, if I call conn.prepareStatement on the same query, jdbc will return the PreparedStatement from cache (it will not be parsed again), but I'm not sure if it's true.
There is a project called c3p0 which was made specifically to handle one of the cases you are having. The website is here https://sourceforge.net/projects/c3p0/ and here http://www.mchange.com/projects/c3p0/ It handles data-source JNDI binding, connection pooling and statement pooling. There also is a reference on stackoverflow on someone using this library with servlets what-is-a-good-strategy-for-caching-prepared-statements-in-tomcat
I have a webapp that uses JNDI lookups to get a connection to the database.
The connection works fine and returns the query no problems. The issue us that the connection does not close properly and is stuck in the 'sleep' mode (according to mysql administrator). This means that they become unusable nad then I run out of connections.
Can someone give me a few pointers as to what I can do to make the connection return to the pool successfully.
public class DatabaseBean {
private static final Logger logger = Logger.getLogger(DatabaseBean.class);
private Connection conn;
private PreparedStatement prepStmt;
/**
* Zero argument constructor
* Setup generic databse connection in here to avoid redundancy
* The connection details are in /META-INF/context.xml
*/
public DatabaseBean() {
try {
InitialContext initContext = new InitialContext();
DataSource ds = (DataSource) initContext.lookup("java:/comp/env/jdbc/mysite");
conn = ds.getConnection();
}
catch (SQLException SQLEx) {
logger.fatal("There was a problem with the database connection.");
logger.fatal(SQLEx);
logger.fatal(SQLEx.getCause());
}
catch (NamingException nameEx) {
logger.fatal("There was a naming exception");
logger.fatal(nameEx);
logger.fatal(nameEx.getCause());
}
}
/**
* Execute a query. Do not use for statements (update delete insert etc).
*
* #return A ResultSet of the execute query. A set of size zero if no results were returned. It is never null.
* #see #executeUpdate() for running update, insert delete etc.
*/
public ResultSet executeQuery() {
ResultSet result = null;
try {
result = prepStmt.executeQuery();
logger.debug(prepStmt.toString());
}
catch (SQLException SQLEx) {
logger.fatal("There was an error running a query");
logger.fatal(SQLEx);
}
return result;
}
SNIP
public void close() {
try {
prepStmt.close();
prepStmt = null;
conn.close();
conn = null;
} catch (SQLException SQLEx) {
logger.warn("There was an error closing the database connection.");
}
}
}
This is inside a javabean that uses the database connection.
public LinkedList<ImportantNoticeBean> getImportantNotices() {
DatabaseBean noticesDBBean = new DatabaseBean();
LinkedList<ImportantNoticeBean> listOfNotices = new LinkedList<ImportantNoticeBean>();
try {
PreparedStatement preStmt = noticesDBBean.getConn().prepareStatement("SELECT pseudonym, message, date_to, date_from " +
"FROM importantnotices, users " +
"WHERE importantnotices.username = users.username " +
"AND NOW() >= date_from AND NOW() <= date_to;");
noticesDBBean.setPrepStmt(preStmt);
ResultSet result = noticesDBBean.executeQuery();
while (result.next()) {
ImportantNoticeBean noticeBean = new ImportantNoticeBean();
noticeBean.setAuthor(result.getString("pseudonym"));
noticeBean.setMessage(result.getString("message"));
noticeBean.setDateTo(result.getDate("date_to"));
noticeBean.setDateFrom(result.getDate("date_from"));
listOfNotices.add(noticeBean);
}
result.close();
} catch (SQLException SQLEx) {
logger.error("There was an error in ImportantNoticesBean.getImportantNotices()");
logger.error(SQLEx);
} finally {
noticesDBBean.close();
}
return listOfNotices;
}
<Context reloadable="true">
<Resource name="jdbc/mysite"
auth="Container"
type="javax.sql.DataSource"
username="user"
password="password"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/mysite"
maxActive="10"
maxIdle="5"
maxWait="6000"
removeAbandoned="true"
logAbandoned="false"
removeAbandonedTimeout="20"
/>
</Context>
You seem to be closing the connection properly - except for the case where prepStmt.close() throws a SQLException, I can't find a connection leak.
What pool implementation are you using? When you close a connection, the pool need not close the underlying MySQL connection immediately - after all that is the point of a connection pool! So from MySQL side, the connections would look alive, although your app is not using any; they might simply be held by the TC connection pool.
You might want to experiment with the settings of the connection pool.Ask it to shrink the pool when the system is idle. Or, ask it to refresh all connections periodically. Or, have a strict upper bound on the number of concurrent connections it ever gets from MySQL etc.
One way to check if your code has a connection leak is to force the ds.getConnection() to always open a new physical connection and conn.close() to release the connection (if your connection pool has settings for those). Then if you watch the connections on MySQL side, you might be able to figure out if the code really has a connection leak or not.
This is a similar question - Connection Pool Settings for Tomcat
This is my response to that question and it fixed the problem for the other guy. It may help you out too.
Tomcat Documentation
DBCP uses the Jakarta-Commons Database Connection Pool. It relies on number of Jakarta-Commons components:
* Jakarta-Commons DBCP
* Jakarta-Commons Collections
* Jakarta-Commons Pool
I'm using the same connection pooling stuff and I'm setting these properties to prevent the same thing it's just not configured through tomcat.
But if the first thing doesn't work try these.
testWhileIdle=true
timeBetweenEvictionRunsMillis=300000
Ok I might have this sorted. I have changed the database config resource to the following:
*SNIP*
maxActive="10"
maxIdle="5"
maxWait="7000"
removeAbandoned="true"
logAbandoned="false"
removeAbandonedTimeout="3"
*SNIP*
This works well enough for now. What is happening, afaik, is that once I reach the ten connections then Tomcat is checking for abandoned connections (idle time > 3). It does this in a batch job each time that max connections is reached. The potential issue with this is if i need more than 10 queries run at the same time (not unique to me). The important thing is that removeAbandonedTimeout is less than maxWait.
Is this what should be happening? ie Is this the way that the pool should operate? If it is is seems, at least to me, that you would wait until something (the connection) is broken before fixing rather than not letting it 'break' in the first place. Maybe I am still not getting it.
The issue us that the connection does not close properly and is stuck in the 'sleep' mode
This was actually only half right.
The problem I ran into was actually that each app was defining a new connection to the database sever. So each time I closed all the connections App A would make a bunch of new connections as per it's WEB.xml config file and run happily. App B would do the same. The problem is that they are independent pools which try to grab up to the server defined limit. It is a kind of race condition I guess. So when App A has finished with the connections it sits waiting to to use them again until the timeout has passed while App B who needs the connection now is denied the resources even though App A has finished with the and should be back in the pool. Once the timeout has passed, the connection is freed up and B (or C etc) can get at it again.
e.g. if the limit is 10 (mySQL profile limit) and each app has been configured to use a max of 10 the there will be 20 attempts at connections. Obviously this is a bad situation.
The solution is to RTFM and put the connection details in the right place. This does make shared posting a pain but there are ways around it (such as linking to other xml files from the context).
Just to be explicit: I put the connection details in the WEB.xml for each app and the had a fight about it.
One thing that #binil missed, you are not closing the result set in the case of an exception. Depending on the driver implementation this may cause the connection to stay open. Move the result.close() call to the finally block.
I am using the same configuration as you are. If the connection in mysql administrator(windows) shows that it is in sleep mode it only means that is pooled but not in use. I checked this running a test program program with multiple threads making random queries to Mysql. if it helps here is my configuration:
defaultAutoCommit="false"
defaultTransactionIsolation="REPEATABLE_READ"
auth="Container"
type="javax.sql.DataSource"
logAbandoned="true"
removeAbandoned="true"
removeAbandonedTimeout="300"
maxActive="-1"
initialSize="15"
maxIdle="10"
maxWait="10000"
username="youruser"
password="youruserpassword"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://yourhost/yourdatabase"/>