Develop Reference

still having You can't operate on a closed ResultSet error after using connection pool

I have many threads accessing MYSQL database, at first I didn't use connection pool so I had this error "You can't operate on a closed ResultSet"
I searched on Google and found out that I should used connection pool so I tried c3p0 API for implementation, but I still have the same problem and nothing changed.
so should I Synchronize getAllcountries method or there's another better solution.
public class DataSource {
private static DataSource datasource;
private ComboPooledDataSource cpds ;
private DataSource() throws IOException, SQLException, PropertyVetoException {
cpds = new ComboPooledDataSource();
cpds.setDriverClass("com.mysql.jdbc.Driver"); //loads the jdbc driver
cpds.setJdbcUrl("jdbc:mysql://localhost/question_game");
cpds.setUser("root");
cpds.setPassword("");
cpds.setMaxPoolSize(500);
}
public static DataSource getInstance() throws IOException, SQLException, PropertyVetoException {
if (datasource == null) {
datasource = new DataSource();
return datasource;
} else {
return datasource;
}
}
public Connection getConnection() throws SQLException {
return this.cpds.getConnection();
}
public List<Country> getAllCountries() {
String query = "SELECT * FROM country order by name ";
List<Country> list = new ArrayList<Country>();
Country country = null;
ResultSet rs = null;
try {
try {
connection = DataSource.getInstance().getConnection();
} catch (IOException e) {
e.printStackTrace();
} catch (PropertyVetoException e) {
e.printStackTrace();
}
statement = connection.createStatement();
rs = statement.executeQuery(query);
while (rs.next()) {
//getting countries
}
} catch (SQLException ex) {
ex.printStackTrace();
} finally {
//DbUtil used to check if null
DbUtil.close(rs);
DbUtil.close(statement);
DbUtil.close(connection);
}
return list;
}

In addition to #stephen-c 's observation,
you basically have two options: either synchronize getAllCountries method, or make the database connection local to that method instead of having it as a class member.
As you have it now, 'connection' is a class member (available to all invocations of getAllCountries(), on all threads), so the connection is probably being overwritten by a second thread. Move it to a variable in the method, and then each invocation of the method will have its own connection.

I see a method called getConnection, I don't see where you are calling it. Instead, I see your getAllCountries method using a static DataSource object that is created like this:
new DataSource();
I searched on Google and found out that I should used connection pool so I tried c3p0 API for implementation, but I still have the same problem and nothing change
Yea ...
It looks like you just cut-and-pasted the code you found into your application without really thinking about it. Obviously ... you need to understand what the code is doing AND figure out how to use it in your application.

Tomcat connection pool & idle connections

We are developing a website using
Tomcat 7
JDBC
PostgreSQL 9.2
We've had some connection leaks and think we corrected them (the database no longer stops responding), but the behaviour of the connection pool still seems leaky, as we have a number of idle connections greater than the maxIdle set in context.xml. I'd like to be sure the problem is fixed.
For testing purposes, I'm using the following context.xml :
<Resource
auth="Container"
name="jdbc/postgres"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
type="javax.sql.DataSource"
username="admin"
password="..."
driverClassName="org.postgresql.Driver"
url="jdbc:postgresql://127.0.0.1:5432/..."
initialSize="1"
maxActive="50"
minIdle="0"
maxIdle="3"
maxWait="-1"
minEvictableIdleTimeMillis="1000"
timeBetweenEvictionRunsMillis="1000"
/>
If I understand correctly, we should have 1 idle connection on startup and from 0 to 3 depending on the load, right ?
What is happening is : 1 connection on startup, up to 3 idle connections if the load is low, and more than 3 idle connections after a high load. Then these connections are not closed immediatly, and we don't know when/if they will be closed (sometime some of them are closed).
So the question is : is this behaviour normal, or not ?
Thanks for your help
EDIT : added factory attribute, didn't change the problem
EDIT 2 : using removeAbandoned & removeAbandonedTimeout make the idle connexions being closed every removeAbandonedTimeout. So we probably still have some connection leaks. Here are some pieces of code we are using to connect to the database and execute requests :
PostgreSQLConnectionProvider, just a static class to provide a connection :
public class PostgreSQLConnectionProvider {
public static Connection getConnection() throws NamingException, SQLException {
String dsString = "java:/comp/env/jdbc/postgres";
Context context = new InitialContext();
DataSource ds = (DataSource) context.lookup(dsString);
Connection connection = ds.getConnection();
return connection;
}
}
DAO abstract class :
public abstract class DAO implements java.lang.AutoCloseable {
// Private attributes :
private Connection _connection;
// Constructors :
public DAO() {
try { _connection = PostgreSQLConnectionProvider.getConnection(); }
catch (NamingException | SQLException ex) {
Logger.getLogger(DAO.class.getName()).log(Level.SEVERE, null, ex);
}
}
// Getters :
public Connection getConnection() { return _connection; }
// Closeable :
#Override
public void close() throws SQLException {
if(!_connection.getAutoCommit()) {
_connection.rollback();
_connection.setAutoCommit(true);
}
_connection.close();
}
}
UserDAO, a small DAO subclass (we have several DAO sublasses to request the database) :
public class UserDAO extends DAO {
public User getUserWithId(int id) throws SQLException {
PreparedStatement ps = null;
ResultSet rs = null;
User user = null;
try {
String sql = "select * from \"USER\" where id_user = ?;";
ps = getConnection().prepareStatement(sql);
ps.setInt(1, id);
rs = ps.executeQuery();
rs.next();
String login = rs.getString("login");
String password = rs.getString("password");
String firstName = rs.getString("first_name");
String lastName = rs.getString("last_name");
String email = rs.getString("email");
user = new User(id, login, password, firstName, lastName, email);
}
finally {
if(rs != null) rs.close();
if(ps != null) ps.close();
}
return user;
}
}
An example of a DAO subclass use :
try(UserDAO dao = new UserDAO()) {
try {
User user = dao.getUserWithId(52);
}
catch (SQLException ex) {
// Handle exeption during getUserWithId
}
}
catch (SQLException ex) {
// Handle exeption during dao.close()
}

Looking at the code it appears the connection is grabbed for the lifetime of the DAO, not the lifetime of the statement, which is the usual expectation. Normally, you would grab a connection from the pool just as you're about to execute the statement, and call close() on it when you're done in order to return it to the pool.
Additionally, in your finally clause, both rs.close() and ps.close() can throw exceptions resulting in missing the last call against the prepared statement.
In Java 7 you can also use a try with resources statement that will close both the prepared statement and the connection for you. According to the spec, the driver is supposed to close the result for you when the statement is closed.

JSP/ DAO error : The method getInstance() is undefined for the type ConnectionPool

As you can see, this code is part of my Data Access Object Layer.
I've never used the ConnectionPool Object before because i'm still studying Java.
Anyway, i'm getting an error message stating :
The method getInstance() is undefined for the type ConnectionPool.
(at line 5)
Should any of you have experienced this before, help would be appreciated.
import java.sql.*;
import java.util.*;
import org.apache.tomcat.jdbc.pool.ConnectionPool;
import music.business.*;
public class ProductDB
{
//This method returns null if a product isn't found.
public static Product selectProduct(String productCode)
{
ConnectionPool pool = ConnectionPool.getInstance(); //<===<====<====<=================
Connection connection = pool.getConnection();
PreparedStatement ps = null;
ResultSet rs = null;
String query = "SELECT * FROM Product " +
"WHERE ProductCode = ?";
try
{
ps = connection.prepareStatement(query);
ps.setString(1, productCode);
rs = ps.executeQuery();
if (rs.next())
{
Product p = new Product();
p.setCode(rs.getString("ProductCode"));
p.setDescription(rs.getString("ProductDescription"));
p.setPrice(rs.getDouble("ProductPrice"));
return p;
}
else
{
return null;
}
}
catch(SQLException e)
{
e.printStackTrace();
return null;
}
finally
{
DBUtil.closeResultSet(rs);
DBUtil.closePreparedStatement(ps);
pool.freeConnection(connection);
}
}
I just found out that i made a mistake:
- the ConnectionPool in my, above mentioned, class was not supposed to be Tomcat imported. Its a JNDI class. See below. The getInstance is actually a method in my JNDI class. Sorry for waisting your time guys. Thank you
import java.sql.*;
import javax.sql.DataSource;
import javax.naming.InitialContext;
public class ConnectionPool
{
private static ConnectionPool pool = null;
private static DataSource dataSource = null;
public synchronized static ConnectionPool getInstance()
{
if (pool == null)
{
pool = new ConnectionPool();
}
return pool;
}
private ConnectionPool()
{
try
{
InitialContext ic = new InitialContext();
dataSource = (DataSource) ic.lookup("java:/comp/env/jdbc/musicDB");
}
catch(Exception e)
{
e.printStackTrace();
}
}
public Connection getConnection()
{
try
{
return dataSource.getConnection();
}
catch (SQLException sqle)
{
sqle.printStackTrace();
return null;
}
}
public void freeConnection(Connection c)
{
try
{
c.close();
}
catch (SQLException sqle)
{
sqle.printStackTrace();
}
}
}

A ConnectionPool can be created using its constructor which accepts some pool properties. Even though this constructor is exposed creating a pool within your application, especially in each DAO may not be advisable.
The point of pooling is to have a single pool of connections that the application can retrieve a connection from when it needs to perform some work with the database. Using this design the code will have multiple connection pools, which defeats the point of pooling.
Usually a datasource is established within Tomcat, which internally handles the building of a connection pool. See these resources for more about connection pooling in Tomcat.

Create it in a 'DBService' class & share it, if you really don't want to define it in Tomcat. As Duffy says, defining it in Tomcat probably would be the best way..
For defining it in Tomcat: http://tomcat.apache.org/tomcat-6.0-doc/jndi-datasource-examples-howto.html#Database_Connection_Pool_(DBCP)_Configurations
The getInstance() syntax appears all wrong. For creating & configuring it directly, see: http://people.apache.org/~fhanik/jdbc-pool/jdbc-pool.html

I would have Tomcat manage that pool for you. It has instructions on how to create a JNDI data source. You should do that and get this out of your code.
It'll have the added benefit of externalizing the connection parameters from your app. They'll live in configuration on the app server.

JDBC Connection pooling issue - Deadlock while getting connection

I am using JDBC connection pooling in Tomcat. To retrieve connections I have defined a connection factory as below:
public class ConnectionManager {
// reference to the ConnectionManager
private static ConnectionManager instance = null;
// Connection to MySQL database
private Connection connect = null;
private static DataSource ds = null;
// Logger
public static final Logger logger = Logger
.getLogger(ConnectionManager.class);
static {
try {
Context initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
ds = (DataSource) envCtx.lookup("jdbc/ConnectionManager");
} catch (NamingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
/**
* Private Constructor .. since its a singleton
*
*/
private ConnectionManager() {
}
public static ConnectionManager getInstance() {
if (instance == null) {
instance = new ConnectionManager();
}
return instance;
}
public Connection getDbConnection() {
Connection conn = null;
try {
synchronized (DataSource.class) {
conn = ds.getConnection();
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return conn;
}
public void closeDbConnection() throws SQLException {
conn.close();
}
}
Now I see that my code always gets stuck at conn = ds.getConnection(); line. Please let me know what I am doing wrong. From my DAO methods I am using the following to get connection: conn = ds.getConnection();
Clearly its a multi-threading issue. What should I do?

Most of your class seems geared around retrieving the JNDI datasource and using it to create connections. Not necessarily a bad idea but in this case you have introduced some bugs into your program with the additional complexity.
First off, your singleton is not a singleton. Your are not synchronizing the getInstance method so its possible to multiple threads to invoke this method at the same time. The best method in Java (unfortunately) for implementing singletons is via an enum:
public enum ConnectionManager {
INSTANCE;
}
Your second significant issue is that you are synchronizing on a class that you don't explicitly control. There is nothing preventing third party JARs or even other classes in your own application from synchronizing on the DataSource class, making it a rife target for deadlocking issues. I would take out all the superfluous methods from the class and remove the synchronize block:
public enum ConnectionManager {
INSTANCE;
private DataSource ds = null;
ConnectionManager() {
try {
final Context initCtx = new InitialContext();
final Context envCtx = (Context) initCtx.lookup("java:comp/env");
ds = (DataSource) envCtx.lookup("jdbc/ConnectionManager");
} catch (NamingException e) {
e.printStackTrace();
}
}
public Connection getConnection() throws SQLException {
if(ds == null) return null;
return ds.getConnection();
}
}
Now, most datasource implementations are thread safe in my experience, so the above code should work most of the time. But, we shouldn't rely on implementations we cannot control, so lets add a safe synchronization to the code, like so:
public enum ConnectionManager {
INSTANCE;
private DataSource ds = null;
private Lock connectionLock = new ReentrantLock();
ConnectionManager() {
try {
final Context initCtx = new InitialContext();
final Context envCtx = (Context) initCtx.lookup("java:comp/env");
ds = (DataSource) envCtx.lookup("jdbc/ConnectionManager");
} catch (NamingException e) {
e.printStackTrace();
}
}
public Connection getConnection() throws SQLException {
if(ds == null) return null;
Connection conn = null;
connectionLock.lock();
try {
conn = ds.getConnection();
} finally {
connectionLock.unlock();
}
return conn;
}
}
You don't have to add wrapper methods to close the connection, that is the responsibility of the calling code. Good luck.

#arya, seems like you are having the problem of connection leak, and because of that the pool is getting exhausted and the code just waits till it gets a new connection, To analyze the problem , use any of the database monitoring tools, or manually try to trace the leak (The point in code where you have consumed a connnection but forgot to return it to the pool after use).

Well I would say first try out your dataSource is working or not with a test source.
I suggest look at Apache Tomcat JNDI Data Resource How To, for Apache Tomcat 6.0 and for Apache Tomcat 7.0.
Look at the instructions carefully and analyse what's going wrong in your code, then update your question with specific problem.

That code is virtually guaranteed to cause connection leaks in a multithreaded system. closeDbConnection() closes only the last connection borrowed from pool - so if 10 threads have called getDbConnection(), and after that closeDbConnection(), only 1 connection is closed and 9 still alive. Repeat that several times and pool is exhausted (unless connection is cleaned up in finalize(), but that's probably not the case). I would get rid of the whole class, or reworked it to act only as a datasource locator.

Java connecting to multiple databases

I am creating a java application that connects to multiple databases. A user will be able to select the database they want to connect to from a drop down box.
The program then connects to the database by passing the name to a method that creates an initial context so it can talk with an oracle web logic data source.
public class dbMainConnection {
private static dbMainConnection conn = null;
private static java.sql.Connection dbConn = null;
private static javax.sql.DataSource ds = null;
private static Logger log = LoggerUtil.getLogger();
private dbMainConnection(String database) {
try {
Context ctx = new InitialContext();
if (ctx == null) {
log.info("JDNI Problem, cannot get InitialContext");
}
database = "jdbc/" + database;
log.info("This is the database string in DBMainConnection" + database);
ds = (javax.sql.DataSource) ctx.lookup (database);
} catch (Exception ex) {
log.error("eMTSLogin: Error in dbMainConnection while connecting to the database : " + database, ex);
}
}
public Connection getConnection() {
try {
return ds.getConnection();
} catch (Exception ex) {
log.error("Error in main getConnection while connecting to the database : ", ex);
return null;
}
}
public static dbMainConnection getInstance(String database) {
if (dbConn == null) {
conn = new dbMainConnection(database);
}
return conn;
}
public void freeConnection(Connection c) {
try {
c.close();
log.info(c + " is now closed");
} catch (SQLException sqle) {
log.error("Error in main freeConnection : ", sqle);
}
}
}
My problem is what happens if say someone forgets to create the data source for the database but they still add it to the drop down box? Right now what happens is if I try and connect to a database that doesn't have a data source it errors saying it cannot get a connection. Which is what I want but if I connect to a database that does have a data source first, which works, then try and connect to the database that doesn't have a data source, again it errors with
javax.naming.NameNotFoundException: Unable to resolve 'jdbc.peterson'. Resolved 'jdbc'; remaining name 'peterson'.
Which again I would expect but what is confusing me is it then grabs the last good connection which is for a different database and process everything as if nothing happened.
Anyone know why that is? Is weblogic caching the connection or something as a fail safe? Is it a bad idea to create connections this way?

You're storing a unique datasource (and connection, and dbMainConnection) in a static variable of your class. Each time someone asks for a datasource, you replace the previous one by the new one. If an exception occurs while getting a datasource from JNDI, the static datasource stays as it is. You should not store anything in a static variable. Since your dbMainConnection class is constructed with the name of a database, and there are several database names, it makes no sense to make it a singleton.
Just use the following code to access the datasource:
public final class DataSourceUtil {
/**
* Private constructor to prevent unnecessary instantiations
*/
private DataSourceUtil() {
}
public static DataSource getDataSource(String name) {
try {
Context ctx = new InitialContext();
String database = "jdbc/" + name;
return (javax.sql.DataSource) ctx.lookup (database);
}
catch (NamingException e) {
throw new IllegalStateException("Error accessing JNDI and getting the database named " + name);
}
}
}
And let the callers get a connection from the datasource and close it when they have finished using it.

You're catching JNDI exception upon lookup of the nonexistent datasource but your singleton still keeps the reference to previously looked up datasource. As A.B. Cade says, null reference to ds upon exception, or even before that.
On a more general note, perhaps using Singleton is not the best idea.