Develop Reference

Redshift server closes connection after 10 minutes

I have a statement that takes about 20 minutes to run, which is of the form:
create table new_table diststyle key distkey(column1) sortkey(column2)
as (select ....);
When I run it using an SQL IDE or with the psql command line client, the statement executes successfully but when I run it from my Java program, the server closes the connection after 10 minutes with the following exception:
org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [create table new_table diststyle key distkey(column1) sortkey(column2) as (select ....);];
SQL state [HY000]; error code [600001]; [Amazon](600001) The server closed the connection.;
nested exception is java.sql.SQLException: [Amazon](600001) The server closed the connection.
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:84) ~[spring-jdbc-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:419) ~[spring-jdbc-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:538) ~[spring-jdbc-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at com.abc.mypackage.MyClass.myMethod(Myclass.java:123) [classes/:?]
Caused by: java.sql.SQLException: [Amazon](600001) The server closed the connection.
at com.amazon.support.channels.TLSSocketChannel.read(Unknown Source) ~[?:?]
Caused by: com.amazon.support.exceptions.GeneralException: [Amazon](600001) The server closed the connection.
at com.amazon.support.channels.TLSSocketChannel.read(Unknown Source) ~[?:?]
I'm using org.apache.commons.dbcp2.BasicDataSource to create connections. I've tried extending the timeout via defaultQueryTimeout, maxConnLifetimeMillis and socketTimeout but to no avail. The server keeps closing the connection after the same 10 minutes.
dataSource = new BasicDataSource();
dataSource.setUsername(dbUser);
dataSource.setPassword(dbPassword);
dataSource.setUrl(dbUrl);
dataSource.setDefaultAutoCommit(true);
dataSource.setTestOnBorrow(true);
dataSource.setTestOnReturn(true);
dataSource.setDriverClassName("com.amazon.redshift.jdbc41.Driver");
dataSource.setDefaultQueryTimeout(7200);
dataSource.setMaxConnLifetimeMillis(7200000);
dataSource.addConnectionProperty("socketTimeout", "7200");
How do I keep the connection alive for longer?
P.S. I do not have any problems establishing connections and running queries that take less than 10 minutes to finish.

You might want to extend your socket timeout.
Current it is 7200ms only:
dataSource.addConnectionProperty("socketTimeout", "7200");

check if the redshift server have a workload management policy that is timing out queries after 10 minutes.
your java code might be setting this policy

You need to set the tcpKeepAlive time to 1 min or less while getting the connection to redshift cluster.
Properties props = new Properties();
props.setProperty("user", user);
props.setProperty("password", password);
props.setProperty("tcpKeepAlive", "true");
props.setProperty("TCPKeepAliveMinutes", "1");
DriverManager.getConnection("jdbc:redshift://"+endpoint+":"
+port+"/"+database, props);

OP here- I was able to make it work by writing wrappers over BasicDataSource and Connection to poll active connection with isValid(int) every few minutes (any frequency more than once-per-10-minutes works). In hindsight, it seems that most timeout-related properties on BasicDataSource apply to connections which are in the pool but are not being used. setDefaultQueryTimeout and tcpKeepAlive + TCPKeepAliveMinutes did not work.
P.S. It has been a while since I resolved this problem and I do not have the code for the wrappers now. Here's a brief description of the wrappers.
WrappedConnection class takes a Connection object (conn) and a TimerTask object (timerTask) in its constructor and implements the Connection interface by simply calling the methods from conn. timerTask calls this.isValid(100) every few minutes as long as the connection is active. WrappedConnection.close stops timerTask and then calls conn.close.
WrappedBasicDataSource implements the DataSource interface, redirecting methods to a BasicDataSource object. BasicDataSourceWrapper.getConnection gets a connection from the aforementioned BasicDataSource and generates a WrappedConnection using the connection and a new TimerTask object.
I might have missed explaining some details but this is the gist of it.

Find real error cause when acquiring connection with c3po ComboPooledDataSource

I am trying to get connection from ComboPooledDataSource, but let's say I made a typo in the password. I would like to get the information about the actual error cause, but all I can from the thrown exception is that something went wrong. Is there a way to find out what the real problem is, in form of a String that can be displayed to the user?
I am using c3po-0.9.1.2.
Let's be concrete, my code looks like this:
ComboPooledDataSource ds = new ComboPooledDataSource();
ds.setDriverClass("org.postgresql.Driver");
ds.setJdbcUrl("jdbc:postgresql://localhost:5432/my_db");
ds.setUser("user1");
ds.setPassword("wrong-password");
try {
Connection c = ds.getConnection();
...
} catch (Exception e) {
e.printStackTrace();
}
The ds.getConnection() code throws an exception, which prints the following stakctrace:
java.sql.SQLException: Connections could not be acquired from the underlying database!
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:529)
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128)
at test.ConnectionErrorDemo.main(ConnectionErrorDemo.java:21)
Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source.
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1319)
at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:557)
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:477)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:525)
... 2 more
So, this exception is not helpful at all, as it does not say anything about the actual cause of the problem. However, in the console you can also see log entries like this, which describe the actual problem:
WARNING: com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask#249939c3 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception:
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "user1"
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:398)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:173)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:136)
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21)
at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31)
at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:393)
at org.postgresql.Driver.connect(Driver.java:267)
at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:134)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:182)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:171)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:137)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1014)
at com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:32)
at com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1810)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
If I understand that correctly, this log entry is done by the c3po library itself, an therefore I thought maybe there is a way to access this information somehow. Any ideas?

I am getting Cannot get a connection, pool error Timeout waiting for idle object, When I try to create more than 250 threads in my web application [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Connection Pool Exception: Cannot get a connection, pool error Timeout waiting for idle object
I am getting Cannot get a connection, pool error Timeout waiting for idle object error, When I try to create more than 250 threads in my web application. I am creating web application using JSF 2.0 and Hibernate.
I have tried with modified hibernate.xml,server.xml,context.xml and also mysql properties.
The followings are I am getting.
WARN (JDBCExceptionReporter.java:233) - SQL Error: 0, SQLState: null
ERROR (JDBCExceptionReporter.java:234) - Cannot get a connection, pool error Timeout waiting for idle object
ERROR (BaseServlet.java:301) - ******** java.lang.Thread.getStackTrace(Thread.java:1426)
Caused by: org.hibernate.exception.GenericJDBCException: Cannot open connection
org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:140)
org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:128)
org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:52)
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:449)
org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167)
org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:160)
org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:81)
org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1473)
sun.reflect.GeneratedMethodAccessor378.invoke(Unknown Source)
Caused by: org.apache.tomcat.dbcp.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object
org.apache.tomcat.dbcp.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:114)
org.apache.tomcat.dbcp.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
org.hibernate.connection.DatasourceConnectionProvider.getConnection(DatasourceConnectionProvider.java:92)
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446)
org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167)
org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:160)
org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:81)
org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1473)
sun.reflect.GeneratedMethodAccessor378.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1144)
org.apache.tomcat.dbcp.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
org.apache.tomcat.dbcp.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
org.hibernate.connection.DatasourceConnectionProvider.getConnection(DatasourceConnectionProvider.java:92)
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446)
org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167)
org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:160)
org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:81)
org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1473)
sun.reflect.GeneratedMethodAccessor378.invoke(Unknown Source)
Please if have any idea about this help me......

Maybe you have set a timeout for "dead" connections and some queries take longer than that. That means that your pool removed a busy connection as "dead" from the pool and requests another from the DB - until the DB pulls the plug.
To debug this, enable logging for your connection pool, so you can see when it requests new connections.
also check your mysql connection settings. and try to close connection when you are done with your db coz next time(beyond maxConnectionAge limit) that connection state will be dead .

I had such issue before, what you need to do is
close hibernate sessions when you are done with it.
e.g.
Session sess = factory.openSession();
Transaction tx;
try {
tx = sess.beginTransaction();
//do some work
...
tx.commit();
}
catch (Exception e) {
if (tx!=null) tx.rollback();
throw e;
}
finally {
sess.close(); // closing session
}

Basics - Troubleshooting Hibernate / JDBC Connection Pool Issue

What is Hibernate's responsibility in regards to database connections it gets from an underlying connection pool. Does it test to see if a connection is closed before it uses it? and if so get another connection from the pool?
I've included error and confirmation info below. Any ideas of where I can start to troubleshoot this would be very helpful. And any advice on the SQL Server driver settings we are using.
from the Catalina log:
04-Nov-2010 21:54:52.691 WARNING org.apache.tomcat.jdbc.pool.ConnectionPool.abandon Connection has been abandoned PooledConnection[ConnectionID:8]:java.lang.Exception
at org.apache.tomcat.jdbc.pool.ConnectionPool.getThreadDump(ConnectionPool.java:926)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:681)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:545)
at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:166)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:106)
from our application log:
2010-11-04 21:54:52,705 [tomcat-http--18] WARN util.JDBCExceptionReporter - SQL Error: 0, SQLState: 08S01
2010-11-04 21:54:52,707 [tomcat-http--18] ERROR util.JDBCExceptionReporter - Socket closed
2010-11-04 21:54:52,708 [tomcat-http--18] ERROR transaction.JDBCTransaction - JDBC rollback failed
java.sql.SQLException: Connection has already been closed.
at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:112)
at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:94)
at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:71)
at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:94)
at org.apache.tomcat.jdbc.pool.interceptor.ConnectionState.invoke(ConnectionState.java:132)
at $Proxy38.rollback(Unknown Source)
at org.hibernate.transaction.JDBCTransaction.rollbackAndResetAutoCommit(JDBCTransaction.java:217)
at org.hibernate.transaction.JDBCTransaction.rollback(JDBCTransaction.java:196)
at org.springframework.orm.hibernate3.HibernateTransactionManager.doRollback(HibernateTransactionManager.java:676)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:845)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:822)
at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:412)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:111)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:625)
The configuration:
<Resource defaultAutoCommit="false" defaultReadOnly="false"
defaultTransactionIsolation="SERIALIZABLE"
driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
fairQueue="false" initialSize="10"
jdbcInterceptors="ConnectionState;StatementFinalizer"
jmxEnabled="true" logAbandoned="true" maxActive="100"
maxIdle="10" maxWait="30000"
minEvictableIdleTimeMillis="10000" minIdle="10"
name="com.ourcompany.ap.shoppingcart/datasource"
password="somePassword" removeAbandoned="true"
removeAbandonedTimeout="60" testOnBorrow="true"
testOnReturn="false" testWhileIdle="false"
timeBetweenEvictionRunsMillis="5000"
type="javax.sql.DataSource"
url="jdbc:sqlserver://approd\approd;databaseName=prod"
useEquals="false" username="AccessPointNet"
validationInterval="30000" validationQuery="SELECT 1"/>`

I had a similar problem which was solved by increasing the removeAbandonedTimeout value to a higher number. The problem we faced was due to the query which took longer time that the above mentioned timeout.

What is Hibernate's responsibility in regards to database connections it gets from an underlying connection pool.
Not much, releasing it when the Session gets closed.
Does it test to see if a connection is closed before it uses it? and if so get another connection from the pool?
No, Hibernate doesn't, checking the validity of connection(s) is the responsibility of a connection pool if you want to.
I've included error and confirmation info below. Any ideas of where I can start to troubleshoot this would be very helpful.
What kind of process are you running exactly? A long transaction? Does it timeout? What does the Caused by: say? About the trace:
2010-11-04 21:54:52,705 [tomcat-http--18] WARN util.JDBCExceptionReporter - SQL Error: 0, SQLState: 08S01
2010-11-04 21:54:52,707 [tomcat-http--18] ERROR util.JDBCExceptionReporter - Socket closed
2010-11-04 21:54:52,708 [tomcat-http--18] ERROR transaction.JDBCTransaction - JDBC rollback failed java.sql.SQLException: Connection has already been closed.
Can you reproduce it in a deterministic way? Any networking problem?
And any advice on the SQL Server driver settings we are using.
I've added a great resource about Tomcat and connection pool configuration below. Not specific to SQL Server though.
Resources
Configuring jdbc-pool for high-concurrency

We usually work around this by using dbcp, and providing a validationQuery when definining our data source. Then, dbcp will verify the usability of pooled connections by issuing that query (and transparently recreate the connection should it no longer work), prior to returning them to the application.
Check out
http://tomcat.apache.org/tomcat-6.0-doc/jndi-datasource-examples-howto.html
for more details.

I am currently using liquibase(v1.9) in my project, and when the changeSets run against a blank schema it always takes longer than 60 seconds which results in the thread being marked abandoned I'm not thrilled with increasing the removeAbandonedTimeout value, but this is the only solution I've been able to find to prevent this issue; however, after the initial schema population is complete this is seldom a problem so I set the value back to 60 seconds.

I worked on an issue in the past where we weren't returning connections back to the pool correctly. So, when a connection was used and not returned, making a database call when it was timing out would throw an exception.
We were able to reproduce the issue by making a call to the database, waited 8 hours (postgres' default time out) and tried to make a call to the database again. It throw the same exception every time. Our solution was to rethink (or better yet, add) a connection management strategy.
So, to sum up, are you actually returning your connections to the pool by closing the Session?

I got the solution for the above exception.
Just close the instance of session factory as well while closing the session .
Look at the below Code:
public class HibernateUtil {
private static final SessionFactory sessionFactory = buildSessionFactory();
private static SessionFactory buildSessionFactory() {
try {
// Create the SessionFactory from hibernate.cfg.xml
return new Configuration().configure("hibernate.cfg.xml").buildSessionFactory();
}
catch (Throwable ex) {
ex.printStackTrace();
// Make sure you log the exception, as it might be swallowed
System.err.println("Initial SessionFactory creation failed." + ex);
throw new ExceptionInInitializerError(ex);
}
}
public static SessionFactory getSessionfactory() {
return sessionFactory;
}
public static Session getSession() {
Session session=sessionFactory.openSession();
session.getTransaction().begin();
return session;
}
public static void closeSession(Session session) {
if(session!=null )
{
if(session.getTransaction().isActive())
{
session.getTransaction().commit();
}
session.close();
getSessionfactory().close();
}
}
}
just call the method HibernateUtil.closeSession(). This will solve the problem.

Java Connection Pooling best practices?

After getting fed up with c3p0's constant locking I'm turning to BoneCP for an alternative connection pool for my database. I have a server app that processes around 7,000 items per minute and needs to log those items into our MySQL database. I currently have 100 worker threads and have set up my pool like such:
BoneCPConfig config = new BoneCPConfig();
config.setJdbcUrl("jdbc:mysql://"+Settings.MYSQL_HOSTNAME+"/"+Settings.MYSQL_DATABASE+"?autoReconnectForPools=true" );
config.setUsername(Settings.MYSQL_USERNAME);
config.setPassword(Settings.MYSQL_PASSWORD);
config.setMinConnectionsPerPartition(5);
config.setMaxConnectionsPerPartition(10);
config.setPartitionCount(5);
config.setAcquireIncrement(5);
connectionPool = new BoneCP(config); // setup the connection pool
Are those acceptable settings for such an app? I'm asking because after a minute or two into running I was getting BoneCP exceptions when trying to call getConnection on the pool. Thanks for the help.
Here is the code I was using for the db calls in my worker threads, it can't fail on the dbConn = this.dbPool.getConnection() line. Am I not closing connections properly?
private void insertIntoDb() {
try {
Connection dbConn = this.dbPool.getConnection();
try {
PreparedStatement ps3 = dbConn.prepareStatement("INSERT IGNORE INTO test_table1 SET test1=?, test2=?, test3=?");
ps3.setString(1, "some string");
ps3.setString(2, "some other string");
ps3.setString(3, "more strings");
ps3.execute();
ps3.close();
PreparedStatement ps4 = dbConn.prepareStatement("INSERT IGNORE INTO test_table2 SET test1=?, test2=?, test3=?");
ps4.setString(1, "some string");
ps4.setString(2, "some other string");
ps4.setString(3, "more strings");
ps4.execute();
ps4.close();
} catch(SQLException e) {
logger.error(e.getMessage());
} finally {
try {
dbConn.close();
} catch (SQLException e) {
logger.error(e.getMessage());
}
}
} catch(SQLException e) {
logger.error(e.getMessage());
}
}
This is the error I was seeing:
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
ERROR pool-2-thread-39 2010-09-04 13:36:19,798 com.test.testpackage.MyTask - null
java.sql.SQLException
at com.jolbox.bonecp.BoneCP.getConnection(BoneCP.java:381)

Are those acceptable settings for such an app? I'm asking because after a minute or two into running I was getting boneCP exceptions when trying to call getConnection on the pool. thanks for the help.
If you have 100 workers, why do you limit the pool to 50 connections (number of partitions x max number of connections per partition i.e. 5 x 10 in your case)?
Am I not closing connections properly?
Looks ok (but maybe enable connectionWatch as hinted to see what the warning is about exactly). Personally, I close all the resources I use, including statement and result sets. Just in case, here is the idiom I use:
Connection conn = null;
PreparedStatement pstmt = null;
ResultSet rs = null;
try {
conn = pool.getConnection();
pstmt = conn.prepareStatement(SOME_SQL);
pstmt.setFoo(1, foo);
...
rs = pstmt.executeQuery();
...
} finally {
if (rs != null) try { rs.close(); } catch (SQLException quiet) {}
if (pstmt != null) try { pstmt.close(); } catch (SQLException quiet) {}
if (conn != null) try { conn.close(); } catch (SQLException quiet) {}
}
You could group the above calls in a static method of a utility class.
Or you could use DbUnit.closeQuietly(Connection, Statement, ResultSet) from Commons DbUtils that already does this.