I use Hikary connection pool with following settings:
HikariDataSource dataSource = new HikariDataSource();
dataSource.setMinimumIdle(0);
dataSource.setMaximumPoolSize(Integer.MAX_VALUE);
dataSource.setJdbcUrl(jdbcConnectionString);
dataSource.setConnectionTestQuery("select 1");
dataSource.setIdleTimeout(TimeUnit.SECONDS.toMillis(60));
dataSource.getConnection();
After getConnection() hikari try to get 2 connections to instance, but put in connection pool just one connection. How can I fix it? The hikari version is 3.4.0
I found the answer. Hikari creates first connection in checkFailFast method. I update this comment when find how to disable this method. The checkFailFast doesn't work if initializationFailTimeout<0. It helps me
After getConnection() hikari try to get 2 connections to instance, but put in connection pool just one connection. How can I fix it?
There is nothing to fix in this behavior. It simple means, that two conenctions were opened and one of them was closed.
The reason why the second connection was closed is that you set setMinimumIdle(0), i.e. no idle connection is maintainend in the pool and all idle connection are closed.
If you want to see both connection in the pool, simple set setMinimumIdle(1). After calling DataSource.getConnection() there will be two connection in the pool - one yours and one idle.
If you don't want to open the second connection at all, set
config.setMinimumIdle( 1 );
config.setMaximumPoolSize( 1 );
But think twice, why do you use a connection pool with only one connection.
You may anyway increase both parameters later, while the pool is running.
HikariConfigMXBean bn = DataSource.ds.getHikariConfigMXBean()
bn.setMaximumPoolSize(10)
bn.setMinimumIdle(10)
This will (not instantly) open 9 additional connections to the database.
Note that while setting the MaximumPoolSize == MinimumIdle the number of connection in the pool remains stable, no connections are opened or closed, which is probably the thing you want to observe.
Tested with Hicari 3.4.0 and Oracle 12.2
Related
This question already has an answer here:
How to release connection from JDBC connection pool in Tomcat?
(1 answer)
Closed 1 year ago.
I have an application which uses oracle.ucp.jdbc.PoolDataSource to maintain pool of JDBC connections. I am able to get a connection and use it.
At the end of my function, I want to return the connection to the pool. I don't find any method to return the the pool.
How to return the connection to the pool?
The fine manual oracle.ucp.jdbc
Class PoolDataSourceImpl
This DataSource uses the Universal Connection Pool to get a connection, creating a proxy to that connection, and returning the proxy to the client. When the client calls close on the connection proxy, the underlying physical connection is returned to the pool.
So, as noted in the comments, use connection.close(). The connection is a proxy and the close method will return the connection to the pool instead of closing the connection as is usual.
/* connection pool created with 5 connections based on the region specific.
with below code it will get connection from connection pool which is already created.*/
Connection con = DatasourceClient.getDataSourceMap.get(region).getConnection();
OracleConnection oConn = con.unwrap(oracle.jdbc.OracleConnection.class);
Will above code will get two connections from pool and do i need to close both con and Oconn ?
i am getting pool exhausted and connection closed exceptions tried many ways by changing pool properties.
So just want to know what above code is doing.
tried closing the above connections but didn't get any difference results.
Using Oracle Jdbc template instead of spring jdbc because in my procedures there are array values which in few cases only input, in some cases only output and other both INOUT.
Can any one help me in this please ? Thank you.
No, it will get only a single connection out, which you then unwrap to it's actual class.
However you will need to call con.close() (and never oCon.close()) to return the connection back to the pool. This is because the wrapper's close() doesn't actually close the connection, it returns it back to the pool.
I'm developing a web app on Tomcat 8 with Maven, I'm using c3p0 to handle connections on the main thread and on 2 other concurrent threads, my connection manager class is asking a DataSource singleton class I've implemented for synchronized connections, like so
public synchronized Connection getConnection() {
try {
return cpds.getConnection();
} catch (SQLException ex) {
logger.error("Error while issuing a pooled connection", ex);
}
return null;
}
, but when I'm trying to use these connections they start to either interrupt
09:47:17.164 [QuartzScheduler_Worker-4] ERROR com.myapp.providers.DataSource - Error while issuing a pooled connection
java.sql.SQLException: An SQLException was provoked by the following failure: java.lang.InterruptedException
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:65) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:62) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:531) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128) ~[c3p0-0.9.1.2.jar:0.9.1.2]
or close in mid transaction and breaking any statements and result sets that are being used at that time
I'm configuring the DataSource object like so
cpds = new ComboPooledDataSource();
cpds.setDriverClass(oracle.jdbc.driver.OracleDriver);
cpds.setJdbcUrl(jdbc:oracle:thin:#xx.xxx.xxx.xxx:1521:XE);
cpds.setUser("username");
cpds.setPassword("password");
// database connection properties
cpds.setInitialPoolSize(10);
cpds.setAcquireIncrement(3);
cpds.setMaxPoolSize(100);
cpds.setMinPoolSize(15);
cpds.setMaxStatements(75);
// connection pool preferences
cpds.setIdleConnectionTestPeriod(60);
cpds.setMaxIdleTime(30000);
cpds.setAutoCommitOnClose(false);
cpds.setPreferredTestQuery("SELECT 1 FROM DUAL");
cpds.setTestConnectionOnCheckin(false);
cpds.setTestConnectionOnCheckout(false);
cpds.setAcquireRetryAttempts(30);
cpds.setAcquireRetryDelay(1000);
cpds.setBreakAfterAcquireFailure(false);
I've also written a small test method that runs in a loop and queries the database for n times but that works fine.
c3p0-0.9.1.2 is very, very old; please consider upgrading to 0.9.5.1, the current production version.
The problem is both clear and not so clear. The clear part is that something is calling interrupt() on client Threads that are waiting to acquire Connections. The not-so-clear part is who is doing that and why.
A guess is that Tomcat itself is doing that because the client Threads are hung too long. If the Threads are hanging at getConnection(), that could be due to a Connection leak and pool exhaustion. We see above how you acquire Connections. Are you vigilant about ensuring that they are reliably close()ed in finally blocks?
A thing you might try is to set a checkoutTimeout, e.g.
cpds.setCheckoutTimeout( 5000 ); // 5 secs
This won't actually solve the problem if Connection checkouts are hanging. But rather than a problem provoked by mysterious interrupts, you'll see c3p0 TimeoutExceptions instead. That will verify that the issue is long hangs on checkout, though, which would most likely be due to pool exhaustion, either from a Connection leak (missing calls to close()), or simply from a maxPoolSize value too low for your load.
If there does seem to be a Connection leak, please see unreturnedConnectionTimeout and debugUnreturnedConnectionStackTraces for help tracking it down. See also "Configuring to Debug and Workaround Broken Client Applications"
In our project we are maintaining our own DB connection pool.
For resolving the issue 'java.sql.SQLRecoverableException: Io exception most of people has suggested to use standard connection pool like apache dbcp.
I am wondering what is the logic those standard pooling mechanism will perform during connection reset?
How do DBConnectionPool know that DB connection has timed out? since we know conn.isClosed() won't help here.
Is it each db connection will have one tcp client socket with DB server?
Finally is it advisable; whenever i return the connection to the pool; pool should close the connection; if the connection is existing more than ~10 mins from it is returned?
[~10 mins server side conn timeout variable]
Kindly answer all my questions.
I am answering this question assuming that you made use of Apache DBCP for connection pooling by using org.apache.commons.pool.impl.GenericObjectPool, org.apache.commons.dbcp.DataSourceConnectionFactory, org.apache.commons.dbcp.PoolableConnectionFactory and org.apache.commons.dbcp.PoolingDataSource classes.
I am wondering what is the logic those standard pooling mechanism
will perform during connection reset?
If GenericObjectPool.testOnBorrow and GenericObjectPool.testOnReturn are set true to The Connection will be validated whether it is active or not using a validationQuery set in PoolableConnectionFactory. If the validation is failed the Connection object is dropped and new one is created and added to the pool
How do DBConnectionPool know that DB connection has timed out? since
we know conn.isClosed() won't help here. Same mechanism as above
Is it each db connection will have one tcp client socket with DB
server? Yes
Finally is it advisable; whenever i return the connection to the
pool; pool should close the connection; if the connection is existing
more than ~10 mins from it is created? [~10 mins server side conn
timeout variable] If you think it should will not create unneccessary network traffic and if you have special reason to do that. You can do it. By setting minEvictableIdleTimeMillis in GenericObjectPool along with timeBetweenEvictionRunsMillis if you want to remove based on idle time
I have a scenario and the question follows
Application server has two connections pools to DB. A and B
A points to -> DatabaseA -> has 128 connections
A has Stored Procedures which access tables residing in DatabaseB over the DB link
B points to -> DatabaseB -> has 36 connections
Now lets say that Java code calls Stored Proc in DatabaseA by using connection pool A. This stored proc is getting data over the DB link from DatabaseB
Question:
Based on this scenario if we get connection closed errors on the front end. Is it viable to say that even though java is calling the SP (in DatabaseA) from pool A (128) but since the SP is bringing data from DatabaseB it has less amount of connections (36).
Basically I want to know when the data is brought over the DB link like this...does it take away from 36 connections assigned to pool B pointint to DatabaseB?
Exact Exception
Exact exception I get is: --- Cause: java.sql.SQLException: Closed Connection
Some Stack trace:
Caused by: java.sql.SQLException:
Closed Connection at
com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryWithCallback(GeneralStatement.java:185)
at
com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryForList(GeneralStatement.java:123)
at
com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:614)
at
com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:588)
at
com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryForList(SqlMapSessionImpl.java:118)
at
org.springframework.orm.ibatis.SqlMapClientTemplate$3.doInSqlMapClient(SqlMapClientTemplate.java:268)
at
org.springframework.orm.ibatis.SqlMapClientTemplate.execute(SqlMapClientTemplate.java:193)
at
org.springframework.orm.ibatis.SqlMapClientTemplate.executeWithListResult(SqlMapClientTemplate.java:219)
at
org.springframework.orm.ibatis.SqlMapClientTemplate.queryForList(SqlMapClientTemplate.java:266)
Also, I am using iBatis ...so don't have try..catch..finally blocks
The stored procedure is running in the database; when it makes the connection to the other database it makes a direct connection and doesn't go through the app server's pool. In fact, it could make a connection to any database that is linked to A, regardless whether or not there's a connection pool to that database maintained by the app server.
This exception indicates a resource leak, i.e. the JDBC code is not properly closing connections in the finally block (to ensure that it's closed even in case of an exception) or the connection is been shared among multiple threads. If two threads share the same connection from the pool and one thread closes it, then this exception will occur when the other thread uses the connection.
The JDBC code should be written so that connections (and statements and resultsets) are acquired and closed (in reversed order) in the very same method block. E.g.
Connection connection = null;
// ...
try {
connection = database.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException logOrIgnore) {}
}
Another possible cause is that the pool is holding connections too long idle and not testing/verifying them before releasing. This is configureable in a decent connection pool. Consult its documentation.
"Basically I want to know when the data is brought over the DB link like this...does it take away from 36 connections assigned to pool B pointint to DatabaseB?"
No. The database server will make a distinct connection to the other database server irrespective of any connection pool.
I have to suffer a firewall that cuts of connections after a period of inactivity so I see this error quite a lot. Look into dbms_session.close_database_link, since the database link connection would generally remain for the duration of the session (and since you have a connection pool, that session probably sits around for a very long time).