My system encounter some connection leak in connection pool. I would like to list down some statistic of the connection pool regularly, how can I do that? For example, Current Capacity, Active Connections High Count, Connections Total Count, Leaked Connection Count and etc.
I am using javax.sql.DataSource to retrieve the connection from connection pool. But I couldn't find any interface that can retrieve those connection pool information. Any ideas?
I am using Oracle DB and Java EE as my server side script.
The javax.sql.DataSource is an interface and it just abstracts a data source. It does not involve providing pooled connections to it.
A connection pool is responsible for providing pooled, reusable connections to a database (data source).
First you need to find out which connection pool you're using. Connection pool implementations usually provide a way to query things like the number of active connections.
For example the Apache DBCP has a BasicDataSource class which is a connection pool, and it has a methods for this:
BasicDataSource.getMaxTotal();
BasicDataSource.getNumActive();
BasicDataSource.getNumIdle();
BasicDataSource.getMinIdle();
BasicDataSource.getMaxIdle();
Since you mentioned you're using Oracle DB, most likely your connection pool is OracleOCIConnectionPool (part of Oracle JDBC driver) which provides:
OracleOCIConnectionPool.getMaxLimit();
OracleOCIConnectionPool.getPoolSize();
OracleOCIConnectionPool.getActiveSize();
OracleOCIConnectionPool.getMinLimit();
Related
/* connection pool created with 5 connections based on the region specific.
with below code it will get connection from connection pool which is already created.*/
Connection con = DatasourceClient.getDataSourceMap.get(region).getConnection();
OracleConnection oConn = con.unwrap(oracle.jdbc.OracleConnection.class);
Will above code will get two connections from pool and do i need to close both con and Oconn ?
i am getting pool exhausted and connection closed exceptions tried many ways by changing pool properties.
So just want to know what above code is doing.
tried closing the above connections but didn't get any difference results.
Using Oracle Jdbc template instead of spring jdbc because in my procedures there are array values which in few cases only input, in some cases only output and other both INOUT.
Can any one help me in this please ? Thank you.
No, it will get only a single connection out, which you then unwrap to it's actual class.
However you will need to call con.close() (and never oCon.close()) to return the connection back to the pool. This is because the wrapper's close() doesn't actually close the connection, it returns it back to the pool.
I use Hikary connection pool with following settings:
HikariDataSource dataSource = new HikariDataSource();
dataSource.setMinimumIdle(0);
dataSource.setMaximumPoolSize(Integer.MAX_VALUE);
dataSource.setJdbcUrl(jdbcConnectionString);
dataSource.setConnectionTestQuery("select 1");
dataSource.setIdleTimeout(TimeUnit.SECONDS.toMillis(60));
dataSource.getConnection();
After getConnection() hikari try to get 2 connections to instance, but put in connection pool just one connection. How can I fix it? The hikari version is 3.4.0
I found the answer. Hikari creates first connection in checkFailFast method. I update this comment when find how to disable this method. The checkFailFast doesn't work if initializationFailTimeout<0. It helps me
After getConnection() hikari try to get 2 connections to instance, but put in connection pool just one connection. How can I fix it?
There is nothing to fix in this behavior. It simple means, that two conenctions were opened and one of them was closed.
The reason why the second connection was closed is that you set setMinimumIdle(0), i.e. no idle connection is maintainend in the pool and all idle connection are closed.
If you want to see both connection in the pool, simple set setMinimumIdle(1). After calling DataSource.getConnection() there will be two connection in the pool - one yours and one idle.
If you don't want to open the second connection at all, set
config.setMinimumIdle( 1 );
config.setMaximumPoolSize( 1 );
But think twice, why do you use a connection pool with only one connection.
You may anyway increase both parameters later, while the pool is running.
HikariConfigMXBean bn = DataSource.ds.getHikariConfigMXBean()
bn.setMaximumPoolSize(10)
bn.setMinimumIdle(10)
This will (not instantly) open 9 additional connections to the database.
Note that while setting the MaximumPoolSize == MinimumIdle the number of connection in the pool remains stable, no connections are opened or closed, which is probably the thing you want to observe.
Tested with Hicari 3.4.0 and Oracle 12.2
I have a java process which is multi threaded using ExecutorService (15 threads). Each thread calls store procedure to insert data to table, my connection to be pooled across 15 threads so that I could see multiple commits on the table at the same time, but i only see one connection established for one active thread even through 15 threads are ready and waiting.
Driver: oracle.jdbc.driver.OracleDriver
Following are the connection details I have in my properties file
url, username, password
Class.forName(DB_DRIVER);
DataSource oracleDataSource = new DriverManagerDataSource(DB_CONNECTION, DB_USER,DB_PASSWORD);
ObjectPool objectPool = new GenericObjectPool();
DataSourceConnectionFactory datasourceConnectionFactory = new DataSourceConnectionFactory(oracleDataSource);
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(datasourceConnectionFactory, objectPool, null, null, false, true);
objectPool.setFactory(poolableConnectionFactory);
PoolingDataSource datasource = new PoolingDataSource(objectPool)
Oracle has Universal Connection Pool (ucp.jar) which is easy to use and is from oracle. All you need is to include ucp.jar in the classpath along with ojdbc6.jar or ojdbc7.jar.
Refer to the UCP Reference Guide: http://docs.oracle.com/database/121/JJUCP/toc.htm
PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();
pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
pds.setMinPoolSize(10);
pds.setMaxPoolSize(50);
Connection conn=pds.getConnection();
You need a connection pool.
Either the object that manages the executor pool will check out a connection, give it to the ExecutorService, and close it when the task completes OR the ExecutorService will manage it. Make sure that you pay careful attention to connection management and SQL resource cleanup or you'll quickly have problems under high request volume.
Usually it's Java EE app servers that manage connection pools for you, but it sounds like you aren't using one. If that's true, perhaps Apache Database Connection Pool will suit your purpose.
In our project we are maintaining our own DB connection pool.
For resolving the issue 'java.sql.SQLRecoverableException: Io exception most of people has suggested to use standard connection pool like apache dbcp.
I am wondering what is the logic those standard pooling mechanism will perform during connection reset?
How do DBConnectionPool know that DB connection has timed out? since we know conn.isClosed() won't help here.
Is it each db connection will have one tcp client socket with DB server?
Finally is it advisable; whenever i return the connection to the pool; pool should close the connection; if the connection is existing more than ~10 mins from it is returned?
[~10 mins server side conn timeout variable]
Kindly answer all my questions.
I am answering this question assuming that you made use of Apache DBCP for connection pooling by using org.apache.commons.pool.impl.GenericObjectPool, org.apache.commons.dbcp.DataSourceConnectionFactory, org.apache.commons.dbcp.PoolableConnectionFactory and org.apache.commons.dbcp.PoolingDataSource classes.
I am wondering what is the logic those standard pooling mechanism
will perform during connection reset?
If GenericObjectPool.testOnBorrow and GenericObjectPool.testOnReturn are set true to The Connection will be validated whether it is active or not using a validationQuery set in PoolableConnectionFactory. If the validation is failed the Connection object is dropped and new one is created and added to the pool
How do DBConnectionPool know that DB connection has timed out? since
we know conn.isClosed() won't help here. Same mechanism as above
Is it each db connection will have one tcp client socket with DB
server? Yes
Finally is it advisable; whenever i return the connection to the
pool; pool should close the connection; if the connection is existing
more than ~10 mins from it is created? [~10 mins server side conn
timeout variable] If you think it should will not create unneccessary network traffic and if you have special reason to do that. You can do it. By setting minEvictableIdleTimeMillis in GenericObjectPool along with timeBetweenEvictionRunsMillis if you want to remove based on idle time
I have a scenario and the question follows
Application server has two connections pools to DB. A and B
A points to -> DatabaseA -> has 128 connections
A has Stored Procedures which access tables residing in DatabaseB over the DB link
B points to -> DatabaseB -> has 36 connections
Now lets say that Java code calls Stored Proc in DatabaseA by using connection pool A. This stored proc is getting data over the DB link from DatabaseB
Question:
Based on this scenario if we get connection closed errors on the front end. Is it viable to say that even though java is calling the SP (in DatabaseA) from pool A (128) but since the SP is bringing data from DatabaseB it has less amount of connections (36).
Basically I want to know when the data is brought over the DB link like this...does it take away from 36 connections assigned to pool B pointint to DatabaseB?
Exact Exception
Exact exception I get is: --- Cause: java.sql.SQLException: Closed Connection
Some Stack trace:
Caused by: java.sql.SQLException:
Closed Connection at
com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryWithCallback(GeneralStatement.java:185)
at
com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryForList(GeneralStatement.java:123)
at
com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:614)
at
com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:588)
at
com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryForList(SqlMapSessionImpl.java:118)
at
org.springframework.orm.ibatis.SqlMapClientTemplate$3.doInSqlMapClient(SqlMapClientTemplate.java:268)
at
org.springframework.orm.ibatis.SqlMapClientTemplate.execute(SqlMapClientTemplate.java:193)
at
org.springframework.orm.ibatis.SqlMapClientTemplate.executeWithListResult(SqlMapClientTemplate.java:219)
at
org.springframework.orm.ibatis.SqlMapClientTemplate.queryForList(SqlMapClientTemplate.java:266)
Also, I am using iBatis ...so don't have try..catch..finally blocks
The stored procedure is running in the database; when it makes the connection to the other database it makes a direct connection and doesn't go through the app server's pool. In fact, it could make a connection to any database that is linked to A, regardless whether or not there's a connection pool to that database maintained by the app server.
This exception indicates a resource leak, i.e. the JDBC code is not properly closing connections in the finally block (to ensure that it's closed even in case of an exception) or the connection is been shared among multiple threads. If two threads share the same connection from the pool and one thread closes it, then this exception will occur when the other thread uses the connection.
The JDBC code should be written so that connections (and statements and resultsets) are acquired and closed (in reversed order) in the very same method block. E.g.
Connection connection = null;
// ...
try {
connection = database.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException logOrIgnore) {}
}
Another possible cause is that the pool is holding connections too long idle and not testing/verifying them before releasing. This is configureable in a decent connection pool. Consult its documentation.
"Basically I want to know when the data is brought over the DB link like this...does it take away from 36 connections assigned to pool B pointint to DatabaseB?"
No. The database server will make a distinct connection to the other database server irrespective of any connection pool.
I have to suffer a firewall that cuts of connections after a period of inactivity so I see this error quite a lot. Look into dbms_session.close_database_link, since the database link connection would generally remain for the duration of the session (and since you have a connection pool, that session probably sits around for a very long time).