I have been trying to go through the c3p0 documentation but not able to understand 'testConnectionOnCheckin' property.
Docs says - "Connections are tested before they are included in pool".
Does this property apply to only new connections that c3p0 creates are tested before they are included in pool? What is point of checking new connections? Wouldn't they generally be valid?
Also couple of days my application logs were showing following:
[managed:2 unused:2 excluded:1]
And my application was throwing exception for one particular connection which I assume is 'excluded' one. Is 'excluded' connection counted in pool and can c3p0 hand it over to application without checking validity? If not, then would setting 'testConnectionOnCheckin' test this excluded connection for validity before it is used by my application?
I apologize for too many questions but it's just that I am confused.
Thanks
Jitendra
testConnectionOnCheckin tests Connections after they are checked-in by clients [ie via Connection.close()], but before they are reintegrated into the Connection pool. I'm not sure what documentation you are looking at, but see
http://www.mchange.com/projects/c3p0/#testConnectionOnCheckin
http://www.mchange.com/projects/c3p0/#configuring_connection_testing
I generally recommend testing Connections with a combination of idleConnectionTestPeriod and testConnectionsOnCheckIn (and a fast preferredTestQuery).
An "excluded" Connection is a Connection currently in use by a client, but which c3p0 has noticed is faulty. c3p0 marks these Connections to be destroyed rather than reintegrated into the pool when they are checked-in by the client.
I hope this helps!
Related
I added setMaxActive(8) on org.apache.tomcat.jdbc.pool.PoolProperties. Every time the DB restarts, the application is unusable because the established connections remain. I get the following error:
org.postgresql.util.PSQLException: This connection has been closed
I've tried using some other settings on the pool to no avail...
Thank you for help!
Use the validationQuery property which will check if the connection is valid before returning the connection.
Ref: Tomcat 6 JDBC Connection Pool
This property is available on latest tomcat versions.
Look at this link:
Postgres connection has been closed error in Spring Boot
Very valid question and this problem is usually faced by many. The
exception generally occurs, when network connection is lost between
pool and database (most of the time due to restart). Looking at the
stack trace you have specified, it is quite clear that you are using
jdbc pool to get the connection. JDBC pool has options to fine-tune
various connection pool settings and log details about whats going on
inside pool.
You can refer to to detailed Apache documentation on pool
configuration to specify abandon timeout
Check for removeAbandoned, removeAbandonedTimeout, logAbandoned parameters
Additionally you can make use of additional properties to further
tighten the validation
Use testXXX and validationQuery for connection validity.
My own $0.02: use these two parameters:
validationQuery=<TEST SQL>
testOnBorrow=true
I'm facing a problem in JavaEE(Glassfish and also TomEE) with connection pooling. When the network/datatier is down and going up again, the JDBC pool seems to "hang". So there are always timeouts while trying to get a connection to the datatier. I'm sure there is something to handle this behaviour. Any suggestions?
Thanks!
you should configure the pool.
here the links: http://tomee.apache.org/datasource-config.html and http://tomee.apache.org/common-datasource-configurations.html
Take care of XXXevictionXXX, validationquery and testXXX properties
Vaadin 7 offers the SQLContainer implementation. The Book of Vaadin says to use either of its implementations of a JDBC connection pool. But I already am using the Tomcat JDBC Connection Pool implementation. Having one pool that draws from another pool seems like a bad thing.
To continue using the Tomcat pool, I implemented the com.vaadin.data.util.sqlcontainer.connection.JDBCConnectionPool interface. That interface requires three methods:
reserveConnectionI return a connection drawn from the Tomcat pool.
releaseConnectionI do nothing. I tried calling close on the connection, but that actually closed the connection rather than returning it to the Tomcat pool. Apparently the SQLContainer already called close once and my second call to close actually closes down the connection. I was getting runtime errors saying the connection was not open.
destroyI do nothing. Supposedly this method is just some workaround for some issue with Postgres (which I'm using), but seems irrelevant given that I am actually using the Tomcat pool.
➜ Is implementing that interface the correct approach?
➜ If implementing that interface is the way to go, did I do so properly? Any other issues I should address?
My Tomcat pool is available via JNDI, so I'm not sure if I should be using the Vaadin class J2EEConnectionPool.
You should in fact be able to use J2EEConnectionPool, exactly as you described in your own answer above. I have used successfully used J2EEConnectionPool with FreeformQuery so I know this works. Unfortunately there is apparently a bug in Vaadin's TableQuery implementation which caused the "connection has been closed" error you saw. See: http://dev.vaadin.com/ticket/12370
The ticket proposes a code change, but in my case I simply replaced the offending TableQuery with a FreeformQuery.
J2EEConnectionPool Is Not A Pool (a misnomer)
Having looked at the source code, it appears the J2EEConnectionPool class is misnamed. It is not an implementation of a pool. It merely draws on a javax.sql.DataSource object (obtained via JNDI if not provided to constructor) to get a java.sql.Connection object.
The "Pool" part of the name must come from an assumption that the DataSource is backed by a connection pool.
Yes, Use J2EEConnectionPool
So, if a real connection pool is in use such as the Tomcat JDBC Connection Pool available via DataSource or JNDI, pass that DataSource or JNDI info to an instance of J2EEConnectionPool for use with the Vaadin SQLContainer.
Forum Thread
See this discussion, “Any chance of bug fix for TableQuery+SQLContainer and Connection pools?”, on the Vaadin Forums.
Bug Ticket
See Ticket # 12370, “SQLContainer does not work with tomcat/BoneCP connection pool”, in the Vaadin issue tracker.
I have a situation where through a Java program, I create a javax.naming.ldap.LdapContext and do a search() operation on it - which makes an underlying connection. Then I put the Java app thread to sleep, during which I restart the LDAP server (OpenLDAP, just to note). When the App thread wakes up and tries to do any operation on the LdapContext created earlier, it throws "CommunicationException: Connection is closed".
What I want is to be able to re-establish the connection.
I see that LdapContext has a reconnect() method - where I pass controls as null. However, this does not have any effect. What I saw in the Sun LDAP implementation that during the time when the LDAP server was restarted, the ConnectionPool maintained by the Sun implementation marked the underlying com.sun.jndi.ldap.LdapClient instance with a "usable=false". Upon reconnect() call - it simply calls ensureOpen(), which again checks if the usable flag is false or not - if it's false; then it throws CommunicationException - so back to square one.
My question is: how does a Java app survive an external LDAP server restart? Is creation of new LdapContext again is the only way out?
Appreciate any insights.
Here is the stacktrace of the exception:
javax.naming.CommunicationException: connection closed [Root exception is java.io.IOException: connection closed]; remaining name 'uid=foo,ou=People,dc=example,dc=com'
at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1979)
at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1824)
at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1749)
at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:368)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:338)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:321)
at javax.naming.directory.InitialDirContext.search(InitialDirContext.java:248)
Caused by: java.io.IOException: connection closed
at com.sun.jndi.ldap.LdapClient.ensureOpen(LdapClient.java:1558)
at com.sun.jndi.ldap.LdapClient.search(LdapClient.java:504)
at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1962)
... 26 more
Just enable JNDI connection pooling and it will all be taken care of for you behind the scenes. See the JNDI Guide to Features and the LDAP Provider documentation. It's controlled by just a couple of properties.
The UnboundID LDAP SDK provides a means to auto-connect wherein that auto-reconnect operation is invisible to the client.
We had this problem at work. The solution we came up with (may not be the best answer). Was to create a watchdog thread that would check the connection at some fixed rate. If the connection did not work, it would re-initialize the connection with LDAP.
You should note that this is related essentially to LDAP connection pooling. As defined here:
A connection is retrieved from the pool, used, returned to the pool, and then, retrieved again from the pool for another Context instance.
Thus, the reuse of a previous connection may cause such problem:
You may test the behavior without using LDAP connection pooling by setting
com.sun.jndi.ldap.connect.pool=false
Also, another possible cause may be the timeout of reading the LDAP operations. In fact, the reading operation is not notified about the closure of the LDAP server after a specific timeout. For more information, you may take a look at this link
We use OpenEJB on Tomcat (used to run on JBoss, Weblogic, etc.). While running load tests we experience significant performance problems with handling JMS messages (queues). Problem was localized to blocking on database connection pool getting or releasing connection to the pool. Blocking prevented concurrent MDB instances (threads) from running hence performance suffered 10-fold and worse. The same code used to run on application servers (with their respective connection pool implementations) with no blocking at all.
Example of thread blocked:
Name: JMS Resource Adapter-worker-23
State: BLOCKED on org.apache.commons.pool.impl.GenericObjectPool#1ea6b4a owned by: JMS Resource Adapter-worker-19
Total blocked: 18,426 Total waited: 0
Stack trace:
org.apache.commons.pool.impl.GenericObjectPool.returnObject(GenericObjectPool.java:916)
org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:91)
- locked org.apache.commons.dbcp.PoolableConnection#1bcba8
org.apache.commons.dbcp.managed.ManagedConnection.close(ManagedConnection.java:147)
com.xxxxx.persistence.DbHelper.closeConnection(DbHelper.java:290)
....
Couple of questions.
I am almost certain that some transactional attributes and properties contribute to this blocking, but MDBs are defined as non-transactional (we use both annotations and ejb-jar.xml). Some EJBs do use container-managed transactions though (and we can observe blocking there as well). Are there any DBCP configurations that may fix blocking?
Is DBCP connection pool implementation replaceable in OpenEJB? How easy (difficult) to replace it with another library?
Just in case this is how we define data source in OpenEJB (openejb.xml):
<Resource id="MyDataSource" type="DataSource">
JdbcDriver oracle.jdbc.driver.OracleDriver
JdbcUrl ${oracle.jdbc}
UserName ${oracle.user}
Password ${oracle.password}
JtaManaged true
InitialSize 5
MaxActive 30
ValidationQuery SELECT 1 FROM DUAL
TestOnBorrow true
</Resource>
My 2 cts...
1 - Are there any DBCP configurations that
may fix blocking?
Although I cannot see it in the doc, I think there should also be a setting attribute named 'WhenExaustedAction' in the Resource node that could take a value "GROW" (value 2) as opposed to "BLOCK" (value 1) or "FAIL" (value 0). This comes straight from the Pools common.
Both Hibernate and Cayenne do use this DBCP setting. Don't know about OpenEJB though.
No need to say that this would work only if all connections are dutifully closed of course (which is sometimes hard to guarantee).
Then you could probably see through JMX how many connections you need at peak activity time and you could then set the maxActive to a higher value evolved from these measures.
2 - Is DBCP connection pool implementation
replaceable in OpenEJB? How easy
(difficult) to replace it with another
library?
Sorry no idea. Would imagine yes. Or possibly DBCP allows another connection pool manager.
UPDATE: Just had a look in the code and it seems DBCP is the only option for connection pooling.
Incidentally I've seen that the whenExhaustedAction settings. is not supported by openejb.xml.
There would however, still be one option left, since you are using an Oracle Database.
One thing you could try is to use Oracle implicit connection caching (assuming version 10g) and leave DBCP with an arbitrary "sufficient" amount of connections.
To do so, you would need to configure in the openejb.xml resource block, ConnectionProperties properties and use Oracle JDBC connection properties. That is connectionCachingEnabled=true and at least connectionCacheName and connectionCacheProperties. In this way I would lure DBCP into believing it's doing the real job and actually using Oracle's pooling mechanism. That would also mean taking little risks with DBCP and thereby a more liberal sizing of the maxActive setting.
Resolved issue with dbcp blocking by changing pool configuration (openejb.xml):
TestOnBorrow false
Thank you, Andy, from OpenEJB team!