I am in the process of experimenting with Quarkus, building a small REST application. For this I have elected to use the Agroal datasource but neither Panache or plain Hibernate (as is shown in their examples), but rather I'm using plaing JDBC.
For database interaction I have created a small service that injects the AgroalDataSource and uses it to open up database connections. Said service exposes two methods one for running no transactional queries and one or transactional ones. For the first part everything works fine, but when I attempt to update a database entry the action fails upon attempting to commit the connection.
public <E> E update(TransactionalRunner<E> runner) {
try (var connection = dataSource.getConnection()) {
return attemptTransactional(runner, connection);
} catch (SQLException e) {
log.error("Establishing a database connection has failed", e);
throw new DataAccessException(e);
}
}
private <E> E attemptTransactional(TransactionalRunner<E> runner, Connection connection) {
try {
connection.setAutoCommit(Boolean.FALSE);
E result = runner.run(new QueryRunner(), connection);
connection.commit();
return result;
} catch (SQLException e) {
try {
log.error("Committing a transaction has failed. Attempting rollback", e);
DbUtils.rollback(connection);
throw new DataAccessException(e);
} catch (SQLException ex) {
log.error("Rolling back a transaction has failed. Giving up...", ex);
throw new DataAccessException(ex);
}
}
}
The stacktrace I'm getting is the following:
2019-12-21 14:25:59,350 ERROR [com.ari.rev.dat.DatabaseAccessService] (vert.x-worker-thread-1) Committing a transaction has failed. Attempting rollback: java.sql.SQLException: Attempting to commit while taking part in a transaction
at io.agroal.pool.wrapper.ConnectionWrapper.commit(ConnectionWrapper.java:183)
at com.ariskourt.revolut.database.DatabaseAccessService.attemptTransactional(DatabaseAccessService.java:44)
at com.ariskourt.revolut.database.DatabaseAccessService.update(DatabaseAccessService.java:33)
at com.ariskourt.revolut.database.DatabaseAccessService_ClientProxy.update(DatabaseAccessService_ClientProxy.zig:114)
at com.ariskourt.revolut.services.AccountTransferService.transferAmount(AccountTransferService.java:66)
at com.ariskourt.revolut.services.AccountTransferService_Subclass.transferAmount$$superaccessor2(AccountTransferService_Subclass.zig:164)
at com.ariskourt.revolut.services.AccountTransferService_Subclass$$function$$2.apply(AccountTransferService_Subclass$$function$$2.zig:51)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:119)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:92)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:32)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:53)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:26)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired_Bean.intercept(TransactionalInterceptorRequired_Bean.zig:168)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
I'm using the default Agroal datasource offered by Quarkus without any custom configuration. As for the QueryRunner these are just part of the Apache DbUtils suite.
Has anyone got any idea on how to resolve this?
We have a large multithreaded Java EE application running on Wildfly 8.
We are using OrientDB 2.1.19.
And we have some problems with the connection leaks. At some point orient server stops responding and all threads working with db stuck on retrieving new connection.
Configuration is following:
OGlobalConfiguration.CLIENT_CONNECT_POOL_WAIT_TIMEOUT.setValue(5000);
OGlobalConfiguration.CLIENT_DB_RELEASE_WAIT_TIMEOUT.setValue(5000);
OGlobalConfiguration.CLIENT_CHANNEL_MAX_POOL.setValue(1000);
OGlobalConfiguration.DB_POOL_MIN.setValue(100);
OGlobalConfiguration.DB_POOL_MAX.setValue(5000);
OGlobalConfiguration.STORAGE_LOCK_TIMEOUT.setValue(5000);
OGlobalConfiguration.DB_POOL_IDLE_TIMEOUT.setValue(5000);
OGlobalConfiguration.DB_POOL_IDLE_CHECK_DELAY.setValue(1000);
OPartitionedDatabasePool we're getting thru OPartitionedDatabasePoolFactory
poolFactory = new OPartitionedDatabasePoolFactory();
documentPool = poolFactory.get(orientDBPath, username, password);
Then depending on our needs we are using ODatabaseDocumentTx or OObjectDatabaseTx using
documentPool.acquire();
Acquired ODatabaseDocumentTx or OObjectDatabaseTx we are passing to so called executor, which executes Runnable or Callable.
public void run(ORunnable<DB> oRunnable) {
// db is the private member of executor containing acquired db.
try {
//...
oRunnable.run(db);
//...
} catch (Exception e) {
} finally {
if(!db.isClosed()) {
db.close();
}
}
}
As you can see we are closing db in finally section, also we are fully detaching all documents recieved from db. But in some cases number of connections to DB is not dropping and after that we're getting following exception
2016-07-28 23:10:45,957 FINE [com.orientechnologies.orient.client.remote.ORemoteConnectionManager] (default task-46) Network connection pool is receiving a closed connection to reuse: discard it
2016-07-28 23:10:45,957 FINE [com.orientechnologies.orient.client.remote.ORemoteConnectionManager] (default task-46) Cannot unlock connection lock: java.lang.IllegalMonitorStateException
at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:155) [rt.jar:1.7.0_13]
at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) [rt.jar:1.7.0_13]
at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:460) [rt.jar:1.7.0_13]
at com.orientechnologies.common.concur.lock.OAdaptiveLock.unlock(OAdaptiveLock.java:123) [orientdb-core-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynchClient.unlock(OChannelBinaryAsynchClient.java:371) [orientdb-enterprise-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.client.remote.ORemoteConnectionManager.remove(ORemoteConnectionManager.java:128) [orientdb-client-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.client.remote.ORemoteConnectionManager.release(ORemoteConnectionManager.java:119) [orientdb-client-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.client.remote.OStorageRemote.endResponse(OStorageRemote.java:1643) [orientdb-client-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.client.remote.OStorageRemote.command(OStorageRemote.java:1240) [orientdb-client-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.client.remote.OStorageRemoteThread.command(OStorageRemoteThread.java:453) [orientdb-client-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.core.sql.query.OSQLQuery.run(OSQLQuery.java:72) [orientdb-core-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.core.sql.query.OSQLSynchQuery.run(OSQLSynchQuery.java:85) [orientdb-core-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.core.query.OQueryAbstract.execute(OQueryAbstract.java:33) [orientdb-core-2.1.19.jar:2.1.19]
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.query(ODatabaseDocumentTx.java:717) [orientdb-core-2.1.19.jar:2.1.19]
Here is the piece of code for the exception above
ODocument storedSession = orient.onDocuments().call(new ODocCallable<ODocument>() {
#Override
public ODocument call(ODatabaseDocumentTx db) {
try {
List<ODocument> list = db.query(new OSQLSynchQuery<ODocument>("select from " + CLAZZ + " where storedSession_sessionID = ?"), sessionId);
if (list.isEmpty()){
return null;
}
return documentUnpin(list.get(0));
} catch (Exception e) {
// PersistentSession may not be in DB on request (for example in workspaces)
}
return null;
}
});
Any suggestions how to improve the situation with connections?
Thanks.
I simply start up a zookeeper server (3.4.6) with using the org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ServerConfig) method, then I try to shut it down. During shutdown I get this:
11:43:11,176 WARN {main} [org.apache.zookeeper.jmx.MBeanRegistry] Failed to unregister MBean InMemoryDataTree
11:43:11,176 WARN {main} [org.apache.zookeeper.jmx.MBeanRegistry] Error during unregister
javax.management.InstanceNotFoundException: org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:115)
at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:132)
at org.apache.zookeeper.server.ZooKeeperServer.unregisterJMX(ZooKeeperServer.java:465)
at org.apache.zookeeper.server.ZooKeeperServer.shutdown(ZooKeeperServer.java:458)
at org.apache.zookeeper.server.NIOServerCnxnFactory.shutdown(NIOServerCnxnFactory.java:271)
at org.apache.zookeeper.server.ZooKeeperServerMain.shutdown(ZooKeeperServerMain.java:132)
...
I have unfortunately no idea what the error Failed to unregister MBean InMemoryDataTree means. I did not find anything with search engines apart from logs from some project builds. I could read the code, but it would take obviously a lot of time to understand that.
Do I have to change something in my startup process to get rid of this, or is this completely normal?
The logs indicate that it is a warning, and from looking into the source and comments, can largely be ignored. In my experience I do not recall seeing those particular messages, but kept running into run time exceptions/errors that kept subsequent unit tests from running and caused my gradle build to exit the test task without generating the report.
I have no idea how you were trying to shut down ZookeeperServerMain, but my work around was to extend ZookeeperServerMain so I could access its protected shutdown method.
public class MyZookeeperServerMain extends ZookeeperServerMain{
...
public void openZoo() {
/**/
Properties startupProperties = createProperties(DEFAULT_LOG_DIR, 2181);
QuorumPeerConfig quorumConfiguration = new QuorumPeerConfig();
try {
quorumConfiguration.parseProperties(startupProperties);
} catch (Exception e) {
throw new RuntimeException(e);
}
zooKeeperServer = new MyZookeeperServerMain();
final ServerConfig configuration = new ServerConfig();
configuration.readFrom(quorumConfiguration);
zooEntrance = new Thread() {
public void run() {
try {
zooKeeperServer.runFromConfig(configuration);
}
catch (IOException e) {
LOG.error("ZooKeeper Failed", e);
}
catch(Exception ie)
{
LOG.debug("shutting down zookeeper", ie);
}
catch(Error e)
{
System.out.println("error stopping zooKeeper: " + e);
}
}
};
zooEntrance.start();
}
public void closeZoo() {
zooKeeperServer.shutdown();
}
}
I'm currently working on a project which was originally not build for high load.
My problem atm is that at some point during the stress test (30 users) the application seems to "get stuck" and when it releases it spits out a lot of exceptions. Unable to get managed connection for [MY_DS]
When I run only one user, it works like a charm so it has something to do with the concurrency.
I also checked if there were any unclosed DB connections at the end of one run and there were none, so at normal usage, there are no connection leaks.
My suspicion goes out to my open and close methods (because they are static). Here are the methods:
public static Connection getConnection() {
if (logger.isDebugEnabled()) logger.debugLog("getConnection()");
try {
return DSUtils.getDefaultDataSource().getConnection();
} catch (SQLException se) {
logger.errorLog("SQLException", se);
throw new ApplicationRuntimeException(MessageCodesConstants.ERROR_SQL_EXCEPTION, se);
}
}
public static void closeConnection(Connection con) {
if (logger.isDebugEnabled()) logger.debugLog("closeConnection");
try {
if (con != null) {
con.close();
}
} catch (SQLException se) {
logger.warnLog("SQLException while closing connection");
}
}
It is an EE application running on JBoss EAP 6.2.0 backed-up by a SQL Server 2008.
Can somebody point me in the right direction to find out where the solution can be found?
I'm experiencing a memory leak due to orphaned threads in Tomcat. Particularly, it seems that Guice and the JDBC driver are not closing threads.
Aug 8, 2012 4:09:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: A web application appears to have started a thread named [com.google.inject.internal.util.$Finalizer] but has failed to stop it. This is very likely to create a memory leak.
Aug 8, 2012 4:09:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: A web application appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak.
I know this is similar to other questions (such as this one), but in my case, the answer of "don't worry about it" won't be sufficient, as it is causing problems for me. I have CI server which regularly updates this application, and after 6-10 reloads, the CI server will hang because Tomcat is out of memory.
I need to be able to clear up these orphaned threads so I can run my CI server more reliably. Any help would be appreciated!
I just dealt with this problem myself. Contrary to some other answers, I do not recommend issuing the t.stop() command. This method has been deprecated, and for good reason. Reference Oracle's reasons for doing this.
However there is a solution for removing this error without needing to resort to t.stop()...
You can use most of the code #Oso provided, just replace the following section
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
Thread[] threadArray = threadSet.toArray(new Thread[threadSet.size()]);
for(Thread t:threadArray) {
if(t.getName().contains("Abandoned connection cleanup thread")) {
synchronized(t) {
t.stop(); //don't complain, it works
}
}
}
Replace it using the following method provided by the MySQL driver:
try {
AbandonedConnectionCleanupThread.shutdown();
} catch (InterruptedException e) {
logger.warn("SEVERE problem cleaning up: " + e.getMessage());
e.printStackTrace();
}
This should properly shutdown the thread, and the error should go away.
I've had the same issue, and as Jeff says, the "don't worry about it approach" was not the way to go.
I did a ServletContextListener that stops the hung thread when the context is being closed, and then registered such ContextListener on the web.xml file.
I already know that stopping a thread is not an elegant way to deal with them, but otherwise the server keeps on crashing after two or three deploys (it is not always possible to restart the app server).
The class I created is:
public class ContextFinalizer implements ServletContextListener {
private static final Logger LOGGER = LoggerFactory.getLogger(ContextFinalizer.class);
#Override
public void contextInitialized(ServletContextEvent sce) {
}
#Override
public void contextDestroyed(ServletContextEvent sce) {
Enumeration<Driver> drivers = DriverManager.getDrivers();
Driver d = null;
while(drivers.hasMoreElements()) {
try {
d = drivers.nextElement();
DriverManager.deregisterDriver(d);
LOGGER.warn(String.format("Driver %s deregistered", d));
} catch (SQLException ex) {
LOGGER.warn(String.format("Error deregistering driver %s", d), ex);
}
}
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
Thread[] threadArray = threadSet.toArray(new Thread[threadSet.size()]);
for(Thread t:threadArray) {
if(t.getName().contains("Abandoned connection cleanup thread")) {
synchronized(t) {
t.stop(); //don't complain, it works
}
}
}
}
}
After creating the class, then register it on the web.xml file:
<web-app...
<listener>
<listener-class>path.to.ContextFinalizer</listener-class>
</listener>
</web-app>
The least invasive workaround is to force initialisation of the MySQL JDBC driver from code outside of the webapp's classloader.
In tomcat/conf/server.xml, modify (inside the Server element):
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
to
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
classesToInitialize="com.mysql.jdbc.NonRegisteringDriver" />
With mysql-connector-java-8.0.x use com.mysql.cj.jdbc.NonRegisteringDriver instead
This assumes you put the MySQL JDBC driver into tomcat's lib directory and not inside your webapp.war's WEB-INF/lib directory, as the whole point is to load the driver before and independently of your webapp.
References:
http://bugs.mysql.com/bug.php?id=68556#c400606
http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html#JRE_Memory_Leak_Prevention_Listener_-_org.apache.catalina.core.JreMemoryLeakPreventionListener
http://markmail.org/message/dmvlkps7lbgpngil
com.mysql.jdbc.NonRegisteringDriver source v5.1
com.mysql.cj.jdbc.NonRegisteringDriver source v8.0
Changes in connector/J v8.0
Effective from MySQL connector 5.1.23 onwards, a method is provided to shut the abandoned connection cleanup thread down, AbandonedConnectionCleanupThread.shutdown.
However, we don't want direct dependencies in our code on the otherwise opaque JDBC driver code, so my solution is to use reflection to find the class and method and invoke it if found. The following complete code snippet is all that's needed, executed in the context of the class loader that loaded the JDBC driver:
try {
Class<?> cls=Class.forName("com.mysql.jdbc.AbandonedConnectionCleanupThread");
Method mth=(cls==null ? null : cls.getMethod("shutdown"));
if(mth!=null) { mth.invoke(null); }
}
catch (Throwable thr) {
thr.printStackTrace();
}
This cleanly ends the thread if the JDBC driver is a sufficiently recent version of the MySQL connector and otherwise does nothing.
Note it has to be executed in the context of the class loader because the thread is a static reference; if the driver class is not being or has not already been unloaded when this code is run then the thread will not be running for subsequent JDBC interactions.
I took the best parts of the answers above and combined them into an easily extensible class. This combines Oso's original suggestion with Bill's driver improvement and Software Monkey's reflection improvement. (I liked the simplicity of Stephan L's answer too, but sometimes modifying the Tomcat environment itself is not a good option, especially if you have to deal with autoscaling or migration to another web container.)
Instead of directly referring to the class name, thread name, and stop method, I also encapsulated these into an private inner ThreadInfo class. Using a list of these ThreadInfo objects, you can include additional troublesome threads to be shutdown with the same code. This is a bit more complex of a solution than most people likely need, but should work more generally when you need that.
import java.lang.reflect.Method;
import java.sql.Driver;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.List;
import java.util.Set;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Context finalization to close threads (MySQL memory leak prevention).
* This solution combines the best techniques described in the linked Stack
* Overflow answer.
* #see Tomcat Guice/JDBC Memory Leak
*/
public class ContextFinalizer
implements ServletContextListener {
private static final Logger LOGGER =
LoggerFactory.getLogger(ContextFinalizer.class);
/**
* Information for cleaning up a thread.
*/
private class ThreadInfo {
/**
* Name of the thread's initiating class.
*/
private final String name;
/**
* Cue identifying the thread.
*/
private final String cue;
/**
* Name of the method to stop the thread.
*/
private final String stop;
/**
* Basic constructor.
* #param n Name of the thread's initiating class.
* #param c Cue identifying the thread.
* #param s Name of the method to stop the thread.
*/
ThreadInfo(final String n, final String c, final String s) {
this.name = n;
this.cue = c;
this.stop = s;
}
/**
* #return the name
*/
public String getName() {
return this.name;
}
/**
* #return the cue
*/
public String getCue() {
return this.cue;
}
/**
* #return the stop
*/
public String getStop() {
return this.stop;
}
}
/**
* List of information on threads required to stop. This list may be
* expanded as necessary.
*/
private List<ThreadInfo> threads = Arrays.asList(
// Special cleanup for MySQL JDBC Connector.
new ThreadInfo(
"com.mysql.jdbc.AbandonedConnectionCleanupThread", //$NON-NLS-1$
"Abandoned connection cleanup thread", //$NON-NLS-1$
"shutdown" //$NON-NLS-1$
)
);
#Override
public void contextInitialized(final ServletContextEvent sce) {
// No-op.
}
#Override
public final void contextDestroyed(final ServletContextEvent sce) {
// Deregister all drivers.
Enumeration<Driver> drivers = DriverManager.getDrivers();
while (drivers.hasMoreElements()) {
Driver d = drivers.nextElement();
try {
DriverManager.deregisterDriver(d);
LOGGER.info(
String.format(
"Driver %s deregistered", //$NON-NLS-1$
d
)
);
} catch (SQLException e) {
LOGGER.warn(
String.format(
"Failed to deregister driver %s", //$NON-NLS-1$
d
),
e
);
}
}
// Handle remaining threads.
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
Thread[] threadArray = threadSet.toArray(new Thread[threadSet.size()]);
for (Thread t:threadArray) {
for (ThreadInfo i:this.threads) {
if (t.getName().contains(i.getCue())) {
synchronized (t) {
try {
Class<?> cls = Class.forName(i.getName());
if (cls != null) {
Method mth = cls.getMethod(i.getStop());
if (mth != null) {
mth.invoke(null);
LOGGER.info(
String.format(
"Connection cleanup thread %s shutdown successfully.", //$NON-NLS-1$
i.getName()
)
);
}
}
} catch (Throwable thr) {
LOGGER.warn(
String.format(
"Failed to shutdown connection cleanup thread %s: ", //$NON-NLS-1$
i.getName(),
thr.getMessage()
)
);
thr.printStackTrace();
}
}
}
}
}
}
}
I went a step further from Oso,improved the code above in two points:
Added the Finalizer thread to the need-to-kill check:
for(Thread t:threadArray) {
if(t.getName().contains("Abandoned connection cleanup thread")
|| t.getName().matches("com\\.google.*Finalizer")
) {
synchronized(t) {
logger.warn("Forcibly stopping thread to avoid memory leak: " + t.getName());
t.stop(); //don't complain, it works
}
}
}
Sleep for a little while to give threads time to stop. Without that, tomcat kept complaining.
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
logger.debug(e.getMessage(), e);
}
Bill's solution looks good, however I found another solution directly in MySQL bug reports:
[5 Jun 2013 17:12] Christopher Schultz
Here is a much better workaround until something else changes.
Enable Tomcat's JreMemoryLeakPreventionListener (enabled by default on Tomcat 7), and add this attribute to the element:
classesToInitialize="com.mysql.jdbc.NonRegisteringDriver"
If "classesToInitialize" is already set on your , just add NonRegisteringDriver to the existing value separated by a comma.
and the answer:
[8 Jun 2013 21:33] Marko Asplund
I did some testing with the JreMemoryLeakPreventionListener / classesToInitialize workaround (Tomcat 7.0.39 + MySQL Connector/J 5.1.25).
Before applying the workaround thread dumps listed multiple AbandonedConnectionCleanupThread instances after redeploying the webapp several times. After applying the workaround there's only one AbandonedConnectionCleanupThread instance.
I had to modify my app, though, and move MySQL driver from the webapp to Tomcat lib.
Otherwise, the classloader is unable to load com.mysql.jdbc.NonRegisteringDriver at Tomcat startup.
I hope it helps for all who still fighting with this issue...
It seems this was fixed in 5.1.41. You could upgrade Connector/J to 5.1.41 or newer.
https://dev.mysql.com/doc/relnotes/connector-j/5.1/en/news-5-1-41.html
The implementation of AbandonedConnectionCleanupThread has now been improved, so that there are now four ways for developers to deal with the situation:
When the default Tomcat configuration is used and the Connector/J jar is put into a local library directory, the new built-in application detector in Connector/J now detects the stopping of the web application within 5 seconds and kills AbandonedConnectionCleanupThread. Any unnecessary warnings about the thread being unstoppable are also avoided. If the Connector/J jar is put into a global library directory, the thread is left running until the JVM is unloaded.
When Tomcat's context is configured with the attribute clearReferencesStopThreads="true", Tomcat is going to stop all spawned threads when the application stops unless Connector/J is being shared with other web applications, in which case Connector/J is now protected against an inappropriate stop by Tomcat; the warning about the non-stoppable thread is still issued into Tomcat's error log.
When a ServletContextListener is implemented within each web application that calls AbandonedConnectionCleanupThread.checkedShutdown() on context destruction, Connector/J now, again, skips this operation if the driver is potentially shared with other applications. No warning about the thread being unstoppable is issued to Tomcat's error log in this case.
When AbandonedConnectionCleanupThread.uncheckedShutdown() is called, the AbandonedConnectionCleanupThread is closed even if Connector/J is shared with other applications. However, it may not be possible to restart the thread afterwards.
If you look at source code, they called setDeamon(true) on thread, so it won't block shutdown.
Thread t = new Thread(r, "Abandoned connection cleanup thread");
t.setDaemon(true);
See To prevent a memory leak, the JDBC Driver has been forcibly unregistered. Bill's answer deregisters all Driver instances as well as instances that may belong to other web applications. I have extended Bill's answer with a check that the Driver instance belongs to the right ClassLoader.
Here is the resulting code (in a separate method, because my contextDestroyed has other things to do):
// See https://stackoverflow.com/questions/25699985/the-web-application-appears-to-have-started-a-thread-named-abandoned-connect
// and
// https://stackoverflow.com/questions/3320400/to-prevent-a-memory-leak-the-jdbc-driver-has-been-forcibly-unregistered/23912257#23912257
private void avoidGarbageCollectionWarning()
{
ClassLoader cl = Thread.currentThread().getContextClassLoader();
Enumeration<Driver> drivers = DriverManager.getDrivers();
Driver d = null;
while (drivers.hasMoreElements()) {
try {
d = drivers.nextElement();
if(d.getClass().getClassLoader() == cl) {
DriverManager.deregisterDriver(d);
logger.info(String.format("Driver %s deregistered", d));
}
else {
logger.info(String.format("Driver %s not deregistered because it might be in use elsewhere", d.toString()));
}
}
catch (SQLException ex) {
logger.warning(String.format("Error deregistering driver %s, exception: %s", d.toString(), ex.toString()));
}
}
try {
AbandonedConnectionCleanupThread.shutdown();
}
catch (InterruptedException e) {
logger.warning("SEVERE problem cleaning up: " + e.getMessage());
e.printStackTrace();
}
}
I wonder whether the call AbandonedConnectionCleanupThread.shutdown() is safe. Can it interfere with other web applications? I hope not, because the AbandonedConnectionCleanupThread.run() method is not static but the AbandonedConnectionCleanupThread.shutdown() method is.