Exception on Spring integration, when shut down the tomcat server - java

I have a Spring Integration application which runs on Tomcat 6. It has a message-driven-channel-adapter and it communicates with remote Active MQ server. So far so good I am able to do my work.
However when I shut down Tomcat I am getting the following exception endlessly looping in the catalina log:
Exception in thread "org.springframework.jms.listener.DefaultMessageListenerContainer#0-178" Exception in thread "org.springframework.jms.listener.DefaultMessageListenerContainer#0-179"
java.lang.NullPointerException
at org.apache.log4j.LogManager.getLogger(LogManager.java:188)
at org.apache.log4j.Logger.getLogger(Logger.java:104)
at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289)
at org.apache.commons.logging.impl.Log4JLogger.trace(Log4JLogger.java:166)
at org.springframework.jms.support.JmsUtils.closeMessageConsumer(JmsUtils.java:158)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1111)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:971)
at java.lang.Thread.run(Thread.java:722)
java.lang.NullPointerException
at org.apache.log4j.LogManager.getLogger(LogManager.java:188)
at org.apache.log4j.Logger.getLogger(Logger.java:104)
at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289)
at org.apache.commons.logging.impl.Log4JLogger.trace(Log4JLogger.java:166)
at org.springframework.jms.support.JmsUtils.closeSession(JmsUtils.java:115)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1112)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:971)
at java.lang.Thread.run(Thread.java:722)
Nov 13, 2012 7:23:37 PM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already. Could not load org.apache.log4j.spi.VectorWriter. The eventual following stack trace is ca
used by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1273)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1233)
at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
at org.apache.log4j.Category.forcedLog(Category.java:388)
at org.apache.log4j.Category.log(Category.java:853)
at org.apache.commons.logging.impl.Log4JLogger.warn(Log4JLogger.java:234)
at org.springframework.jms.listener.DefaultMessageListenerContainer.handleListenerSetupFailure(DefaultMessageListenerContainer.java:831)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:981)
at java.lang.Thread.run(Thread.java:722)
Do you guys have any idea bout this? do i need to close or destroy Spring context before I shut down Tomcat?
Appreciate your help
Thanks,
Keth

Well, looks like you are correct:
do i need to close or destroy Spring context before I shut down Tomcat?
From Spring JSM source code it looks like your Tomcat closes its log4j.LogManager earlier, than Spring stops its context.

Related

How to fix java.lang.NoClassDefFoundError in tomcat

I'm getting below error message often and due to this error I'm suspecting my application creating outofMemory Error in permGen space.
Can someone help how to fix the below exception.
I'm using pure servlet and tomcat 7.
INFO: Illegal access: this web application instance has been stopped already. Could not load com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1745)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1703)
at com.mchange.v2.resourcepool.BasicResourcePool.checkIdleResources(BasicResourcePool.java:1481)
at com.mchange.v2.resourcepool.BasicResourcePool.access$2000(BasicResourcePool.java:32)
at com.mchange.v2.resourcepool.BasicResourcePool$CheckIdleResourcesTask.run(BasicResourcePool.java:1964)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Exception in thread "Timer-0" java.lang.NoClassDefFoundError: com/mchange/v2/resourcepool/BasicResourcePool$AsyncTestIdleResourceTask
at com.mchange.v2.resourcepool.BasicResourcePool.checkIdleResources(BasicResourcePool.java:1481)
at com.mchange.v2.resourcepool.BasicResourcePool.access$2000(BasicResourcePool.java:32)
at com.mchange.v2.resourcepool.BasicResourcePool$CheckIdleResourcesTask.run(BasicResourcePool.java:1964)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Caused by: java.lang.ClassNotFoundException: com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1854)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1703)
... 5 more
That message just occurs because the web application is still trying to load things after you have shut it down.
As such, it should not have "business impact" (you don't care about errors in an app that you shut down), especially if it happens in the process of shutting down the whole app server.
You may want to figure out why it does not shutdown more cleanly (maybe some background processing that needs to be stopped when the webapp is asked to terminate). If there is stuff still going on in there, that might cause problems for any other web apps still running.
If you did not intend to shut down the webapp, then there is probably another error earlier in the log.

Could not load org.jgroups.protocols.pbcast.GmsImpl$Request

I have no idea about this wrong when I use J2Cache:
INFO: Illegal access: this web application instance has been stopped already. Could not load org.jgroups.protocols.pbcast.GmsImpl$Request. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1612)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:855)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:636)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:103)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:147)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:185)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:301)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:209)
at org.jgroups.protocols.Discovery.up(Discovery.java:379)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1412)
at org.jgroups.protocols.TP$MyHandler.run(TP.java:1598)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jun 11, 2015 2:47:30 PM org.apache.catalina.connector.CoyoteAdapter log
WARNING: Exception while attempting to add an entry to the access log
java.lang.NullPointerException
at org.apache.catalina.connector.CoyoteAdapter.log(CoyoteAdapter.java:557)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:182)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:620)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
The above of this wrong happened with tomcat's starting.
I have googled about it.
Someone said:
If you use a QUIT signal on the Java process it should output a thread
dump that should identify which threads is still running and
preventing a proper shutdown. I have seen this on occasion with some
versions of HSQLDB that had a driver that didn't shutdown properly but
it could be anything else deployed in your install.
To send the QUIT signal : kill -QUIT JAVA_PID
where JAVA_PID is the process ID of your Java process you can retrieve
using : ps aux
But unfortunately, I can't understand it.
Can anybody help me?
This sounds familiar to this issue - https://issues.jboss.org/browse/JGRP-826. My guess is that tomcat already shuts down while there are still JGroups threads running. How do you shutdown Tomcat? Do you explicitly shutdown the cache as well? I would try implementing a ServletContextListener to receive a callback when the application gets shutdown. At this point I would try to explicitly close the cache as well. This all depends of course how and where you use the cache.

Hazelcast keep running after application stops on Weblogic (and also Tomcat)

I have just been playing around to get what Hazelcast provides. There are only a few things using Hazelcast in the applcation.
On Weblogic server 10.3.6, I saw that there is something wrong on one of the Maps. It kept giving exception about no class definiton error even though it has been running maybe more than a few days without a problem. Then I stoppped the application on running 8 nodes on Oracle Weblogic. I assumed the Hazelcast clusters/instances would shutdown as well but even though the application stops on all the nodes of Weblogic, I saw Hazelcast merge exceptions on the logs (keep throwing).
I also test my spring based application on Tomcat 7, even though I shutdown the application, Hazelcast resists shutting down somehow.
Is it normal behaviour? How can we shutdown all Hazelcast instances even after shutting down the application?
Note that I call Hazelcast when my application starts, there is no special client, only 8 nodes of Weblogic server.
Edit: Here is the stacktrace of the migration problem
SEVERE: Problem while reading DataSerializable, namespace: 0, id: 0, class: 'com.hazelcast.partition.impl.MigrationRequestOperation', exception: com.hazelcast.partition.impl.MigrationRequestOperation
com.hazelcast.nio.serialization.HazelcastSerializationException: Problem while reading DataSerializable, namespace: 0, id: 0, class: 'com.hazelcast.partition.impl.MigrationRequestOperation', exception: com.hazelcast.partition.impl.MigrationRequestOperatio
at com.hazelcast.nio.serialization.DataSerializer.read(DataSerializer.java:120)
at com.hazelcast.nio.serialization.DataSerializer.read(DataSerializer.java:39)
at com.hazelcast.nio.serialization.StreamSerializerAdapter.toObject(StreamSerializerAdapter.java:65)
at com.hazelcast.nio.serialization.SerializationServiceImpl.toObject(SerializationServiceImpl.java:260)
at com.hazelcast.spi.impl.NodeEngineImpl.toObject(NodeEngineImpl.java:186)
at com.hazelcast.spi.impl.BasicOperationService$OperationPacketHandler.loadOperation(BasicOperationService.java:638)
at com.hazelcast.spi.impl.BasicOperationService$OperationPacketHandler.handle(BasicOperationService.java:621)
at com.hazelcast.spi.impl.BasicOperationService$OperationPacketHandler.access$1500(BasicOperationService.java:614)
at com.hazelcast.spi.impl.BasicOperationService$BasicDispatcherImpl.dispatch(BasicOperationService.java:566)
at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.process(BasicOperationScheduler.java:466)
at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.processPriorityMessages(BasicOperationScheduler.java:480)
at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.doRun(BasicOperationScheduler.java:457)
at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.run(BasicOperationScheduler.java:432)
Caused by: java.lang.ClassNotFoundException: com.hazelcast.partition.impl.MigrationRequestOperation
at weblogic.utils.classloaders.GenericClassLoader.findLocalClass(GenericClassLoader.java:297)
at weblogic.utils.classloaders.GenericClassLoader.findClass(GenericClassLoader.java:270)
at weblogic.utils.classloaders.ChangeAwareClassLoader.findClass(ChangeAwareClassLoader.java:64)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:179)
at weblogic.utils.classloaders.ChangeAwareClassLoader.loadClass(ChangeAwareClassLoader.java:43)
at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:124)
at com.hazelcast.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:113)
at com.hazelcast.nio.ClassLoaderUtil.newInstance(ClassLoaderUtil.java:66)
at com.hazelcast.nio.serialization.DataSerializer.read(DataSerializer.java:109)
... 12 more
Mar 16, 2015 5:31:29 PM com.hazelcast.spi.OperationService
The credible Hazelcast docs says:
As a final step, if you are done with your client, you can shut it down as shown below. This will release all the used resources and will close connections to the cluster.
client.shutdown();
which should be called in destroy() method of your bean:
public class ExampleBean implements DisposableBean {
public void destroy() {
client.shutdown();
}
}
The exception you have pasted occurs due to the fact that your app class loader is destroyed upon shutdown of your app.

Tomcat memory leak - Stopping Guava Finalizer thread

I have a Spring web application running on Tomcat.
When trying to shutdown Tomcat with the shutdown.sh script, the java process doesn't end because it has a Thread that's still running. The catalina.log contains
Jul 22, 2013 2:07:50 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/myapp] appears to have started a thread named [com.google.common.base.internal.Finalizer] but has failed to stop it. This is very likely to create a memory leak.
Apparently BoneCP has some Guava integration that causes this thread to block. Is there a clean or intended way to kill this thread?
There's an open issue on Guava for this problem - the issue itself is pretty old but if you check the latest comments you'll see some suggestions from other BoneCP users to resolve this issue
Also check this other issue and this patch

web instance already stopped

i get this error
Mar 22, 2011 12:36:01 AM org.apache.catalina.loader.WebappClassLoader findResourceInternal
INFO: Illegal access: this web application instance has been stopped already. Could not load META-INF/services/javax.xml.parsers.DocumentBuilderFactory. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
Mar 22, 2011 12:36:01 AM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already. Could not load com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1562)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1521)
at javax.xml.parsers.FactoryFinder.getProviderClass(Unknown Source)
at javax.xml.parsers.FactoryFinder.newInstance(Unknown Source)
at javax.xml.parsers.FactoryFinder.find(Unknown Source)
at javax.xml.parsers.DocumentBuilderFactory.newInstance(Unknown Source)
at com.xmlparser.XmlParser.parseXmlFile(XmlParser.java:33)
at com.xmlparser.XmlParser.<init>(XmlParser.java:25)
at com.jobs.SendRoutineMessagesJob.execute(SendRoutineMessagesJob.java:29)
at org.quartz.core.JobRunShell.run(JobRunShell.java:199)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)
could some one please guide me as to what mistake i am committing here. i am not sure if i would need to paste the source code as well.
This because of you are deploying new instance of your application to server but the some thread or connection of previous instance attempts doing some business logic you have written. That's why, the main part of JVM called ClassLoader informs you about "Illegal access: this web application instance has been stopped already" and throws appropriate exception ... You can switch off auto deploy feature of Application Server which preferable on Production Environment and after deploying application restart server manually. Or you can in Class level control Servlet Life Cycle :))) during "pre deploy" close all connection or stop some threads. Or use sophisticated way controlling Threads with ManagedExecutorService. Good luck!!!
seems like i was creating instance of something which was already created..i removed the instance and now it works fine
Are you using Netbeans? I had a similar error except it was could not load oracle.sql.lnxlib. Like many posters have pointed out, this error usually has to do with Tomcat caching multiple instances of the application. A coworker was able to solve the problem for me by undeploying old instances of the application within Netbeans. To do so, go to the Services Tab > Servers and then double click on Apache Tomcat or TomEE and after running the application once since opening netbeans, a Web Applications Folder should appear. If you double click on that, you should see instances of your application appear. You should be able to right click them and then click undeploy (you may have to right click them and hit stop first). Finally, you should stop and then start tomcat before building and running your application.
Hope this helps.
See attached screenshot for clarity
If you use gradle there is an error
Illegal access: this web application instance has been stopped already. Could not load [].
The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
You can change mysql-connect-java to
mysql: mysql-connector-java: 8.0.13
Good luck
I got this error on Tomcat on an Ubuntu server ....finally resolved it by doing a Tomcat shutdown with "systemctl stop tomcat" and then "sudo reboot" and then once the box came back up, starting Tomcat again

Categories

Resources