Develop Reference

OpenAM is hung and high CPU utilization by Java

I recently started working in a project which uses OpenAM for single sign on top of tomcat 7 and server running on Apache 2.4.12.
Installed Java version details:
java version "1.7.0_65"
OpenJDK Runtime Environment (rhel-2.5.1.2.el6_5-x86_64 u65-b17)
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)
OpenAM 11.0.0 (2013-November-08 10:40)
Most of the time 95% of CPU utilization is by java. I can't find the reason for this high CPU utilization.
We had an issue today with OpenAM, OpenAM Tomcat processes seems to have hung in two servers out of 8 and not listening to the port 8180. After restart, the service is up and running fine with ports listening.
However, I could see below error in openAM logs, I tried to find the root cause but unfortunately I can't, could anyone please help?
Server 1:
Jul 28, 2015 8:33:36 AM com.sun.xml.ws.transport.http.servlet.WSServletContextListener contextDestroyed
INFO: WSSERVLET13: JAX-WS context listener destroyed
Jul 28, 2015 8:33:43 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sso] appears to have started a thread named [com.google.inject.internal.util.$Finalizer] but has failed to stop it. This is very likely to create a memory leak.
Jul 28, 2015 8:33:43 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sso] appears to have started a thread named [Monitor Provider State Updater] but has failed to stop it. This is very likely to create a memory leak.
Jul 28, 2015 8:33:43 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/sso] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal#4ee2edc3]) and a value of type [org.glassfish.grizzly.ThreadCache.ObjectCache] (value [org.glassfish.grizzly.ThreadCache$ObjectCache#1467c5ab]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Jul 28, 2015 8:33:43 AM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-bio-8180"]
Jul 28, 2015 8:33:43 AM org.apache.coyote.AbstractProtocol destroy
Server 2:
Jul 28, 2015 8:34:50 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sso] appears to have started a thread named [com.google.inject.internal.util.$Finalizer] but has failed to stop it. This is very likely to create a memory leak.
Jul 28, 2015 8:34:50 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/sso] appears to have started a thread named [Monitor Provider State Updater] but has failed to stop it. This is very likely to create a memory leak.
Jul 28, 2015 8:33:43 AM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/sso] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal#4ee2edc3]) and a value of type [org.glassfish.grizzly.ThreadCache.ObjectCache] (value [org.glassfish.grizzly.ThreadCache$ObjectCache#1467c5ab]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Jul 28, 2015 8:33:43 AM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-bio-8180"]
Jul 28, 2015 8:33:43 AM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-bio-8180"]
Jul 28, 2015 8:33:43 AM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["ajp-bio-8109"]
Jul 28, 2015 8:33:43 AM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["ajp-bio-8109"]
Jul 28, 2015 8:33:50 AM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Jul 28, 2015 8:33:50 AM org.apache.coyote.AbstractProtocol init
Other WARNING in Server 2:
Jul 28, 2015 8:35:13 AM com.sun.jersey.server.impl.application.WebApplicationImpl newResourceClass
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.forgerock.openam.forgerockrest.authn.AuthenticationRestService.getMethodNotSupported(), with URI template, "/", is treated as a resource method
Jul 28, 2015 8:35:13 AM com.sun.jersey.server.impl.application.WebApplicationImpl newResourceClass
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.forgerock.openam.forgerockrest.authn.AuthenticationRestService.authenticate(javax.ws.rs.core.HttpHeaders,javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.String,java.lang.String,java.lang.String,java.lang.String), with URI template, "/", is treated as a resource method
Jul 28, 2015 8:35:23 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /usr/local/tomcat7/openam/webapps/manager

Getting error "Tomcat was unable to start within 45 seconds." [duplicate]

This question already has answers here:
Setting property 'source' to 'org.eclipse.jst.jee.server:JSFTut' did not find a matching property
(8 answers)
Closed 7 years ago.
I am getting an error while running apache tomcat server in ecllipse and the error description is"Server apache-tomcat-7.0.54 at localhost was unable to start within 45 seconds. If the server requires more time, try increasing the timeout in the server editor".
After getting this message i have increased the time for server to start but the same message i am getting again.
Console prints error is
Jun 23, 2014 11:05:20 AM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'source' to 'org.eclipse.jst.jee.server:example' did not find a matching property.
Jun 23, 2014 11:05:20 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8080"]
Jun 23, 2014 11:05:20 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-8009"]
Jun 23, 2014 11:05:20 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 868 ms
Jun 23, 2014 11:05:20 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Jun 23, 2014 11:05:20 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.54
Jun 23, 2014 11:05:21 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8080"]
Jun 23, 2014 11:05:21 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
Jun 23, 2014 11:05:21 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 458 ms

Eclipse has a timeout for starting and stopping Tomcat. You can change it, by opening 'Servers' view, double clicking the server name. As I can see form your logs your server did start in 458ms. But this problem might also be caused by components running lazily after server startup, f.e. servlets having a load-on-startup value greater than 0.
First check if your server really starts up completly. This is when the green 'play' icon ('Start server') turns into a red 'stop' icon and the message 'starting server...' dissapears in the 'Progress' view. If that is not working increase the startup time, as mentioned before. If even that is not working check your servlets for endless slopes or connection attempts to a webservice or a database.

Memory Leak Grails application in Tomcat 7

I have a grails (v2.1.5) app in production, it works fine and i can redeploy without problems, but i have updated the app (a few changes), when I deploy the new war, i get a memory leak error, the server starts but the app doesn´t work (the page in the browser is only a withe page), I tried to reboot the instance, stop all services,....
I can redeploy only the old war. How could i fix this problem?
Thanks.
The errors:
Mar 09, 2014 4:02:14 PM org.apache.catalina.core.StandardContext startInternal SEVERE: Error listenerStart
Mar 09, 2014 4:07:17 PM org.apache.catalina.core.StandardContext startInternal SEVERE: Context [] startup failed due to previous errors
Mar 09, 2014 4:07:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc SEVERE: The web application [] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Mar 09, 2014 4:07:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak.
Mar 09, 2014 4:07:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [] appears to have started a thread named [net.sf.ehcache.CacheManager#61783b69] but has failed to stop it. This is very likely to create a memory leak.
Mar 09, 2014 4:07:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [] appears to have started a thread named [com.padelsix.SecRole.data] but has failed to stop it. This is very likely to create a memory leak.

The web application [/services] appears to have started a thread named [net.sf.ehcache.CacheManager#37ce6a10] but has failed to stop it. This is very likely to create a memory leak.
The solution, in our groovy based spring web app that uses annotation based configuration, was to register the net.sf.ehcache.constructs.web.ShutdownListener class with the context in our class that extends WebApplicationInitializer via an overridden onStartup() method, something like this:
import net.sf.ehcache.constructs.web.ShutdownListener
import org.springframework.web.WebApplicationInitializer
...
public class CustomWebApplicationInitializer implements WebApplicationInitializer {
...
#Override
public void onStartup(ServletContext servletContext) throws ServletException {
...
createContext(...)
...
}
private AnnotationConfigWebApplicationContext createContext(final Class... annotatedClasses) {
AnnotationConfigWebApplicationContext context = new AnnotationConfigWebApplicationContext()
context.register(annotatedClasses)
context.register(ShutdownListener)
return context
}
}

The main problem is not the potential memory leaks, it is the first part of the log:
Mar 09, 2014 4:02:14 PM org.apache.catalina.core.StandardContext
startInternal SEVERE: Error listenerStart
Mar 09, 2014 4:07:17 PM org.apache.catalina.core.StandardContext
startInternal SEVERE: Context [] startup failed due to previous errors
The error "SEVERE: Error listenerStart" usually means that there was an error in a servlet context listener (run on startup) that prevented the webapp to start.
Look for errors both in catalina.out and localhost.<date>.log. You may need to enable additional logging to find the root cause of this error. Google for "SEVERE: Error listenerStart" and look for specific instructions for your version of Tomcat.

Tomcat 7 Goes down without notice.Please Respond Critical Issue

I'm using Tomcat 7 to deploy my app.
Tomcat goes down randomly and requires me to restart the server time and again to keep it up and running.
Sometimes when i restart ,i check the logs to see waiting for instances to be deallocated.
Other times when i restart ,i get the ERROR 1053.
I tried deleting my temp/work folders.I checked my database connections through hibernate ,i made sure that i have all the required jars in lib under tomcat.
The way i deploy is copy my war file and its respective Applicationfolder into Webapps under Tomcat.Please direct me as to what needs to be done to keep the server stable.
Another important note is that the exception in the logs below,changes from Nullpointer to Illegal state Exception
Any help to identify the root cause will be truly appreciated as this has been an impending and critical issue that has been haunting,inspite of trying different solutions.
My Logs are as follows
INFO: Waiting for 3 instance(s) to be deallocated
Jan 27, 2014 2:17:27 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/cep4] registered the JDBC driver [oracle.jdbc.OracleDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Jan 27, 2014 2:17:27 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/cep4] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak.
Jan 27, 2014 2:17:27 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/cep4] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation.
Jan 27, 2014 2:17:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/cep4] created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Locale context]) and a value of type [org.springframework.web.servlet.DispatcherServlet$1] (value [en_US]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Jan 27, 2014 2:17:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/cep4] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal#1655d52d]) and a value of type [org.springframework.security.core.context.SecurityContextImpl] (value [org.springframework.security.core.context.SecurityContextImpl#388100c: Authentication: org.springframework.security.cas.authentication.CasAuthenticationToken#388100c: Principal: com.java.service.MyUserDetails#9154e84; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails#ffff4c9c: RemoteIpAddress: 128.32.111.117; SessionId: CF331A8C27CEA68FAC797ED7E0E654B5; Granted Authorities: com.java.service.MyUserDetails$1#1aca2c25 Assertion: org.jasig.cas.client.validation.AssertionImpl#1a6d280b Credentials (Service/Proxy Ticket): ST-89206-bOamLKnOoLTLDJoIxYqs-cas-p3]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Jan 27, 2014 2:17:27 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/cep4] created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Request attributes]) and a value of type [org.springframework.web.context.request.ServletRequestAttributes] (value [SecurityContextHolderAwareRequestWrapper[ org.springframework.security.web.savedrequest.SavedRequestAwareWrapper#5d1e88cf]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Jan 27, 2014 2:17:27 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-bio-80"]
Jan 27, 2014 2:17:27 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-bio-8443"]
Jan 27, 2014 2:17:27 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["ajp-bio-8009"]
Jan 27, 2014 2:17:27 PM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already. Could not load org.hibernate.cfg.ImprovedNamingStrategy. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
at org.springframework.util.ClassUtils.forName(ClassUtils.java:258)
at org.springframework.beans.factory.support.AbstractBeanDefinition.resolveBeanClass(AbstractBeanDefinition.java:417)
at org.springframework.beans.factory.support.AbstractBeanFactory.doResolveBeanClass(AbstractBeanFactory.java:1283)
at org.springframework.beans.factory.support.AbstractBeanFactory.resolveBeanClass(AbstractBeanFactory.java:1254)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:433)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:270)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:120)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1360)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1118)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:322)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Jan 27, 2014 2:17:29 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'maxSpareThreads' to '75' did not find a matching property.
Jan 27, 2014 2:17:29 PM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'source' to 'org.eclipse.jst.jee.server:cep4' did not find a matching property.
Jan 27, 2014 2:17:29 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-80"]
Jan 27, 2014 2:17:29 PM org.apache.coyote.AbstractProtocol init

Really my colleagues and I faced with a similar issue with nearly the same error logs when using Apache Tomcat 7 as application server. Finally we could solve the problem using helps of contents at Stack Overflow and Apache Tomcat websites. Here I would like to share our experiences in a step by step form of debugging:
Try to find out crashing time due to server time.
Goto log folder of Tomcat and check errors just before the crashing time to the end.
Try to find out the reason of each error and try to solve the problems which caused the errors.
Start to monitor Tomcat behaviour using available tools. We used jConsole, but do not forget to link jConsole to Tomcat.
Try to control important parameters such as changes in Heap Memory and Threads especially in peak time of using your system. Maybe you have to increase specified or default values of Heap Memory or Threads.
P.S.: Do not panic on PS Survivor space! That's full most of time. ;-)
Final Word
Finally in our case by monitoring Tomcat with jConsole, we found out the problem was
insufficient Heap Memory in peak time. The solution was increasing the amount of allocated heap memory of Tomcat.

Severe Tomcat crash has me stumped

I'm pretty used to crashing my tomcat server. But I don't think I made any changes at all and seem to have broken it completely. Was working one min then not the next.
I really hope someone can help here is the Catalina.out:
Dec 20, 2012 1:35:56 AM org.apache.catalina.startup.Embedded initDirs
SEVERE: Cannot find specified temporary folder at
Dec 20, 2012 1:35:57 AM org.apache.catalina.core.AprLifecycleListener init
INFO: Loaded APR based Apache Tomcat Native library 1.1.23.
Dec 20, 2012 1:35:57 AM org.apache.catalina.core.AprLifecycleListener init
INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
Dec 20, 2012 1:35:57 AM org.apache.coyote.http11.Http11AprProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8040
Dec 20, 2012 1:35:57 AM org.apache.coyote.ajp.AjpAprProtocol init
INFO: Initializing Coyote AJP/1.3 on ajp-8042
Dec 20, 2012 1:35:57 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 587 ms
Dec 20, 2012 1:35:57 AM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Dec 20, 2012 1:35:57 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.33
Dec 20, 2012 1:35:57 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory docs
Dec 20, 2012 1:35:57 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory ROOT
Dec 20, 2012 1:35:57 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory myApp
Dec 20, 2012 1:35:57 AM org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/lhome/me/tomcat6/webapps/myApp/WEB-INF/lib/javax.servlet.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
Dec 20, 2012 1:35:58 AM org.apache.coyote.http11.Http11AprProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8040
Dec 20, 2012 1:35:58 AM org.apache.coyote.ajp.AjpAprProtocol start
INFO: Starting Coyote AJP/1.3 on ajp-8042
Dec 20, 2012 1:35:58 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 880 ms
Error Reading TLE line 1: java.lang.Exception: TLE line 1 not valid first line
Error Creating SGP4 Satellite
Dec 20, 2012 1:36:26 AM org.apache.coyote.http11.Http11AprProtocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8040
Dec 20, 2012 1:36:26 AM org.apache.coyote.ajp.AjpAprProtocol pause
INFO: Pausing Coyote AJP/1.3 on ajp-8042
Dec 20, 2012 1:36:27 AM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
Dec 20, 2012 1:36:30 AM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/myApp] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Dec 20, 2012 1:36:30 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/myApp] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation.
Dec 20, 2012 1:36:30 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/myApp] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak.
Dec 20, 2012 1:36:30 AM org.apache.coyote.http11.Http11AprProtocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-8040
Dec 20, 2012 1:36:30 AM org.apache.coyote.ajp.AjpAprProtocol destroy
INFO: Stopping Coyote AJP/1.3 on ajp-8042
Dec 20, 2012 1:37:12 AM org.apache.catalina.startup.Catalina stopServer
SEVERE: Catalina.stop:
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:208)
at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:338)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:416)
And I have a new log that has been created, I've never seen it before but it seems to think its something severe. A long one but here it is:
Massive Error Log I've never seen before

Your JVM's got SIGBUS. This is either a hardware problem, or a low-level JVM code bug.
siginfo:si_signo=SIGBUS: si_errno=0,
si_code=2 (BUS_ADRERR), si_addr=0x00007f0a9bb06fe5
One suspicious entry is -Djava.io.tmpdir= in your startup (i.e. temp dir is empty string). Could do funny things to JVM, though I did not test it. May be it is OK to have it like that.
jvm_args: -Dcatalina.base=/lhome/me/tomcat6
-Dcatalina.home=/usr/share/tomcat6
-Djava.endorsed.dirs=
-Djava.io.tmpdir=
-Djava.util.logging.config.file=/lhome/me/tomcat6/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
Bottom line: I believe your hardware goes down. Backup everything.

The bomb seems to have occurred in the native method java.util.zip.ZipFile.getZipMessage() which is called by getEntry(). ZipFile had experienced an error and was trying to construct an exception message. This occurs while stopping the running webapp as part of reload.
Did you move the .war file? That can cause the problem.
These kinds of problems are hard to diagnose and fix. However, the problem is very likely to be java.io.tmpdir not being set properly. http://efod.se/blog/archive/2011/05/02/java-sigbus is a similar issue.
If that doesn't work, use apt-get to upgrade zlib and OpenJDK, and force ZipFile.getEntry() to be interpreted:
-X:CompileCommand=exclude,java/util/zip/ZipFile,getEntry
As OSB Stats said, don't trust your hardware (although it's not likely a hardware problem).

Thanks to all who gave help here.
However I fell I should answer my own question. Although I still don't fully understand why it caused such a severe exception. The problem seemed to lie with me having dropped a table on my database that the program was reading. This table was full of data that the program used to compile other data. Once I re-populated this table it was fine.
I don't know why something so simple caused such a lot of errors and never showed up in the logs as a table not found error. However I can only assume that the program was trying to run with blank data and this someone caused a severe error.
So thanks again to those who offered help!