Develop Reference

java.lang.IllegalStateException while reloading Tomcat

I'm working on Spring with Hibernate application, It's working fine but while starting or reloading tomcat server, i'm getting java.lang.IllegalStateException Exception.
Can anybody explain, why this exception occurring and how to resolve it??
INFO: Illegal access: this web application instance has been stopped already. Could not load java.net.BindException. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
at com.mysql.jdbc.SQLError.createLinkFailureMessageBasedOnHeuristics(SQLError.java:1220)
at com.mysql.jdbc.exceptions.jdbc4.CommunicationsException.<init>(CommunicationsException.java:57)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1074)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3270)
at com.mysql.jdbc.MysqlIO.quit(MysqlIO.java:1659)
at com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4296)
at com.mysql.jdbc.ConnectionImpl.cleanup(ConnectionImpl.java:1265)
at com.mysql.jdbc.ConnectionImpl.finalize(ConnectionImpl.java:2667)
at java.lang.System$2.invokeFinalize(Unknown Source)
at java.lang.ref.Finalizer.runFinalizer(Unknown Source)
at java.lang.ref.Finalizer.access$100(Unknown Source)

I want to give you couple of choices. You can try it. Any option can fulfill your demand.
Restart your tomcat and apache server because a long time of using,
it keeps older version of your application.
Clean your tomcat temp directory and restart
As stated in error,
The eventual following stack trace is caused by an error thrown for
debugging purposes as well as to attempt to terminate the thread which
caused the illegal access, and has no functional impact.
It may be the real cause that at this moment you are in debugging mode and it doesn't clear the running thread. so remove your break point and run it not debugging
If your code contains any kind of thread which are not properly terminated, then this type of error may occur.
That you have init() method but no destroy() method, then this type of error may occur. For details, you can follow the link - http://www.javaspecialists.eu/archive/Issue056.html
if the webapp has stopped, or is stopping, that means that the .war file or WEB-INF/web.xml timestamp changed, and the webapp might be reloading. Please check timestamp is OK or not.
To turn it off, set reloadable="false" in the context definition of
your app. It might be tomcat's server.xml.
In details solution:
The tomcat's server.xml of the reloadable Context is set to false.
For example:
Context path="/expert" docBase="expert" debug="0" reloadable ="false"/>
The solution is easy, as long as the tomcat's server.xml in reloadable = "true" into false on the line, but to do so would lose the advantage of hot deployment, and for the development is not very convenient, simply change it or not. This error does not matter.
Error principle:
The reason is because the tomcat restart, because the previous tomcat thread has not completely shut down, restart tomcat will report this exception, but this does not affect the normal use, just jump abnormal annoying people. Used hibernate, spring or other large components, when a WEB application system has a lot of class,
if you turned the Tomcat reloadable = true, then whenever the relevant documents change, Tomcat stops web app and frees up memory, and then reload web app. it may be a huge project. So we always think if there is only a certain class of overloaded functions, will greatly meet our debugger.
UPDATE: For code related issue
First, I want to tell you that I have given you some solutions for tomcat basis. Now I want to give you a solution for code basis. Would you please cross check your code with this issue? Please follow the URL.
http://www.coderanch.com/t/660126/Wiki/Illegal-State-Exception
UPDATE: For MySQL related issue
There are 2 issues.
This web application instance has been stopped already. Could not load java.net.BindException.
This web application instance has been
stopped already.Could not load com.mysql.jdbc.
This is because the MySQL JDBC driver on the application under the WEB-INF/lib directory, in the re-release of its loaded twice, so long as it can be copied to %TOMCAT_HOME%/lib can solve the problem.
We can solve these two anomalies MySQL drivers from the WEB-INF/lib folder moved to %TOMCAT_HOME%/lib.
I suspect this might be happening after the web application is restarted, where it's down for a short period of time. Then some finalize() method in the code is probably trying to do some cleanup too late. Whether or not that's in your code or the MySQL driver I can't say. You definitely should only have one version of a jar in a directory at a time. You might want to upgrade it to the latest (5.1.38 right now) in case something has been fixed that might be affecting you.(Number 9 is copied from #WhiteFang34)
Related link for 9: tomcat 6.0.24 Exception: Could not load com.mysql.jdbc.SQLError

The issue was because of the server cache. Follow the below steps to fix this issue
Find the directory called "work" under your tomcat's root directory
Delete it completely
Restart the server
Issue will be fixed.
Note: In case you have default tomcat installation in linux machine, you have to delete the folder /var/cache/tomcat7/Catalina

I think that the root cause of this problem is that something is leaking JDBC Connection objects.
These objects are actually instances of com.mysql.jdbc.ConnectionImpl ... which is a class with a finalize(). Normally, object finalization tidies up the leaked connections. In this case, it looks like the sequence of events is as follows:
the GC runs after the webapp has shutdown
a ConnectionImpl object is found to be unreachable
the object's finalize() method attempts to shutdown the MySQL database connection ... which entails sending a message to the server
when the message send fails for some reason, and the comms-level code tries to create an exception by calling com.mysql.jdbc.SQLError.createCommunicationsException
that tries to create an exception reflectively ...
The last step is failing because it is trying to use the webapp's classloader for the webapp ... which has been shut down. Apparently, this is detected by a check that is intended to help programmers diagnose webapps that don't shutdown cleanly.
The best solution is to track down the source of the Connection leak(s) and fix it. If Connection objects did not leak, then they would be closed while the webapp's classloader was still active.
Alternatively, you could simply turn off the check, as described in #Skywalker's answer.

The problem could be a simple SQL Error. tThe
java.lang.IllegalStateException
means that an object or class was called on when it was not ready to be called on. These errors at the bottom of the stack
at java.lang.ref.Finalizer.runFinalizer(Unknown Source)
at java.lang.ref.Finalizer.access$100(Unknown Source)
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
look like they come from the Java Garbage Collector
attempting to clean up MySQL Object Pointers
that were temporarily unreferenced during a reboot. It seems to me that your error here is just a few MySQL Object Pointers being halted in the middle of execution when the GC calls their finalize methods and then pauses the application to attempt a clean up the object. (leading to a fail and a throw of this exception)
This error is most likely harmless, and could maybe be fixed by adding a
JNI_DestroyJavaVM();
to your main thread to be executed on shutdown.

Java RMI NoClassDefFoundError for javax.json.JsonValue in Remote object

Running into Exception caused during call to UnicastRemoteObject.exportObject().
javax.json.jar is on the classpath and is used in many other places in the application without any problems.
This part of the application worked fine until I added a method that returned a JsonValue to the remote object.
Any ideas?
java.rmi.ServerError: Error occurred in server thread; nested exception is:
java.lang.NoClassDefFoundError: javax/json/JsonValue
at sun.rmi.server.UnicastServerRef.oldDispatch(UnicastServerRef.java:416)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:267)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:378)
at sun.rmi.registry.RegistryImpl_Stub.bind(Unknown Source)
NOTE:
I also tried running the rmiregistry with codebase pointed directly at the javax.json.jar but the exception remains
rmiregistry -J-Djava.rmi.server.codebase=file:///JarLibrary/javax.json.jar &

Running into Exception caused during call to UnicastRemoteObject.exportObject().
No you aren't. See the stack trace. It's happening in Registry.bind().
You need to run the server with the java.rmi.server.codebase property set, but a file:// codebase URL isn't going to work unless either all the clients are running in the server host, in which case you don't really need the codebase feature at all, or it points to a shared folder in a form that both the Registry and the clients can use. It's usually HTTP.
But I question whether you need the codebase feature at all. You just have to ensure that the relevant jar file is on the CLASSPATH of both the Registry and the clients. The simplest way to ensure that for the Registry is to use LocateRegistry.createRegistry() in the server JVM instead of the external rmiregistry program.
I'm also wondering why you're using JSON at all. RMI is built over Object Serialization. You don't need to add another serializer.

RMI server does not dynamically load missing classes from its HTTP codebase server

Context
This is an application using Java RMI, with:
A client on host A and a server on host B.
A HTTP server on host A, it holds a copy of all classes used by the client and the server.
The RMI registry is created from the server code, it shares server JVM and CLASSPATH.
For the server and the client applications:
The java.rmi.server.codebase property is set (in the code) to the HTTP server URL.
A security manager is in place with the appropriate policies.
The java.rmi.server.hostname is set to the LAN address.
Method invocation:
A server method is defined with a parameter of type WorkRequest (abstract class).
The client invokes this method using a subclass WorkRequestSquare.
WorkRequestSquare is never mentioned on the server code
Server code
Object execute(WorkRequest work) throws RemoteException { return work.execute(); }
Client code:
try { servant.execute(new WorkRequestSquare(123)); }
catch (RemoteException e) {
System.out.println("Error while submitting work request:");
e.printStackTrace();
}
This code works when the server has all related classes in the CLASSPATH and no dynamic loading is necessary.
Problem
Removing WorkRequestSquare from the CLASSPATH on the server side throws an exception on the client side (nothing is thrown on the server side):
Error while submitting work request:
java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
java.lang.ClassNotFoundException: WorkRequestSquare
at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
...
WorkRequestSquare being never explicitly mentioned on the server code, it should be dynamically loadable, but this doesn't happen.
Bypass
I was made aware by Stuart Marks that the property java.rmi.server.useCodebaseOnly should be set to false.
The default value for this property has been changed from false to true in JDK 7u21. useCodebaseOnly set to true prevents the JVM from dynamically loading classes from any location, except the one defined locally using java.rmi.server.codebase. This means that the codebase value set by the client JVM when marshalling the WorkRequestSquare parameter is ignored by a server which useCodebaseOnly is true.
When useCodebaseOnly is set to false on the server, and WorkRequestSquare class is removed from the server CLASSPATH, the server now gets the class definition from the HTTP server. This is a valid bypass.
Question
There is still something abnormal as the client and the server have the same value for the codebase property.
When useCodebaseOnly was defaulted to true the server JVM should have ignored client codebase added to the RMI stream, but it should have anyway used its own codebase to retrieve the WorkRequestSquare class.
This should have been successful anyway because client and server codebase values are identical.
Could someone cast some light on that matter?

What is the minimal set of classes that need to be in the classpath?
The classes that are actually used by name in the client code. For example, the remote interface, the types of the formal arguments and the result, exception types named in remote method signatures.
/ what are the classes that can be missing in the classpath, provided they are available using the codebase server?
Classes that implement classes or interfaces above and that aren't used by name in the client code.
Is there a tool to debug RMI communication (e.g. Eclipse plugin) and avoid using a network sniffer?
There certainly was an eclipse plugin, I saw it many years ago, but I've never actually needed one.

Java: no security manager: RMI class loader disabled

Hi I have RMI application and now I try to invoke some methods at server from my client. I have following code:
public static void main(final String[] args) {
try {
//Setting the security manager
System.setSecurityManager(new RMISecurityManager());
IndicatorsService server = (IndicatorsService) Naming
.lookup("rmi://localhost/" + IndicatorsService.SERVICE_NAME);
DataProvider provider = new OHLCProvider(server);
server.registerOHLCProvider(provider);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (RemoteException e) {
e.printStackTrace();
} catch (NotBoundException e) {
e.printStackTrace();
}
}
server Is correctly loaded, but when I am trying to call server.registerOHLCProvider(provider); I get these errors:
java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
java.lang.ClassNotFoundException: sk.xorty.client.providers.OHLCProvider (no security manager: RMI class loader disabled)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:336)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:255)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:233)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:142)
at sk.fri.statistics.service.impl.IndicatorsServiceImpl_Stub.registerOHLCProvider(Unknown Source)
at sk.fri.statistics.service.Client.main(Client.java:61)
Caused by: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
java.lang.ClassNotFoundException: sk.xorty.client.providers.OHLCProvider (no security manager: RMI class loader disabled)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:296)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.ClassNotFoundException: sk.xorty.client.providers.OHLCProvider (no security manager: RMI class loader disabled)
at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:375)
at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:165)
at java.rmi.server.RMIClassLoader$2.loadClass(RMIClassLoader.java:620)
at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:247)
at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:197)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1574)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:306)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:290)
... 9 more
I have added my policy file as VM argument, here is how it looks like:
grant {
permission java.security.AllPermission;
}
It keeps saying something about disabled classloading, so I guess problem is somewhere there ...
Thanks!

Remote class loading can be tricky.
The original post doesn't include any information about the code base. It may be that the client's security configuration is correct, but it has no access to the remote code. The classes are loaded directly from the "code base" by the client. They are not presented to the client by the service over the RMI connection. The service merely references an external source for the classes.
The server should specify the system property java.rmi.server.codebase. The value must be a URL that is accessible to the client, from which the necessary classes can be loaded. If this is a file: URL, the file system must be accessible to the client.
And the other way around: If the server should be able to load classes from the client (like here), the client must set the code base property to a URL that is accessible to the server.

Every time you invoke a method on an RMI dynamic proxy, the MarshalInputStream (which extends ObjectInputStream to override resolveClass and resolveProxyClass) delegates to LoaderHandler to look in 3 places for the ClassLoader to use:
The ClassLoader of the proxy that is being invoked (technically, it uses a hack called latestUserDefinedLoader(): it walks up the stack, looking for the first method on the stack that is not part of JRE).
Thread-local contextClassLoader of the caller
Codebase ClassLoader if SecurityManager is enabled
If System property java.rmi.server.useCodebaseOnly=false, then the codebase ClassLoader uses URLs in the remote java.rmi.server.codebase. Note that the default value of useCodebaseOnly changed in JDK 7u21 so that remote codebase is not used anymore unless you change it!
Otherwise, the codebase ClassLoader uses URLs in the local java.rmi.server.codebase.
So there are a few possible reasons that you would get a ClassNotFoundException when invoking a Remote method:
If stack contains “no security manager: RMI class loader disabled”, then make sure to set a SecurityManager as described by others if you need remote class loading for both sides to get all the Remote interfaces and serializable classes.
If you are using remote class loading and it stopped working when you upgraded to JRE 7u21, then either set -Djava.rmi.server.useCodebaseOnly=true to match previous behavior, or set -Djava.rmi.server.codebase to a space-separated list of URLs on both the local and the remote sides. And make sure that computer can access those URLs.
If you are using a custom ClassLoader locally whose parent classloader defines some Remote interfaces, then make sure to call Thread.setContextClassLoader(ClassLoader) so that RMI will use that ClassLoader. (This was my problem: I had a SwingWorker that happened to be scheduled onto a worker thread that was created before the contextClassLoader was set on the EventDispatchThread). For example, A and C belong to your custom ClassLoader but B belongs to the parent ClassLoader, then when you call a.getB().getC(), the getB() call will use the custom classloader, but the getC() call will fail to find C in the latestUserDefinedClassLoader and will have to fall back to the contextClassLoader.
All of this is a cautionary tale on poor API design of ObjectInputStream. ObjectInputStream should have required you to pass a ClassLoader parameter, not try to find one haphazardly using latestUserDefinedLoader, contextClassLoader, and codebase.

I want to add something which may be helpful for some people, especially begginers.
I came here and searched for a solution for the above error, but being a beginner, I didn't know how to use security policies and specify the "java.rmi.server.codebase" attribute.
The simplest way to fix that error is to make sure that the classes of the objects to be sent over RMI are in the same path of packages.. In this way, both applications have the class in the same location relative to their main folder and the error will solve.
Example:
If you want to send an object of type MedicationDTO (which is serializable) from server to client, make sure that it is in the same package path.
In my case, in the server app, the object was in com.example.springdemo.dto and in the client app, it was in com.example.springdemo.service.dto.. The problem was that, using IntelliJ, because the service package had nothing in it, but an other package, their name was concatenated (service.dto) and I could not see that the path was not the same.
So, make sure that your classes have the same package path. (Solution for my case: MedicationDTO class has to be in both application in package: com.example.springdemo.dto.
I know this is not the best solution, it's just a 'little trick', but I would have been extremely happy to find this solution then, because it had saved me from a lot of wasted time to solve the problem.
I hope this will be helpful for those who want a quick fix to that error, because I think learning to use security managers and including codebase could be a little tricky and will take time.

You need the security manager at the server side, not only at the client side.
Without this, the server's RMI engine refuses to load classes from the client, as it can't guarantee that these won't do evil things on the server.
Do you need the RMI class loading at all? Couldn't the server already have the classes which the client tries to send?

I know my case is extremely special but maybe it helps others. I had the case that there were multiple applications on one server sharing the same registry with different paths of course. Wherever this registry is created (usually from the first application) the full classpath of all later applications must be specified in this first application. To sum it up:
Make sure policy is provided
Make sure classpath is full specified for all applications, clients and server

I know why it happens.
for example you start server in the project A,
but you use the Client in the project B to request this server,this is wrong.
So you should put the server and client in the same project.

RMI: Using a foreign remote object

Just when I thought I know how RMI works, it comes back and tell me I do not. The following situation:
I have given:
an interface: http://pastebin.com/KxXH1jQ9
an adress where the rmiregistry is runnung
The interface offers cleary 3 getter methods to gain informations about the object. I simply want to lookup the object call the three methods and print out theirs values:
However the following implementation ( http://pastebin.com/d4JS0Wai ) does not do the trick resulting in the following exception:
Exception in thread "main" java.rmi.UnmarshalException: error unmarshalling return; nested exception is:
java.lang.ClassNotFoundException: Article
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at rmi.ArticleDisplayer.main(ArticleDisplayer.java:21)
Caused by: java.lang.ClassNotFoundException: Article
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
...
Also I don't know why I should need a SecurityManager anyway, as my machine is not executing the code. The code is executed on the machine running the rmiregistry. Anyway this is my policy file: http://pastebin.com/chGEqceA

This problem occurs because you should have the same Article class accessible by both the client and the source applications.
You can do a Jar with the common classes used by the 2 modules (client and server) and use it (I generally give the name "commons" to these type of packages).

Okay, the answer to the problem:
The implementation, as well as the interface are inside the default package. I wrote the program inside my own package: rmi
Also the interfaces I downloaded where in the package rmi. As a matter of fact rmi.Article != Article.