Im' running this code on a Linux Red Hat with the sun/oracle JVM 1.6_23, inside a VMWare server.
After some times the JVM seem to be unable to access my anonymous inner classes.
My classpath is fine, since it works for a period.
All i got is errors like this one :
java.lang.NoClassDefFoundError: com/mycompany/impl/MyClassImpl$1
at com.mycompany.impl.MyClassImpl.markAsDeletable(MyClassImpl.java:45).
line 45 is the first line below, it can't find my new Predicate
DomaineVO domaineVO = Iterables.find(domainesVO, new Predicate<DomaineVO>() {
#Override
public boolean apply(DomaineVO input) {
return input.getId().equals(domaine.getIdentifier().toString());
}
});
Any ideas ?
Finally, i think we might have put the finger on the problem.
We are running this code on jetty, and we use automatic deploy of the .war files.
By default jetty use the java.io.tmpdir to deploy the .war files.
Our problem was only on linux, and mostly early in the morning (as soon as the first office worker use the app).
The cause was the cleaning of the /tmp at night (made by a LOGROTATE command on our servers).
Rules of thumb : Never use /tmp for too long time, and make jetty deploy war in a directory of your own.
Thanks everyone
It sounds like the JVM can't find the class file for the anonymous class. This would be named 'MyClassImpl$1.class' - if it's not present in the classpath, something must have deleted it. If it is present then there's something wrong with the JVM.
This sounds very odd. Firstly if the code works for a time, as you say, the file must be there. Secondly, it is rare for a JVM to unload a class from memory once it has been used. Some JVMs will do it in tight memory situations or as part of a GC, but as an optimisation they normally stick around.
My only guess is you are using the JVM in a situation where ClassLoaders are changing. If you are using Netbeans (especially) but I think also eclipse, then if you partially recompile the code then classloaders might not match. Is this running within an IDE?
An alternative is a ClassLoader changing. If you re-release to a running webserver or application server, then an old class will not have a matching classloader for the new instance. The ClassLoader may not be able to find the old version, even though the file is there. Are you re-releasing to an Application/Webserver?
Finally, I guess this might be possible with Serialization. If the class is Serializable, and the serialVersionUIDs don't match I guess this might be able to happen. Are you doing any object serialisation here?
Related
After Updating log4j - to mitigate the log4shell vulnerability (CVE-2021-44228) - in class path, (or any other library generally), Do I need to restart JVM to make updates get into count?
Here, is mentioned that the "newly added classes" inside class path are getting loaded to the JVM automatically without restart, but what about the classes that are already loaded with the same name via class loader? Do they get overwritten?
The same question applies for tomcat (Although I guess it would be the same as JVM?)
Even if the new classpath would immediately be used by the JVM, there may be a number of objects instantiated from the old classes in memory. The new classes would then only apply to new instances. AFAIK log4j would not throw away it's objects during runtime.
To be on the safe side you definitely want to restart the JVM.
Is it required to restart JVM after updating log4j in classpath?
Probably yes. It depends on which classloader loads log4j.
If the log4j libraries are exclusively part of your webapps, you might be able to get away with hot-loading all of the webapps.
But you said "in classpath" and I guess that mean's in Tomcat's classpath; i.e. the shared libraries.
My advice would be not to take the risk. Restart Tomcat.
(Your systems should be designed so that Tomcat restarts are not a significant problem. There are various ways to do that. Indeed, one could argue that if downtime of your (single) Tomcat instance is an operational concern, then you should be running multiple copies.)
... but what about the classes that are already loaded with the same name via class loader? Do they get overwritten?
A classloader won't notice things have changed in its classpath. They are not designed to work that way.
And even if it did, a classloader cannot reload a class. The JVM architecture / runtime type safety don't allow it.
The hot-loading feature that (some) people use to avoid Tomcat restarts actually involves creating a brand new classloader to load the new version. The old version of the class will still exist in its original classloader, and other code will remain bound to it.
I want my java application to be able to automatically keep itself up to date, i already made all the code to download the latest jar file and put it in the designated path but since my program has to be open to actually check if there are updates available and then update them it gives me this error:
Exception in thread "main" java.nio.file.FileSystemException: name.jar: The process cannot access the file because it is being used by another process
Now my question is not why i get this error since it is quite obvious why, the question is: how would i go about actually updating the .jar file succesfully since it has to be open to actually download the update? I'd rather not make another .jar to act like a standalone updater if there are other options.
Example of the code i'm using to test:
URL url = new URL("<working url to the .jar file>");
InputStream in = url.openStream();
Files.copy(in, Paths.get("app.jar"), StandardCopyOption.REPLACE_EXISTING);
in.close();
First of all: the other answer is correct, there are ways to silently update/restart JAR files. And downloading new JARs, with a full restart, that is fine.
But the question asks about updating a JAR "in use", and for that I have a distinct non-answer: you are going down the very wrong rabbit hole!
Even if you somehow hack your way into overriding the JAR file on the file system while one (or more!) JVMs that have (or have not) have loaded classes from that JAR are running, the result is not what you would expect it to be: replacing the JAR file content doesn't magically reload all classes into a running JVM. The classes are already loaded! Changing their "load origin" in the file system does not affect "running" classes.
If at all, you enable situations such as:
class A is loaded from Jar X (version N)
Jar X is updated
class B is loaded from Jar X (version N+m)
But snap, that class B expects class A to look differently. And all of a sudden you have a versioning conflict within that JVM instance. Because it loaded some classes from X (version N), and other classes from X (version N+1, or N+5 because that customer skipped downloads for 2 weeks). And it gets worse: the exact error scenario might totally depend on the workload that your JVM has seen so far, and how it is used after the update. One customer might not have any issues, and the other might crash within 5 minutes (crashing at some point is very likely though).
Long story short: don't do this. Instead:
either tell your users that they have to restart the application when updates came in
or look into the "real" JVM hotswapping mechanism. But honestly: this is more of a debug feature, you do that in a development environment, for example using a tool like JRebel. You do not want to use it in a production environment at your customer. Because as said, runtime versioning issues are bad.
First of all it is close to impossible to replace a running application, without restarting it, but there are techniques to transition fluidly. Here's a popular one (simplistic):
You launch application.exe, which checks version and finds that there's a new version.
application.exe launches updater.exe and closes itself (or waits, until updated version is downloaded and then closes itself.
updater.exe replaces the file and launches application.exe again.
So this part is a staple (replacing the core application), without some sort of hardcode memory hacks, to my knowledge.
If you don't need to update the actual core application and are willing to invest time into developing a dynamic library/asset management in your application, you can essentially unload a library or an asset, update it from application.exe and then reload it when it has finished updating, without needing to restart the application.
This might be the thing you are looking for, because if your application.exe is just a loader, and core logic of the application is 1 of the libraries, you can replace essentially any part of the application. You still have to "restart" that part of the application, but you can save and restore important data before restart and restore it after to make the transition somewhat fast and painless.
It's will most likely be time consuming to learn and implement.
Here's an answer with some insight into the second portion of the answer.
PS: I used ".exe" as a reference everywhere. Just imagine it's about jars, same principles apply.
Some Confusion about the source code of Tomcat
Some day before, I began to read the source code of the tomcat
First, the server starts from the main method in the org.apache.catalina.startup.BootStrap, but when I go into the code bootstrap.init(),
and I was confused by the following code.Like:
Just for easy to debug the tomcat, I change the code like this:
The code can still work. And in the Tomcat source code, there are many code block like this, For example,in org.apache.catalina.startup.BootStrap.start()
we can find the following code:
Still for easy to debug, I turn the code into:
>
The code still works fine.So I was confused, Here are my questions.
what's difference between the two kinds of code?
Why the coder of Tomcat do not write the code like what I write? what's the benefit?
If the Tomcat with my kind code works in a production environment, what will happen or nothing will happen?
1) what's difference between the two kinds of code?
The Tomcat code eliminates a static dependency on the Catalina class. In general, that has a couple of potential benefits:
It may allow you to substitute an alternative version of that class ... without changing the code. (But not here, because the FQN for the class is hard-wired.)
For some JVMs, it may result in deferred class loading which can give faster JVM startup. (I'm not sure if that is relevant here.)
In addition, the Tomcat code is explicitly creating a new classloader and loading the Catalina using that ... rather than the default one for the Bootstrap class. I'm not sure of the significance of that.
2) Why the coder of Tomcat do not write the code like what I write? what's the benefit?
Ask the coder. The (possible) benefits are as I described above, but it may have been done for some other reason ...
3) If the Tomcat with my kind code works in a production environment, what will happen or nothing will happen?
Probably nothing, though it might affect the classloader organization for Tomcat in some way that impacts on your webapps.
But if you are worried, don't change the code! There is no real need to change this ... based on why you said you are doing it. There's a saying:
"If it ain't broken, don't fix it."
Ok, here is the issue. We are using an old version of Tomcat (6.0.35) because our version of linux (ubuntu 12.04) doesn't have a repository with a newer version of it. Unfortunately, there is a bug in 6.0.35 that is affecting us ( https://issues.apache.org/bugzilla/show_bug.cgi?id=53725 ). IT doesn't want to/can't change tomcat versions. They want to use the package management system, they don't want to rebuild tomcat. They can't maintain something like this on a whole bunch of servers and we don't particularly want to disable Gzip. We can deal with the corruption until we can finally upgrade.
So with that background, the real question. The changes which fix this bug reside in one single class. We came up with a crazy solution (which will probably never see production, but was more of a "this would be so terrible, lets try it!" sort of idea), How about we compile the fixed version of the class and hotswap it in. After all, tomcat and our project use the same JVM.
Is this doable/possible/more than a little crazy?
Some hurdles. We are using Java 6. The new class adds methods and fields. I'm not sure that tomcat keeps it around.
In most cases this is unreasonable to actually do. The only way to cleanly unload a class and then load a replacement is to let the ClassLoader used to load that particular class be garbage collected. This will require you to remove all references to any of the classes that ClassLoader loads (which is probably at least all of TomCat) and reload them using a ClassLoader that loads all classes like normal, except for the class you 'fixed'. You would have to make it so that it loads that class instead of the old one, while still allowing the other classes to load normally.
In short, it is possible, however it will require digging into java ClassLoaders. I'd recommend that you do not actually attempt this, as it will be very annoying to get right and it will certainly not work without shutting down the TomCat server first (since you have to reload all of TomCat's classes).
Who is doing the unzipping, Tomcat or the application? If its the application, just make a copy of the class in error to the root of your project classpath in src/main/java and put there the patched version of the class.
Upon packaging the patched class will be in WEB-INF/classes, and override the version in the server. But its overridden only from the point of view of the application, the server cannot see the patched version and still sees the buggy version.
If its tomcat itself that needs to unzip something, then its not possible to fix that without tinkering with the server.
But if its the application doing the unzipping its for sure possible, there is no obligation to use the libraries of the server we can use our own more updated copies, usually not in patches but in jars in WEB-INF/lib - the server is designed for that.
The situation at hand is not as simple as the title seems to indicate.
Java 1.6_17 running via JWS.
I have a class, lets say MyClass and one of its instance member variables is a Type from an errant 3rd party library where during class initialization it dynamically tries loading some of its own classes with Class.forName(String). In one of these cases it happens to dynamically call: Class.forName("foo/Bar").This class name doesn't follow the JLS for binary names and ultimately leads to a java.lang.NoClassDefFoundError: foo/Bar.
We have a custom ClassLoader which I've added a sanitize method to ClassLoader.findClass(String) and ClassLoader.loadClass(String) which fixes this problem.
I can call stuff like:
myCustomClassLoader.findClass("foo/Bar")
Which then loads the class without any problems. But even if I load the class ahead of time, I still get the exception later. This is because during initialization of MyClass which refers to Bar - their code ends up calling Class.forName("foo/Bar") in a static block somewhere. This actually would be OK if the ClassLoader it was trying to use was my custom class loader. But it isn't. It is the com.sun.jnlp.JNLPClassLoader which doesn't do such sanitation, thus my problem.
I've made sure that Thread.currentThread().getContextClassLoader() is set to my custom class loader. But this (as you know) has no effect. I even set it as the first thing i do in main() due to some stuff I read and still, MyClass.class.getClassLoader() - is the JNLPClassLoader. If I could force it to NOT be the JNLPClassLoader and to use mine instead, problem solved.
How can I control which ClassLoader is used to load the class via their static Class.forName("foo/Bar") call made during class initialization? I believe if I can force MyClass.class.getClassLoader() to return my custom class loader, my problem will be resolved.
I'm open to other options if anyone has ideas.
TL;DR: Help me force all Class.forName(String) calls in a third party library which are referenced by MyClass - to use the classloader of my choosing.
This reminds me of an article I read 10 years ago about the classloading arrangements in Java. It's still there on JavaWorld.
The article won't answer your question directly, but it may help understand your problem. You need to cause MyClass to be loaded through your custom class loader and trump the default class loading behavior, which is to first delegate class loading to the parent classloader and only attempt to load a class if that fails.
Allowing MyClass to get loaded by a classloader other than yours will store a relationship from the instantiated class to that classloader (via getClassLoader) and cause Java to use that other classloader to try to discover any referenced classes found at compile time, effectively bypassing your custom class loader by virtue of the class loader hierarchy and the delegation model. If MyClass is instead defined by your class loader, you get a second chance.
It sounds like a job for something like URLClassLoader, overriding loadClass and trumping the delegation model for classes residing in your JARs. You'll probably want to use a bootstrap approach (as suggested by Thomas in a comment above) to force a single entrypoint class to be loaded through your custom class loader, dragging all the others with it.
Also informative is this other JavaWorld article by the same guy, which warns you about the caveats of Class.forName. That too may trip your classloading arrangements.
I hope this helps and proves informative. In any case, it sounds like a difficult solution that is easy to break as your code evolves.
I think everyone gave good solid attempts at answering the problem. However, it turns out that I misdiagnosed the problem.
I had a coworker take over the problem and asked him to get a JDK with debug flags on so we could debug the JNLPClassLoader to see what was going on as I had tried all of the suggestions here + some.
We ended up getting OpenJDK because recompiling the JDK from scratch is a total nightmare (we tried). After getting OpenJDK working with our product and debugging through the JNLPClassLoader - it turns out that it was still using a REALLY old .jnlp from months earlier that had the resource path wrong and thus why it couldn't find the class.
We were confused why it was still using the ancient .jnlp even though we had redeployed the server correctly many times with the correct .jnlp and lots of code changes between which were reflected in our client application when run.
Well, it turns out that on client machines, Java caches the .jnlp file. Even if your application changes and it redownloads your application, it still won't re-download the new .jnlp for whatever reason. So it will use all of the new code, but look up resources/class paths using the cached .jnlp.
If you run:
javaws -uninstall
On the client machine then that will clear the .jnlp cache and next time it will use the correct .jnlp file.
Really sad that this was the problem. Hopefully, this saves someone else endless hours of frustration like it caused us.
If you run out of ideas with patching the ClassLoaders themselves, you might consider rewriting the library bytecode itself -- just replace the "foo/bar" constant with the correct value, and then you don't need to customize further class loading at all!
You could do this either at runtime or beforehand.