When using ServiceLoader.load(X.class) and there are 2 jar files which contain definitions for X.class which one will get deployed?
The issue I'm seeing is that this appears to be intermittent in terms of class load order.
This is a issue as the definitions are slightly different.
Ideally this would only appear once on the class path and avoid all problems but in the intermediate time I'm trying to understand whats loads first.
It depends on java version, application server type and version, probably even file system settings and who knows what else.
I think it's enough to say that there is no guarantee about the order, and it's something you should never rely on.
Related
I want to select different implementations of classes dynamically, based on a runtime condition. Let's say I have a class with fully qualified class name C. My running system may have many definitions of class C, each of which is in its own jar. I have a runtime condition (held in a ThreadLocal) that tells which definition should be chosen.
I was asked in a comment to clarify the original requirement, so I will clarify the requirement as best I can. There are multiple teams writing software to contribute to this system - something like 4000 classes in many independent modules. What's more they can change over time. They are currently running in separate JVMs so there is no issue with classes overlapping. Now we are considering running them in the same JVM with multiple releases running simultaneously on the same JVM; which specific set of implementations used being differentiated by the ThreadLocal. So the original problem was how to allow a thread to at one time run one set of implementations and at another time run another.
I have a tomcat application that is currently using OpenJDK 8.
I believe I can write a custom ClassLoader that manipulates the classpath to choose the definition of C differently based on the ThreadLocal. But I'm afraid the results will be cached somewhere such as JVM Code Cache. Unless I can override that behavior too, the next time the class is needed, the runtime condition may have changed and the version in the cache would be wrong.
Is there any way to do what I need to do?
Well, the simple solution would be to NOT have multiple definitions of class C, but instead have 'class C1', 'class C2', etc (i.e. they don't overlap and can be simultaneously loaded) and then your runtime property just picks the right one as appropriate. That is way-way the easiest solution, so strongly consider it first. But it may not meet your needs.
If you truly need to have multiple separate implementations of a single 'class C', then what you are effectively talking about is a 'hot swap' scenario. Fortunately, Tomcat and other tools got good (with limitations) at hotswapping long-time-back. You likely already know this, but 'hot swapping' meets the need of a developer who codes 'class C', deploys it to a container, tries it out, realizes it has a glitch, makes a quick code edit and wants to run the modified code without relaunching the container. Hotswapping does this by basically overlaying the new implementation in the JVM. It only works up to a point because each 'redeploy' pollutes the 'class space' of the JVM and you eventually run of of 'class space memory' and/or the JVM starts going unstable. Depending on your needs and tolerances though, hotswapping might work.
The clean way to do what you want is to define C as an interface, then you can load and use any class that implements interface C
I'm seeing an interesting behavior in some of the code I inherited where there's a project with a dependency jar in which there exists a class with the same name and in the same package as a class in the project:
Eclipse project:
src/com.abc.d.E
depends on XYZ.jar in which there exists a com.abc.d.E.class
Just curious if this setup is legal in Java. According to Eclipse, which doesn't mark it as an error and allows to create a new class that shadows an existing class in a dependency jar, except when one tries to rename such an existing class in the workspace - then it produces the following warning:
"Binary references to a refactored element have been found. They will
not be updated, which may lead to problems if you proceed."
And then this forward refactoring is allowed. However, if one would like to refactor backwards to a conflicting name, the following message appears in Eclipse:
"Type named 'E' already exists in package 'com.abc.d'
So is it
a correct Java
an allowed inconsistency in Eclipse's behavior or
.an Eclipse bug?
Thank you.
If both classes where accessible to the same classloader (for example if both jar files find themselves on the same classpath), then only one of them would be loaded (probably the one that's first in the list), which would lead to all kinds of nasty results, esepcially if they don't behave the same.
Assume library 1 uses E.frobnicate() while library 2 expects there to be a E.frob() method: one of those will get a NoSuchMethodError and generally just fail spectacularly.
Generally speaking, the package name and class name should uniquely identify a class. And if that's no longer the case then you'll get into trouble.
You can work around the issue as long as you never need those two libraries to be accessible from the same classloader, then the JVM can handle it just fine (because FQCN plus the classloader is used internally by the JVM to uniquely identify a class during runtime).
The only case where this is intentionally done is if some library re-implements classes of another one in a binary-compatible manner. See log4j-over-slf4j for an example.
There's nothing that states you cannot have two classes with the same package/class name.
This happens all the time when you accidentally have two versions of a library on your classpath, leading to things like IncompatibleClassChangeError or getting the wrong version because you're at the mercy of class load order.
Eclipse, OTOH, has to do something with those classes other than just loading the first one encountered on the classpath. You can't rename to something that already exists, because that wouldn't make any sense.
Renaming something out from under something that already exists is likely an error, but can't be guaranteed to be an error, because you might be fixing a naming problem, rather than just creating a new/different one.
I'm working on a sandbox feature for my java antivirus, and I've come into a question: Does the specified package on a class matter for compilation?
Example:
I'm running a program that wants to use Runtime.getRuntime().exec(), when the classloader attempts to load that to run a method, does it check the package qualified in the file, if they exist? I would prefer not to try and change files in the JVM, but to simply load ones from a different package. I can accomplish the loading and such, but my only dilemma, will it crash and burn? Inside the java, it would be registered as say, java.lang.Runtime, but the compiled code will say for example pkg.pkg.Runtime and will it need to extend the old runtime? My guess is that extending the old runtime would just break it. Does anyone know anything about this? I'm working on making a testable example, but I'm still a bit away and wanted to get some answers, as well as this might benefit some people.
Does the specified package on a class matter for compilation?
Yes it does matter. A class called pkg.pkg.Runtime() cannot be loaded as if it was java.lang.Runtime.
Furthermore, if my memory is correct, the JVM has some additional security measures in it to prevent normal applications from injecting classes into core packages such as java.lang.
If you need to change the behaviour of the java.lang.Runtime class (for experimental purposes!) then I think you will need to put your modified version on the boot classpath, ahead of the "rt.jar" file.
However:
This level of tinkering can easily result in JVM instability; i.e. hard JVM crashes that are difficult to diagnose.
If your aim is to produce a "production quality" tool, then you will find that things that involve tinkering with the JVM are not considered acceptable. People are going to be very suspicious of installation instructions that say things like "add this to your installed JVM's bootclasspath".
Distributing a "tinkered with" JVM may fall foul of Oracle's Java licensing agreement.
My advice would be to look for a less intrusive way of doing what you are trying to do. For instance, if you are trying to do virus checking, either do it outside of the JVM, or in a custom application classloader.
You commented:
I have a custom classloader, my question is: If I compile a class that is labelled as say, pkg.pkg.Runtime, can I register in my classloader as java.lang.Runtime?
As I said above, no you can't. A bytecode file has the classname embedded in it. If you attempt to "pull a swifty" by loading a class with a different name, the JVM will throw an Error.
And:
If not, then how can I replace the class? If the compiled package name has to equal the request referenced naming, then can I modify the .class file to to match, or perhaps compile it as if it were in the java.lang package?
That's what you would have to do. You need to name the class java.lang.Runtime in the source code and compile it as such.
But what I meant by my advice above is that you should use do the virus checking in the class loader. Forget about trying to replace / modify the behaviour of Runtime. It is a bad idea for the reasons I listed above.
The situation at hand is not as simple as the title seems to indicate.
Java 1.6_17 running via JWS.
I have a class, lets say MyClass and one of its instance member variables is a Type from an errant 3rd party library where during class initialization it dynamically tries loading some of its own classes with Class.forName(String). In one of these cases it happens to dynamically call: Class.forName("foo/Bar").This class name doesn't follow the JLS for binary names and ultimately leads to a java.lang.NoClassDefFoundError: foo/Bar.
We have a custom ClassLoader which I've added a sanitize method to ClassLoader.findClass(String) and ClassLoader.loadClass(String) which fixes this problem.
I can call stuff like:
myCustomClassLoader.findClass("foo/Bar")
Which then loads the class without any problems. But even if I load the class ahead of time, I still get the exception later. This is because during initialization of MyClass which refers to Bar - their code ends up calling Class.forName("foo/Bar") in a static block somewhere. This actually would be OK if the ClassLoader it was trying to use was my custom class loader. But it isn't. It is the com.sun.jnlp.JNLPClassLoader which doesn't do such sanitation, thus my problem.
I've made sure that Thread.currentThread().getContextClassLoader() is set to my custom class loader. But this (as you know) has no effect. I even set it as the first thing i do in main() due to some stuff I read and still, MyClass.class.getClassLoader() - is the JNLPClassLoader. If I could force it to NOT be the JNLPClassLoader and to use mine instead, problem solved.
How can I control which ClassLoader is used to load the class via their static Class.forName("foo/Bar") call made during class initialization? I believe if I can force MyClass.class.getClassLoader() to return my custom class loader, my problem will be resolved.
I'm open to other options if anyone has ideas.
TL;DR: Help me force all Class.forName(String) calls in a third party library which are referenced by MyClass - to use the classloader of my choosing.
This reminds me of an article I read 10 years ago about the classloading arrangements in Java. It's still there on JavaWorld.
The article won't answer your question directly, but it may help understand your problem. You need to cause MyClass to be loaded through your custom class loader and trump the default class loading behavior, which is to first delegate class loading to the parent classloader and only attempt to load a class if that fails.
Allowing MyClass to get loaded by a classloader other than yours will store a relationship from the instantiated class to that classloader (via getClassLoader) and cause Java to use that other classloader to try to discover any referenced classes found at compile time, effectively bypassing your custom class loader by virtue of the class loader hierarchy and the delegation model. If MyClass is instead defined by your class loader, you get a second chance.
It sounds like a job for something like URLClassLoader, overriding loadClass and trumping the delegation model for classes residing in your JARs. You'll probably want to use a bootstrap approach (as suggested by Thomas in a comment above) to force a single entrypoint class to be loaded through your custom class loader, dragging all the others with it.
Also informative is this other JavaWorld article by the same guy, which warns you about the caveats of Class.forName. That too may trip your classloading arrangements.
I hope this helps and proves informative. In any case, it sounds like a difficult solution that is easy to break as your code evolves.
I think everyone gave good solid attempts at answering the problem. However, it turns out that I misdiagnosed the problem.
I had a coworker take over the problem and asked him to get a JDK with debug flags on so we could debug the JNLPClassLoader to see what was going on as I had tried all of the suggestions here + some.
We ended up getting OpenJDK because recompiling the JDK from scratch is a total nightmare (we tried). After getting OpenJDK working with our product and debugging through the JNLPClassLoader - it turns out that it was still using a REALLY old .jnlp from months earlier that had the resource path wrong and thus why it couldn't find the class.
We were confused why it was still using the ancient .jnlp even though we had redeployed the server correctly many times with the correct .jnlp and lots of code changes between which were reflected in our client application when run.
Well, it turns out that on client machines, Java caches the .jnlp file. Even if your application changes and it redownloads your application, it still won't re-download the new .jnlp for whatever reason. So it will use all of the new code, but look up resources/class paths using the cached .jnlp.
If you run:
javaws -uninstall
On the client machine then that will clear the .jnlp cache and next time it will use the correct .jnlp file.
Really sad that this was the problem. Hopefully, this saves someone else endless hours of frustration like it caused us.
If you run out of ideas with patching the ClassLoaders themselves, you might consider rewriting the library bytecode itself -- just replace the "foo/bar" constant with the correct value, and then you don't need to customize further class loading at all!
You could do this either at runtime or beforehand.
I have a web application and two class files,
First class is MyClass.class which is inside the abc.jar file (WEB-INF/lib/abc.jar) and
Second class is YourClass.class which is inside classes folder (WEB-INF/classes/ YourClass.class).
My question is which class would load first when the Application starts? And Why ?
In my experience you can't predict the order in which the classes are loaded by the JVM.
Once I made a test runner (kinda Maven's Surefire) and with the same JVM and OS it loaded classes in different order when run in different machines. The lesson learnt:
You shouldn't build your applications
to depend on class loading order
Classes are loaded as needed, for some definition of "needed". Exactly when a class is loaded is dependent upon the JRE implementation, javac implementation, exactly what thread are up to, server code and, of course, application code. It's a bad idea to make assumptions in this area. If you want to see what happens for a particular run, you can use -verbose:class
Sun's class loader docs always say WEB-INF/classes OR WEB-INF/lib, but doesn't say which one will be checked first.
From IBM docs:
"The rules for loading classes are spelled out in detail in the JVM specification. The basic principle is that classes are only loaded when needed (or at least appear to be loaded this way -- the JVM has some flexibility in the actual loading, but must maintain a fixed sequence of class initialization). Each class that gets loaded may have other classes that it depends on, so the loading process is recursive."
So I think the answer is: It depends on which classes is needed in your application first.
As duffymo points out, this can vary. One way you might ascertain the sequence for this specific app is to insert Response.Write text in the class constructors and web web app page loading methods. "Instantiated object in Class A", "Opened web page MyPage", and so on.
Once you've figured out the sequence, comment out the code for those so you can reuse them later to verify that you haven't made a change (such as instantiating an object earlier or later) that affected the sequence.