If I have multiple files of the same name on classpath (e.g. I have multiple .jar with log4j.properties), what are the rules JVM follows to chose one?
It is specified by the order in which the resources (i.e. usually jar files) are specified using -classpath option. Resources 'earlier' on the classpath take precedence over resources that are specified after them. This can be also set in the manifest file of your application and then you don't need to provide -classpath option. You may want to check these articles on how to work with manifest files.
The exhaustive description of "how classes are found" can be found here, where the section on JAR-class-path Classes describes the logic of JAR-files searching.
The ClassLoader determines where a resource will be located (taken from ClassLoader JavaDoc):
The ClassLoader class uses a delegation model to search for classes and resources. Each instance of ClassLoader has an associated parent class loader. When requested to find a class or resource, a ClassLoader instance will delegate the search for the class or resource to its parent class loader before attempting to find the class or resource itself. The virtual machine's built-in class loader, called the "bootstrap class loader", does not itself have a parent but may serve as the parent of a ClassLoader instance.
So wherever in your code Class#getResource or Class#getResourceAsStream is called, this happens (taken from Class.java)
public java.net.URL getResource(String name) {
name = resolveName(name);
ClassLoader cl = getClassLoader0();
if (cl==null) {
// A system class.
return ClassLoader.getSystemResource(name);
}
return cl.getResource(name);
}
ClassLoader.java:
public URL getResource(String name) {
URL url;
if (parent != null) {
url = parent.getResource(name);
} else {
url = getBootstrapResource(name);
}
if (url == null) {
url = findResource(name);
}
return url;
}
where ClassLoader#findResource is actually to be overwritten by the ClassLoader implementation. This implies that the behavior is different on an application server, a TomCat or if you are running from a jar file, it depends on the ClassLoader implementations of the environment you are currently in.
Here is an example that you may use to trace what's going under the hood in your particular case.
I am contributing a proven case that if classpath is, say, all jars in a folder, and you want to prioritize one (or some) of them, this does not work:
Windows:
bin/prioritized.jar;bin/*
Linux:
bin/prioritized.jar:bin/*
It appears that the first path bin/prioritized.jar is ignored just because the second one with a wildcard includes it in its own scope. This is what effectivelly breaks the specified order of classpaths.
Therefore, in order to have multiple resources prioritized (tested on Java 10.0.1), you need to put them in non-overlapping scopes and then they will work.
Related
What I understand is that if I use:
Instrumentation#getAllLoadedClasses()
I do get a selection of all loaded classes by the target JVM. But If I do:
Class.forName("my.class.name")
This will not be the same class as the class loaded by VM. Yes, I can add this particular class as a jar in the agent MANIFEST.MF Class-Path - but that does not look the same to me as getAllLoadedClasses().
Could someone please confirm whether this is correct i.e. I would not be able to find a specific class using Class.forName() when instrumenting? My objective was not to iterate over all loaded classes using getAllLoadedClasses() - But if there is no alternative, I guess that's okay for now.
** UPDATE
What I made a mistake in writing is the Boot-Class-Path which I have now corrected in my manifest. Using -verbose:class logging I managed to see that my jars are being loaded as
[Opened C:\fullpath\someother.jar]
[Opened C:\fullpath\another.jar]
[Opened C:\fullpath\different.jar]
But I don't see any corresponding loading information. I tried adding a Class.forName("a.package.in.someother.jar.classname") and got NoClassDefFoundError. As soon as I jump into the agent jar, I cannot use Class.forName() to check if the class is loaded by the target VM. I am getting a NoClassDefFoundError.
FURTHER UPDATE
Okay I have "Fattened" the manifest to look up all classes in my WEB-INF/lib and tomcat's lib directory. What I can see is below:
1) When my custom class MyClass is loaded for the first time. -verbose shows:
[Loaded my.pkg.MyClass from file:/C:/base/webapps/ROOT/WEB-INF/lib/mypkg.jar]
2) If I try to load the class again, it is correctly showing the above order.
3) My agent jar is manifested with all classes for my tomcat lib and my web-inf/lib directory. And I can also confirm that the loader sees the jars correctly.
4) Now I inject the agent, and call Class.forName("my.pkg.MyClass") from within the agent class. I get the below results.
[Loaded my.pkg.MyClass from file:/C:/base/webapps/ROOT/WEB-INF/lib/mypkg.jar]
I acknowledge that it's system class loader loding it inside my agent code as #RafaelWinterhalter pointed out in one of his answers. Is there any way I can force a "Delegation" so that the a different classloader loads the agent class and therefore, correctly redefines a class.
Any help is appreciated.
As it is stated in the javadoc:
Invoking this method is equivalent to:
Class.forName(className, true, currentLoader)
where currentLoader denotes the defining class loader of
the current class.
You can also see from the source code that the method is marked #CallerSensitive which means that you get a different result based on the class loader that invokes the method.
When calling Instrumentation::getAllLoadedClasses, the returned array contains classes of any class loader and not only of the current class loader which is the system class loader when running a Java agent. Therefore:
for (Class<?> type : instrumentation.getAllLoadedClasses()) {
assert type == Class.forName(type.getName());
}
is not generally true.
After a bit of run around, and Thanks to #Holger who reminded me what the problem was - incorrect class loader.
before I inject the agent, I have done the following:
// Get the current context class loader, which is app ext. classLoader
ClassLoader original = Thread.currentThread().getContextClassLoader().getSystemClassLoader();
// Set the system classloader to app classloader which won't delegate anything
Field scl = ClassLoader.class.getDeclaredFields();
scl.setAccessible(true);
scl.set(null, Thread.currentThread().getContextClassLoader());
// Now inject agent
try {
vm.loadAgent(agentPath, args);
} catch (all sorts of errors/exceptions in chain) {
// Log them and throw them back up the stack.
} finally {
vm.detach();
// Put back the classLoader linkage
sc.set(null, original);
}
How I have confirmed
When it goes in my Agent Class - Thread.currentThread().getContextClassLoader() becomes my application extn loader. But the system classloader now becomes `ParallelWebappClassLoader".
I am assuming this is how it works, but could be totally worng:
i) When I say Class.forName("my.pkg") it will check the system class loader which is pointing to my loader now. If the class is not found (i.e. not loaded), it will go to parents etc. I believe this is more or less the delegation model is.
ii) In this way, the class is loaded in VM by the same class loader which would also load the class in my webapp under normal circumstances.
iii) I will not instrument anything else apart from my own classes so the classloader will always be the same.
So far I have not seen any LinkageError happening. But I still feel this is too risky and if I break the link I am screwed.
Using Class.forName in a java profiler must be avoided to escape from the NoClassDef error. JVM Loads the class files in the different level of class loaders based on their classpath setting and the class file demand.
Java Cre libraries + Boot path defended libraries will be loaded in bootstrap Level
Java Agent will be loaded in system level and it goes on. Class.forName() will look the class files from the parent loaders, the current loader will not check the child loader (Until unless we implement our own loaders)
Java core classes will be accessible from your application code but our application code will not be accessible by the Java core classes. Its called class loader hierarchy.
You have three options.
Lookup the class files from the Instrumentation.GetLoadedClassFiles()
Through Transformers you can get all the loaders classes and you can track them and look for your class in every loader until you find.
Have the Class.forname implementation in the lowest level of the hierarchy so that it can internally access all the path.
Maintain the hierarchy properly to avoid too many weird errors.
Assuming you are looking for the Class<?> in order to re-transform a class, it seems to me that you could save the ClassLoader passed to your transformer, and later use ClassLoader.loadClass(String). Something like:
class MyTransformer implements ClassFileTransformer {
Map<String, ClassLoader> _name2loader = new ...;
...
public byte[] transform(ClassLoader loader, String className,
Class<?> classBeingRedefined,
ProtectionDomain pd,
byte[] classfileBuffer) throws ... {
...
_name2loader.put(className.replace("/","."), classLoader);
...
}
...
Class<?> getClass(String name) throws ClassNotFoundException {
ClassLoader cl = _name2loader.get(name);
if (cl == null) {
throw ClassNotFoundException("No loader for class " + name);
}
return cl.loadClass(name);
}
}
Note that the className passed to transform uses slashes, not dots... A better alternative than the String.replace may be to actually read the class name from the classfileBuffer using your bytecode library (such as javaassist or ASM, but if you're transforming bytecode, you're likely already using such a library).
Note: I'm not sure if you'd see the same class being passed for transformation with different ClassLoaders, but it would be good to look out for that (or research it).
I have a web service we'll call service.war. It implements an interface we'll call ServicePluginInterface. During the startup of service.war, it reads in environment variables and uses them to search for a jar (MyPlugin.jar). When it finds that jar, it then uses a second environment variable to load the plugin within the jar. The class that it loads looks like this:
public class MyPlugin implements ServicePluginInterface {...}
The servlet attempts to load the plugin using code like:
try {
if (pluginClass == null) {
plugin = null;
}
else {
ZipClassLoader zipLoader = new ZipClassLoader(Main.class.getClassLoader(), pluginJar);
plugin = (ServicePluginInterface)zipLoader.loadClass(pluginClass).newInstance();
plugin.getAccount(null,null);
}
} catch (Exception e) {
...
}
The trick is that I don't have source or a jar for ServicePluginInterface. Not wanting to give up so easily, I pulled the class files out of the service.war files. By using those class files as dependencies, I was able to build, without compiler warnings, MyPlugin. However, when actually executed by Tomcat, the section of code above generates a runtime exception:
java.lang.ClassCastException: com.whatever.MyPlugin cannot be cast to com.whomever.ServicePluginInterface
As a second point of reference, I am also able to construct a synthetic class loader (separate java executable that uses the same class loading mechanism. Again, since I do not have the original source to ServicePluginInterface, I used the class files from the WAR. This second, synthetic loader, or faux-servlet if you will, CAN load MyPlugin just fine. So I would postulate that the Tomcat JVM seems to be detecting some sort of difference between the classes found inside the WAR, and extracted class files. However, since all I did to extract the class files was to open the WAR as a zip and copy them out, it is hard to imagine what that might be.
Javier made a helpful suggestion about removing the definition of ServicePluginInterface, the problem with that solution was that the ZipClassLoader that the servlet uses to load the plugin out of the jar overrides the ClassLoader findClass function to pull the class out of the JAR like so:
protected Class<?> findClass(String name) throws ClassNotFoundException
{
ZipEntry entry = this.myFile.getEntry(name.replace('.', '/') + ".class");
if (entry == null) {
throw new ClassNotFoundException(name);
}
...
}
The class ZipClassLoader then recursively loads all parent objects and interfaces from the jar. This means that if the plugin jar does not contain the definition for ServicePluginInterface, it will fail.
Classes defined by different class loaders are different:
At run time, several reference types with the same binary name may be
loaded simultaneously by different class loaders. These types may or
may not represent the same type declaration. Even if two such types do
represent the same type declaration, they are considered distinct. JLS
In that case zipLoader returns an instance of MyPlugin that implements the other ServicePluginInterface (is it loaded from the zip too?):
(ServicePluginInterface)zipLoader.loadClass(pluginClass).newInstance();
It seems that the application server already has a definition of ServicePluginInterface, then you don't need to redeploy it. It should be enough to add the required files (ServicePluginInterface, etc.) as non-deployed dependecies of your project.
Another approach goes by living with the fact, and accessing methods in ServicePluginInterface via reflection (use the Class object returned by zipLoader, instead of ServicePluginInterface.class).
I'm trying to use a resource from another module to import a file. My goal is to pass the filename by each custom class, and let the base class of another module fetch the file.
But I always get a Nullpointer Exception.
What am I doing wrong?
Module A:
src/main/java/foo/bar/MyBaseClass.java
src/main/resources/foo/bar/test.xml
Module B:
src/main/java/other/path/MyCustomClass extends MyBaseClass
classes:
abstract class MyBaseClass {
public static String TESTFILE = "foo/bar/test.xml";
getData(String filename) {
InputStream inputStream = MyBaseClass.class.getResourceAsStream(String filename); //NPE
}
}
class MyCustomClass extends MyBaseClass() {
doSomething() {
getData(TESTFILE);
}
}
/edit:
should I maybe use something like this?
super.getClass().getResourceAsStream(..)
It's very likely you should be using ClassLoader.getResourceAsStream()
e.g.
Thread.currentThread().getContextClassLoader().getResourceAsStream()
(probably safer, might work in different environments more correctly, i.e., where a special classloader is being used, Java EE?)
or at least
aClass.getClassLoader().getResourceAsStream()
this is how you should load resources on the classpath which may be in a different JAR (or classpath entry) than the given class you're calling getResourceXXX on.
If you're using a class that's in module B and you want to load resources from module A, you need to use ClassLoader.getResourceXXX.
So in Java you should generally use this approach (unless you care about restricting resource loading to a smaller area...)
Another thing to be careful about: pay attention to the need for a leading "/", always double-check the javadocs of whichever method you're using
see also: http://www.xyzws.com/servletfaq/what-is-different-between-classloadergetresourceasstream-and-classgetresourceasstream/21
The getResourcesAsStream() expects a name which is NOT a filename cause it's a resource name. Furthermore it looks like you are trying to access a resource from an other maven module. And not to forget you are trying to access the resource via a relative path which should be changed into an absolute resources path like /foo/bar/test.xml instead of foo/bar/test.xml.
I have an application that allows, using an abstract class, people to write their own implementations. I load these implementations as .class-files from a directory. Currently, I have this solution:
File classDir = new File("/users/myproject/classes/");
URL[] url = { classDir.toURI().toURL() };
URLClassLoader urlLoader = new URLClassLoader(url);
String filename;
for (File file : classDir.listFiles()) {
filename = string.getFilenameWithoutExtension(file);
if (filename.equals(".") || filename.equals("..") || filename.startsWith("."))
continue;
AbstractClass instance = (AbstractClass)urlLoader
.loadClass("org.mypackage." + filename)
.getConstructor(ConfigUtil.class, DatabaseUtil.class, StringUtil.class)
.newInstance(config, database, string));
instance.doSomething();
}
As you see - I need to specify the package the classes are located in in order to correctly load them. Omitting the package, I get an
java.lang.NoClassDefFoundError:
MyClass (wrong name: org/mypackage/MyClass)
error.
Now, from a architectural POV, I think it is very ill-designed that classes other people designed have to be compiled to MY package when loading them.
So I ask you: Is there a way I can load classes form the file system without having to specify the package they reside in?
Yes; implement an interface (or use an annotation).
Then use any class-scanning library (there are lots of SO questions about this, like this one) to load the particular class in question. Searching for "Java class scanning" or "Java plugin mechanism" will help.
You might also just want to use the Java Plugin Framework and avoid some effort. Although it's not clear to me that it's maintained any more, I know people are still using it.
You can use the ServiceProvider to load implementations which you don't know.
I saw both Class.getResource and ClassLoader.getSystemResource used to locate a resource in Java. Is there any reason to prefer one to another?
There are several ways to load resources, each with a slightly different meaning -
ClassLoader::getSystemResource() uses the system classloader. This uses the classpath that was used to start the program. If you are in a web container such as tomcat, this will NOT pick up resources from your WAR file.
Class<T>#getResource() prepends the package name of the class to the resource name, and then delegates to its classloader. If your resources are stored in a package hierarchy that mirrors your classes, use this method.
ClassLoader#getResource() delegates to its parent classloader. This will eventually search for the resource all the way upto the system classloader.
If you are confused, just stick to ClassLoader#getResource()
From Class.getResource( )
This method delegates the call to its
class loader, after making these
changes to the resource name: if the
resource name starts with "/", it is
unchanged; otherwise, the package name
is prepended to the resource name
after converting "." to "/". If this
object was loaded by the bootstrap
loader, the call is delegated to
ClassLoader.getSystemResource.
and ClassLoader.getSystemResource( )
Find a resource of the specified name
from the search path used to load
classes. This method locates the
resource through the system class
loader
ClassLoader.getSystemResource() will use the bootstrap (system) classloader.
Class.getResource() will use that particular instance of Class's classloader, in other words whatever classloader was used to load that class. This may be a different classloader than the system classloader.
Using Methods of java.lang.Class:
public java.net.URL getResource(String name) {
name = resolveName(name);
ClassLoader cl = getClassLoader();
if (cl==null) {
return ClassLoader.getSystemResource(name); // A system class.
}
return cl.getResource(name);
}