Develop Reference

How to load classes dynamically from jar file

If i have a jar file that contain many class, how to get classes, and create instances at runtime. In this code i don't understand the Class.forName line, the MyClass is that class what contain the jar file or that class what will create after the jar file load?
URLClassLoader child = new URLClassLoader(
new URL[] {myJar.toURI().toURL()},
this.getClass().getClassLoader()
);
Class classToLoad = Class.forName("com.MyClass", true, child);
Method method = classToLoad.getDeclaredMethod("myMethod");
Object instance = classToLoad.newInstance();
Object result = method.invoke(instance);

Class.forName loads a class.
The first parameter is the (canonical) name of the class. In this case, you try to load the class com.MyClass.
The second parameter specifies that the class should be initialized at that point(static variables are initialized and static blocks are run.
The third parameter is the ClassLoader the class will be loaded from. In your case, it will try to find the class from myJar but if the class isn't found there, it will try to load the class from the same ClassLoader the calling class has been loaded.

Using Class.forName() in Java Instrumentation Agent

What I understand is that if I use:
Instrumentation#getAllLoadedClasses()
I do get a selection of all loaded classes by the target JVM. But If I do:
Class.forName("my.class.name")
This will not be the same class as the class loaded by VM. Yes, I can add this particular class as a jar in the agent MANIFEST.MF Class-Path - but that does not look the same to me as getAllLoadedClasses().
Could someone please confirm whether this is correct i.e. I would not be able to find a specific class using Class.forName() when instrumenting? My objective was not to iterate over all loaded classes using getAllLoadedClasses() - But if there is no alternative, I guess that's okay for now.
** UPDATE
What I made a mistake in writing is the Boot-Class-Path which I have now corrected in my manifest. Using -verbose:class logging I managed to see that my jars are being loaded as
[Opened C:\fullpath\someother.jar]
[Opened C:\fullpath\another.jar]
[Opened C:\fullpath\different.jar]
But I don't see any corresponding loading information. I tried adding a Class.forName("a.package.in.someother.jar.classname") and got NoClassDefFoundError. As soon as I jump into the agent jar, I cannot use Class.forName() to check if the class is loaded by the target VM. I am getting a NoClassDefFoundError.
FURTHER UPDATE
Okay I have "Fattened" the manifest to look up all classes in my WEB-INF/lib and tomcat's lib directory. What I can see is below:
1) When my custom class MyClass is loaded for the first time. -verbose shows:
[Loaded my.pkg.MyClass from file:/C:/base/webapps/ROOT/WEB-INF/lib/mypkg.jar]
2) If I try to load the class again, it is correctly showing the above order.
3) My agent jar is manifested with all classes for my tomcat lib and my web-inf/lib directory. And I can also confirm that the loader sees the jars correctly.
4) Now I inject the agent, and call Class.forName("my.pkg.MyClass") from within the agent class. I get the below results.
[Loaded my.pkg.MyClass from file:/C:/base/webapps/ROOT/WEB-INF/lib/mypkg.jar]
I acknowledge that it's system class loader loding it inside my agent code as #RafaelWinterhalter pointed out in one of his answers. Is there any way I can force a "Delegation" so that the a different classloader loads the agent class and therefore, correctly redefines a class.
Any help is appreciated.

As it is stated in the javadoc:
Invoking this method is equivalent to:
Class.forName(className, true, currentLoader)
where currentLoader denotes the defining class loader of
the current class.
You can also see from the source code that the method is marked #CallerSensitive which means that you get a different result based on the class loader that invokes the method.
When calling Instrumentation::getAllLoadedClasses, the returned array contains classes of any class loader and not only of the current class loader which is the system class loader when running a Java agent. Therefore:
for (Class<?> type : instrumentation.getAllLoadedClasses()) {
assert type == Class.forName(type.getName());
}
is not generally true.

After a bit of run around, and Thanks to #Holger who reminded me what the problem was - incorrect class loader.
before I inject the agent, I have done the following:
// Get the current context class loader, which is app ext. classLoader
ClassLoader original = Thread.currentThread().getContextClassLoader().getSystemClassLoader();
// Set the system classloader to app classloader which won't delegate anything
Field scl = ClassLoader.class.getDeclaredFields();
scl.setAccessible(true);
scl.set(null, Thread.currentThread().getContextClassLoader());
// Now inject agent
try {
vm.loadAgent(agentPath, args);
} catch (all sorts of errors/exceptions in chain) {
// Log them and throw them back up the stack.
} finally {
vm.detach();
// Put back the classLoader linkage
sc.set(null, original);
}
How I have confirmed
When it goes in my Agent Class - Thread.currentThread().getContextClassLoader() becomes my application extn loader. But the system classloader now becomes `ParallelWebappClassLoader".
I am assuming this is how it works, but could be totally worng:
i) When I say Class.forName("my.pkg") it will check the system class loader which is pointing to my loader now. If the class is not found (i.e. not loaded), it will go to parents etc. I believe this is more or less the delegation model is.
ii) In this way, the class is loaded in VM by the same class loader which would also load the class in my webapp under normal circumstances.
iii) I will not instrument anything else apart from my own classes so the classloader will always be the same.
So far I have not seen any LinkageError happening. But I still feel this is too risky and if I break the link I am screwed.

Using Class.forName in a java profiler must be avoided to escape from the NoClassDef error. JVM Loads the class files in the different level of class loaders based on their classpath setting and the class file demand.
Java Cre libraries + Boot path defended libraries will be loaded in bootstrap Level
Java Agent will be loaded in system level and it goes on. Class.forName() will look the class files from the parent loaders, the current loader will not check the child loader (Until unless we implement our own loaders)
Java core classes will be accessible from your application code but our application code will not be accessible by the Java core classes. Its called class loader hierarchy.
You have three options.
Lookup the class files from the Instrumentation.GetLoadedClassFiles()
Through Transformers you can get all the loaders classes and you can track them and look for your class in every loader until you find.
Have the Class.forname implementation in the lowest level of the hierarchy so that it can internally access all the path.
Maintain the hierarchy properly to avoid too many weird errors.

Assuming you are looking for the Class<?> in order to re-transform a class, it seems to me that you could save the ClassLoader passed to your transformer, and later use ClassLoader.loadClass(String). Something like:
class MyTransformer implements ClassFileTransformer {
Map<String, ClassLoader> _name2loader = new ...;
...
public byte[] transform(ClassLoader loader, String className,
Class<?> classBeingRedefined,
ProtectionDomain pd,
byte[] classfileBuffer) throws ... {
...
_name2loader.put(className.replace("/","."), classLoader);
...
}
...
Class<?> getClass(String name) throws ClassNotFoundException {
ClassLoader cl = _name2loader.get(name);
if (cl == null) {
throw ClassNotFoundException("No loader for class " + name);
}
return cl.loadClass(name);
}
}
Note that the className passed to transform uses slashes, not dots... A better alternative than the String.replace may be to actually read the class name from the classfileBuffer using your bytecode library (such as javaassist or ASM, but if you're transforming bytecode, you're likely already using such a library).
Note: I'm not sure if you'd see the same class being passed for transformation with different ClassLoaders, but it would be good to look out for that (or research it).

Instantiating classes from a jar that implement a common interface, and then assigning the instance to the interface causes ClassCastException

This is a classloader issue that I am struggling with. I understand the root cause of the issue (different classloaders), but I'm not sure about the best way to fix it.
I have project with some common interfaces; let's call it api. I have two other projects called runner and module that both use api as a dependency.
The job of runner is to dynamically load a module artifact (from a jar; it's a fat one that includes its dependencies) and then execute it. runner expects module to provide certain concrete implementations from api. To make sure that classes from different versions of module.jar don't clobber each other, I create a new classloader with a URL to module.jar, and set the parent classloader to the classloader of the class that loads and processes module.jar. This works without any issues.
The problem arose when I used runner as a dependency inside a webapp (a spring boot app to be specific), and quickly found that I couldn't load some classes from module.jar because they conflict with classes that already exist in the current classpath (from other dependencies in the webapp).
Since module.jar really only needs the classes from api, I thought that I could create a new URLClassLoader (without a parent) that only has classes from api.jar, and then use that as the parent classloader when I load up the module. This is where I started running into trouble:
CommonInterface commonInterface = null;
Class<CommonInterface> commonInterfaceClass = null;
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL, apiClassesClassLoader);
//...
//...
//clazz is a concrete implementation from module.jar
if(myClassLoader.loadClass(CommonInterface.class.getName()).isAssignableFrom(clazz)) {
commonInterfaceClass = clazz;
}
commonInterface = commonInterfaceClass.newInstance(); //ClassCastException
I understand that my original problem is due to the fact that the classloader first checks to see if the class has already been loaded before attempting to load it, which meant that when it was resolved using the name from module.jar, it was linking against an incompatible version of the class.
What's a good way to deal with this issue? Instead of creating a URL classloader that only has classes from api, does it make sense to create my own implementation that delegates to the parent only if the requested class is one from api?

You have loaded CommonInterface from two different class loaders. Classes with the same name but different class loaders are different classes to the JVM. (Even if the classes are 100% identical in the .class file - the problem is not incompatibility but the fact that they're from different class loaders)
If you do a
System.out.println(CommonInterface.class == myClassLoader.loadClass(CommonInterface.class.getName()));
You'll find that this prints false.
The way your create your classloader:
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL, apiClassesClassLoader);
.. would only work if apiClassesClassLoader is also a parent class loader of the class that contains this code.
You could try:
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL,
getClass().getClassLoader());
But from your description (it's a "fat" jar that contains its own dependencies) and the intricacies of the web classloader (child first) this may not solve your problem.
In that case, the only solution is to make your module jar "lean" to ensure that you only load each class once with one class loader only.

I forgot to update this question with my solution. I was able to solve this issue by creating a custom class-loader that extends URLClassLoader. This classloader does not have a parent.
I then overrode loadClass to control how classes were being loaded. I first check to see if the class exists in module.jar. If so, I load it from there. Otherwise, I load it using the current classloader. Since my custom classloader doesn't have a parent, it can load classes from module.jar even if they were already loaded by the main classloader, because they do not exist in my custom classloader's hierarchy.
The basic approach was like this:
public class MyClassLoader extends URLClassLoader {
private final ClassLoader mainClassLoader = MyClassLoader.class.getClassLoader();
private final Set<String> moduleClasses;
private MyClassLoader(URL url) {
super(new URL[]{ url });
try {
JarURLConnection connection = (JarURLConnection) url.openConnection();
this.moduleClasses = connection.getJarFile().stream()
.map(JarEntry::getName)
.filter(name -> name.endsWith(".class"))
.map(name -> name.replace(".class", "").replaceAll("/", "."))
.collect(Collectors.toSet());
} catch(IOException e) {
throw new IllegalArgumentException(String.format("Unexpected error while reading module jar: %s", e.getMessage()));
}
}
public static MyClassLoader newInstance(JarFile libraryJar) {
try {
return new MyClassLoader(new URL(String.format("jar:file:%s!/", libraryJar.getName())));
} catch(MalformedURLException e) {
throw new IllegalArgumentException(String.format("Path to module jar could not be converted into proper URL: %s", e.getMessage()));
}
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
if(moduleClasses.contains(name)) {
Class<?> clazz = findLoadedClass(name);
if(clazz != null) {
return clazz;
} else {
return findClass(name);
}
} else {
return mainClassLoader.loadClass(name);
}
}
}

How to add a class to the ClassPath at runtime?

A have a jar file which has a main method. I compile a new class and I want to add this new class to CP.
/folder
/my_jar_file.jar
/MyClass.class
How can I do that kind of thing in Java without recompiling of jar file?

Adding to class path is will not load your class at runtime.
You need to explicitly load your jar in the code as below,
URLClassLoader classLoader = new URLClassLoader (yourJar.toURL(), this.getClass().getClassLoader());
Class classToLoad = Class.forName ("com.company.abc", true, classLoader);

Classloader java.lang.NoClassDefFoundError?

So I have a classloader loading a class like so:
ClassLoader cl = new URLClassLoader(urls);
Class cls = cl.loadClass("modules.Test");
Method method = cls.getDeclaredMethod("getModule", noparams);
Class<?> type = method.getReturnType();
if(type.newInstance() instanceof Module){
System.out.println("Accessed field with type: Module");
}
The class Module is in another jar at runtime. And the Test.class was generated within that main jar then i unarchived it, so the dependency would be there.
How can I access other dependencies from the external .class file I have loaded?
The exception:
java.lang.NoClassDefFoundError: com/xxxxxxx/xxxx/objects/Module
Caused by: java.lang.ClassNotFoundException: com.xxxxxxx.xxxx.objects.Module

I think that the problem is happening because your modules.Test class depends the Modules class, but your custom class loader can't find that class.
I think that is because you have instantiated the custom classloader incorrectly. You wrote:
ClassLoader cl = new URLClassLoader(urls);
That creates a classloader whose parent classloader is the default system classloader. But the error implies that the default classloader is not the one that knows about Modules. Try this instead:
Classloader cl = new URLClassLoader(
urls, this.getClass().getClassLoader());
This should at least give you a classloader that knows about Modules.
Note: adding the URL for the JAR containing Modules to the urls array is a non-solution. You are liable to end up loading the Modules class twice, and that is liable to lead to other problems. (The instanceof won't work, for example.)