I am trying to use a custom class loader to load all the dependencies needed for the application. I've implemented the customerClassLoader following the site: https://www.javacodegeeks.com/2013/03/java-handmade-classloader-isolation.html
However, I dont understand how to tell my application to use the custom classLoader whenever needed.
For instance: Lets say, I have a method to make http request like below. How can I tell the application to use the custom classLoader to load the required jars?
private HttpResponse get() {
HttpClient client = HttpClientBuilder.create().build();
HttpGet request = new HttpGet(url);
HttpResponse response = client.execute(request);
return response;
}
Java uses ClassLoader implicitly when you use new, import keyword, the jvm will use the current class's classloader to load the dependent classes, so you can use the custom classloader to load a bootstrap class explicitly by using classloader.loadclass, and the bootstrap just runs a method belonging to your target class instance. An example follows.
There is a class Target that depends on the class DateFormatter which is included in the spring-context, and has a method named start.
import org.springframework.format.datetime.DateFormatter;
public class Target {
private static DateFormatter dateFormatter;
public void start(){
System.out.println(this.getClass().getClassLoader());
dateFormatter=new DateFormatter();
System.out.println(dateFormatter);
}
}
Next, we compile and package the above code as a jar named target.jar, which is stored at D:\\test\\target.jar.
Next, we declare a class BootStrap in another jar that will call the method start of Target instance. The BootStrap class will dynamically load the target.jar and spring-context jar files by the same classloader which is a URLClassLoader instance. Because of this, the method start in Target instance can access the DateFormatter class that is defined in spring-context.
public class BootStrap {
public static void main(String[] args) throws Exception{
URL url = new URL("http://maven.aliyun.com/nexus/content/groups/public/org/springframework/spring-context/4.3.1.RELEASE/spring-context-4.3.1.RELEASE.jar?spm=0.0.0.0.kG1Pdw&file=spring-context-4.3.1.RELEASE.jar");
URL url2= (new File("D:\\test\\target.jar").toURI().toURL());
URLClassLoader classLoader = new URLClassLoader(new URL[]{url,url2});
Class<?> clz = classLoader.loadClass("com.zhuyiren.Target");
Object main = clz.newInstance();
Method test = clz.getMethod("start");
test.invoke(main);
}
}
Finally, run the BootStrap main method. There are two important thing:
The BootStrap class and Target class don't belong to a same jar file.
The target.jar is not stored in CLASSPATH path.
These 2 point can make sure that the AppClassLoader can not find and load the Target class. Because of the mechanism of class loader, jvm will use the custom load the Target. Of course, you can guarantee it by changing the URLClassLoader classLoader = new URLClassLoader(new URL[]{url,url2}); to URLClassLoader classLoader = new URLClassLoader(new URL[]{url, url2}, ClassLoader.getSystemClassLoader().getParent());
And we can see the result:
java.net.URLClassLoader#e9e54c2
org.springframework.format.datetime.DateFormatter#4dd8dc3
That means we can access the DateFormatter instance which is defined in spring-context jar file successfully, while the spring-context is not stored in CLASSPATH, but we are using the custom clasloader to load and use it.
Related
I have a main method that creates custom classloader and instantiates a class, called Test, with it.
public class App {
public static void main(String[] args) throws Exception {
try {
Class.forName("com.mycompany.app2.Test2"); // We ensure that Test2 is not part of current classpath
System.err.println("Should have thrown ClassNotFound");
System.exit(1);
} catch (ClassNotFoundException e) {
// ignore
}
String jar = "C:\\experiments\\classloader-test2\\target\\classloader-test2-1.0-SNAPSHOT.jar"; // Contains Test2
URL[] classPaths = new URL[] { new File(jar).toURI().toURL() };
ClassLoader classLoader = new URLClassLoader(classPaths, App.class.getClassLoader());
Thread.currentThread().setContextClassLoader(classLoader);
Class.forName("com.mycompany.app2.Test2", true, classLoader); // Check that custom class loader can find the wanted class
Test test = (Test) Class.forName("com.mycompany.app.Test", true, classLoader).getDeclaredConstructor().newInstance();
test.ex(); // This throws ClassNotFound for Test2
}
}
This class then itself instantiates another class that is not part of the original classpath, but is part of the custom one.
public class Test {
public void ex() {
new Test2().test();
}
}
In my understanding of classloader, since Test was created with the custom classloader any class loadings within should be done with the same loader. But this does not seem to be the case.
Exception in thread "main" java.lang.NoClassDefFoundError: com/mycompany/app2/Test2
at com.mycompany.app.Test.ex(Test.java:7)
at com.mycompany.app.App.main(App.java:28)
What do I need to do in the main method to make Test#ex work, without changing Test?
I'm using Java 17.
You create the URLClassLoader using App.class.getClassLoader() as the parent class loader. Hence, the request to load Test through the custom class loader is resolved through the parent loader, ending up at exactly the same class you’d get with Test.class in your main method.
You could pass a different parent loader, e.g. null to denote the bootstrap loader, to forbid resolving the Test class though the parent loader but this would result in either of two unhelpful scenarios
If the custom class loader has no com.mycompany.app.Test class on its own, the loading attempt would simply fail.
If the custom class loader has a com.mycompany.app.Test class, i.e. inside classloader-test2-1.0-SNAPSHOT.jar, it would be a different class than the Test class referenced in your main method, loaded by the application class loader. In this case, the type cast (Test) would fail.
In other words, the Test class referenced by you main method can not be affected by another, unrelated class loader at all.
There is an entirely different approach which may work in some scenarios. Do not create a new class loader, when all you want to do, is to inject a new class.
byte[] code;
try(var is = new URL("jar:file:C:\\experiments\\classloader-test2\\target\\" +
"classloader-test2-1.0-SNAPSHOT.jar!/com/mycompany/app2/Test2.class").openStream())
{
code = is.readAllBytes();
}
MethodHandles.lookup().defineClass(code);
Test test = new Test();
test.ex();
This adds the class to the current class loading context, so subsequent linking will succeed. With the following catches:
No previous attempt to link this class must have been made so far
It only works for a classes without dependencies to other absent classes (as there’s no class loader resolving those to the jar file outside the class path).
In some cases, when the dependencies are non-circular or resolved lazily, you could add all the classes with multiple define calls, if you know which you need and in which order.
The class must be in the same package, otherwise, you’d have to move the lookup context to the right package, with the documented restrictions
An entirely different approach to add the classes to the existing environment, would be via Java Agents, as they can add jar files to the class path.
We have a reporting aplication that by default generates pdf outputs, but you can write your own classes to generate any other output format. This way I have generated xls files using apache poi 10.0. But, now came a request to generate xlsx file. When I try to create a workbook with this code:
XSSFWorkbook wbTemplate=new XSSFWorkbook()
I got the error:
java.lang.NoSuchMethodError: org.apache.xmlbeans.XmlOptions.setSaveAggressiveNamespaces()Lorg/apache/xmlbeans/XmlOptions;
I have discovered that the application already uses a very old version of the xmlbeans file that of course doesn't contain the above method. First I tryed to replace the xml bean file with a newer version just in case I have luck, but the application freezes.
My next idea is to use classLoader and when the app runs my class to generate the xlsx file I load the above method. To do so I have implemented this solution found on the internet:
URL[] classLoaderUrls = new URL[]{new URL("file:/C:/HOME/Installs/Apache POI/poi-3.10/ooxml-lib/xmlbeans-2.6.0.jar")};
URLClassLoader urlClassLoader = new URLClassLoader(classLoaderUrls);
Class<?> beanClass = urlClassLoader.loadClass("org.apache.xmlbeans.XmlOptions");
Constructor<?> constructor = beanClass.getConstructor();
Object beanObj = constructor.newInstance();
Method[] m=beanClass.getMethods();
Method method = beanClass.getMethod("setSaveAggressiveNamespaces");
method.invoke(beanObj);
But what a surprise when it wants to get the "setSaveAggressiveNamespaces" method name I got again the error that this function doesn't exist.
Then I have written into a file all the function names of this class and it is true, that name doesn't exist. But exist another one called "setSaveAggresiveNamespaces" with one S! If I invoke this it works, but of course when I wan't to create the XSSF workbook I still get the message that the setSaveAggressiveNamespaces (with double SS) doesn't exist.
But the setSaveAggressiveNamespaces should be in the class since this is coming with the apache poi package.
What can I do in this case to make it work?
The application runs under java 1.6
Thanks in advance for the answers.
UPDATE
Axel, this is how I load now the class:
public void customClassLoader() throws Exception
{
URL[] classLoaderUrls = new URL[]{new URL("file:/C:/HOME/Installs/Apache POI/poi-3.10/ooxml-lib/xmlbeans-2.3.0.jar")};
URLClassLoader urlClassLoader = new URLClassLoader(classLoaderUrls,null);
Class<?> beanClass = urlClassLoader.loadClass("org.apache.xmlbeans.XmlOptions");
log("RESOURCES:" +beanClass.getResource("/org/apache/xmlbeans/XmlOptions.class"));
Constructor<?> constructor = beanClass.getConstructor();
Object beanObj = constructor.newInstance();
Method[] m=beanClass.getMethods();
for (int i=0;i<m.length;++i)
log("QQQ:" +String.valueOf(i)+".: "+ m[i].getName());
Method method = beanClass.getMethod("setSaveAggressiveNamespaces");
method.invoke(beanObj);
}
And then I call the above function at the first row of the class that generates the report. There is nothing before it.
The RESOURCE is written in the log as:
"RESOURCES:jar:file:/C:/HOME/Installs/Apache POI/poi-3.10/ooxml-lib/xmlbeans-2.3.0.jar!/org/apache/xmlbeans/XmlOptions.class"
URLClassLoader(java.net.URL[]) states:
Constructs a new URLClassLoader for the specified URLs using the
default delegation parent ClassLoader.
So the default delegation parent ClassLoader will also be used and so the org.apache.xmlbeans.XmlOptions will be loaded from there if found and not from the additional given URL.
So we need not using the default delegation parent ClassLoader. URLClassLoader(java.net.URL[], null) is doing this.
Example:
import java.net.URL;
import java.net.URLClassLoader;
import java.lang.reflect.Constructor;
public class UseURLClassLoader {
public static void main(String[] args) throws Exception {
URL[] classLoaderUrls;
URLClassLoader urlClassLoader;
Class<?> beanClass;
classLoaderUrls = new URL[]{new URL("file:/home/axel/Dokumente/JAVA/poi/poi-3.10.1/ooxml-lib/xmlbeans-2.6.0.jar")};
urlClassLoader = new URLClassLoader(classLoaderUrls); //default delegation parent ClassLoader is used
beanClass = urlClassLoader.loadClass("org.apache.xmlbeans.XmlOptions");
System.out.println(beanClass.getResource("/org/apache/xmlbeans/XmlOptions.class")); //class is loaded using default parent class loader
URL context = new URL("file:/home/axel/Dokumente/JAVA/poi/poi-3.10.1/");
classLoaderUrls = new URL[] {
new URL(context, "poi-3.10.1-20140818.jar"),
new URL(context, "poi-ooxml-3.10.1-20140818.jar"),
new URL(context, "poi-ooxml-schemas-3.10.1-20140818.jar"),
// maybe others also necessary
new URL(context, "lib/commons-codec-1.5.jar"),
// maybe others also necessary
new URL(context, "ooxml-lib/xmlbeans-2.6.0.jar")
// maybe others also necessary
};
for (int i = 0; i < classLoaderUrls.length; i++) {
System.out.println(classLoaderUrls[i]);
}
urlClassLoader = new URLClassLoader(classLoaderUrls, null); //set default parent class loader null
beanClass = urlClassLoader.loadClass("org.apache.xmlbeans.XmlOptions");
System.out.println(beanClass.getResource("/org/apache/xmlbeans/XmlOptions.class")); //class is loaded using this class loader
}
}
For me called as follows:
axel#arichter:~/Dokumente/JAVA/poi/poi-4.0.0$ java -cp .:./*:./lib/*:./ooxml-lib/* UseURLClassLoader
it produces:
jar:file:/home/axel/Dokumente/JAVA/poi/poi-4.0.0/ooxml-lib/xmlbeans-3.0.1.jar!/org/apache/xmlbeans/XmlOptions.class
file:/home/axel/Dokumente/JAVA/poi/poi-3.10.1/poi-3.10.1-20140818.jar
file:/home/axel/Dokumente/JAVA/poi/poi-3.10.1/poi-ooxml-3.10.1-20140818.jar
file:/home/axel/Dokumente/JAVA/poi/poi-3.10.1/poi-ooxml-schemas-3.10.1-20140818.jar
file:/home/axel/Dokumente/JAVA/poi/poi-3.10.1/lib/commons-codec-1.5.jar
file:/home/axel/Dokumente/JAVA/poi/poi-3.10.1/ooxml-lib/xmlbeans-2.6.0.jar
jar:file:/home/axel/Dokumente/JAVA/poi/poi-3.10.1/ooxml-lib/xmlbeans-2.6.0.jar!/org/apache/xmlbeans/XmlOptions.class
So at first the class is loaded using default parent class loader. For me it loads org.apache.xmlbeans.XmlOptions farther from the newer xmlbeans-3.0.1.jar. For you it loads farther from the older xmlbeans-1.*.jar. That is because those jars are in class path of the default parent class loader.
The second code part then sets the default parent class loader null and so class is loaded only using this class loader.
But messing around with the class loaders is a mess. As implied in my code, having the default parent class loader set null, we need giving the current class loader all the needed class sources. This often becomes very expensive. So not having the old jars in the class path will always be the better solution than messing around with the class loaders.
My project structure is the following (very simplified of course):
So under lib-ext i download on a daily basis from a Jenkins server 2 jar files 'jar1 and jar2' to be checked by my program, i need one file from 'jar1' lets call it: "Class2Bloaded".
The issue is that this file implements an interface that is to be found in 'jar2', lets call this 'Dependency'
What i would like to do is, from my class under src "ClassThatLoads.java", load 'Class2Bloaded.class' and tell the class loader to look into 'jar2' to search for the implementing interface "Dependency.class"
My code so far (omitting exceptions handling):
//Create the URL pointing to Jar1
private URL getJarUrl(JarFile jarFile)
{
return new File(jarFile.getName()).toURI().toURL();
}
URL jar1Url = getJarUrl(jar1);
ClassLoader jar1classLoader = new URLClassLoader(new URL[] { jar1Url });
Class<?> Class2Bloaded = Class.forName(fullClassName, false, jar1classLoader );
So the problem happens within the Class.forName invocation, because the class i want to load implements an interface that is in jar 2.
Exception in thread "main" java.lang.NoClassDefFoundError: com/packagewithinJar2/Dependency
So eventually i have prepared another class loader that points to 'jar2', and i have even got the actual Interface i need:
URL jar2Url = getJarUrl(jar2);
ClassLoader jar2classLoader = new URLClassLoader(new URL[] { jar2Url });
Class<?> Interface2Bloaded = Class.forName(fullClassName, false, jar2classLoader );
Where 'fullClassName' in the second case is the fully qualified name of the interface from which 'Class2Bloaded' depends on.
Is just that i cant find anything in the javadocs of ClassLoader that allows me to 'inject' an additional class loader for the dependencies.
I hope my explanation is clear.
The first thing to do would be to add jar2 to the list of jars your URLClassLoader reads:
ClassLoader jarclassLoader = new URLClassLoader(new URL[] { jar1Url, jar2Url });
BUT the normal thing to do would be to add jar1 and jar2 on your classpath from the beginning.
To do so you would use the -cp parameter of the java executable.
for example, if you compile your classes into the bin directory:
java -cp libext/jar1.jar:libext/jar2.jar:bin ClassThatLoads
That way, you could use the classes seamless in your own java source and get rid of the cumbersome loading part :
public class ClassThatLoads {
public static void main(String[] args) {
Class2Bloaded stuff = new Class2Bloaded();
//use stuff from here...
}
}
I'd like to create a classloader that extends classpath by several third-party jars. Currently my code looks like this:
public class Main {
public static void main(String[] args) throws ClassNotFoundException {
ClassLoader originalClassLoader = Main.class.getClassLoader();
URL[] jarUrls = ...;
ClassLoader customClassLoader = URLClassLoader.newInstance(jarUrls, originalClassLoader);
Class classFromJar = customClassLoader.loadClass("class.from.Jar");
originalClassLoader.equals(classFromJar.getClassLoader()); // true
customClassLoader.equals(classFromJar.getClassLoader()); // false
// I want classFromJar.getClassLoader() to return customClassLoader
}
}
The problem is that even though the class loading using ClassLoader#loadClass works fine, method #getClassLoader of loaded classes returns original classloader instead of the custom one. Such state causes calls like classFromJar.newInstance() to throw ClassNotFoundExeption because originalClassLoader doesn't know about additional jars and so dependencies of classFromJar can't be loaded.
How to make custom classloader to propagate itself to results of Class#getClassLoader for loaded classes?
I'd like to achieve that purely in runtime, without customizing "java.class.path" system property or JVM -Djava.system.class.loader switch.
This is a classloader issue that I am struggling with. I understand the root cause of the issue (different classloaders), but I'm not sure about the best way to fix it.
I have project with some common interfaces; let's call it api. I have two other projects called runner and module that both use api as a dependency.
The job of runner is to dynamically load a module artifact (from a jar; it's a fat one that includes its dependencies) and then execute it. runner expects module to provide certain concrete implementations from api. To make sure that classes from different versions of module.jar don't clobber each other, I create a new classloader with a URL to module.jar, and set the parent classloader to the classloader of the class that loads and processes module.jar. This works without any issues.
The problem arose when I used runner as a dependency inside a webapp (a spring boot app to be specific), and quickly found that I couldn't load some classes from module.jar because they conflict with classes that already exist in the current classpath (from other dependencies in the webapp).
Since module.jar really only needs the classes from api, I thought that I could create a new URLClassLoader (without a parent) that only has classes from api.jar, and then use that as the parent classloader when I load up the module. This is where I started running into trouble:
CommonInterface commonInterface = null;
Class<CommonInterface> commonInterfaceClass = null;
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL, apiClassesClassLoader);
//...
//...
//clazz is a concrete implementation from module.jar
if(myClassLoader.loadClass(CommonInterface.class.getName()).isAssignableFrom(clazz)) {
commonInterfaceClass = clazz;
}
commonInterface = commonInterfaceClass.newInstance(); //ClassCastException
I understand that my original problem is due to the fact that the classloader first checks to see if the class has already been loaded before attempting to load it, which meant that when it was resolved using the name from module.jar, it was linking against an incompatible version of the class.
What's a good way to deal with this issue? Instead of creating a URL classloader that only has classes from api, does it make sense to create my own implementation that delegates to the parent only if the requested class is one from api?
You have loaded CommonInterface from two different class loaders. Classes with the same name but different class loaders are different classes to the JVM. (Even if the classes are 100% identical in the .class file - the problem is not incompatibility but the fact that they're from different class loaders)
If you do a
System.out.println(CommonInterface.class == myClassLoader.loadClass(CommonInterface.class.getName()));
You'll find that this prints false.
The way your create your classloader:
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL, apiClassesClassLoader);
.. would only work if apiClassesClassLoader is also a parent class loader of the class that contains this code.
You could try:
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL,
getClass().getClassLoader());
But from your description (it's a "fat" jar that contains its own dependencies) and the intricacies of the web classloader (child first) this may not solve your problem.
In that case, the only solution is to make your module jar "lean" to ensure that you only load each class once with one class loader only.
I forgot to update this question with my solution. I was able to solve this issue by creating a custom class-loader that extends URLClassLoader. This classloader does not have a parent.
I then overrode loadClass to control how classes were being loaded. I first check to see if the class exists in module.jar. If so, I load it from there. Otherwise, I load it using the current classloader. Since my custom classloader doesn't have a parent, it can load classes from module.jar even if they were already loaded by the main classloader, because they do not exist in my custom classloader's hierarchy.
The basic approach was like this:
public class MyClassLoader extends URLClassLoader {
private final ClassLoader mainClassLoader = MyClassLoader.class.getClassLoader();
private final Set<String> moduleClasses;
private MyClassLoader(URL url) {
super(new URL[]{ url });
try {
JarURLConnection connection = (JarURLConnection) url.openConnection();
this.moduleClasses = connection.getJarFile().stream()
.map(JarEntry::getName)
.filter(name -> name.endsWith(".class"))
.map(name -> name.replace(".class", "").replaceAll("/", "."))
.collect(Collectors.toSet());
} catch(IOException e) {
throw new IllegalArgumentException(String.format("Unexpected error while reading module jar: %s", e.getMessage()));
}
}
public static MyClassLoader newInstance(JarFile libraryJar) {
try {
return new MyClassLoader(new URL(String.format("jar:file:%s!/", libraryJar.getName())));
} catch(MalformedURLException e) {
throw new IllegalArgumentException(String.format("Path to module jar could not be converted into proper URL: %s", e.getMessage()));
}
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
if(moduleClasses.contains(name)) {
Class<?> clazz = findLoadedClass(name);
if(clazz != null) {
return clazz;
} else {
return findClass(name);
}
} else {
return mainClassLoader.loadClass(name);
}
}
}