I have a use case where I need to dynamically load and share a predefined set of packages/classes between multiple classloaders. The goal is to increase overall performance every time we load a jar file.
We are trying to load and execute "apps". Each app contains our SDK libraries. Each app that is loaded is done so in it's own URLClassLoader to keep it isolated from other code as well as to prevent library version conflicts. Since our SDKs libraries are in each and every app, we would like to cache these specific sets of packages/classes so that the next time we load another app, all of the SDK classes do not need to be loaded again and thus removing this overhead.
Since we have several versions of our SDK out there, we are attempting to do this dynamically. Such that, when an app is loaded, we already know what version of the SDK it is. So we would like to have some implementation of a parent Classloader that is specific to this sdk version (we will call it the SDKCacheClassLoader). When the URLClassLoader (or some custom subclass of it) loads and recognizes a package/class from the sdk, we would like to forward that class to the shared parent SDKCacheClassLoader. Then, the next time we load another jar file with the URLClassLoader (setting SDKCacheClassLoader as the shared parent classloader), the parent should already have that class found and loaded from the last execution and the new jar file should not need to be scanned looking for the SDK classes again.
Anyone have any ideas on how to accomplish this goal?
Also, to be clear, I do not need to share instantiated objects, just simply the class definition since I understand that the JVM considers the same classes loaded by 2 different class loaders as completely separate classes.
Related
I have a custom URLClassLoader which loads a couple of classes from jar files outside the normal classpath. So far, so good.
My problem is that I can't call this ClassLoader each and every time when I need a class loaded by my ClassLoader because these classes might by used by third party libs. For that reason I added the classes to the current Thread ClassLoader by reflection:
ProtectionDomain pd = getClass().getProtectionDomain();
final ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
final java.lang.reflect.Method clM = ClassLoader.class.getDeclaredMethod("defineClass", Class.class);
clM.invoke(classLoader, className, byteContent, 0, byteContent.length, pd);
But when it comes to a call to findClass inside that Thread ClassLoader and the requested class has not been added by me yet, I get a NoClassDefFoundException. Of course.
My question is now, is there a way to put my ClassLoader in the global class loading chain of my application? Using the Java option for system ClassLoader is not possible as my ClassLoader can be contained in a web application and deployed as a war file. I also tried using Thread.currentThread().setContextClassLoader() but once a new Thread is created, it of course doesn't have my ClassLoader set.
I searched the web for days but couldn't find any solution.
Thanks for your help and any suggestions.
Best regards,
Gerry
Not really, especially if you want this to be compatible with running as a web application.
You've got multiple problems to solve here. Web containers are permitted to run you under a security manager that will make your code not work (i.e., they can forbid access to the filesystem and/or forbid the creation of ClassLoaders). Java was designed to allow the safe loading of mobile code (it was originally targeting, of all things, cable set-top boxes), and so you can imagine the ability for an application to modify the system class loader on the fly was not part of the architecture.
For the application, what you ought to be doing is either (1) putting your JAR files from which you're specially loading these classes in the system classpath, or (2) loading your application in a custom class loader, then having that custom class loader load your classes (in essentially the same way you are now). Load the third-party libraries in the same class loader. Now you're going to be OK, regardless of what you mean by "might be used by third-party libraries" (this was confusing to me and I am not sure what technique they might be using, and it matters).
For the webapp, you should bundle the needed classes in the web application. You're defeating the web application design by attempting to load classes from elsewhere; the idea of the web application and .war is to make these things self-contained.
So you're doing this the wrong way. If you insist on doing it the wrong way, there may be various non-portable hacks you can start using in order to get your code working properly in particular situations, but I'm only going to write one answer for now and see what you think. :)
I'm using Play 2.2.2, and I have an external jar that tries to load an XML resource from the same jar's root. It does so using System.class.getClassLoader().getResource("/Blabla.xml").
This fails, since apparently Play has a weird hierarchy of classloaders: ReloadableClassLoader and several parents. This hierarchy doesn't include the system class loader, nor the extensions and bootstrap classloaders.
How is this possible?
I found out that using ClassLoader.getSystemClassLoader() returns a classloader hierarchy that resembles the standard classloading hierarchy. But I would prefer not to touch the external jar mentioned above, and make System.class.getClassLoader().getResource("/Blabla.xml") work.
How can I make System.class.getClassLoader().getResource("/Blabla.xml") return the standard system classloader?
Play's classloader set-up is rather complex due to the compile-reload mechanism in development mode. Rather than trying to use the normal System.class.getClassLoader() you may want to use built-in API methods to get to a resource or to the classloader.
I have one jar file in my application's class path. At run time, I add new classes to the jar file and sometimes also modify the fields/methods of the existing classes. Currently I am using URLClassLoader to load the classes dynamically. The new classes added dynamically are loaded correctly and I am able to use them at runtime. But it fails to reload the existing classes that are modified at runtime. I read many articles which states we need to explicitly handle reloading because class once loaded will not be reloaded until all the references to the class are destroyed. Also I tried out sample code that I found but none of them worked.
Can anyone suggest me a proper approach for reloading ? Any sample code for the same will be highly appreciated.
Normally to reload a class you need to unload the entire class loader. i.e. remove all references to all classes loaded for that class loader.
Another option is to use instrumentation to change the byte code of an existing class. This usually comes with limitations and changing fields is something you cannot do. i.e. the objects of that type would have to be translated somehow.
What I normally do is have services which are very quick to start/restart. This way to you easily restart a process which needs updated code ideally by pressing the Run in my IDE. This minimises deployment time as well.
In principle, a class that has already been loaded cannot be reloaded with the same classloader.
For a new load, it is necessary to create a new classloader and thus load the class.
Using URLClassLoader has one problem and that is that the jar file remains open.
If you have multiple classes loaded from one jar file by different instances of URLClassLoader and you change the jar file at runtime, you will usually get this error: java.util.zip.ZipException: ZipFile invalid LOC header (bad signature). The error may be different.
In order for the above errors not to occur, it is necessary to use the close method on all URLClassLoaders using the given jar file. But this is a solution that actually leads to a restart of the entire application.
A better solution is to modify the URLClassLoader so that the contents of the jar file are loaded into the RAM cache. This no longer affects other URLClassloaders that read data from the same jar file. The jar file can then be freely changed while the application is running. For example, you can use this modification of URLClassLoader for this purpose: in-memory URLClassLoader
I have solution split into 2 projects:
The independent project contains interface ExampleInf and declares some services needed for the application. Those services are provided by the third party API (Hadoop client API). This project contains GUI components and other application logic but does not link third party libraries that provide services declared by ExampleInf. There is no class implementing ExampleInf in this project.
The dependent project that contains links to third party libraries. This project contains class ExampleImpl that encapsulates third party API and implements ExampleInf.
In the independent project there is class (let's call it class A) that consumes (uses) services declared by ExampleInf. Because the independent does not link the dependent project, in order to use ExampleInf it needs to load its implementation ExampleImpl dynamically in runtime. Also it needs to dynamically load all the third party libraries required by ExampleImpl.
Currently this is done by a bunch of constants (public static final String attributes) that contain paths to dependent project where dynamically loaded resources are located and a lot of messy ClassLoader code. I do not consider this to be a good solution. Is there any pattern, best practice or common way how this can be done? What would you recommend in your experience?
This pattern reminds me a bit of dependency injection in Java EE. At least I think it is good idea to externalize the locations of classes and libraries (.jar-s) that need to be loaded dynamically to XML and then load them all in cycle instead of calling ClassLoader.loadClass separately for each constant. Is there any nice clean way how to load XML in the same package and load classes and jars specified by that XML? Code example would be much appreciated.
You can use the ServiceLoader utility to do this (this is how many of the jdk services are loaded, e.g. xml libraries and modern jdbc driver libraries). If the dependent project is part of the classpath at startup, then you are good to go (assuming it is setup correctly). otherwise, you would need to load the dependent project in a nested classloader and pass that to the load(Class,ClassLoader) method (or set the classloader as the current context classloader before calling load(Class)).
I'm working on a web application where I have dependencies on two different jars containing two different versions of the same class. The jar files are supplied by an external vendor and cannot be changed.
I've created a custom class loader, which first first tries to load classes from a specific set of jars, and if that fails it just loads the class in the standard manner. This makes it possible to ensure that a specific set of jar files are always used first. This solves my problem.
However, I was wondering if there was an easier way.
Other than rearchitecting your app for OSGi, I'd say that's the best solution.