Here is the problem I'm running into. There's a huge legacy application that runs on java 1.3 and uses external API, say, MyAPI v1.0. The exact implementation of MyAPI 1.0 is located somewhere in the classpath used by the app. There's also a mechanism that allows that app to use external code (some kind of plugin mechanism). Now I have another java library (MyLib.jar) that uses MyAPI v2.0 (which is NOT 100% backward compatible with v1.0) and I have to use it from the original app using that plugin mechanism. So I have to somehow let two (incompatible!) versions of the same API work together. Specifically, I want to use MyAPI v2.0 when API classes are invoked from MyLib.jar classes and use MyAPI 1.0 in all other cases.
MyAPI 1.0 is in the classpath, so it will be used by default, that's ok. I can create my own version of class loader to load classes from MyAPI 2.0 - no problem. But how do I fit it all together? Questions:
MyLib.jar objects instantiate a lot(!) of instances of classes from MyAPI 2.0. Does this mean that I will have to do ALL these instantiations via reflection (specifying my own classloader)? That's a hell of a work!
If some of MyAPI 2.0 objects gets instantiated and it internally instantiates another object from MyAPI, what classloader will it use? Would it use my class loader or default one?
Just generally, does my approach sound reasonable? Is there a better way?
Let's start from answering your 2nd question: when referring from some class to another class it will be loaded by the same classloader that loaded the original class (unless of course the classloader didn't succeed in finding the class and then it will delegate to its parent classloader).
Having said that, why won't your entire MyLib.jar be loaded by a custom classloader, and then it can refer to the newer version of the API in a regular way. Otherwise you have a problem, because you will have to work with the Object type and reflection all the way through.
You need to be careful with class loaders. If you do what you were suggesting, you would almost always end up MyAPI 1.0 even when using your class loader for MyAPI 2.0. The reason for this is how classes are loaded using the class loader. Classes are always loaded from the parent class loader first.
"The ClassLoader class uses a delegation model to search for classes and resources. Each instance of ClassLoader has an associated parent class loader. When requested to find a class or resource, a ClassLoader instance will delegate the search for the class or resource to its parent class loader before attempting to find the class or resource itself. The virtual machine's built-in class loader, called the "bootstrap class loader", does not itself have a parent but may serve as the parent of a ClassLoader instance. " (http://java.sun.com/javase/6/docs/api/java/lang/ClassLoader.html)
To provide isolation between the two APIs properly, you would need 2 class loaders (or 2 in addition to the main application one).
Parent - System classloader
|- Classloader1 - Used to load MyAPI 1.0
|- Classloader2 - Used to load MyAPI 2.0
Now to your questions. What you probably want to do is move most of the logic that uses the API into the classloaders. In addition to MyAPI 1.0/2.0, you should load the part of the application that uses those. Then the parent application just has to call a method that uses the API. This way you make a single reflection call to start the application and everything inside that application just uses standard references.
You can do this with a fancy ClassLoader without reflection.
Basically, the classloader has to say "if the loading class is from this jar, load from classpath B, otherwise use the main ClassLoader".
It's a little more complicated than that, but if you start from that idea you'll get it worked out.
This sounds reasonable. In the [API javadoc for loadClass][1] it says:
"Loads the class with the specified binary name. The default implementation of this method searches for classes in the following order:
Invoke findLoadedClass(String) to check if the class has already been loaded.
Invoke the loadClass method on the parent class loader. If the parent is null the class loader built-in to the virtual machine is used, instead.
Invoke the findClass(String) method to find the class."
If CL1 is for MyAPI1, CL2 is for MyAPI2, and CL3 is for MyLib it sounds like you want them to be checked in the order: CL3, CL2, CL1. Which from the above quote (as parents are checked first) suggests you want CL1 to have parent CL2, and CL2 to have parent CL3. As the constructor with a parent classloader is protected, you'll have to use URL classloader with the parents set appropriately.
Something like
URLCLassLoader cl3 = new URLClassLoader(new URL[]{ path to MyLib});
URLCLassLoader cl2 = new URLClassLoader(new URL[]{ path to API2}, cl3);
URLCLassLoader cl1 = new URLClassLoader(new URL[]{ path to API1}, cl2);
then use cl1 everywhere.
[1]: http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ClassLoader.html#loadClass(java.lang.String, boolean)
Related
I have a problem loading class from JAR file.
I can load them with URLClassLoader using a JarFile etc etc like in this answer; but if later in code I try to instantiate them with reflection:
Object obj = Class.forName(className).newInstance()
I get a ClassNotFoundException.
Can I dinamically load a class to use them later, when I need them, just like classes in ClassPath?
Thank you!
You need to provide class loader to Class.forName method - as otherwise it will look in this same class loader as your class is in.
Object obj = Class.forName("name", true, loader).newInstance() .
But you can't just load a class and then use it in your code like MyLoadedType - as here java does not know where to look for that class, unless you will ensure that your code and loaded code is in this same class loader - you can do this by running all your code from custom class loader that allows for adding new sources in runtime. (URLClassLoader allows for this but method is protected - so you need to extend it and make it public, in java 8 system class loader is also URLClassLoader - but this was changed in java 9) .
But you can operate on that code using reflections like I showed you.
What I understand is that if I use:
Instrumentation#getAllLoadedClasses()
I do get a selection of all loaded classes by the target JVM. But If I do:
Class.forName("my.class.name")
This will not be the same class as the class loaded by VM. Yes, I can add this particular class as a jar in the agent MANIFEST.MF Class-Path - but that does not look the same to me as getAllLoadedClasses().
Could someone please confirm whether this is correct i.e. I would not be able to find a specific class using Class.forName() when instrumenting? My objective was not to iterate over all loaded classes using getAllLoadedClasses() - But if there is no alternative, I guess that's okay for now.
** UPDATE
What I made a mistake in writing is the Boot-Class-Path which I have now corrected in my manifest. Using -verbose:class logging I managed to see that my jars are being loaded as
[Opened C:\fullpath\someother.jar]
[Opened C:\fullpath\another.jar]
[Opened C:\fullpath\different.jar]
But I don't see any corresponding loading information. I tried adding a Class.forName("a.package.in.someother.jar.classname") and got NoClassDefFoundError. As soon as I jump into the agent jar, I cannot use Class.forName() to check if the class is loaded by the target VM. I am getting a NoClassDefFoundError.
FURTHER UPDATE
Okay I have "Fattened" the manifest to look up all classes in my WEB-INF/lib and tomcat's lib directory. What I can see is below:
1) When my custom class MyClass is loaded for the first time. -verbose shows:
[Loaded my.pkg.MyClass from file:/C:/base/webapps/ROOT/WEB-INF/lib/mypkg.jar]
2) If I try to load the class again, it is correctly showing the above order.
3) My agent jar is manifested with all classes for my tomcat lib and my web-inf/lib directory. And I can also confirm that the loader sees the jars correctly.
4) Now I inject the agent, and call Class.forName("my.pkg.MyClass") from within the agent class. I get the below results.
[Loaded my.pkg.MyClass from file:/C:/base/webapps/ROOT/WEB-INF/lib/mypkg.jar]
I acknowledge that it's system class loader loding it inside my agent code as #RafaelWinterhalter pointed out in one of his answers. Is there any way I can force a "Delegation" so that the a different classloader loads the agent class and therefore, correctly redefines a class.
Any help is appreciated.
As it is stated in the javadoc:
Invoking this method is equivalent to:
Class.forName(className, true, currentLoader)
where currentLoader denotes the defining class loader of
the current class.
You can also see from the source code that the method is marked #CallerSensitive which means that you get a different result based on the class loader that invokes the method.
When calling Instrumentation::getAllLoadedClasses, the returned array contains classes of any class loader and not only of the current class loader which is the system class loader when running a Java agent. Therefore:
for (Class<?> type : instrumentation.getAllLoadedClasses()) {
assert type == Class.forName(type.getName());
}
is not generally true.
After a bit of run around, and Thanks to #Holger who reminded me what the problem was - incorrect class loader.
before I inject the agent, I have done the following:
// Get the current context class loader, which is app ext. classLoader
ClassLoader original = Thread.currentThread().getContextClassLoader().getSystemClassLoader();
// Set the system classloader to app classloader which won't delegate anything
Field scl = ClassLoader.class.getDeclaredFields();
scl.setAccessible(true);
scl.set(null, Thread.currentThread().getContextClassLoader());
// Now inject agent
try {
vm.loadAgent(agentPath, args);
} catch (all sorts of errors/exceptions in chain) {
// Log them and throw them back up the stack.
} finally {
vm.detach();
// Put back the classLoader linkage
sc.set(null, original);
}
How I have confirmed
When it goes in my Agent Class - Thread.currentThread().getContextClassLoader() becomes my application extn loader. But the system classloader now becomes `ParallelWebappClassLoader".
I am assuming this is how it works, but could be totally worng:
i) When I say Class.forName("my.pkg") it will check the system class loader which is pointing to my loader now. If the class is not found (i.e. not loaded), it will go to parents etc. I believe this is more or less the delegation model is.
ii) In this way, the class is loaded in VM by the same class loader which would also load the class in my webapp under normal circumstances.
iii) I will not instrument anything else apart from my own classes so the classloader will always be the same.
So far I have not seen any LinkageError happening. But I still feel this is too risky and if I break the link I am screwed.
Using Class.forName in a java profiler must be avoided to escape from the NoClassDef error. JVM Loads the class files in the different level of class loaders based on their classpath setting and the class file demand.
Java Cre libraries + Boot path defended libraries will be loaded in bootstrap Level
Java Agent will be loaded in system level and it goes on. Class.forName() will look the class files from the parent loaders, the current loader will not check the child loader (Until unless we implement our own loaders)
Java core classes will be accessible from your application code but our application code will not be accessible by the Java core classes. Its called class loader hierarchy.
You have three options.
Lookup the class files from the Instrumentation.GetLoadedClassFiles()
Through Transformers you can get all the loaders classes and you can track them and look for your class in every loader until you find.
Have the Class.forname implementation in the lowest level of the hierarchy so that it can internally access all the path.
Maintain the hierarchy properly to avoid too many weird errors.
Assuming you are looking for the Class<?> in order to re-transform a class, it seems to me that you could save the ClassLoader passed to your transformer, and later use ClassLoader.loadClass(String). Something like:
class MyTransformer implements ClassFileTransformer {
Map<String, ClassLoader> _name2loader = new ...;
...
public byte[] transform(ClassLoader loader, String className,
Class<?> classBeingRedefined,
ProtectionDomain pd,
byte[] classfileBuffer) throws ... {
...
_name2loader.put(className.replace("/","."), classLoader);
...
}
...
Class<?> getClass(String name) throws ClassNotFoundException {
ClassLoader cl = _name2loader.get(name);
if (cl == null) {
throw ClassNotFoundException("No loader for class " + name);
}
return cl.loadClass(name);
}
}
Note that the className passed to transform uses slashes, not dots... A better alternative than the String.replace may be to actually read the class name from the classfileBuffer using your bytecode library (such as javaassist or ASM, but if you're transforming bytecode, you're likely already using such a library).
Note: I'm not sure if you'd see the same class being passed for transformation with different ClassLoaders, but it would be good to look out for that (or research it).
I am trying to debug a very strange class error by looking at the ClassLoaders for some dynamically created components. ClassLoaders are something I've never played much with - and im surprised that standard JDK classes have null Class loader instances.
Can somebody explain the output of this simple main method in terms of the classes whose loaders I am attempting to print, and also more generally:
the way ClassLoaders work on the JVM and
how we can debug missing classes using ClassLoaders.
public class MyClass {
/**
* #param args
*/
public static void main(String[] args) {
System.out.println(relfect.MyClass.class.getClassLoader());
System.out.println(String.class.getClassLoader());
System.out.println(ArrayList.class.getClassLoader());
System.out.println(JButton.class.getClassLoader());
System.out.println(System.class.getClassLoader());
Boolean b = new Boolean(true);
System.out.println(b.getClass().getClassLoader());
}
}
Output
sun.misc.Launcher$AppClassLoader#1f7182c1
null
null
null
null
null
The javadoc for getClassLoader() says
Returns the class loader for the class. Some implementations may use null to represent the bootstrap class loader. This method will return null in such implementations if this class was loaded by the bootstrap class loader.
So, that at least explains why you get that result. But it does not explain why the implementors decided to do it that way.
EDIT:
After testing adding my own classes to the bootclasspath then they also show up as null class loader.
Classloader of the bootstrap classes is null, it's not a java class.
Do not mistake the classes found of the classpath and the ones loaded by the bootstrap loader. The latter is responsible for the core JDK classes usually found in rt.jar. It's a native classloader, hence no reference towards.
The classes on the classpath are loaded by the System classloader, and the class of it can be specified via property.
Morealso the null classloader is considered a security issue and there are checks based on the caller class having null classloader.
This is how it works . Whenever JVM try to load any class it's checks below conditions.
If Class is loaded from Bootstrap ClassPath i.e; jdk\jre\lib\rt.jar , BootStrap ClassLoader will be called.
If Class is loaded from Extension Classpath i.e; jdk\jre\lib\ext*.jar , Extension ClassLoader will be called.
If Class is loaded from Application ClassPath i.e; as specified in Environment Variable , Application ClassLoader is called .
Since Bootstrap ClassLoader is not implemented in java , it's either implemented in c or c++ so there is no reference for it that's why it returns null . But Extension and Application class Loader is written in java so you will get the reference as sun.misc.Launcher$ExtClassLoader#someHexValue and sun.misc.Launcher$AppClassLoader#someHexValue .
So, if you do something like this System.out.println(String.class.getClassLoader()) you will get null since this class is been called by BootStrap ClassLoader, On the other hand if you do the same thing for a class in Ext or App Class path you will get $ExtClassLoader#someHexValue and sun.misc.Launcher$AppClassLoader#someHexValue respectively .
In Java docs I see such statements:
The system class loader's resource
lookup algorithm is used to find the
resource.
But where can I learn how this "lookup algorithm" actually works, especially in context of Android.
you have a hierarchy of class loaders in java, e.g.:
Bootstrap CL
|
|
Extension CL
|
|
SystemClassLoader CL
|
|
Application Specific CL
When invoking a CL's findClass() Method a ClassLoader normally delegates to it's parent first which will try to load the Class.
E.g. if you're trying to load a Class which is in the ext-folder of your JVM you could use
Class clazz=ClassLoader.getSystemClassLoader().findClass("org.pack.ExtClass");
this will first delegate to it's parent, the Extension ClassLoader, which will delegate to it's parent, the Bootstrap CL. The Bootstrap CL will return null to the Extension ClassLoader since it could not find the class, then the Extension CL will look for the Class on it's own. Since it will find the class "org.pack.ExtClass" in the classpath it will load, prepare and return the class to the SystemClassLoader, which will set the reference clazz to the returned class object.
So as a general rule a ClassLoader will always delegate to it's parent first and the try to find the class on it's own.
But there are Exceptions to this rule, the most prominent being the WebApp Classloaders of Tomcat et al. where the WebAppClassLoader behaves a bit differently. I think there's a bootstrap.jar in the Tomcat Directory with a CL which will load classes first and only delegate to it's parent if the requested class could not be found.
Some Links:
http://www.developer.com/java/other/article.php/2248831/Java-Class-Loading-The-Basics.htm
http://tomcat.apache.org/tomcat-5.5-doc/class-loader-howto.html
http://www.ibm.com/developerworks/java/library/j-dyn0429/
hope that helped..
In Android (Dalvik) system classloader is PathClassLoader.
You will have to look at its source code to figure out the algo. It would be a some form of tree traversal across available Jar files and directories.
How do you change the CLASSPATH of a Java process from within the Java process?
Before you ask me "Why would you want to do that?" I'll explain it shortly.
When you have a Clojure REPL running it is common to need more jars in your CLASSPATH to load a Clojure source file, and I'd like to do it without having to restart Clojure itself (which is not really an option when using it on Slime on Emacs).
That's the reason but I don't want this question tagged as some-weird-language some-weird-editor and be disregarded by the majority of Java developers that may have the answer.
Update Q4 2017: as commented below by vda8888, in Java 9, the System java.lang.ClassLoader is no longer a java.net.URLClassLoader.
See "Java 9 Migration Guide: The Seven Most Common Challenges"
The class loading strategy that I just described is implemented in a new type and in Java 9 the application class loader is of that type.
That means it is not a URLClassLoader anymore, so the occasional (URLClassLoader) getClass().getClassLoader() or (URLClassLoader) ClassLoader.getSystemClassLoader() sequences will no longer execute.
java.lang.ModuleLayer would be an alternative approach used in order to influence the modulepath (instead of the classpath). See for instance "Java 9 modules - JPMS basics".
For Java 8 or below:
Some general comments:
you cannot (in a portable way that's guaranteed to work, see below) change the system classpath. Instead, you need to define a new ClassLoader.
ClassLoaders work in a hierarchical manner... so any class that makes a static reference to class X needs to be loaded in the same ClassLoader as X, or in a child ClassLoader. You can NOT use any custom ClassLoader to make code loaded by the system ClassLoader link properly, if it wouldn't have done so before. So you need to arrange for your main application code to be run in the custom ClassLoader in addition to the extra code that you locate.
(That being said, cracked-all mentions in the comments this example of extending the URLClassLoader)
And you might consider not writing your own ClassLoader, but just use URLClassLoader instead. Create a URLClassLoader with a url that are not in the parent classloaders url's.
URL[] url={new URL("file://foo")};
URLClassLoader loader = new URLClassLoader(url);
A more complete solution would be:
ClassLoader currentThreadClassLoader
= Thread.currentThread().getContextClassLoader();
// Add the conf dir to the classpath
// Chain the current thread classloader
URLClassLoader urlClassLoader
= new URLClassLoader(new URL[]{new File("mtFile").toURL()},
currentThreadClassLoader);
// Replace the thread classloader - assumes
// you have permissions to do so
Thread.currentThread().setContextClassLoader(urlClassLoader);
If you assume the JVMs system classloader is a URLClassLoader (which may not be true for all JVMs), you can use reflection as well to actually modify the system classpath... (but that's a hack;)):
public void addURL(URL url) throws Exception {
URLClassLoader classLoader
= (URLClassLoader) ClassLoader.getSystemClassLoader();
Class clazz= URLClassLoader.class;
// Use reflection
Method method= clazz.getDeclaredMethod("addURL", new Class[] { URL.class });
method.setAccessible(true);
method.invoke(classLoader, new Object[] { url });
}
addURL(new File("conf").toURL());
// This should work now!
Thread.currentThread().getContextClassLoader().getResourceAsStream("context.xml");
I don't believe you can - the right thing to do (I believe) is create a new classloader with the new path. Alternatively, you could write your own classloader which allows you to change the classpath (for that loader) dynamically.
There's no need to write your own class loader! There's clojure.lang.DynamicClassLoader.
http://blog.japila.pl/2011/01/dynamically-redefining-classpath-in-clojure-repl/
You may want to look into using java.net.URLClassLoader. It allows you to programmatically load classes that weren't originally in your classpath, though I'm not sure if that's exactly what you need.
It is possible as seen from the two links below, the method VonC gives seems to be the best but check out some of these posts and google for "Java Dynamic Classpath" or "Java Dynamic Class Loading" and find out some info from there.
I'd post in more depth but VonC has pretty much done the job.
From Dynamic loading of class and Jar files.
Also check this sun forum post.
String s="java -classpath abcd/ "+pgmname+" "+filename;
Process pro2 = Runtime.getRuntime().exec(s);
BufferedReader in = new BufferedReader(new InputStreamReader(pro2.getInputStream()));
is an example of changin the classpath in java program