Develop Reference

Java cast from subclass to superclass under different classloader

I know that Class instance loaded by different class loader can't be cast to each other.
But what if the one Class extends the other? I did an experiment and the result is confusing. Here is the ClassLoader I define:
public class MyClassLoader extends ClassLoader {
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
try {
if (name.startsWith("java")) {
return super.loadClass(name);
}
String filename = "/" + name.replaceAll("\\.", "/") + ".class";
InputStream is = getClass().getResourceAsStream(filename);
if (is == null) {
return super.loadClass(name);
}
byte[] b = new byte[is.available()];
is.read(b);
return defineClass(name, b, 0, b.length);
} catch (Throwable e) {
throw new ClassNotFoundException(name);
}
}
}
And the experiment code:
// These classes will be loaded by MyClassLoader
class Parent { }
class Child extends Parent { }
class MyCalendarData_aa_DJ extends CalendarData_aa_DJ { }
class MyAppleScriptEngine extends AppleScriptEngine { }
class MyBufferedReader extends BufferedReader {
public MyBufferedReader(Reader in) {
super(in);
}
}
public class DifferentClassLoaderCast {
public static void main(String[] args) throws Exception {
ClassLoader classLoader = new MyClassLoader();
Class<?> pClass = classLoader.loadClass(Parent.class.getName());
Class<?> cClass = classLoader.loadClass(Child.class.getName());
// true, as pClass and cClass are loaded by same classloader
System.out.println(pClass.isAssignableFrom(cClass));
// false, different classloader
System.out.println(Parent.class.isAssignableFrom(cClass));
// true, why?
System.out.println(Object.class.isAssignableFrom(pClass));
Class<?> myCalendarData_aa_DJClass = classLoader.loadClass(MyCalendarData_aa_DJ.class.getName());
// false, CalendarData_aa_DJ is loaded by JAVA ext-classloader
System.out.println(CalendarData_aa_DJ.class.isAssignableFrom(myCalendarData_aa_DJClass));
Class<?> myAppleScriptEngine = classLoader.loadClass(MyAppleScriptEngine.class.getName());
// false, why? AppleScriptEngine is loaded by JAVA bootstrap-classloader
System.out.println(AppleScriptEngine.class.isAssignableFrom(myAppleScriptEngine));
Class<?> myBufferedReader = classLoader.loadClass(MyBufferedReader.class.getName());
// true, why? BufferedReader is loaded by JAVA bootstrap-classlaoder
System.out.println(BufferedReader.class.isAssignableFrom(myBufferedReader));
}
}
It seems that subclass loaded by MyClassLoader can be cast to superclass loaded by bootstrap class loader under package starts with java or built-in class?

// true, why?
System.out.println(Object.class.isAssignableFrom(pClass));
this one should be entirely obvious. Object is java.lang.Object and you rather clumsily call super.loadClass if the fully qualified name starts with java. Which means the loader of Object.class is the system loader, and this is true for all load ops: Whether classLoader loads Parent, or the system loader does, they both work off of the notion that j.l.Object.class is loaded by the system loader: The same type, therefore, compatible.
// false, why? AppleScriptEngine is loaded by JAVA bootstrap-classloader
System.out.println(AppleScriptEngine.class.isAssignableFrom(myAppleScriptEngine));
same reason. In reverse: the fully qualified name of AppleScriptEngine is not starting with "java".
Class<?> myBufferedReader = classLoader.loadClass(MyBufferedReader.class.getName());
// true, why? BufferedReader is loaded by JAVA bootstrap-classlaoder
System.out.println(BufferedReader.class.isAssignableFrom(myBufferedReader));
you guessed it. Because the FQN of BufferedReader starts with "java".
Perhaps you've misunderstood the classloading model.
The model that classloaders employ is a parent/child relationship. A classloader has a parent.
Any class is loaded by some classloader; if it hits any other class in its source code it will ask its own classloader to load it. But that loader may defer the job to any other loader. That's important. Your code will defer for any class whose FQN starts with "java" (and not even "java.", which is a peculiar choice). Otherwise, it loads itself. The classloader that is on record as THE loader of a class is the one that invoked defineClass. In your code, if you go via the if block that checks for starting with "java", your loader does NOT invoke defineClass, and therefore isn't the loader. If that if is not taken, you always end up invoking defineClass, making you the loader.
The common model for classloaders is this:
Ask your parent(s) to load the class, in order. If it can, great. We return that result, and that means the loader of said class is the parent and not you!
If not, then this loader will load it. Conflicts are unlikely; after all, the system loader couldn't even find it. Now you are the loader.
ClassLoader itself supports this model, but you get it by overriding findClass and NOT loadClass. The default impl of loadClass will do precisely as above: First calls the parents' loadClass methods, and only if those can't find it, will it invoke findClass to finish the job.
I strongly recommend you follow this flow, and update your code to extend findClass, not loadClass.
If you really want to load it yourself and NOT delegate to your parent loaders, then, yeah, overriding loadClass is how you do it. But now you have to deal with the fact that if it is a class that your parent can also find, that you can run into the scenario where your loader loaded, say, com.foo.Example, and parent did too, and whilst those classes have exactly the same name, as far as the JVM is concerned, they are completely unrelated and entirely incompatible with each other. The JVM doesn't mind, but it leads to highly confusing scenarios, where an object of type com.foo.Example cannot be assigned to a variable of type... com.foo.Example.
If you must do this, note that checking if it starts with "java" is highly suboptimal. For starters, "java." is a better fit, and for seconds, not all system classes start with "java". Ask the system loader first, if it can load it, defer to that (just return what it found), at the very least.
What are you trying to accomplish by writing a loader? With that insight, I can give more advice on which method (loadClass or findClass) is appropriate to override.

How to understand "Every Class object contains a reference to the ClassLoader that defined it. "?

I know I can get a the classloader of a class by
xxxclass.class.getClassLoader(), but where exactly does the xxxclass
hold the reference of its classloader who defines it?
e.g.
public class ClassA {
System.out.println("I am class A");
}
I don't see any clue of the classloader reference in ClassA. However,
I can get the classloader of ClassA by using
ClassA.class.getClassLoader().
How does ClassA.class.getClassLoader() work?

The sentence as it appears in the documentation of ClassLoader is:
Every Class object contains a reference to the ClassLoader that defined it.
Note the two links? What they tell you is that the documentation refers to the Class object, not to the plain object of class ClassA.
Every class that you define in Java has a Class object associated with it, which allows you to look at that class in the meta level. That is, treat the class itself as an object, pass it as parameter, use reflection on it etc.
As you have noticed, one way to access the Class object is use ClassA.class. If you have a reference to an object of type ClassA:
ClassA myObj = new ClassA();
Then you can get the Class object using myObj.getClass().
So there is no reference to the ClassLoader in the myObj object, Only in its associated Class object.
Now, the other link tells you how to get the reference to the ClassLoader object once you have a Class object - through the getClassLoader() method.
Now, if you look at the source code of the Class class, you will see:
#CallerSensitive
public ClassLoader getClassLoader() {
ClassLoader cl = getClassLoader0();
if (cl == null)
return null;
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
ClassLoader.checkClassLoaderPermission(cl, Reflection.getCallerClass());
}
return cl;
}
So it calls getClassLoader0() to get the class loader. Its source is:
native ClassLoader getClassLoader0();
That is, the class loader reference is actually part of the native structure of this Java class, and it is not available to see using Java language tools. Nevertheless, it exists there, and available to you through the aforesaid method.

If you write this:
public class Ball {
private Person thrower;
public Ball(Person thrower) {
this.thrower = thrower;
}
public Person getThrower() {return thrower;}
}
then every Ball object contains a reference to the Person that threw it, right?
Similarly, the Class class has something like this: (although I'm not showing how classLoader gets assigned)
public class Class {
... other stuff ...
private ClassLoader classLoader;
public ClassLoader getClassLoader() {return classLoader;}
... other stuff ...
}
and so every Class object has a reference to the ClassLoader that loaded it.
In your example, ClassA is not a Class object, so the statement doesn't apply to it. It does apply to ClassA.class which is a Class object (or refers to one at least).

In fact, it's not so simple in this case.
(Or at least, it was not so simple, until a recent update)
Java is a very high-level language, and the JVM is a rather complex beast, which is (fortunately!) hiding many details that you don't want to be concerned with when using a high-level, object-oriented language.
As already pointed out in the other answers, the Class#getClassLoader() method delegates to a private native method getClassLoader0(). Usually, you simply don't know (and should not have to care about) what a private native method does.
But thanks to the open source JDK, one can trace the path of this method call (here, for a recent version of the JDK8) :
The native getClassLoader0 method of Class is bound to JVM_GetClassLoader
The JVM_GetClassLoader method eventually calls Klass::class_loader()
The Klass::class_loader() method only returns the result of ClassLoaderData::class_loader()
The ClassLoaderData::class_loader() method finally returns the field that you are looking for
However, note that this has changed in a recent commit of the JDK9: Now, the ClassLoader is stored as a private instance field in the Class class, in order to improve performance.

If you look at the source code of java.lang.Class it appears that it delegates to a native method called getClassLoader0. So the implementation details are down to the JVM.
I'm no expert on this, but I suppose this might allow garbage collection to work by not having reference cycles in Java.

Dynamically creating a subclass at runtime

I'm currently developing a custom ORM framework and utilising ASM to dynamically generate sub classes at runtime. The generation process seems to complete OK, however when I try to instantiate the resulting class I'm getting a "NoClassDefFoundError".
The error seems to pertain to the Super class rather then the actual subclass. Here is an excerpt from the subclass generation method:
private Class generateProxyClass(Class anEntitySuperClass,
List<Field> fieldsToIntercept) throws ClassNotFoundException{
String entitySuperClassName = this.convertToInternalName(anEntitySuperClass.getName());
//String entityProxySubClassName = "com/flux/dynamic/".concat(anEntitySuperClass.getSimpleName()).concat("Proxy");
String entityProxySubClassName = anEntitySuperClass.getSimpleName().concat("Proxy");
ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_MAXS);
cw.visit(V1_6,ACC_PUBLIC+ACC_SUPER,entityProxySubClassName,null,entitySuperClassName,null);
cw.visitSource(entityProxySubClassName.concat(".java"),null);
//create constructor
MethodVisitor mv = cw.visitMethod(ACC_PUBLIC,"<init>","()V",null,null);
mv.visitCode();
//have our consturctor initailise its super class.
mv.visitVarInsn(ALOAD,0);
mv.visitMethodInsn(INVOKESPECIAL,entitySuperClassName,"<init>","()V");
mv.visitInsn(RETURN);
mv.visitMaxs(0,0);
mv.visitEnd();
this.generateAndAppendProxyAccessorMethods(anEntitySuperClass,fieldsToIntercept, cw);
cw.visitEnd();
//at this point our class should be fully generated an include all required fields. next we
//convert the class to a byte array and pass it in to our helper method to load an
//actual class from the byte array.
return this.loadProxyClass(cw.toByteArray(),entityProxySubClassName);
}
The "loadProxyClass" method called above is a helper method that basically instantiates
and calls a custom ClassLoader in order to load the dynamically created class:
/**loads the generated proxy class from the provided bytes. */
private Class loadProxyClass(byte[] aGeneratedProxyClass,String proxyClassName) throws ClassNotFoundException{
return new ProxyClassLoader(Thread.currentThread().getContextClassLoader(),aGeneratedProxyClass)
.loadClass(this.convertToExternalName(proxyClassName));
}
The ProxyClassLoader simply extends ClassLoader and overrides the "findClass" method in order to load the Dynamically Generated class bytes:
public class ProxyClassLoader extends ClassLoader {
private byte[] rawClassBytes;
public ProxyClassLoader(ClassLoader parentClassLoader,byte[] classBytes){
super(parentClassLoader);
this.rawClassBytes = classBytes;
}
#Override
public Class findClass(String name) {
return defineClass(name,this.rawClassBytes, 0,this.rawClassBytes.length);
}
}
The error I get is: Exception in thread "main" java.lang.NoClassDefFoundError: DummyEntity (wrong name: DummyEntityProxy)
Where the DummyEntity is the super class I pass into the generateProxyClass method and the DummyEntityProxy is the class I'm attempting to generate. I'm stumped, any help would be greatly appreciated.

Generally, it isn’t a good idea to implement a ClassLoader that tries to return the same class regardless of what it has been asked for. This is perfectly illustrated by the error you get: NoClassDefFoundError: DummyEntity (wrong name: DummyEntityProxy). The system asked your ClassLoader for a class named DummyEntity and you returned a class named DummyEntityProxy.
The remaining question is why your loader has been asked for that class as usually the parent loader is asked first. It seems that the parent loader has not found the super class which indicates that the parent class loader you have used (Thread.currentThread().getContextClassLoader()) has no access to your super class. It would have been easier if you used anEntitySuperClass.getClassLoader() as parent loader.
Of course, you have to ensure that all other classes used by your generated proxy are accessible by anEntitySuperClass’s class loader. If not, you might need a very complex loader delegation structure to make both group of classes available. It might even be impossible (that depends on what your proxy actually ought to do).

The problem is revealed by your exception's message:
Exception in thread "main" java.lang.NoClassDefFoundError: DummyEntity (wrong name: DummyEntityProxy)
Your class loader expected to load a class DummyEntity but the linked resource contained a class named DummyEntityProxy. How could that happen? It is your class loader's findClass method's implementation:
#Override
public Class findClass(String name) {
return defineClass(name, this.rawClassBytes, 0, this.rawClassBytes.length);
}
You do not distinguish what class is attempted to be loaded but you define any class of name with the only class it knows, the DummyEntityProxy's byte representation. Rather implement:
#Override
public Class findClass(String name) {
if (!name.equals(entityProxySubClassName)) {
throw new ClassNotFoundException(name);
}
return defineClass(name, this.rawClassBytes, 0, this.rawClassBytes.length);
}
This way, you are making sure that you are not defining a class of another name. It seems however as if the ProxyClassLoader should not be queried for the class in the first place but that one of its parents should have successfully resolved it.
It seems like ASM is quite a low-level API for your needs. Have you considered a more high-level API like for example my library Byte Buddy? Other ORMs like Hibernate or Eclipse link also use an API on that level, simply because the things you are struggling with are difficult to get right.

Thank you all very much for your suggestions. After many hours of tinkering I managed to resolve the error. It appears that the error was attributed to the method:
this.generateAndAppendProxyAccessorMethods(anEntitySuperClass,fieldsToIntercept, cw);
More specifically, some of the code generated by this method incorrectly referenced the super class by its simple name rather than its internal fully qualified class name. I omitted the implementation of this method from my question for brevity and also because I genuinely didn't expect that the problem was associated with this method. Generally, when errors occur in dynamically generated byte code logic it can be immensely difficult to pinpoint the cause, simply because JVM error messages are so ambiguous.

When exactly is NoClassDefFoundError thrown?

How do runtime dependencies in Java work exactly. For example, is code like this possible if Impl1 or Impl2 are not in the classpath at runtime:
Thinger t;
if (classIsAvailable(Impl1.class)) t = new Impl1();
else t = new Impl2();
t.doThing();
Or if there is no common interface:
if (classIsAvailable(Impl1.class)) Impl1.doThingThisWay();
else Impl2.doThingTheOtherWay();

You can't do it exactly like this, because in order to evaluate Impl1.class, said class must be available (i.e. loaded). You can however try to load a specific class by its name
Class aClass = classLoader.forName("Impl1");
If this does not fail (throw an Exception) you can create an instance of this class using newInstance().
Of course, in order to be able to use your class, you have to make sure it implements an Interface, which is known at compile time. In this case you can cast your created object to that interface type and continue using it.
This article has some sample code.

ClassNotFoundException is thrown when
an application tries to load in a class through its string name using:
* The forName method in class Class.
* The findSystemClass method in class ClassLoader .
* The loadClass method in class ClassLoader.
but no definition for the class with the specified name could be found.
You may also find http://www.xyzws.com/javafaq/what-does-classforname-method-do/17 useful.
HTH

Your example would fail with a NoClassDefFoundError when your class is loaded with either Impl1 or Impl2 not in the classpath, so none of the code would execute in that case.

load class not in classpath dynamically in web application - without using custom classloader

I am developing a web application.
The web application generates java classes on the fly. For example it generates class com.people.Customer.java
In my code, I dynamically compile this to get com.people.Customer.class and store in some directory say repository/com/people/Customer.class which is not on the classpath of my application server.My application server(I am using WebSphere Application Server/Apache Tomcat etc) picks up the classes from the WEB-INF/classes directory. The Classloader would use this to load the classes.
After compilation I need to load this class so that it becomes accessible to other classes using it after its creation.
When I use Thread.currentThread().getContextClassLoader().loadClass(com.people.Customer) obviously the Classloader is not able to load the class, since its not on the classpath(not in WEB-INF/classes). Due to similar reasons, getResource(..) or getResourceAsStream(..) also does not work.
I need a way to :
Read the class Customer.class maybe as a stream (or any other way would do) and then load it. Following are the constraints:
I cannot add the repository folder to the WEB-INF/classes folder.
I cannot create a new Custom ClassLoader. If I create a new ClassLoader and this loads the class, it will not be accessible to its parent ClassLoader.
Is there any way of achieving this?
If not this, in the worse case, is there a way of overriding the default class loader with a custom class loader for web applications the same classloader should be used to load applications throughout entire lifecycle of my web application.
Appreciate any solution :)

You need a custom class loader to do this, and in this classloader you need to re-define a method findClass(String name)
An example:
public class CustomClassLoader extends ClassLoader {
final String basePath = "/your/base/path/to/directory/named/repository/";
#Override
protected Class<?> findClass(final String name) throws ClassNotFoundException {
String fullName = name.replace('.', '/');
fullName += ".class";
String path = basePath + fullName ;
try {
FileInputStream fis = new FileInputStream(path);
byte[] data = new byte[fis.available()];
fis.read(data);
Class<?> res = defineClass(name, data, 0, data.length);
fis.close();
return res;
} catch(Exception e) {
return super.findClass(name);
}
}
}
Then, you'll be load classes from custom location. For example:
Class<?> clazz = Class.forName("my.pretty.Clazz", true, new CustomClassLoader());
Object obj = clazz.newInstance();
Doing this, you tell JVM that class named my.pretty.Clazz should be loaded by your custom class loader, which knows how and where from to load your custom class.
It resolves full class name (like my.pretty.Clazz) to file name (in our case: /your/base/path/to/directory/named/repository/my/pretty/Clazz.class), then loads obtained resource as a byte array, and finally converts this array to a Class instance.
This example is very simple and demonstrates a general technique about how to load custom classes as in your case.
I suggest you to read some articles about class loading, for example this one.

Short answer: No
Without a custom ClassLoader, you cannot dynamically load classes.
However, your assumption that you cannot use a custom ClassLoader because your other objects loaded by the WebApp ClassLoader would be unable to use these newly loaded classes is incorrect. All you need is a generic way to use these newly created classes - like a common interface or a meta-description (Beans Introspector for accessing bean properties).
But if you are using third-party libraries like Hibernate and you are dynamically loading entities at runtime which are to be persisted, then you will have a hard time, but imho it is possible.

Sure you can do this. Just get the web classloader and call the defineClass() method using reflection (it is protected, so be sure to call setAccessible(true) on the method. defineClass() takes a byte array, so it doesn't make any difference where you class is from. Make sure that the class name is unique and you're loading it only once, or you'll have complicated classloading problems.