I've some JavaDoc doclet, just rewritten for the new Java Doclet API, which is run by the Maven plugin, which does this:
1) Loads some jars from a directory and adds them to the context class path. This is like:
URL[] urls = <all .jar URLs>
URLClassLoader xClsLoader = URLClassLoader.newInstance ( urls, Thread.currentThread ().getContextClassLoader () );
Thread.currentThread ().setContextClassLoader ( xClsLoader );
2) Searches for all the instances of a root class and, for each of them, tries to pick a given annotation:
String classFQNs = ...
for ( String fqn: classFQNs )
{
Class dynCls = Thread.currentThread ().getContextClassLoader ().loadClass ( fqn );
Status statusAnnotation = dynCls.getAnnotation ( Status.class );
if ( statusAnnotation == null )
// DAMN IT! IT'S ALWAYS NULL!
}
The problem seems to be that Status.class is both in the classpath loaded by Maven together with the plugin, AND in the additional .jar that I add to the classpath seen by the doclet. I cannot remove it from the Maven Java Javadoc's dependencies, cause that would break the code doing 2), if I don't load that same .jar in 1), the JavaDoc scan fails, because many classes in the other jars involved in 2) extend a root class in this troubling .jar.
Yes, I could rearrange the code, split the .jar, use an alternative to javadoc, etc, but all of those options would imply huge changes in a legacy project, which isn't be best quality code you can imagine.
So, what's going on? This used to work up to Java 8, I understand Java >= 9 now sees the same class coming from different jars as different classes, even different unnamed modules, so that the annotation in the loaded class isn't considered the same as the one that the doclet tries to fetch, despite Status is exactly the same, just loaded differently, once by Maven the other time by my code in 1). How could I overcome that? Is there another way to load those .jars? Is there a way to reconcile those same annotations?
Related
What I understand is that if I use:
Instrumentation#getAllLoadedClasses()
I do get a selection of all loaded classes by the target JVM. But If I do:
Class.forName("my.class.name")
This will not be the same class as the class loaded by VM. Yes, I can add this particular class as a jar in the agent MANIFEST.MF Class-Path - but that does not look the same to me as getAllLoadedClasses().
Could someone please confirm whether this is correct i.e. I would not be able to find a specific class using Class.forName() when instrumenting? My objective was not to iterate over all loaded classes using getAllLoadedClasses() - But if there is no alternative, I guess that's okay for now.
** UPDATE
What I made a mistake in writing is the Boot-Class-Path which I have now corrected in my manifest. Using -verbose:class logging I managed to see that my jars are being loaded as
[Opened C:\fullpath\someother.jar]
[Opened C:\fullpath\another.jar]
[Opened C:\fullpath\different.jar]
But I don't see any corresponding loading information. I tried adding a Class.forName("a.package.in.someother.jar.classname") and got NoClassDefFoundError. As soon as I jump into the agent jar, I cannot use Class.forName() to check if the class is loaded by the target VM. I am getting a NoClassDefFoundError.
FURTHER UPDATE
Okay I have "Fattened" the manifest to look up all classes in my WEB-INF/lib and tomcat's lib directory. What I can see is below:
1) When my custom class MyClass is loaded for the first time. -verbose shows:
[Loaded my.pkg.MyClass from file:/C:/base/webapps/ROOT/WEB-INF/lib/mypkg.jar]
2) If I try to load the class again, it is correctly showing the above order.
3) My agent jar is manifested with all classes for my tomcat lib and my web-inf/lib directory. And I can also confirm that the loader sees the jars correctly.
4) Now I inject the agent, and call Class.forName("my.pkg.MyClass") from within the agent class. I get the below results.
[Loaded my.pkg.MyClass from file:/C:/base/webapps/ROOT/WEB-INF/lib/mypkg.jar]
I acknowledge that it's system class loader loding it inside my agent code as #RafaelWinterhalter pointed out in one of his answers. Is there any way I can force a "Delegation" so that the a different classloader loads the agent class and therefore, correctly redefines a class.
Any help is appreciated.
As it is stated in the javadoc:
Invoking this method is equivalent to:
Class.forName(className, true, currentLoader)
where currentLoader denotes the defining class loader of
the current class.
You can also see from the source code that the method is marked #CallerSensitive which means that you get a different result based on the class loader that invokes the method.
When calling Instrumentation::getAllLoadedClasses, the returned array contains classes of any class loader and not only of the current class loader which is the system class loader when running a Java agent. Therefore:
for (Class<?> type : instrumentation.getAllLoadedClasses()) {
assert type == Class.forName(type.getName());
}
is not generally true.
After a bit of run around, and Thanks to #Holger who reminded me what the problem was - incorrect class loader.
before I inject the agent, I have done the following:
// Get the current context class loader, which is app ext. classLoader
ClassLoader original = Thread.currentThread().getContextClassLoader().getSystemClassLoader();
// Set the system classloader to app classloader which won't delegate anything
Field scl = ClassLoader.class.getDeclaredFields();
scl.setAccessible(true);
scl.set(null, Thread.currentThread().getContextClassLoader());
// Now inject agent
try {
vm.loadAgent(agentPath, args);
} catch (all sorts of errors/exceptions in chain) {
// Log them and throw them back up the stack.
} finally {
vm.detach();
// Put back the classLoader linkage
sc.set(null, original);
}
How I have confirmed
When it goes in my Agent Class - Thread.currentThread().getContextClassLoader() becomes my application extn loader. But the system classloader now becomes `ParallelWebappClassLoader".
I am assuming this is how it works, but could be totally worng:
i) When I say Class.forName("my.pkg") it will check the system class loader which is pointing to my loader now. If the class is not found (i.e. not loaded), it will go to parents etc. I believe this is more or less the delegation model is.
ii) In this way, the class is loaded in VM by the same class loader which would also load the class in my webapp under normal circumstances.
iii) I will not instrument anything else apart from my own classes so the classloader will always be the same.
So far I have not seen any LinkageError happening. But I still feel this is too risky and if I break the link I am screwed.
Using Class.forName in a java profiler must be avoided to escape from the NoClassDef error. JVM Loads the class files in the different level of class loaders based on their classpath setting and the class file demand.
Java Cre libraries + Boot path defended libraries will be loaded in bootstrap Level
Java Agent will be loaded in system level and it goes on. Class.forName() will look the class files from the parent loaders, the current loader will not check the child loader (Until unless we implement our own loaders)
Java core classes will be accessible from your application code but our application code will not be accessible by the Java core classes. Its called class loader hierarchy.
You have three options.
Lookup the class files from the Instrumentation.GetLoadedClassFiles()
Through Transformers you can get all the loaders classes and you can track them and look for your class in every loader until you find.
Have the Class.forname implementation in the lowest level of the hierarchy so that it can internally access all the path.
Maintain the hierarchy properly to avoid too many weird errors.
Assuming you are looking for the Class<?> in order to re-transform a class, it seems to me that you could save the ClassLoader passed to your transformer, and later use ClassLoader.loadClass(String). Something like:
class MyTransformer implements ClassFileTransformer {
Map<String, ClassLoader> _name2loader = new ...;
...
public byte[] transform(ClassLoader loader, String className,
Class<?> classBeingRedefined,
ProtectionDomain pd,
byte[] classfileBuffer) throws ... {
...
_name2loader.put(className.replace("/","."), classLoader);
...
}
...
Class<?> getClass(String name) throws ClassNotFoundException {
ClassLoader cl = _name2loader.get(name);
if (cl == null) {
throw ClassNotFoundException("No loader for class " + name);
}
return cl.loadClass(name);
}
}
Note that the className passed to transform uses slashes, not dots... A better alternative than the String.replace may be to actually read the class name from the classfileBuffer using your bytecode library (such as javaassist or ASM, but if you're transforming bytecode, you're likely already using such a library).
Note: I'm not sure if you'd see the same class being passed for transformation with different ClassLoaders, but it would be good to look out for that (or research it).
In a custom plugin (or task) I would like to read all compiled classes (preferrably those that have changed from last compilation) with a classloader so that I'll be able to use reflection on them.
Is that possible?
1) It would be great to have a cook right after a Java class was compiled so that I could read it, but I found no way to do this.
2) I'm thinking of something like this ...
compileJava.doLast {
ClassLoader parent = getClass().getClassLoader();
GroovyClassLoader loader = new GroovyClassLoader(parent);
// retrieve all class files
// for each class file, loader.parseClass(classFile)
}
In a gradle script getClass().getClassloader() will get you the classloader of the gradle script. This will NOT contain the compiled classes or compile/runtime jars. I think you want to do something similar to:
Collection<URL> urls = sourceSets.main.runtimeClasspath.files.collect { it.toURI().toURL() }
Classloader parent = new URLClassLoader(urls.toArray());
If you want to only act on the classes that have changed you are best to do do that in an incremental task
I have a web service we'll call service.war. It implements an interface we'll call ServicePluginInterface. During the startup of service.war, it reads in environment variables and uses them to search for a jar (MyPlugin.jar). When it finds that jar, it then uses a second environment variable to load the plugin within the jar. The class that it loads looks like this:
public class MyPlugin implements ServicePluginInterface {...}
The servlet attempts to load the plugin using code like:
try {
if (pluginClass == null) {
plugin = null;
}
else {
ZipClassLoader zipLoader = new ZipClassLoader(Main.class.getClassLoader(), pluginJar);
plugin = (ServicePluginInterface)zipLoader.loadClass(pluginClass).newInstance();
plugin.getAccount(null,null);
}
} catch (Exception e) {
...
}
The trick is that I don't have source or a jar for ServicePluginInterface. Not wanting to give up so easily, I pulled the class files out of the service.war files. By using those class files as dependencies, I was able to build, without compiler warnings, MyPlugin. However, when actually executed by Tomcat, the section of code above generates a runtime exception:
java.lang.ClassCastException: com.whatever.MyPlugin cannot be cast to com.whomever.ServicePluginInterface
As a second point of reference, I am also able to construct a synthetic class loader (separate java executable that uses the same class loading mechanism. Again, since I do not have the original source to ServicePluginInterface, I used the class files from the WAR. This second, synthetic loader, or faux-servlet if you will, CAN load MyPlugin just fine. So I would postulate that the Tomcat JVM seems to be detecting some sort of difference between the classes found inside the WAR, and extracted class files. However, since all I did to extract the class files was to open the WAR as a zip and copy them out, it is hard to imagine what that might be.
Javier made a helpful suggestion about removing the definition of ServicePluginInterface, the problem with that solution was that the ZipClassLoader that the servlet uses to load the plugin out of the jar overrides the ClassLoader findClass function to pull the class out of the JAR like so:
protected Class<?> findClass(String name) throws ClassNotFoundException
{
ZipEntry entry = this.myFile.getEntry(name.replace('.', '/') + ".class");
if (entry == null) {
throw new ClassNotFoundException(name);
}
...
}
The class ZipClassLoader then recursively loads all parent objects and interfaces from the jar. This means that if the plugin jar does not contain the definition for ServicePluginInterface, it will fail.
Classes defined by different class loaders are different:
At run time, several reference types with the same binary name may be
loaded simultaneously by different class loaders. These types may or
may not represent the same type declaration. Even if two such types do
represent the same type declaration, they are considered distinct. JLS
In that case zipLoader returns an instance of MyPlugin that implements the other ServicePluginInterface (is it loaded from the zip too?):
(ServicePluginInterface)zipLoader.loadClass(pluginClass).newInstance();
It seems that the application server already has a definition of ServicePluginInterface, then you don't need to redeploy it. It should be enough to add the required files (ServicePluginInterface, etc.) as non-deployed dependecies of your project.
Another approach goes by living with the fact, and accessing methods in ServicePluginInterface via reflection (use the Class object returned by zipLoader, instead of ServicePluginInterface.class).
Problem
I'm writing a standalone utility program which, given a jar containing a JPA-2 annotated persistence unit, needs to programmatically get a list of all my #Entity classes in a particular persistence unit.
I'd like to decide which of 2 approaches would be the way to go to get this information, and why; or if there is another better way I haven't thought of.
Solution 1
Java program puts jar on the classpath, creates persistence unit from the classes in the jar using JavaSE methodologies. Then it uses the javax.persistence classes to get the JPA Metamodel, pull back list of class tokens from that.
EntityManagerFactory emf = Persistence.createEntityManagerFactory("MY_ PERSISTENCE_UNIT");
Metamodel mm = emf.getMetamodel();
// loop these, using getJavaType() from Type sub-interface to get
// Class tokens for managed classes.
mm.getManagedTypes();
Solution 2
Program scan the directories and files inside the specified jar for persistence.xml files, then finds one with the specified persistence unit name. Then XPath the file to get the list of <class> XML elements and read the fully qualified class names from there. From names, build class tokens.
Constraints/Concerns
I'd like to go with approach 1 if possible.
This utility will NOT run inside a container, but the jar is an EJB project designed to run inside one. How will this be a problem?
The utility will have Open-EJB available on the classpath to get implementations of all the Java EE 6 classes.
Even though the EJB project is built to run on Hibernate, the utility should not be Hibernate-specific.
Are there any stumbling blocks?
In case anyone's interested, Solution 1 worked. Here's essentially what I had to do:
public MySQLSchemaGenerator() throws ClassNotFoundException {
Properties mySQLDialectProps = new Properties();
mySQLDialectProps.setProperty("javax.persistence.transactionType", "RESOURCE_LOCAL");
mySQLDialectProps.setProperty("javax.persistence.jtaDataSource", "");
final EntityManagerFactory emf = Persistence.createEntityManagerFactory("<persistence_unit_name>", mySQLDialectProps);
final Metamodel mm = emf.getMetamodel();
for (final ManagedType<?> managedType : mm.getManagedTypes()) {
managedType.getJavaType(); // this returns the java class of the #Entity object
}
}
The key was to override my transaction type and blank out the jtaDataSource which had been defined in my persistence.xml. Turns out everything else was unnecessary.
If Your jar is well-formed (persistence.xml at the right place - in the META-INF folder), then all looks fine.
It is not necessary to run your utility inside a container, JPA is not a part of JavaEE specs.
I have an application that allows, using an abstract class, people to write their own implementations. I load these implementations as .class-files from a directory. Currently, I have this solution:
File classDir = new File("/users/myproject/classes/");
URL[] url = { classDir.toURI().toURL() };
URLClassLoader urlLoader = new URLClassLoader(url);
String filename;
for (File file : classDir.listFiles()) {
filename = string.getFilenameWithoutExtension(file);
if (filename.equals(".") || filename.equals("..") || filename.startsWith("."))
continue;
AbstractClass instance = (AbstractClass)urlLoader
.loadClass("org.mypackage." + filename)
.getConstructor(ConfigUtil.class, DatabaseUtil.class, StringUtil.class)
.newInstance(config, database, string));
instance.doSomething();
}
As you see - I need to specify the package the classes are located in in order to correctly load them. Omitting the package, I get an
java.lang.NoClassDefFoundError:
MyClass (wrong name: org/mypackage/MyClass)
error.
Now, from a architectural POV, I think it is very ill-designed that classes other people designed have to be compiled to MY package when loading them.
So I ask you: Is there a way I can load classes form the file system without having to specify the package they reside in?
Yes; implement an interface (or use an annotation).
Then use any class-scanning library (there are lots of SO questions about this, like this one) to load the particular class in question. Searching for "Java class scanning" or "Java plugin mechanism" will help.
You might also just want to use the Java Plugin Framework and avoid some effort. Although it's not clear to me that it's maintained any more, I know people are still using it.
You can use the ServiceProvider to load implementations which you don't know.