I'm using a Java Agent (Agent.class) to transform a method in a program (Program.class) in a way that includes a call to the Agent class.
public Program.getMultiplier()F:
ALOAD 1
ALOAD 2
FDIV
+ INVOKESTATIC Agent.getCustomMultiplier(F)F
FRETURN
I've inspected the class loaders and their parents of both Agent and Program classes, and their hierarchy looks like this:
Agent.class: AppClassLoader <- PlatformClassLoader <- null
Program.class: URLClassLoader <- PlatformClassLoader <- null
When the Program executes the added INVOKESTATIC instruction, it throws a ClassNotFoundException -- it cannot find the Agent class as it was loaded by a different class loader.
As a temporary solution, I've tried forcing AppClassLoader to become a parent of URLClassLoader with reflection, which works in older Java versions but has been removed since Java 12.
Is there a more reliable way to make sure my Agent class is visible from any class loader?
You can add classes to the bootstrap class loader using appendToBootstrapClassLoaderSearch. This makes the classes of the specified jar file available to all classes whose defining class loader follows the standard delegation pattern.
But this requires the classes to be packaged in a jar file. When you specify the Agent’s own jar file, you have to be aware that classes loaded through the bootstrap loader are distinct from the classes loaded through the app loader, even when they originate from the same jar file. Further, the classes loaded by the bootstrap loader must not have dependencies to classes loaded by by other class loaders.
If your getCustomMultiplier method is supposed to interact with the running Agent, you have to separate the agent and the class containing this method.
Have your Agent listen to the creation of new ClassLoaders and then attach instances of them to the new ClassLoaders.
This way you preserve your "Agent listens to ClassLoader" interface, even if it now extends beyond the one platform class loader you expected the Agent to listen to.
You may be able to do something specific that works for URLClassLoader, but not all classes are loaded by an instance of URLClassLoader. Any OSGi project won't, most web servers also use their own classloaders in order to support hot reload, etc.
As far as I know there's no way to just casually update some 'global parent of all classloaders' or inject one; there's no such parent, and even if there was, a classloader is free to ignore its parent entirely.
Therefore the general answer is: No, you can't do that.
But, let's get our hacking hats on!
You're an agent already. One of the things you get to do as agent is to 'witness' classes as they are being loaded. Just invoke .addTransformer on the instance of Instrumentation you get in your agentmain and register one.
When you notice the Program class being loaded, do the following:
Take the bytecode and toss it through ASM, BCEL, Bytecode Buddy, or any other java 'class file reader/transformer' framework.
Also open up a class from within your agent's code (I wouldn't use Agent itself, I'd make a class called ProgramAddonMethods or whatnot as a container - everything inside is for the program to use / for your agent to 'inject' into that program.
Add every static member in ProgramAddonMethods directly to Program. As you do so, modify the typename on all accesses (both INVOKESTATIC and the read/write field opcodes) where the etypename is ProgramAddonMethods and make it the fully qualified name of the targeted class instead.
inject the INVOKESTATIC as you already do, but, rewrite it so that it's going to its own class, as you just copied all the static methods and fields over there.
Then return the bytecode of that modified class from your transformer.
This 100% guarantees you cannot possibly run into any module or classpath boundary issues and it will work with any classloader abstraction, guaranteed, but there are some caveats:
Just don't attempt to futz with instance anything. Make it all static methods and fields. You can make fake instance fields using an IdentityHashMap if you must (e.g. a static IdentityHashMap<Foo, String> names; is effectively identical to adding private String name; to the Foo class.. except it's a bit slower of course; presumably as you're already in a mess o reflection that's acceptable here).
Your code has to be 'dependency free'. It cannot rely on anything else, no libraries other than java.*, not even a helper class. This idea quickly runs out of steam if the job you're injecting becomes complicated. If you must, make a classloader for your own agent jar using the appropriate 'thread-safely initialize it only once' guards, and have that load in a bundle that does have the benefit of allowing dependencies.
This is all highly complicated stuff but you appear to have already worked out how to inject INVOKESTATIC calls, so, I think you know how to do this.
This is precisely what lombok does to 'patch' some methods in eclipse to ensure that things like save actions, auto-formatting, and syntax highlighting don't break - lombok injects knowledge of generated notes where appropriate and does it in this exact manner because eclipse uses a classloader platform called Equinox which makes any other solution problematic. You can look at it for inspiration or guidelines, though it's not particularly well documented. You're looking in particular at:
The lombok.eclipse.agent package in the eclipseAgent source root.
The lombok.patcher project which is lombok's only actual dependency, in particular the lombok.patcher.PatchScript.transplantMethod method.
Note that the next method may also interest you: lombok.patcher's 'insert' doesn't move the method - it injects the body of the method directly in there (it 'inlines'). This requires some serious finagling of the stack and is only advised for extremely simple one-liner-esque methods, and probably is excessive and unneccessary firepower for this problem.
DISCLAIMER: I wrote most of that.
Related
I have been trying to use a Java agent to apply a bytecode transformation with ASM.
I implemented an Agent with the premain method adding a transformer to the Instrumentation.
I added the "Premain-Class" line in the .jar manifest
Premain-Class: <MyAgentPath>
Then I tried to run the application with the agent.
There I have a problem : my transformer modifies some method calls, so if not all involved classes are modified too, it cannot work. And there are some classes which are not modified, like "org.apache.commons.math3.util.FastMath".
Of course then, I got the error :
java.lang.NoSuchMethodError: org.apache.commons.math3.util.FastMath.floor<new_descriptor>
I checked a lot of posts saying it could be the bootstrap loader which does not know the path to this class so I tried to add it using different ways :
Adding the "Boot-Class-Path" line to the manifest :
Boot-Class-Path: <...>/commons-math3<...>.jar
Using the method "appendToBootstrapClassLoaderSearch(JarFile)"
inst.appendToBootstrapClassLoaderSearch("<...>/commons-math3<...>.jar");
Using the JVM argument "-Xbootclasspath/a:"
-Xbootclasspath/a:<...>/commons-math3<...>.jar
None of this changed anything.
I also used the Instrumentation class method getAllLoadedClasses() to see which ones were loaded, and all classes involved in the application process where loaded, including FastMath.
for(Class<?> clazz : MyAgent.getInstInstance().getAllLoadedClasses()){
buffWrite.write(clazz.getName());
As the class "FastMath" gave an error and as the Bootstrap Loader should have its path, I tried adding some method calls to methods from other classes in the same package. It appears the problem does not show for every class of the package.
For example: MathUtils is transformed and a call to the modified method checkFinite(D)V -> checkFinite<new_descriptor>.
So I guess the problem has nothing to do with the paths given to the bootstrap loader.
If you have some ideas about what is happening, I would be glad to hear about it!
A NoSuchMethodError is most likely not caused by not adding something to the bootstrap class loader. The only chance that this could be a problem would be if there were suddenly two such jars available where one was instrumented and the other was not.
If you call change a method checkFinite(D)V to become another method checkFinite<new_descriptor>, then you need to make sure that any class using FastMath.floor updates the descriptor to this method. This means that you need to walk through every method of every class looking for visitMethodIns calls of ASM. It seems like you are missing some. Since you are changing the layout of the FastMath class, you must instrument it while loading it for the first time and you cannot redefine it.
The Java internal classes do not know of FastMath as this is a third-party dependency. Therefore, it should be possible to instrument any call from your agent. It seems to me like you are loading FastMath prematurely.
I am using apache felix osgi. When I am loading class using classLoader.loadClass(..) it is giving ClassNotFoundException
but when I am using Class.forName() to load a class it is working fine.
What difference it makes when we use classLoader.loadClass()?. Why we need to use only Class.forName() or how to use classLoader.loadClass() properly?
In any modular environment, such as OSGi, it is not sufficient to load a class only by name, because many modules might have a class with that name. Therefore a class must be uniquely identified using the class name AND the module that should load it.
If you call Class.forName() in its single-arg form then you are failing to provide the module information, so Java falls back to using the caller's class loader. This is no better than just taking a random guess. In your case it just happened to work (it found the class), but only through pure luck!
If you call ClassLoader.loadClass() – note this is not a static method – then you are actually providing a class loader. This is much better! Unfortunately you provided the wrong loader, i.e., not the one that actually knows about the class. You didn't specify where you got this loader from so I can only speculate why it was wrong.
Better is to use OSGi's Bundle.loadClass() method, which allows you to explicitly load a class from a bundle. Again you need to know which bundle the class is supposed to come from, but that's an inevitable consequence of working in a modular environment.
Finally, even better still is to forget about dynamic class loading completely and learn how to use Services. You should never have to use dynamic class loading in OSGi, except when dealing with 3rd party or legacy libraries that require it.
The method Class.forName(String) uses the caller's class loader. For example, if you are doing
class MyClass {
void someMethod() {
Class.forName("my.pkg.SomeClass");
}
}
then the class loader, which loaded the class MyClass, also loads the class "my.pkg.SomeClass". In standalone applications this is normally the so-called system class loader.
If you see a difference between using Class.forName and ClassLoader.loadClass, then you are using another class loader.
Class loading in OSGI is more difficult, as OSGI does a good job sepearting class loaders for all OSGI bundles to not get in conflict loading all their resources and classes.
As an additional tip: since you got the exception, you probably got the class loader you refer to as classLoader via a call to Thread.currentThread().getContextClassLoader(). In an OSGi environment that is probably the worst class loader to get since it is absolutely undefined what it is set to. The loading may fail at one time and succeed at another time, resulting in hard-to-find problems.
ClassLoader.loadClass or Class.forName seem to be synonyms for the same basic operation: request a dynamic class load. Yet calling Class.forName does additional "checking" which is not very useful (certainly in OSGi).
When doing a dynamic class load, the returned type has no implied type by the code. The code must use reflection to access static members or to create an instance. A created instance can either be reflected upon or cast to a type which must already implicitly know by the code. This cast will result in a runtime type check which will ensure type safety.
This is very different than an implicit class load done by the VM to resolve a class constant pool entry. Since it is the goal of these implicit class loads to avoid runtime type checks, the loader constraints are used to ensure type safety.
It does not seem necessary or reasonable to impose loader constraint checks on some dynamic class load requests. That is, code calling Class.forName (or ClassLoader.loadClass) and the VM resolving a class constant pool entry have different type safety needs. The former does not require loader constraint checks since that will be done at runtime by a type cast if needed. The latter does require loader constraint checks to avoid the need for runtime type checks.
So it does seem reasonable to have Class.forName behavior altered to avoid loader constraint checks. Only internal VM class load requests need to check loader constraints.
For more understanding Click here
Is there a feasible way to get my own code run whenever any class is loaded in Java, without forcing the user explicitly and manually loading all classes with a custom classloader?
Without going too much into the details, whenever a class implementing a certain interface read its annotation that links it with another class, and give the pair to a third class.
Edit: Heck, I'll go to details: I'm doing an event handling library. What I'm doing is having the client code do their own Listener / Event pairs, which need to be registered with my library as a pair. (hm, that wasn't that long after all).
Further Edit: Currently the client code needs to register the pair of classes/interfaces manually, which works pretty well. My intent is to automate this away, and I thought that linking the two classes with annotations would help. Next, I want to get rid of the client code needing to keeping the list of registrations up to date always.
PS: The static block won't do, since my interface is bundled into a library, and the client code will create further interfaces. Thus, abstract classes won't do either, since it must be an interface.
If you want to base the behavior on an interface, you could use a static initializer in that interface.
public interface Foo{
static{
// do initializing here
}
}
I'm not saying it's good practice, but it will definitely initialize the first time one of the implementing classes is loaded.
Update: static blocks in interfaces are illegal. Use abstract classes instead!
Reference:
Initializers (Sun Java Tutorial)
But if I understand you right, you want the initialization to happen once per implementing class. That will be tricky. You definitely can't do that with an interface based solution. You could do it with an abstract base class that has a dynamic initializer (or constructor), that checks whether the requested mapping already exists and adds it if it doesn't, but doing such things in constructors is quite a hack.
I'd say you cleanest options are either to generate Code at build time (through annotation processing with apt or through bytecode analysis with a tool like asm) or to use an agent at class load time to dynamically create the mapping.
Ah, more input. Very good. So clients use your library and provide mappings based on annotations. Then I'd say your library should provide an initializer method, where client code can register classes. Something like this:
YourLibrary.getInstance().registerMappedClasses(
CustomClass1.class,
CustomClass2.class,
CustomClass3.class,
CustomClass4.class
)
Or, even better, a package scanning mechanism (example code to implement this can be found at this question):
YourLibrary.getInstance().registerMappedClassesFromPackages(
"com.mycompany.myclientcode.abc",
"com.mycompany.myclientcode.def"
)
Anyway, there is basically no way to avoid having your clients do that kind of work, because you can't control their build process nor their classloader for them (but you could of course provide guides for classloader or build configuration).
If you want some piece of code to be run on any class loading, you should:
overwrite the ClassLoader, adding your own custom code at the loadClass methods (don't forget forwarding to the parent ClassLoader after or before your custom code).
Define this custom ClassLoader as the default for your system (here you got how to do it: How to set my custom class loader to be the default?).
Run and check it.
Depending on what kind of environment you are, there are chances that not all the classes be loaded trouugh your custom ClassLoader (some utility packages use their own CL, some Java EE containers handle some spacific areas with specific classLoaders, etc.), but it's a kind of aproximation to what you are asking.
I'm trying to get a reference to a java.lang.Class which is on my classpath and look at the members on it with reflection. The trouble is, I'm trying to do this with only an incomplete classpath. By this I mean that the classpath contains all the classes I am interested in, but does not necessarily contain further classes which these classes might depend on. Therefore it is not possible to fully load the class in the normal way.
Is this possible?
You can use ASM to parse the .class file without loading the class
It should be fine, you should only encounter ClassNotFound errors when those classes are invoked and need to be loaded. I don't think this will be the case if you are not instantiating instances of the classes, even if those classes import the others that are missing.
However, if your class makes any static initializer references to the unavailable classes then exceptions may be thrown even without class instantiations.
The classes won't be loaded until they are referenced, which means that if you don't instantiate your class only the classes referenced in the classes static initialization block (or referenced in another static initialization expression) should need to be loaded.
If these classes which might be needed are not in the classpath (for example if it needs another jar which is not referenced), you are likely to encounter an error.
It won't happen when you will load the class, however, but will happen if you call a method on it which requires these unreferenced classes. So, depending on what you want to do (basing on title, accessing basic information from the type), it might work, as long as what you need doesn't involve a call to these other classes.
You have to "extend" your classpath. You can create a custom class loader (you'll have to read the doc, I did it 5 years ago, but I don't remember the details).
So you can create that class loader (only override some very simple methods that maps the name of the class to some URL to read the bytes from), instance it, and ask it for classes.
You can access that loaded classes vía reflection. But never statically.
If you give the classloader a parent it will "inherit" the classes already in classpath. If not it will use the bootstrap classloader.
Edit if there are not depended classes, this will raise an error. You should create a class loader that has ALL the directly referenced classes.
This question already has answers here:
How can I get a list of all the implementations of an interface programmatically in Java?
(11 answers)
Closed 9 years ago.
Some time ago, I came across a piece of code, that used some piece of standard Java functionality to locate the classes that implemented a given interface. I know the functions were hidden in some non-logical place, but they could be used for other classes as the package name implied. Back then I did not need it, so I forgot about it, but now I do, and I can't seem to find the functions again. Where can these functions be found?
Edit: I'm not looking for any IDE functions or anything, but rather something that can be executed within the Java application.
Awhile ago, I put together a package for doing what you want, and more. (I needed it for a utility I was writing). It uses the ASM library. You can use reflection, but ASM turned out to perform better.
I put my package in an open source library I have on my web site. The library is here: http://software.clapper.org/javautil/. You want to start with the with ClassFinder class.
The utility I wrote it for is an RSS reader that I still use every day, so the code does tend to get exercised. I use ClassFinder to support a plug-in API in the RSS reader; on startup, it looks in a couple directory trees for jars and class files containing classes that implement a certain interface. It's a lot faster than you might expect.
The library is BSD-licensed, so you can safely bundle it with your code. Source is available.
If that's useful to you, help yourself.
Update: If you're using Scala, you might find this library to be more Scala-friendly.
Spring can do this for you...
BeanDefinitionRegistry bdr = new SimpleBeanDefinitionRegistry();
ClassPathBeanDefinitionScanner s = new ClassPathBeanDefinitionScanner(bdr);
TypeFilter tf = new AssignableTypeFilter(CLASS_YOU_WANT.class);
s.addIncludeFilter(tf);
s.scan("package.you.want1", "package.you.want2");
String[] beans = bdr.getBeanDefinitionNames();
N.B. The TypeFilter is important if you want the correct results!
You can also use exclusion filters here instead.
The Scanner can be found in the spring-context jar, the registry in spring-beans, the type filter is in spring-core.
I really like the reflections library for doing this.
It provides a lot of different types of scanners (getTypesAnnotatedWith, getSubTypesOf, etc), and it is dead simple to write or extend your own.
The code you are talking about sounds like ServiceLoader, which was introduced in Java 6 to support a feature that has been defined since Java 1.3 or earlier. For performance reasons, this is the recommended approach to find interface implementations at runtime; if you need support for this in an older version of Java, I hope that you'll find my implementation helpful.
There are a couple of implementations of this in earlier versions of Java, but in the Sun packages, not in the core API (I think there are some classes internal to ImageIO that do this). As the code is simple, I'd recommend providing your own implementation rather than relying on non-standard Sun code which is subject to change.
Package Level Annotations
I know this question has already been answered a long time ago but another solution to this problem is to use Package Level Annotations.
While its pretty hard to go find all the classes in the JVM its actually pretty easy to browse the package hierarchy.
Package[] ps = Package.getPackages();
for (Package p : ps) {
MyAno a = p.getAnnotation(MyAno.class)
// Recursively descend
}
Then just make your annotation have an argument of an array of Class.
Then in your package-info.java for a particular package put the MyAno.
I'll add more details (code) if people are interested but most probably get the idea.
MetaInf Service Loader
To add to #erickson answer you can also use the service loader approach. Kohsuke has an awesome way of generating the the required META-INF stuff you need for the service loader approach:
http://weblogs.java.net/blog/kohsuke/archive/2009/03/my_project_of_t.html
You could also use the Extensible Component Scanner (extcos: http://sf.net/projects/extcos) and search all classes implementing an interface like so:
Set<Class<? extends MyInterface>> classes = new HashSet<Class<? extends MyInterface>>();
ComponentScanner scanner = new ComponentScanner();
scanner.getClasses(new ComponentQuery() {
#Override
protected void query() {
select().
from("my.package1", "my.package2").
andStore(thoseImplementing(MyInterface.class).into(classes)).
returning(none());
}
});
This works for classes on the file system, within jars and even for those on the JBoss virtual file system. It's further designed to work within standalone applications as well as within any web or application container.
In full generality, this functionality is impossible. The Java ClassLoader mechanism guarantees only the ability to ask for a class with a specific name (including package), and the ClassLoader can supply a class, or it can state that it does not know that class.
Classes can be (and frequently are) loaded from remote servers, and they can even be constructed on the fly; it is not difficult at all to write a ClassLoader that returns a valid class that implements a given interface for any name you ask from it; a List of the classes that implement that interface would then be infinite in length.
In practice, the most common case is an URLClassLoader that looks for classes in a list of filesystem directories and JAR files. So what you need is to get the URLClassLoader, then iterate through those directories and archives, and for each class file you find in them, request the corresponding Class object and look through the return of its getInterfaces() method.
Obviously, Class.isAssignableFrom() tells you whether an individual class implements the given interface. So then the problem is getting the list of classes to test.
As far as I'm aware, there's no direct way from Java to ask the class loader for "the list of classes that you could potentially load". So you'll have to do this yourself by iterating through the visible jars, calling Class.forName() to load the class, then testing it.
However, it's a little easier if you just want to know classes implementing the given interface from those that have actually been loaded:
via the Java Instrumentation framework, you can call Instrumentation.getAllLoadedClasses()
via reflection, you can query the ClassLoader.classes field of a given ClassLoader.
If you use the instrumentation technique, then (as explained in the link) what happens is that your "agent" class is called essentially when the JVM starts up, and passed an Instrumentation object. At that point, you probably want to "save it for later" in a static field, and then have your main application code call it later on to get the list of loaded classes.
If you were asking from the perspective of working this out with a running program then you need to look to the java.lang.* package. If you get a Class object, you can use the isAssignableFrom method to check if it is an interface of another Class.
There isn't a simple built in way of searching for these, tools like Eclipse build an index of this information.
If you don't have a specific list of Class objects to test you can look to the ClassLoader object, use the getPackages() method and build your own package hierarchy iterator.
Just a warning though that these methods and classes can be quite slow.