Develop Reference

Make a Java class visible from any ClassLoader

I'm using a Java Agent (Agent.class) to transform a method in a program (Program.class) in a way that includes a call to the Agent class.
public Program.getMultiplier()F:
ALOAD 1
ALOAD 2
FDIV
+ INVOKESTATIC Agent.getCustomMultiplier(F)F
FRETURN
I've inspected the class loaders and their parents of both Agent and Program classes, and their hierarchy looks like this:
Agent.class: AppClassLoader <- PlatformClassLoader <- null
Program.class: URLClassLoader <- PlatformClassLoader <- null
When the Program executes the added INVOKESTATIC instruction, it throws a ClassNotFoundException -- it cannot find the Agent class as it was loaded by a different class loader.
As a temporary solution, I've tried forcing AppClassLoader to become a parent of URLClassLoader with reflection, which works in older Java versions but has been removed since Java 12.
Is there a more reliable way to make sure my Agent class is visible from any class loader?

You can add classes to the bootstrap class loader using appendToBootstrapClassLoaderSearch. This makes the classes of the specified jar file available to all classes whose defining class loader follows the standard delegation pattern.
But this requires the classes to be packaged in a jar file. When you specify the Agent’s own jar file, you have to be aware that classes loaded through the bootstrap loader are distinct from the classes loaded through the app loader, even when they originate from the same jar file. Further, the classes loaded by the bootstrap loader must not have dependencies to classes loaded by by other class loaders.
If your getCustomMultiplier method is supposed to interact with the running Agent, you have to separate the agent and the class containing this method.

Have your Agent listen to the creation of new ClassLoaders and then attach instances of them to the new ClassLoaders.
This way you preserve your "Agent listens to ClassLoader" interface, even if it now extends beyond the one platform class loader you expected the Agent to listen to.

You may be able to do something specific that works for URLClassLoader, but not all classes are loaded by an instance of URLClassLoader. Any OSGi project won't, most web servers also use their own classloaders in order to support hot reload, etc.
As far as I know there's no way to just casually update some 'global parent of all classloaders' or inject one; there's no such parent, and even if there was, a classloader is free to ignore its parent entirely.
Therefore the general answer is: No, you can't do that.
But, let's get our hacking hats on!
You're an agent already. One of the things you get to do as agent is to 'witness' classes as they are being loaded. Just invoke .addTransformer on the instance of Instrumentation you get in your agentmain and register one.
When you notice the Program class being loaded, do the following:
Take the bytecode and toss it through ASM, BCEL, Bytecode Buddy, or any other java 'class file reader/transformer' framework.
Also open up a class from within your agent's code (I wouldn't use Agent itself, I'd make a class called ProgramAddonMethods or whatnot as a container - everything inside is for the program to use / for your agent to 'inject' into that program.
Add every static member in ProgramAddonMethods directly to Program. As you do so, modify the typename on all accesses (both INVOKESTATIC and the read/write field opcodes) where the etypename is ProgramAddonMethods and make it the fully qualified name of the targeted class instead.
inject the INVOKESTATIC as you already do, but, rewrite it so that it's going to its own class, as you just copied all the static methods and fields over there.
Then return the bytecode of that modified class from your transformer.
This 100% guarantees you cannot possibly run into any module or classpath boundary issues and it will work with any classloader abstraction, guaranteed, but there are some caveats:
Just don't attempt to futz with instance anything. Make it all static methods and fields. You can make fake instance fields using an IdentityHashMap if you must (e.g. a static IdentityHashMap<Foo, String> names; is effectively identical to adding private String name; to the Foo class.. except it's a bit slower of course; presumably as you're already in a mess o reflection that's acceptable here).
Your code has to be 'dependency free'. It cannot rely on anything else, no libraries other than java.*, not even a helper class. This idea quickly runs out of steam if the job you're injecting becomes complicated. If you must, make a classloader for your own agent jar using the appropriate 'thread-safely initialize it only once' guards, and have that load in a bundle that does have the benefit of allowing dependencies.
This is all highly complicated stuff but you appear to have already worked out how to inject INVOKESTATIC calls, so, I think you know how to do this.
This is precisely what lombok does to 'patch' some methods in eclipse to ensure that things like save actions, auto-formatting, and syntax highlighting don't break - lombok injects knowledge of generated notes where appropriate and does it in this exact manner because eclipse uses a classloader platform called Equinox which makes any other solution problematic. You can look at it for inspiration or guidelines, though it's not particularly well documented. You're looking in particular at:
The lombok.eclipse.agent package in the eclipseAgent source root.
The lombok.patcher project which is lombok's only actual dependency, in particular the lombok.patcher.PatchScript.transplantMethod method.
Note that the next method may also interest you: lombok.patcher's 'insert' doesn't move the method - it injects the body of the method directly in there (it 'inlines'). This requires some serious finagling of the stack and is only advised for extremely simple one-liner-esque methods, and probably is excessive and unneccessary firepower for this problem.
DISCLAIMER: I wrote most of that.

Automatically call static block without explicitly calling Class.forName

Asume the following code:
public class Main {
public static final List<Object> configuration = new ArrayList<>();
public static void main(String[] args) {
System.out.println(configuration);
}
}
I now want to be able, to provide "self-configuring" classes. This means, they should be able to simply provide something like a static block, that will get called automatically like this:
public class Custom {
static {
Main.configuration.add(Custom.class);
}
}
If you execute this code, the configuration list is empty (because of the way static blocks are executed). The class is "reachable", but not "loaded". You could add the following to the Main class before the System.out
Class.forName("Custom");
and the list would now contain the Custom class object (since the class is not initialized yet, this call initializes it). But because the control should be inverse (Custom should know Main and not the other way around), this is not a usable approach. Custom should never be called directly from Main or any class, that is associated with Main.
What would be possible though is the following: You could add an Annotation to the class and collect all classes with said annotation, using something like the ClassGraph framework and call Class.forName on each of them.
TL;DR
Is there a way, to automatically call the static block without the need to analyze all classes and the need of knowing the concrete, "self configuring" class? Perfect would be an approach, that, upon starting the application, automatically initializes a classes (if they are annotated with a certain annotation). I thought about custom ClassLoaders, but from what i understand, they are lazy and therefor not usable for this approach.
The background of this is, that i want to incorporate it into an annotation processor, which creates "self configuring code".
Example (warning: design-talk and in depth)
To make this a little less abstract, imagine the following:
You develop a Framework. Let's call it Foo. Foo has the classes GlobalRepository and Repository. GlobalRepository follows the Singleton design pattern (only static methods). The Repository as well as the GlobalRepository have a method "void add(Object)" and " T get(Class)". If you call get on the Repository and the Class cannot be found, it calls GlobalRepository.get(Class).
For convenience, you want to provide an Annotation called #Add. This Annotation can be placed on Type-Declarations (aka Classes). An annotation-processor creates some configurations, which automatically add all annotated classes to the GlobalRepository and therefor reduce boilerplate code. It should only (in all cases) happen once. Therefor the generated code has a static initializer, in which the GlobalRepository is filled, just like you would do with the local repository. Because your Configurations have names that are designed to be as unique as possible and for some reason even contain the date of creation (this is a bit arbitrary, but stay with me), they are nearly impossible to guess.
So, you also add an annotation to those Configurations, which is called #AutoLoad. You require the using developer to call GlobalRepository.load(), after which all classes are analyzed and all classes with this annotation are initialized, and therefor their respective static-blocks are called.
This is a not very scalable approach. The bigger the application, the bigger the realm to search, the longer the time and so on. A better approach would be, that upon starting the application, all classes are automatically initialized. Like through a ClassLoader. Something like this is what i am looking for.

First, don’t hold Class objects in your registry. These Class objects would require you to use Reflection to get the actual operation, like instantiating them or invoking certain methods, whose signature you need to know before-hand anyway.
The standard approach is to use an interface to describe the operations which the dynamic components ought to support. Then, have a registry of implementation instances. These still allow to defer expensive operations, if you separate them into the operational interface and a factory interface.
E.g. a CharsetProvider is not the actual Charset implementation, but provides access to them on demand. So the existing registry of providers does not consume much memory as long as only common charsets are used.
Once you have defined such a service interface, you may use the standard service discovery mechanism. In case of jar files or directories containing class files, you create a subdirectory META-INF/services/ containing a file name as the qualified name of the interface containing qualified names of implementation classes. Each class path entry may have such a resource.
In case of Java modules, you can declare such an implementation even more robust, using
provides service.interface.name with actual.implementation.class;
statements in your module declaration.
Then, the main class may lookup the implementations, only knowing the interface, as
List<MyService> registered = new ArrayList<>();
for(Iterator<MyService> i = ServiceLoader.load(MyService.class); i.hasNext();) {
registered.add(i.next());
}
or, starting with Java 9
List<MyService> registered = ServiceLoader.load(MyService.class)
.stream().collect(Collectors.toList());
The class documentation of ServiceLoader contains a lot more details about this architecture. When you go through the package list of the standard API looking for packages have a name ending with .spi, you get an idea, how often this mechanism is already used within the JDK itself. The interfaces are not required to be in packages with such names though, e.g. implementations of java.sql.Driver are also searched through this mechanism.
Starting with Java 9, you could even use this to do something like “finding the Class objects for all classes having a certain annotation”, e.g.
List<Class<?>> configuration = ServiceLoader.load(MyService.class)
.stream()
.map(ServiceLoader.Provider::type)
.filter(c -> c.isAnnotationPresent(MyAnnotation.class))
.collect(Collectors.toList());
but since this still requires the classes to implement a service interface and being declared as implementations of the interface, it’s preferable to use the methods declared by the interface for interacting with the modules.

How to prevent client from seeing internal private classes in Android library ?

I have a library with several packages-
lets say
package a;
package b;
inside package a I have public a_class
inside package b I have public b_class
a_class uses b_class.
I need to generate a library from this , but I do not want the Client to see b_class.
The only solution I know of is to flatten my beautifully understandable packages to single package and to use default package access for b_class.
Is there another way to do so ? maybe using interfaces or some form of design pattern ??

If you reject to move the code to an individual, controlled server, all you can do is to hinder the client programmer when trying to use your APIs. Let's begin applying good practices to your design:
Let your packages organized as they are now.
For every class you want to "hide":
Make it non-public.
Extract its public API to a new, public interface:
public interface MyInterface {...}
Create a public factory class to get an object of that interface type.
public class MyFactory
{
public MyInterface createObject();
}
So far, you have now your packages loosely coupled, and the implementation classes are now private (as good practices preach, and you already said). Still, they are yet available through the interfaces and factories.
So, how can you avoid that "stranger" clients execute your private APIs? What comes next is a creative, a little complicated, yet valid solution, based on hindering the client programmers:
Modify your factory classes: Add to every factory method a new parameter:
public class MyFactory
{
public MyInterface createObject(Macguffin parameter);
}
So, what is Macguffin? It is a new interface you must define in your application, with at least one method:
public interface Macguffin
{
public String dummyMethod();
}
But do not provide any usable implementation of this interface. In every place of your code you need to provide a Macguffin object, create it through an anonymous class:
MyFactory.getObject(new Macguffin(){
public String dummyMethod(){
return "x";
}
});
Or, even more advanced, through a dynamic proxy object, so no ".class" file of this implementation would be found even if the client programmer dares to decompile the code.
What do you get from this? Basically is to dissuade the programmer from using a factory which requires an unknown, undocumented, ununderstandable object. The factory classes should just care not to receive a null object, and to invoke the dummy method and check the return value it is not null either (or, if you want a higher security level, add an undocumented secret-key-rule).
So this solution relies upon a subtle obfuscation of your API, to discourage the client programmer to use it directly. The more obscure the names of the Macguffin interface and its methods, the better.

I need to generate a library from this , but I do not want the Client to see b_class. The only solution I know of is to flatten my beautifully understandable packages to single package and to use default package access for b_class. Is there another way to do so ?
Yes, make b_class package-private (default access) and instantiate it via reflection for use in a_class.
Since you know the full class name, reflectively load the class:
Class<?> clz = Class.forName("b.b_class")
Find the constructor you want to invoke:
Constructor<?> con = clz.getDeclaredConstructor();
Allow yourself to invoke the constructor by making it accessible:
con.setAccessible(true);
Invoke the constructor to obtain your b_class instance:
Object o = con.newInstance();
Hurrah, now you have an instance of b_class. However, you can't call b_class's methods on an instance of Object, so you have two options:
Use reflection to invoke b_class's methods (not much fun, but easy enough and may be ok if you only have a few methods with few parameters).
Have b_class implement an interface that you don't mind the client seeing and cast your instance of b_class to that interface (reading between the lines I suspect you may already have such an interface?).
You'll definitely want to go with option 2 to minimise your pain unless it gets you back to square one again (polluting the namespace with types you don't want to expose the client to).
For full disclosure, two notes:
1) There is a (small) overhead to using reflection vs direct instantiation and invocation. If you cast to an interface you'll only pay the cost of reflection on the instantiation. In any case it likely isn't a problem unless you make hundreds of thousands of invocations in a tight loop.
2) There is nothing to stop a determined client from finding out the class name and doing the same thing, but if I understand your motivation correctly you just want expose a clean API, so this isn't really a worry.

When using Kotlin, you can use the internal modifier for your library classes.

If I understand correctly you are asking about publishing your library for 3rd party usage without disclosing part of your source? If that's the case you can use proguard, which can obfuscate your library. By default everything will be excluded/obfuscated, unless you specify things you want to exclude from being obfuscated/excluded.

If you want to distribute [part of] your code without the client being able to access it at all, that means that the client won't be able to execute it either. :-O
Thus, you just have one option: Put the sensible part of your code into a public server and distribute a proxy to access it, so that your code would be kept and executed into your server and the client would still be able to execute it through the proxy but without accessing it directly.
You might use a servlet, a webservice, a RMI object, or a simple TCP server, depending on the complexity level of your code.
This is the safest approach I can think of, but it also deserves a price to pay: In addition to complexing your system, it would introduce a network delay for each remote operation, which might be big deal depending on the performance requirements. Also, you should securize the server itself, to avoid hacker intrussions. This could be a good solution if you already have a server that you could take advantage of.

How can I create dynamic proxy for final Class?

In short:
1. I have some final class that I want to create dynamic proxy for it. How can I do it?
2. Can I convert MethodHandle to Method?
Details
First of all, does exists any API to convert MethodHandle to Method? Something like in java.lang.invoke.MethodHandles
public MethodHandle unreflect(Method m) throws IllegalAccessException;
but the opposite way arrond?
Let say I want to create dynamic java.lang.reflect.Method. It is defiend as
public final
class Method extends AccessibleObject implements GenericDeclaration,
Member ;
So, if I want to use JDK Dynamic proxy I must use some interface (Member for example). There 2 main drawabacks though. First, method such as
public Class<?>[] getParameterTypes();
and such as
public Class<?> getReturnType();
are not part of any interface, while they are extensively used.
The second drawback is that it fails to provide drop-in replacement. That is, I can't pass my dynamic proxy to the code that expects java.lang.reflect.Method.
Another approach is to use CGLIB or Javaassist. AFAIK, CGLIB can't proxy final class, does he? Can Javaassist proxy final class? How can I "remove" final identifier from the class? AFAIL, Javvassist can somehow do it...

It depends on what kind of proxy you need. There are basically three aproaches of how you can achieve this, of which two are feasible in production code. As #probrekely stated, the problem of cglib or javassist ist that they dynamically create a subclass what is not possible for final classes. You can avoid this by:
Disabling byte code verification. The Java run time verifies byte code in order to secure that no malicious byte code is loaded. This is important when for example receiving classes over the network or the internet, for example an applet. This way, you could create a subclass of a final class since the byte code verifier would not stop you. Hypothetically, you can disable this verification if you run trusted code only. This can be done by running:
java -Xverify:none ApplicationName
This is however the solution I would recommend you the least. I would not use this aproach for production code but it is most certainly the easiest solution to implement.
Remove the final modifier from loaded classes, either before or after the classes are loaded. This can be achieve by using a Java agent. A Java agent can be installed at application startup via the command line or at runtime via the Attach API. With a byte code tool like ASM, you could parse the original byte array and remove the final modifier from all classes of interest. It is also possible to redefine classes that were already loaded. Remove a final modifier does not introduce conflicts with old class versions such that such a redefinition is always possible.
Do the same as I described with removing the final modifier but redefine the loaded class to actually contain all your instrumentation logic within the original class. This aporach will most certanly require the biggest effort but this will make your instrumentation transperent to all other code. This would be the cleanest solution of all solutions.

Sorry, what you want isn't possible:
You can use CGLIB or Javassist to create proxies for concrete classes because these libraries dynamically generate a subclass of the class you're trying to proxy. A final class can't be subclassed, so you can't create a proxy this way.
PowerMock does let you proxy final classes and methods, but this is because it runs your tests under its special ClassLoader which uses Javassist to modify the bytecode of the classes you wish to proxy as they're being loaded. (You wouldn't want to use this sort of thing in production, as generally the modified "zombie" version of the class that results won't be good for much else than running a specific mock unit test.)
The PowerMock approach wouldn't work here, however - you want to proxy java.lang.reflect.Method, which is on the bootstrap classpath, and so would load before any PowerMock/Javassist type tool and therefore not be proxiable.

Find Java classes implementing an interface [duplicate]

This question already has answers here:
How can I get a list of all the implementations of an interface programmatically in Java?
(11 answers)
Closed 9 years ago.
Some time ago, I came across a piece of code, that used some piece of standard Java functionality to locate the classes that implemented a given interface. I know the functions were hidden in some non-logical place, but they could be used for other classes as the package name implied. Back then I did not need it, so I forgot about it, but now I do, and I can't seem to find the functions again. Where can these functions be found?
Edit: I'm not looking for any IDE functions or anything, but rather something that can be executed within the Java application.

Awhile ago, I put together a package for doing what you want, and more. (I needed it for a utility I was writing). It uses the ASM library. You can use reflection, but ASM turned out to perform better.
I put my package in an open source library I have on my web site. The library is here: http://software.clapper.org/javautil/. You want to start with the with ClassFinder class.
The utility I wrote it for is an RSS reader that I still use every day, so the code does tend to get exercised. I use ClassFinder to support a plug-in API in the RSS reader; on startup, it looks in a couple directory trees for jars and class files containing classes that implement a certain interface. It's a lot faster than you might expect.
The library is BSD-licensed, so you can safely bundle it with your code. Source is available.
If that's useful to you, help yourself.
Update: If you're using Scala, you might find this library to be more Scala-friendly.

Spring can do this for you...
BeanDefinitionRegistry bdr = new SimpleBeanDefinitionRegistry();
ClassPathBeanDefinitionScanner s = new ClassPathBeanDefinitionScanner(bdr);
TypeFilter tf = new AssignableTypeFilter(CLASS_YOU_WANT.class);
s.addIncludeFilter(tf);
s.scan("package.you.want1", "package.you.want2");
String[] beans = bdr.getBeanDefinitionNames();
N.B. The TypeFilter is important if you want the correct results!
You can also use exclusion filters here instead.
The Scanner can be found in the spring-context jar, the registry in spring-beans, the type filter is in spring-core.

I really like the reflections library for doing this.
It provides a lot of different types of scanners (getTypesAnnotatedWith, getSubTypesOf, etc), and it is dead simple to write or extend your own.

The code you are talking about sounds like ServiceLoader, which was introduced in Java 6 to support a feature that has been defined since Java 1.3 or earlier. For performance reasons, this is the recommended approach to find interface implementations at runtime; if you need support for this in an older version of Java, I hope that you'll find my implementation helpful.
There are a couple of implementations of this in earlier versions of Java, but in the Sun packages, not in the core API (I think there are some classes internal to ImageIO that do this). As the code is simple, I'd recommend providing your own implementation rather than relying on non-standard Sun code which is subject to change.

Package Level Annotations
I know this question has already been answered a long time ago but another solution to this problem is to use Package Level Annotations.
While its pretty hard to go find all the classes in the JVM its actually pretty easy to browse the package hierarchy.
Package[] ps = Package.getPackages();
for (Package p : ps) {
MyAno a = p.getAnnotation(MyAno.class)
// Recursively descend
}
Then just make your annotation have an argument of an array of Class.
Then in your package-info.java for a particular package put the MyAno.
I'll add more details (code) if people are interested but most probably get the idea.
MetaInf Service Loader
To add to #erickson answer you can also use the service loader approach. Kohsuke has an awesome way of generating the the required META-INF stuff you need for the service loader approach:
http://weblogs.java.net/blog/kohsuke/archive/2009/03/my_project_of_t.html

You could also use the Extensible Component Scanner (extcos: http://sf.net/projects/extcos) and search all classes implementing an interface like so:
Set<Class<? extends MyInterface>> classes = new HashSet<Class<? extends MyInterface>>();
ComponentScanner scanner = new ComponentScanner();
scanner.getClasses(new ComponentQuery() {
#Override
protected void query() {
select().
from("my.package1", "my.package2").
andStore(thoseImplementing(MyInterface.class).into(classes)).
returning(none());
}
});
This works for classes on the file system, within jars and even for those on the JBoss virtual file system. It's further designed to work within standalone applications as well as within any web or application container.

In full generality, this functionality is impossible. The Java ClassLoader mechanism guarantees only the ability to ask for a class with a specific name (including package), and the ClassLoader can supply a class, or it can state that it does not know that class.
Classes can be (and frequently are) loaded from remote servers, and they can even be constructed on the fly; it is not difficult at all to write a ClassLoader that returns a valid class that implements a given interface for any name you ask from it; a List of the classes that implement that interface would then be infinite in length.
In practice, the most common case is an URLClassLoader that looks for classes in a list of filesystem directories and JAR files. So what you need is to get the URLClassLoader, then iterate through those directories and archives, and for each class file you find in them, request the corresponding Class object and look through the return of its getInterfaces() method.

Obviously, Class.isAssignableFrom() tells you whether an individual class implements the given interface. So then the problem is getting the list of classes to test.
As far as I'm aware, there's no direct way from Java to ask the class loader for "the list of classes that you could potentially load". So you'll have to do this yourself by iterating through the visible jars, calling Class.forName() to load the class, then testing it.
However, it's a little easier if you just want to know classes implementing the given interface from those that have actually been loaded:
via the Java Instrumentation framework, you can call Instrumentation.getAllLoadedClasses()
via reflection, you can query the ClassLoader.classes field of a given ClassLoader.
If you use the instrumentation technique, then (as explained in the link) what happens is that your "agent" class is called essentially when the JVM starts up, and passed an Instrumentation object. At that point, you probably want to "save it for later" in a static field, and then have your main application code call it later on to get the list of loaded classes.

If you were asking from the perspective of working this out with a running program then you need to look to the java.lang.* package. If you get a Class object, you can use the isAssignableFrom method to check if it is an interface of another Class.
There isn't a simple built in way of searching for these, tools like Eclipse build an index of this information.
If you don't have a specific list of Class objects to test you can look to the ClassLoader object, use the getPackages() method and build your own package hierarchy iterator.
Just a warning though that these methods and classes can be quite slow.