How Service Provider Interfaces work in Java. I have been working with JDBC and found out that SPI takes care for loading the Driver class.
The following line provides the implementation classes for Driver which we can iterate using iterator().
ServiceLoader.load(Driver.class); // returns the ServiceLoader for Driver class, we can iterate to get all implementation classes
I wanted to know how this method works internally in Java.
There's quite a bit of documentation provided with the class. Are you looking for information beyond that?
Services in Java are defined by a service provider configuration file included in the service jar. The name of this file based on the name of the service interface and is something like META-INF/services/com.example.CodecFactory (to use the example from the ServiceLoader documentation) or META-INF/services/java.sql.Driver (for the JDBC example that I think you're using).
The key thing to understand here is that all the service provider configuration files for a particular service have the same name. That means that the classpath contains multiple resources with that name, one per service provider. The method ClassLoader.getResources (note it's getResources not getResource) returns a Enumeration that ServiceLoader can use to iterate over all of the config files and identify each provider.
Each configuration file just contains the name of the implementation class. If you look at the Postgres jar, for example, which is named META-INF/services/java.sql.Driver, it just has one line: org.postgresql.Driver. ServiceLoader just reads the implementation class name from the file, then calls Class.forName with the class name and then newInstance to create an instance of the implementation class, which it can then cast to java.sql.Driver or whatever the service interface is.
If the app needs more information about each service provider, it can usually get that information through the service interface. For example, the interface java.sql.Driver includes methods like acceptsURL, getMinorVersion/getMajorVersion, etc. that the app can use to get information about JDBC drivers.
Related
When trying to inject a class which is in the java.lang namespace via java.lang.instrument.Instrumentation#appendToBootstrapClassLoaderSearch on a OpenJDK 11, nothing happens and no error is thrown. When placing the class to inject into a different package, it works as expected.
JarFile jar = new JarFile(new File("file/to/bootstrap.jar));
instrumentation.appendToBootstrapClassLoaderSearch(jar);
// throws ClassNotFoundException java/lang/Dispatcher
Class.forName("java.lang.Dispatcher", false, null);
bootstrap.jar
└─ java/lang/Dispatcher.class
The reason I want to do this is to overcome issues with some OSGi containers. They typically restrict delegation to the bootstrap class loader to only certain packages. By default that obviously always includes java.* which is why I want to put my Dispatcher class there. I'm aware of org.osgi.framework.bootdelegation but that property only gets read during initialization. That means when attaching an agent at runtime, it's already too late to override this value.
An alternative would be to instrument all known OSGi class loaders and to white-list the agent classes. But doing that for each framework and test that for each version seems less feasible.
How can I inject a custom class like java.lang.Dispatcher into the bootstrap class loader? Are there other patterns or best practices to avoid OSGi bootdelegation issues?
To provide some more context:
My idea is to only inject this one Dispatcher class into the bootstrap class loader. The dispatcher basically just holds a static Map. The rest of the agent's classes would be loaded by a dedicated URLClassLoader which is a child of the bootstrap class loader. The agent would then register MethodHandles in the dispatcher's Map so that the injected byte code can get ahold of the MethodHandles which enable accessing the agent's classes loaded in the agent class loader.
It is possible by using unsafe API. Since Java 9, the boot class loader's implementation has changed to only check a designated jmod for a known package, but the boot search path is no longer checked.
Java 11 also removed the sun.misc.Unsafe#defineClass method but the same method is still available in jdk.internal.misc.Unsafe.
You do have to open that class's module which is internal. You can either do so by using sun.misc.Unsafe which allows you to write a field value (accessible) without accessibility checks or by using Instrumentation's official API.
If you are using Byte Buddy, have a look at the ClassInjector implementations which offer implementations for all approaches.
There is an open ticket for adressing the need of Java agents to inject helper classes but until it is resolved, this is a common workaround.
I have an application that consists of a grid of hazelcast nodes which uses extensive runtime bytecode generation (asm). Specifically I am dynamically building predicate<> Java functions from user entered filter expressions. I would like to store the predicates in a map so that they are available across the cluster without having to be recompiled. Predicates are not the only instance of this. I also have ORM style mapping classes that are generated at runtime which need to be shared across the cluster. These classes are loaded by a custom classloader called DynamiClassLoader.
At first I was unable to store my custom generated classes in Hazelcast IMaps, getting a ClassNotFoundexception. However, I found if tell Hazelcast to use my custom DynamicClassLoader using Config.setClassLoader() then it is able to find my dynamic classes and use them in the local member so I have no issues serializing and deserializing instances of my custom classes in IMaps in the same member.
However, I am still unable to deserialize instances of my predicates which were inserted into the map by a different member. I have enabled UserCodeDeployment and stepped through the code in my debugger to confirm that if it cannot find the class locally then it is hitting UserCodeDeploymentClassLoader.java and on the classNotFoundException it is proceeding to check other members for the classes but it is unable to find them. I haven't been able to discover how exactly this works. it seems to look in an internal map for which member any given class can be found on and it does not find my classes in there. I believe it dispatches an operation to other members then to look for the class, but in this case it looks like my custom classloader isn't used so it's not able to find my custom classes.
How can I make dynamically generated classes work with UserCodeDeployment on Hazelcast? Is there a way I can 'register' my dynamic classes with the member's code service or something like that?
Thankyou,
Troy.
I've finally figured this out after extensive debugging. It turns out that the operation in Hazelcast on the target member to find a class calls loadBytecodeFromParent() in ClassDataProvider.java. This looks for a .class file using getResourceAsStream on the classloader:
String resource = className.replace('.', '/').concat(".class");
...
is = parent.getResourceAsStream(resource);
Which basically looks for the class file on the filesystem. Since my dynamic classes are entirely in memory there is no .class file resource for it to find.
I resolved this, by putting a hashmap in my DynamicClassLoader to keep the generated bytecode in and overriding getResourceAsStream to return that bytecode when it's available before looking further. Now it works!
I'm trying to create a framework for plugin/apps, and I think I've settled on using a Service Provider Interface (SPI) in Java to achieve that. The ultimate vision is that people can download plugins (like an app store) and then just drop the jars into a folder, and they'll work with the existing program.
So I have a class PluginInterface that will be my SPI. I want all plugins to implement this class (it has basic methods like doesAccept, handle, etc.). Since 3rd parties will be creating the Service Providers, how can I let them reference this Java interface in their code (so they can implement it), without actually giving them the source code of the main application?
Create two JARs api and impl where you put your application in the impl and put whatever the code you need to share with third party developer in api.
Example: jaxb-api.jar and jaxb-impl.jar
In my library's code, I load class names from an XML file using JAXB, in order to instanciate them later on using Class.forName(). A fictive example to illustrate the case:
public void libraryMethod() throws Exception {
List<String> classNames = loadClassNamesFromXML();
for (String className : classNames) {
Class.forName(className).newInstance().doThings();
}
}
Now, some users use OSGi to configure their applications, and they load my library with a different class loader than the classes they configure using my XML structure. This means that loading may fail, as the class cannot be found.
Is there any more reliable way to load such classes? Or is there any other way to configure these instances? I'm open to suggestions that result in this:
public void libraryMethod() throws Exception {
// Spring does things like this:
List<SomeType> instances = loadInstancesFromXML();
for (SomeType instance : instances) {
instance.doThings();
}
}
Some constraints:
From a library perspective, the lifecycle of those instances is unimportant. If they have (user) state, my library would not notice.
I want to keep things simple in this library, so I want to avoid creating external dependencies on configuration frameworks like spring, for instance. So I'm only interested in solutions that can be achieved with standard JDK 6+ distributions.
I would really like to keep my simple XML configuration file (slight adaptations to the XML structure are OK).
If you want your library to be flexible and work in both OSGi and non-OSGi environment, you should allow users provide their own ClassLoader's or let them tell your library which class names they have. Read Neil Bartlett blog post.
Original link returns a 404. You can access the article on the Wayback Machine.
Thanks for the explanation. Spring would not make this any easier in OSGi. You can not simply inject an implementation class from a package you do not import. In OSGi you typically use OSGi services to inject implementations that originate outside your bundle and are unknown to you at compile time.
So your user would implement an interface you specify and publish his implementation as an OSGi service. You could then either pick up all such services or let the user specify an ldap filter for his service in the xml config.
The advantage of this aproach is that you do not have to load classes and care about classloaders. So this is the recommended way in OSGi. If you want the same solution for inside and outside OSGi Ivan's aproach with specifying a classloader + classname is an alternative.
In general, in OSGi you should use a service. The reason that Class.forName/XML configuration is so popular is that only a single class gets control. To configure the rest, it needs to know the classes to initialize/call.
In OSGi this problem does not exist since each module (bundle) (can) get control through declarative services (or in the old fashioned way through an activator). So in OSGi you have a peer to peer model. Anybody can register a service and depend on other services.
So instead of specifying class names and assuming they are globally unique (they are not in large systems) it is a lot easier to use services and not leave the Java compiler; these class names are very error prone. In general this means that you often just register your service and wait to be called since there is no need to initialize your clients. However, the whiteboard pattern address the situation when you want to find out about your clients (with bndtools and bnd annotations):
The "server"
#Component
public class MyLib {
#Reference(type='*')
void addSomeType(SomeType st ) {
st.doThings();
}
}
The client
#Component
public class MyClient implements SomeType {
public void doThings() { ... }
}
Hope this helps.
JDBC4 drivers include META-INF/services/java.sql.Driver in the jar which use the ServiceProvider mechanism to register the Driver implementation with the JVM (see java.util.ServiceLoader javadocs). Having the driver on the class path will register the driver automatically, obviating the need to use Class.forName. Instead app code uses ServiceLoader.load to discover registered drivers. Same mechanism can be used for other config. Perhaps something like that could be used? As an aside, when registering one's own implementations with the Service Provider mechanism, using an annotation like the spi looks pretty convenient.
I'd like to implement a dynamic plugin feature in a Java application. Ideally:
The application would define an interface Plugin with a method like getCapabilities().
A plugin would be a JAR pluginX.jar containing a class PluginXImpl implementing Plugin (and maybe some others).
The user would put pluginX.jar in a special directory or set a configuration parameter pointing to it. The user should not necessarily have to include pluginX.jar in their classpath.
The application would find PluginXImpl (maybe via the JAR manifest, maybe by reflection) and add it to a registry.
The client could get an instance of PluginXImpl, e.g., by invoking a method like getPluginWithCapabilities("X"). The user should not necessarily have to know the name of the plugin.
I've got a sense I should be able to do this with peaberry, but I can't make any sense of the documentation. I've invested some time in learning Guice, so my preferred answer would not be "use Spring Dynamic Modules."
Can anybody give me a simple idea of how to go about doing this using Guice/peaberry, OSGi, or just plain Java?
This is actually quite easy using plain Java means:
Since you don't want the user to configure the classpath before starting the application, I would first create a URLClassLoader with an array of URLs to the files in your plugin directory. Use File.listFiles to find all plugin jars and then File.toURI().toURL() to get a URL to each file. You should pass the system classloader (ClassLoader.getSystemClassLoader()) as a parent to your URLClassLoader.
If the plugin jars contain a configuration file in META-INF/services as described in the API documentation for java.util.ServiceLoader, you can now use ServiceLoader.load(Plugin.class, myUrlClassLoader) to obatin a service loader for your Plugin interface and call iterator() on it to get instances of all configured Plugin implementations.
You still have to provide your own wrapper around this to filter plugin capabilites, but that shouldn't be too much trouble, I suppose.
OSGI would be fine if you want to replace the plugins during runtime i.g. for bugfixes in a 24/7 environment. I played a while with OSGI but it took too much time, because it wasn't a requirement, and you need a plan b if you remove a bundle.
My humble solution then was, providing a properties files with the class names of plugin descriptor classes and let the server call them to register (including quering their capabilities).
This is obvious suboptimal but I can't wait to read the accepted answer.
Any chance you can leverage the Service Provider Interface?
The best way to implement plug-ins with Guice is with Multibindings. The linked page goes into detail on how to use multibindings to host plugins.
Apologize if you know this, but check out the forName method of Class. It is used at least in JDBC to dynamically load the DBMS-specific driver classes runtime by class name.
Then I guess it would not be difficult to enumerate all class/jar files in a directory, load each of them, and define an interface for a static method getCapabilities() (or any name you choose) that returns their capabilities/description in whatever terms and format that makes sense for your system.