I have a large desktop Java application and I want to allow other developers to develop plugins for. Plugins will be jars placed in a specified dir. They will not be on the classpath at startup. I will load and deploy them at runtime.
The complication is that some plugins will have dependencies on each other, as well as the core application. So I cannot load each plugin/jar in its own URLClassLoader. Therefore I want to load all plugins into 1 URLClassLoader. Furthermore, some plugins may fail to initialise for various reasons. And I only want a ClassLoader at the end of day that knows about the successfully loaded plugins. The reasons are quite bizarre and relate to some legacy stuff that is using reflection to instantiate classes. This needs to fail if the plugin doesn't initialise for classes defined inside the plugin jar that failed.
Without this requirement, the solution would be:
Collect the jar URLs and build a ClassLoader based on them
Try to initialise a plugin class from each jar (defined in config in the manifest)
Now the ClassLoader here would be passed to the legacy system for it to use for its reflection stuff. However, it's my understanding that it will still be able to instantiate classes from plugin jars whose plugin failed to initialise (since the jar will still in the URL[] of the ClassLoader). Hence this breaks my requirement above.
The only solution I have come up with so far is to create a custom URLClassLoader as follows (simply to allow access to findClass()):
public class CustomURLClassLoader extends URLClassLoader {
public CustomURLClassLoader(final URL[] urls, final ClassLoader parent) {
super(urls, parent);
}
#Override
protected Class<?> findClass(final String name) throws ClassNotFoundException {
return super.findClass(name);
}
}
And then I made another custom ClassLoader that essentially knows about multiple child ClassLoaders:
public class MultiURLClassLoader extends ClassLoader {
private Set<CustomURLClassLoader> loaders = new HashSet<CustomURLClassLoader>();
public MultiURLClassLoader(final ClassLoader parent) {
super(parent);
}
#Override
protected Class<?> findClass(final String name) throws ClassNotFoundException {
Iterator<CustomURLClassLoader> loadersIter = loaders.iterator();
boolean first = true;
while (first || loadersIter.hasNext()) {
try {
if (first) {
return super.findClass(name);
} else {
return loadersIter.next().findClass(name);
}
} catch (ClassNotFoundException e) {
first = false;
}
}
throw new ClassNotFoundException(name);
}
public void addClassLoader(final CustomURLClassLoader classLoader) {
loaders.add(classLoader);
}
public void removeClassLoader(final CustomURLClassLoader classLoader) {
loaders.remove(classLoader);
}
}
Then my loading plugin alogorithm will be something like
MultiURLClassLoader multiURLClassLoader = new MultiURLClassLoader(ClassLoader.getSystemClassLoader());
for (File pluginJar : new File("plugindir").listFiles()) {
CustomURLClassLoader classLoader = null;
try {
URL pluginURL = pluginJar.toURI().toURL();
final URL[] pluginJarUrl = new URL[] { pluginURL };
classLoader = new CustomURLClassLoader(pluginJarUrl, multiURLClassLoader);
multiURLClassLoader.addClassLoader(classLoader);
Class<?> clazz = Class.forName("some.PluginClass", false, multiURLClassLoader);
Constructor<?> ctor = clazz.getConstructor();
SomePluginInterface plugin = (SomePluginInterface)ctor1.newInstance();
plugin.initialise();
} catch (SomePluginInitialiseException e) {
multiURLClassLoader.removeClassLoader(classLoader);
}
}
Then I can pass the multiURLClassLoader instance onto the legacy system and it will only be able to find classes (via reflection) whose plugin successfully loaded.
I've done some basic testing and it seems to work as I'd like so far. But I would very much like someones opinion on whether this seems like a good idea or not? I have never played this much with ClassLoaders before and I am wanting to avoid getting myself in too deep before its too late.
Thanks!
The problem I see is that if you don't know in advance which plugin depends on which, it's very hard to do anything reasonable, to debug problems, to isolate non-functional or bad-behaving plugins, etc.
Therefore I'd suggest another option: Add another field into each plugin's manifest, which will say on what other plugins it depends. Perhaps just a list of other plugin JARs it needs to function. (The core application classes would be always available.) I believe this would make the design much more robust and simplify many things.
Then, you could choose from different designs, for example:
For each plugin you could create a separate ClassLoader that would load just the JARs it needs. Probably the most robust solution. But I see a drawback: plugins that act as dependencies for many other ones will be loaded repeatedly in different class-loaders. It depends on circumstances (plugin count, JARs size, ...) if this could be a problem or not, it could even be an advantage.
You could have one big ClassLoader for all plugins, as you suggest, but you could ask it for plugin classes in the order of their dependencies. The ones that don't depend on anything first, then the ones that depend on those first ones etc. If some plugin class fails to load/initialize, you could immediately discard all plugins that depend on it.
Are you looking for something like the OSGi approach?
You could do something like Petr Pudlák has said, however you should take in account the fact that one of the solutions you have can create cyclic dependencies...
Related
In a custom plugin (or task) I would like to read all compiled classes (preferrably those that have changed from last compilation) with a classloader so that I'll be able to use reflection on them.
Is that possible?
1) It would be great to have a cook right after a Java class was compiled so that I could read it, but I found no way to do this.
2) I'm thinking of something like this ...
compileJava.doLast {
ClassLoader parent = getClass().getClassLoader();
GroovyClassLoader loader = new GroovyClassLoader(parent);
// retrieve all class files
// for each class file, loader.parseClass(classFile)
}
In a gradle script getClass().getClassloader() will get you the classloader of the gradle script. This will NOT contain the compiled classes or compile/runtime jars. I think you want to do something similar to:
Collection<URL> urls = sourceSets.main.runtimeClasspath.files.collect { it.toURI().toURL() }
Classloader parent = new URLClassLoader(urls.toArray());
If you want to only act on the classes that have changed you are best to do do that in an incremental task
This is a classloader issue that I am struggling with. I understand the root cause of the issue (different classloaders), but I'm not sure about the best way to fix it.
I have project with some common interfaces; let's call it api. I have two other projects called runner and module that both use api as a dependency.
The job of runner is to dynamically load a module artifact (from a jar; it's a fat one that includes its dependencies) and then execute it. runner expects module to provide certain concrete implementations from api. To make sure that classes from different versions of module.jar don't clobber each other, I create a new classloader with a URL to module.jar, and set the parent classloader to the classloader of the class that loads and processes module.jar. This works without any issues.
The problem arose when I used runner as a dependency inside a webapp (a spring boot app to be specific), and quickly found that I couldn't load some classes from module.jar because they conflict with classes that already exist in the current classpath (from other dependencies in the webapp).
Since module.jar really only needs the classes from api, I thought that I could create a new URLClassLoader (without a parent) that only has classes from api.jar, and then use that as the parent classloader when I load up the module. This is where I started running into trouble:
CommonInterface commonInterface = null;
Class<CommonInterface> commonInterfaceClass = null;
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL, apiClassesClassLoader);
//...
//...
//clazz is a concrete implementation from module.jar
if(myClassLoader.loadClass(CommonInterface.class.getName()).isAssignableFrom(clazz)) {
commonInterfaceClass = clazz;
}
commonInterface = commonInterfaceClass.newInstance(); //ClassCastException
I understand that my original problem is due to the fact that the classloader first checks to see if the class has already been loaded before attempting to load it, which meant that when it was resolved using the name from module.jar, it was linking against an incompatible version of the class.
What's a good way to deal with this issue? Instead of creating a URL classloader that only has classes from api, does it make sense to create my own implementation that delegates to the parent only if the requested class is one from api?
You have loaded CommonInterface from two different class loaders. Classes with the same name but different class loaders are different classes to the JVM. (Even if the classes are 100% identical in the .class file - the problem is not incompatibility but the fact that they're from different class loaders)
If you do a
System.out.println(CommonInterface.class == myClassLoader.loadClass(CommonInterface.class.getName()));
You'll find that this prints false.
The way your create your classloader:
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL, apiClassesClassLoader);
.. would only work if apiClassesClassLoader is also a parent class loader of the class that contains this code.
You could try:
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL,
getClass().getClassLoader());
But from your description (it's a "fat" jar that contains its own dependencies) and the intricacies of the web classloader (child first) this may not solve your problem.
In that case, the only solution is to make your module jar "lean" to ensure that you only load each class once with one class loader only.
I forgot to update this question with my solution. I was able to solve this issue by creating a custom class-loader that extends URLClassLoader. This classloader does not have a parent.
I then overrode loadClass to control how classes were being loaded. I first check to see if the class exists in module.jar. If so, I load it from there. Otherwise, I load it using the current classloader. Since my custom classloader doesn't have a parent, it can load classes from module.jar even if they were already loaded by the main classloader, because they do not exist in my custom classloader's hierarchy.
The basic approach was like this:
public class MyClassLoader extends URLClassLoader {
private final ClassLoader mainClassLoader = MyClassLoader.class.getClassLoader();
private final Set<String> moduleClasses;
private MyClassLoader(URL url) {
super(new URL[]{ url });
try {
JarURLConnection connection = (JarURLConnection) url.openConnection();
this.moduleClasses = connection.getJarFile().stream()
.map(JarEntry::getName)
.filter(name -> name.endsWith(".class"))
.map(name -> name.replace(".class", "").replaceAll("/", "."))
.collect(Collectors.toSet());
} catch(IOException e) {
throw new IllegalArgumentException(String.format("Unexpected error while reading module jar: %s", e.getMessage()));
}
}
public static MyClassLoader newInstance(JarFile libraryJar) {
try {
return new MyClassLoader(new URL(String.format("jar:file:%s!/", libraryJar.getName())));
} catch(MalformedURLException e) {
throw new IllegalArgumentException(String.format("Path to module jar could not be converted into proper URL: %s", e.getMessage()));
}
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
if(moduleClasses.contains(name)) {
Class<?> clazz = findLoadedClass(name);
if(clazz != null) {
return clazz;
} else {
return findClass(name);
}
} else {
return mainClassLoader.loadClass(name);
}
}
}
I have a web service we'll call service.war. It implements an interface we'll call ServicePluginInterface. During the startup of service.war, it reads in environment variables and uses them to search for a jar (MyPlugin.jar). When it finds that jar, it then uses a second environment variable to load the plugin within the jar. The class that it loads looks like this:
public class MyPlugin implements ServicePluginInterface {...}
The servlet attempts to load the plugin using code like:
try {
if (pluginClass == null) {
plugin = null;
}
else {
ZipClassLoader zipLoader = new ZipClassLoader(Main.class.getClassLoader(), pluginJar);
plugin = (ServicePluginInterface)zipLoader.loadClass(pluginClass).newInstance();
plugin.getAccount(null,null);
}
} catch (Exception e) {
...
}
The trick is that I don't have source or a jar for ServicePluginInterface. Not wanting to give up so easily, I pulled the class files out of the service.war files. By using those class files as dependencies, I was able to build, without compiler warnings, MyPlugin. However, when actually executed by Tomcat, the section of code above generates a runtime exception:
java.lang.ClassCastException: com.whatever.MyPlugin cannot be cast to com.whomever.ServicePluginInterface
As a second point of reference, I am also able to construct a synthetic class loader (separate java executable that uses the same class loading mechanism. Again, since I do not have the original source to ServicePluginInterface, I used the class files from the WAR. This second, synthetic loader, or faux-servlet if you will, CAN load MyPlugin just fine. So I would postulate that the Tomcat JVM seems to be detecting some sort of difference between the classes found inside the WAR, and extracted class files. However, since all I did to extract the class files was to open the WAR as a zip and copy them out, it is hard to imagine what that might be.
Javier made a helpful suggestion about removing the definition of ServicePluginInterface, the problem with that solution was that the ZipClassLoader that the servlet uses to load the plugin out of the jar overrides the ClassLoader findClass function to pull the class out of the JAR like so:
protected Class<?> findClass(String name) throws ClassNotFoundException
{
ZipEntry entry = this.myFile.getEntry(name.replace('.', '/') + ".class");
if (entry == null) {
throw new ClassNotFoundException(name);
}
...
}
The class ZipClassLoader then recursively loads all parent objects and interfaces from the jar. This means that if the plugin jar does not contain the definition for ServicePluginInterface, it will fail.
Classes defined by different class loaders are different:
At run time, several reference types with the same binary name may be
loaded simultaneously by different class loaders. These types may or
may not represent the same type declaration. Even if two such types do
represent the same type declaration, they are considered distinct. JLS
In that case zipLoader returns an instance of MyPlugin that implements the other ServicePluginInterface (is it loaded from the zip too?):
(ServicePluginInterface)zipLoader.loadClass(pluginClass).newInstance();
It seems that the application server already has a definition of ServicePluginInterface, then you don't need to redeploy it. It should be enough to add the required files (ServicePluginInterface, etc.) as non-deployed dependecies of your project.
Another approach goes by living with the fact, and accessing methods in ServicePluginInterface via reflection (use the Class object returned by zipLoader, instead of ServicePluginInterface.class).
We're using an Android Library Project to share core classes and resources across different builds (targets) of our Android application. The Android projects for each specific target reference the Core library project (behind the scenes, Eclipse creates and references a jar from the referenced library project).
Overriding resources such as images and XML layouts is easy. Resource files placed in the target project, such as the app icon or an XML layout, automatically override the core library's resources with the same name when the app is built. However, sometimes a class needs to be overridden to enable target-specific behavior. For example, the Amazon target preferences screen cannot contain a link to the Google Play app page, requiring a change in the Amazon project's preferences.xml and preferences Activity class.
The goal is to reduce the amount of duplicate code among target projects while removing as much target-specific code from the Core library as possible. We've come up with a couple of approaches to implement logic specific to different targets:
Write the target-specific functions within Core library classes and use if/switch blocks to select behavior based on product SKU. This approach is not very modular and bloats the Core library codebase.
Extend the particular Core class in a target project and override the base (Core) class functions as needed. Then keep a reference to the base-class object in the Core library and instantiate it with an extended class object (from How to override a class within an Android library project?)
Are there other strategies to override or extend an Android library project class? What are some of the best practices for sharing and extending common classes among Android app targets?
Library project is referenced as a raw project dependency (source-based mechanism), not as a compiled jar dependency (compiled-code based library mechanism).
#yorkw this is not true for the latest versions of ADT Plugin for Eclipse
http://developer.android.com/sdk/eclipse-adt.html
From version 17 Change log
New build features
Added feature to automatically setup JAR dependencies. Any .jar files in the /libs folder are added to the build configuration (similar to how the Ant build system works). Also, .jar files needed by library projects are also automatically added to projects that depend on those library projects. (more info)
More info http://tools.android.com/recent/dealingwithdependenciesinandroidprojects
Before that, update overwriting of the Activity from Library project was easy, just exclude the class. Now the library is included as jar file, and there is no way to exclude class file from jar dependency.
EDIT:
My solution to overwrete/extend Activity from library jar:
I created a simple util class:
public class ActivityUtil {
private static Class getActivityClass(Class clazz) {
// Check for extended activity
String extClassName = clazz.getName() + "Extended";
try {
Class extClass = Class.forName(extClassName);
return extClass;
} catch (ClassNotFoundException e) {
e.printStackTrace();
// Extended class is not found return base
return clazz;
}
}
public static Intent createIntent(Context context, Class clazz) {
Class activityClass = getActivityClass(clazz);
return new Intent(context, activityClass);
}
}
In order to overwrite a library's "SampleActivity" class it a the project which depends on that library, create a new class with the name SampleActivityExtended in the project in the same package and add the new activity to your AndroidManifest.xml.
IMPORTANT: all intents referencing overwritten activities should be created through the util class in the following manner:
Intent intent = ActivityUtil.createIntent(MainActivity.this, SampleActivity.class);
...
startActivity(intent);
behind the scenes, Eclipse creates and references a jar from the referenced library project.
This is not quite accurate. Library project is referenced as a raw project dependency (source-based mechanism), not as a compiled jar dependency (compiled-code based library mechanism). Currently Android SDK does not support exporting a library project to a self-contained JAR file. The library project must always be compiled/built indirectly, by referencing the library in the dependent application and building that application. When build dependent project, the compiled source and raw resources that need to be filtered/merged from Library project are copied and properly included in the final apk file. Note that Android team had started revamping the whole Library Project design (move it from ource-based mechanism to compiled-code based library mechanism) since r14, as mentioned in this earlier blog post.
What are some of the best practices for sharing and extending common classes among Android app targets?
The solution given by Android is Library Project.
The solution given by Java is Inheritance and Polymorphism.
Come together, the best practice IMO is the second option you mentioned in the question:
2.Extend the particular Core class in a target project and override the base (Core) class functions as needed. Then keep a reference to the base-class object in the Core library and instantiate it with an extended class object (from Android library project - How to overwrite a class?)
From my personal experience, I always use Android Library Project (Sometimes with Regular Java Project, for implementing/building common-lib.jar that contains only POJO) manage common code, for instance SuperActivity or SuperService, and extends/implements proper classes/interfaces in the dependent project for Polymorphism.
Solution based on PoisoneR's solution and Turbo's solution.
public static Class<?> getExtendedClass(Context context, String clsName) {
// Check for extended activity
String pkgName = context.getPackageName();
Logger.log("pkgName", pkgName);
String extClassName = pkgName + "." + clsName + "Extended";
Logger.log("extClassName", extClassName);
try {
Class<?> extClass = Class.forName(extClassName);
return extClass;
} catch (ClassNotFoundException e) {
e.printStackTrace();
// Extended class is not found return base
return null;
}
}
The benefits of this is that
The extended class can be in the project's package, not the library's package. Thanks to Turbo for this part.
By taking a String as an argument instead of a Class object, this method is able to be used even with ProGuard. getName() is where the problem is with ProGuard, as that will return something like "a" instead of the name of the original class. So in the original solution instead of looking for ClassExtended it will look for aExtended instead, something which does not exist.
What about using a callback approach here? (Okay, callback is a little bit misleading but I currently have no other word for it:
You could declare an interface in every Activity which should/may be expanded by the user. This interface will have methods like List<Preference> getPreferences(Activity activity) (pass whatever parameters you need here, I would use an Activity or at least a Context to be futureproof).
This approach could give you what you want when I have understood it correctly. While I haven't done this before and don't know how other people handle this I would give it a try and see if it works.
Could you, please, clarify what is different in Kindle and regular Android?
I think - they are the same.
What you need is different resources for Kindle and other devices. Then use appropriate resource.
For example I use 2 links to store:
<string name="appStore"><a href=http://market.android.com/details?id=com.puzzle.jigsaw>Android Market</a> or <a href=http://www.amazon.com/gp/mas/dl/android?p=com.puzzle.jigsaw>Amazon Appstore</a> <br>http://market.android.com/details?id=com.puzzle.jigsaw <br>href=http://www.amazon.com/gp/mas/dl/android?p=com.puzzle.jigsaw</string>
<string name="appStore_amazon"><a href=http://www.amazon.com/gp/mas/dl/android?p=com.puzzle.jigsaw>Amazon Appstore</a> <br>href=http://www.amazon.com/gp/mas/dl/android?p=com.puzzle.jigsaw</string>
and use appStore for all none Amazone product and appStore_amazon for Kindle.
How to determine where are you on run time - that would be another question which was answered here many times.
I was inspired by PoinsoneR's answer to create a Utility class to do the same thing for Fragments - override a fragment in an android Library. The steps are similar to his answer so I won't go into great detail, but here is the class:
package com.mysweetapp.utilities;
import android.support.v4.app.Fragment;
public class FragmentUtilities
{
private static Class getFragmentClass(Class clazz)
{
// Check for extended fragment
String extClassName = clazz.getName() + "Extended";
try
{
Class extClass = Class.forName(extClassName);
return extClass;
}
catch (ClassNotFoundException e)
{
e.printStackTrace();
// Extended class is not found return base
return clazz;
}
}
public static Fragment getFragment(Class clazz)
{
Class fragmentClass = getFragmentClass(clazz);
Fragment toRet = null;
try
{
toRet = (Fragment)fragmentClass.newInstance();
return toRet;
}
catch (InstantiationException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (IllegalAccessException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
return toRet;
}
}
Usage:
FragmentUtilities.getFragment(MySpecialFragment.class)
You can also use an Activity factory if you need to provide extended activitys for differnt build variants and have your library deal with the abstract factory alone. This can be set in your build variants Application file.
I have an application that allows, using an abstract class, people to write their own implementations. I load these implementations as .class-files from a directory. Currently, I have this solution:
File classDir = new File("/users/myproject/classes/");
URL[] url = { classDir.toURI().toURL() };
URLClassLoader urlLoader = new URLClassLoader(url);
String filename;
for (File file : classDir.listFiles()) {
filename = string.getFilenameWithoutExtension(file);
if (filename.equals(".") || filename.equals("..") || filename.startsWith("."))
continue;
AbstractClass instance = (AbstractClass)urlLoader
.loadClass("org.mypackage." + filename)
.getConstructor(ConfigUtil.class, DatabaseUtil.class, StringUtil.class)
.newInstance(config, database, string));
instance.doSomething();
}
As you see - I need to specify the package the classes are located in in order to correctly load them. Omitting the package, I get an
java.lang.NoClassDefFoundError:
MyClass (wrong name: org/mypackage/MyClass)
error.
Now, from a architectural POV, I think it is very ill-designed that classes other people designed have to be compiled to MY package when loading them.
So I ask you: Is there a way I can load classes form the file system without having to specify the package they reside in?
Yes; implement an interface (or use an annotation).
Then use any class-scanning library (there are lots of SO questions about this, like this one) to load the particular class in question. Searching for "Java class scanning" or "Java plugin mechanism" will help.
You might also just want to use the Java Plugin Framework and avoid some effort. Although it's not clear to me that it's maintained any more, I know people are still using it.
You can use the ServiceProvider to load implementations which you don't know.