I am working on an existing application which has quite a lot of external JAR dependencies. I need to enhance it to integrate with a third-party application which has an API. Sadly, the API is not well contained and also comes with a large number of its own dependencies some of which clash with mine.
I believe I should solve this using Classloaders, but I'm struggling to see how to structure them correctly.
To keep it simple, assume we have myapp.jar with a hibernate3.jar dependency, and vendor-api.jar with a hibernate2.jar dependency (and assume these are incompatible).
My new piece of code will reside in the myapp.jar library (although it could be in a separate jar if this would help). Due to the way the vendor API works, my new code needs to extend a class from the vendor-api.jar library.
How can I structure the Classloaders in such a way that anything within the vendor-api.jar accesses only its own dependencies, and anything on my side accesses only the myapp.jar and dependencies?
Thanks, Jon
I've not tried this myself, but from memory each clashing class needs to be in a sibling classloader and any communication between the two needs to go through a common ancestor. However, the ancestor cannot (AFAIK) "directly" reference classes from its children and must access them through the reflection API.
Something along these lines ought to work (dry-coded) YMMV. Comments and error-spotting welcome.
class Orchestrator {
URL[] otherAppClasspath = new URL[] { new URL("file:///vendor-api.jar"),
new URL("file:///hibernate2.jar"),
new URL("file:///code-extending-vendor-api.jar" };
URLClassLoader otherAppLoader = new URLClassLoader(otherAppClasspath);
URL[] yourAppClasspath = new URL[] { new URL("file:///myapp.jar"),
new URL("file:///hibernate3.jar") };
URLClassLoader yourAppLoader = new URLClassLoader(yourAppClasspath);
public void start() {
Method yourAppEntryPoint = yourAppLoader.findClass("com/company/Main").getMethod("start", new Class[] { Orchestrator.class } );
yourAppEntryPoint.invoke(null, new Object[] { this });
}
public static void main(String[] args) {
new Orchestrator().start();
}
// define some abstracted API here that can be called from your app
// and calls down into classes in the other app
public String getSomeResultFromOtherApp(int someArgument) {
Method otherAppAPI = otherAppLoader.findClass("com/company/ExtendingAPIClass").getMethod("getSomeResult", new Class[] { Integer.class });
return (String)otherAppAPI.invoke(null, new Object[] { someArgument });
}
}
#fd's answer gives a technical mechanism that ought to work - give or take some typos, exception handling, etc.
However, I think you would be better off not trying to do this ... unless your dependencies on the 3rd party APIs are restricted to a very small number of methods on a very small number of classes. Each class that you depend on has to be dynamically loaded and each method has to be looked up and invoked reflectively. Too much of that and your codebase will suffer.
If I were you, I'd try to resolve the dependency issue some other way:
Try to get the 3rd party vendor to use hibernate3.jar
Change your application to use hibernate2.jar
Refactor so that your application code and the 3rd party library are in separate JVMs or separate webapps.
From what you say, this might be hard.
Using OSGi may help you in the long term. Here is an implementation I am trying now- http://felix.apache.org
Related
I am developing a component based game engine in java, right now when i have changes to the components I need to rebuild and restart the editor for the changes to take effect (or i can use some limited hot code injection if the application is running in debug mode).
I am looking for a way to allow the user to modify the source of the components and reload them without having to restart the application (maybe just exit and enter game mode instead). Also a important feature that I need is that the final exported code should be native Java code(so no interpreter should be used in the final result)).
Can you give me any pointers on how to integrate the beanshell interpreter in the project? I can manually monitor the source folder for changes and feed the updated java classes to it, but how is the hotswap going to occur really?
First of all, the title is a bit confusing. You don't need to integrate a BeanShell. What you actually need are:
to define a proper architecture
to use Java Compiler API in order to work with java classes
Architecture
Let's say you have an object graph. There are lots of objects, references, etc. So it will be really a tricky task to replace some instance with the new one. Instead of solving this problem you can hide dynamic part behind a "static" proxy. Proxy will handle all reloading stuff (including source folder monitoring).
Before reload:
After reload:
Having that done you can easily track changes and update dynamic part when needed.
Java Compiler API
Instead of using interpreted languages you can use Java, compiling it on the fly and loading using 'Class.forName()'. There are a lot of different examples due to the fact this approach was there for a while.
Here are some details:
Add dynamic Java code to your application
Compiling fully in memory with javax.tools.JavaCompiler
Basically you want to implement extensibility or plugin design pattern. There are multiple ways to implement this scenario.
Which ever the component you want to allow someone else to reload the module, define an interface and implement your own implementation as a default one. For example, Here I am trying to provide a HelloInterface which each country can implement and load anytime,
public interface HelloInterface {
public String sayHello(String input);
..
}
public class HelloImplDefault implements HelloInterface {
public String sayHello(String input) {
return "Hello World";
}
}
Now allow user to add a plugin (custom implementation) files to a pre-configured path. You can either user FileSystemWatcher or a background thread to scan this path and try to compile and load the file.
To compile java file,
private void compile(List<File> files) throws IOException{
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
StandardJavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);
Iterable<? extends JavaFileObject> compilationUnits = fileManager
.getJavaFileObjectsFromFiles(files);
JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, diagnostics, null,
null, compilationUnits);
boolean success = task.call();
fileManager.close();
}
To load class file,
private void load(List<File> files) throws MalformedURLException, InstantiationException, IllegalAccessException, ClassNotFoundException{
ClassLoader cl = Thread.currentThread().getContextClassLoader();
for(File f: files){
if(f.getName().endsWith(".class") && !loadedClass.contains(f.getName())){
URL url = f.toURL();
URL[] urls = new URL[]{url};
Object obj = cl.loadClass(f.getName().replace(".class", "")).newInstance();
if(obj instanceof HelloInterface){
HelloProviders.addProvider((HelloInterface)obj);
System.out.println("Loaded "+ ((HelloInterface)obj).getProviderName());
}else{
//Add more classes if you want
}
loadedClass.add(f.getName());
}
}
}
At this point you can read custom implementation and loaded in system class loader. Now you are ready to go. There are security implications to this approach which you need learn from internet.
I implemented one sample code and posted in github, please take a look. Happy coding!
Take a look at the tapestry-ioc inversion of control container which supports live-reloading.
When in development mode (tapestry.production-mode=false) you can live reload your services. Note that if a service interface changes, you will need a restart. But any changes to the service implementation that do not alter the service interface can be live-reloaded.
I'm working on a JSR-303 validation framework for GWT. Some of you may have heard of it even though it is a small project. Here is gwt-validation.
In the old days (v1.0) it used a marker interface for each class and each class had metadata generated separately. This was bad because it was not part of the JSR-303 standard and we moved on to the next idea.
In version 2.0 it scans the classpath at runtime using Reflections. This is great. The downside is that it doesn't seem to be able to work inside of containerized environments or those with special restrictions.
This is probably my fault, look at the following code:
//this little snippet goes through the classpath urls and ommits jars that are on the forbidden list.
//this is intended to remove jars from the classpath that we know are not ones that will contain patterns
Set<URL> classPathUrls = ClasspathHelper.forJavaClassPath();
Set<URL> useableUrls = new HashSet<URL>();
for(URL url : classPathUrls) {
boolean use = true;
for(String jar : this.doNotScanJarsInThisList) {
if(url.toString().contains(jar)) {
use = false;
break;
}
}
if(use) {
useableUrls.add(url);
}
use = false;
}
ConfigurationBuilder builder = new ConfigurationBuilder()
.setUrls(useableUrls)
.setScanners( new TypeAnnotationsScanner(),
new FieldAnnotationsScanner(),
new MethodAnnotationsScanner(),
new SubTypesScanner()
)
.useParallelExecutor()
;
this.reflections = new Reflections(builder);
I'm using the filter to remove jars that I know can't have annotations in them that I'm interested in. As I mention this gives a huge speed boost (especially on large classpaths) but the ClasspathHelper.forJavaClassPath() that I'm basing this on probably isn't the best way to go in container environments. (e.g. Tomcat, JBoss)
Is there a better way or at least a way that will work with a container environment and still let my users filter out classes they don't want?
I've looked, some, into how the Hibernate Validation project (the reference implementation for JSR-303) and they appear to at least be using (at least in part) the Annotations Processing in Java 6. This can't be all of the story because that didn't show up until JDK6 and Hibernate Validator is JDK5 compatible. (See: hibernate documentation)
So, as always, there's more to the story.
I've read these threads, for reference:
About Scannotation which has been pretty much replaced by Reflections.
This one but it uses File and I'm not sure what the implications are of that in things like GAE (Google App Engine) or Tomcat.
Another that goes over a lot of the things I've talked about already.
These threads have only helped so much.
I've also read about the annotation processing framework and I must be missing something. It appears to do what I want but then again it appears to only work at compile time which I know isn't what is done by Hibernate Validator. (Can anyone explain how it does scanning? It works on GAE which means it can't use any of the IO packages.)
Further, would this code work better than what I have above?
Set<URL> classPathUrls = ClasspathHelper.forClassLoader(Thread.currentThread().getContextClassLoader());
Could that correctly get the classloader inside of a Tomcat or JBoss container? It seems scan a smaller set of classes and still finish okay.
So, in any case, can anyone help me get pointed in the right direction? Or am I just stuck with what I've got?
You could take a look at Spring's annotation support.
Spring can scan annotations in files (using asm IIRC), and works in and out of a container.
It may not be easy because it goes through Spring's Resource abstraction, but it should be doable to reuse (or extract) the relevant code.
I recently discovered the library kryonet, which is super awesome and fits my needs excellently.
However, the one problem that I am having is developing a good strategy for registering all of the classes that can be transferred.
I know that I can write a static method in each object that will return a list of all of the classes that it uses, but I would really rather not have to do that (for my own time purposes, as well as those who will be extending these objects).
I was playing around with trying to see if there was a way to get all of the classes that an object references (in it's fields as well as it's inheritance) from the getClass() method, but I was unable to have any success.
Finally, I know that kryo has kryo.setRegistrationOptional(true) but I am having a very difficult time trying to figure out how to use it. When this option is turned on, kryo still seems to throw exceptions if I haven't registered classes. Also, this method supposed is much slower than being able to register all of the classes. I'm fine if the first time you need to send an object using this method is slow, but I don't know if I'm okay with serious performance degradation every time that I want to send an object.
Ideally, I'll have a package of objects that I will want to send using kryonet. If there was just some was to scan that package and determine all of the classes that I need to register, that would be excellent. Now not all of my clients would need to register every object, but that's something of a separate issue, and I don't know if there is a solution to that.
If anyone could point me in the right direction that would be excellent.
Classes may come from different places such as disk, network, memory (dynamically generated). Therefore, obtaining information about classes to be registered with Kryo has to be handled separately for each specific case.
If you can read classes from a jar file then the following snippet should get you started.
private static List<Class<?>> getFromJarFile(final String jar, final String packageName) throws ClassNotFoundException, IOException {
final List<Class<?>> classes = new ArrayList<Class<?>>();
final JarInputStream jarFile = new JarInputStream(new FileInputStream(jar));
JarEntry jarEntry = null;
do {
jarEntry = jarFile.getNextJarEntry();
if (jarEntry != null) {
String className = jarEntry.getName();
if (className.endsWith(".class")) {
className = className.substring(0, className.lastIndexOf('.')); // strip filename extension
if (className.startsWith(packageName + "/")) { // match classes in the specified package and its subpackages
classes.add(Class.forName(className.replace('/', '.')));
}
}
}
} while (jarEntry != null);
return classes;
}
I am writing a static analysis tool for an assignment, it analyses Java bytecode using the ASM library. One of the parts of ASM that we use requires (or at least, appears to require) that the class be loaded from the ClassLoader.
We were hoping the tool would be able to analyse .class files without requiring them on the classpath. We already load the .classes from a specified directory at run time and read them in using an InputStream. This is acceptable for ASM in most cases. There are some classes, such as SimpleVerifier, which attempt to load the classes though.
Is it possible, under this scenario, to register the .class files to be loaded so that calls to Class.forName() will load them? Or is there an easy way to extend the ClassLoader to allow this?
Edit: the information on URLClassLoader was useful. Unfortunately, using Thread.currentThread().setContextClassLoader() to an instance of that didn't work in this scenario. The library code I'm calling into uses a loader it retrieves on instance initialisation using getClass().getClassLoader().
By the time I set the URLClassLoader the class hasn't been initialised so I guess the contextClassLoader does not load that class.
Have I understand the responses correctly? Would using the URLClassLoader to load the 3rd party class be a possibility?
Almost.
If you have classes compiled somewhere, you can load them with a URLClassLoader. You can then set this ClassLoader to be the ClassLoader for the current Thread: Thread.setContextClassLoader(ClassLoader)
Users can that get the current threads context class loader and use that to access the class definition.
First of all, ASM can be used in a such way that it won't use ClassLoader to obtain information about classes.
There are several places in ASM framework where it loads classes by default but all those places can be overridden in your own subclasses. Out of the top of my head:
ClassWriter.getCommonSuperClass() method is called only when ClassWriter.COMPUTE_FRAMES flag is used and can be overwriten to not use ClassLoader to get inforamtion about classes. You can find an example of that in ClassWriterComputeFramesTest that introduces a ClassInfo abstraction
Similarly SimpleVerifier.getClass() method is used by SimpleVerifier.isAssignableFrom() and you can overwrite the latter and use the ClassInfo abstraction to find the common super type. If I am not mistaken, AspectWerkz project had implemented similar thing in its type pattern matching code. Also note that there is SimpleVerifier.setClassLoader() method, which you can use if you still want to load your own classes.
On a side note, on a Sun's JVMs, loaded classes gets to PermGen area and can't be unloaded, so it is not a good idea to load classes only for static code analysis purposes if you can avoid that, especially if tool would be integrated into a long-live process, such as IDE.
You can't, as far as I know, extend the System class loader at runtime, but you can dynamically load classes from an arbitrary location (jar or directory) using URLClassLoader.
You could try to setup a "launcher" in the startup of your application that creates an URLClassLoader passing it the locations on the classpath and your own .class locations and start the application from that classloader.
When the SimpleVerifier is loaded by the URLClassLoader it will also be able to load the classes from the extra locations.
Yes, you can use URLClassLoader
I have a test where I do load the class at runtime. This class is not in the classpath (nor even exist when the test is run for that matter ), later is it loaded and works great.
Here's the code.
void testHello() throws MalformedURLException, ClassNotFoundException {
URL[] url = {
new URL("file:/home/oreyes/testwork/")
};
try {
new URLClassLoader(url).loadClass("Hello");
throw new AssertionError("Should've thrown ClassNotFoundException");
} catch ( ClassNotFoundException cnfe ){}
c.process();// create the .class file
new URLClassLoader(url).loadClass("Hello");
// it works!!
}
Taken from this question.
I created my own ClassLoader its quite simple.
/**
* Used to hold the bytecode for the class to be loaded.
*/
private final static ThreadLocal<byte[]> BYTE_CODE = new ThreadLocal<byte[]>();
#Override
protected Class<?> findClass(final String name) throws ClassNotFoundException {
final byte[] bytes = BYTE_CODE.get();
if (null == bytes) {
throw new ClassNotFoundException(name);
}
return this.defineClass(null, bytes, 0, bytes.length);
}
From the perspective of a cross application/applet java accessibility service, how would you link to a package but only optionally execute an action based on existence/availability of a package (being already loaded) at runtime?
I think what I'm interested in here is a way to resolve the class identity crisis but rather than the issue being between 2 apps sharing objects, being a service loaded at a higher level of the class loaders.
It seems like reflection is the way to go, but I am not sure how or if I can implement a derived class this way. I need to add a specific listener derived from the specific optional classes, I can load the listener using the applet class loader but the internals still fail. Say you wanted to add an JInternalFrameListener, but Swing wasn't guaranteed to be available, using reflection you can find the method to add the listener, but how can you create and have the frame listener work if it cannot find any of the related classes because they can't be found in the base classloader! Do I need to create a thread and use setContextClassLoader to the classloader that knows about swing so that I can get the class to be loaded reliably? simply trying to set the class loader on my existing thread didn't seem to work.
Earlier description of issues
Sorry, I'm not quite sure what to ask or how to make this clear, so it rambles on a bit.
Say a class uses some feature of another, but the other class may not always be available - say finding the website from JNLP if this is a JNLP app.
At one stage I thought that simply compiling against JNLP would mean that my class would not load unless JNLP was available, and so to identify this optional section I simply wrapped a try{} catch( NoClassDefFoundError ) around it.
Later something changed (perhaps changing jdk or ?? I don't recall) and it seemed that I should also use a try{} catch( ClassNotFoundException ).
Now I wanted to extend this idea to other optional features, but it doesn't seem to work consistently.
Say I wanted to add some feature to do something more advanced in a JRE1.6 runtime using the same jar and classes as I run in a JRE1.3, or say I want to handle some controls in a specific gui toolkit which may not always be used like SWT or oracle.forms.
Is there some way of doing this more reliably? It just seems wrong to cause an exception and catch it to ignore it all the time.
The current issue comes down to being able to compile against oracle.forms but then the accessibility component installed in ext is unable to access the oracle.forms classes even though objects from the oracle.forms package have been created. If I throw the frmall.jar into the ext directory to test then the accessibility component works up to the point that the whole lot gets flakey because of the different versions of the same package.
I seem to be caught up on an issue with the class loader not being the right one or something (??). How do I find the right one?
Edit:
The answers so far are kindof interesting but not quite getting me where I want to be.
In the case of the gui components I currently compile in the form of a factory something like...
import oracle.forms.ui.*;
import java.awt.*;
static public IComponentNode newNode( INode parent, Component component ) {
System.out.println( component.getClass().toString() );
try{
if( component instanceof FormDesktopContainer )
... does stuff here like return new FormDesktopNode( parent, (FormDesktopContainer) component )
} catch ( NoClassDefFoundError a ) {
System.out.println( a.getMessage() );
}
where it prints out class oracle.forms.ui.FormDesktopContainer and then throws and exception on the instanceof call with NoClassDefFound thus printing out oracle/forms/ui/FormDesktopContainer
So how can it have an instance of a class yet not be able to find it?
How about this? messy, but it ought to work:
public boolean exists(String className){
try {
Class.forName(className);
return true;
}
catch (ClassNotFoundException){
return false;
}
}
You can check the availability of a class by calling
ClassLoader.getSystemClassLoader().loadClass("my.package.MyClass")
if it throws a ClassNotFoundException, it's not available. If you get the Class object, it is. You can then choose behaviour based on whether or not the class is available.
I suggest compiling the majority of your code against your minimum target. Have code that uses particular optional libraries clearly separated, but dependent upon the bulk of your code. Dynamically load the code that uses optional libraries once. The main class should do something that checks for the presence of the required library/version in its static initialiser.
In the case of JNLP, your JNLP main class load the JNLP dependent code statically.
(Note that attempting to catch class loading related exceptions from normally linked code is unreliable.)
getSystemClass loader was not useful for this purpose as there where multiple possible class loaders to interact with based on which applet the given window was in. The accessibility components being loaded at a more base class loader cannot see the applet specific classes.
To interact with the objects reflection does the job, though it does add so much more to maintain.
// statically linking would be
return component.getText();
// dynamically is
try {
return (String)component.getClass().getMethod("getText", new Class [] {}).invoke(component, new Object [] {});
} catch (Throwable e) {
e.printStackTrace();
}
The trickier bit is in writing a class derived from an interface that is not directly accessible, using the Proxy service allows this to be accomplished, providing the proxy service the applet specific class loader and the dynamically loaded class for the interface.
public void addListener(Container parent) {
if (parent == null) { return; }
if ("oracle.forms".equals(parent.getClass().getName())) {
// Using the class loader of the provided object in the applet
// get the "class" of the interface you want to implement
Class desktopListenerClass = Class.forName( "oracle.DesktopListener"
, true, parent.getClass().getClassLoader());
// Ask the proxy to create an instance of the class,
// providing your implementation through the InvocationHandler::invoke
Object desktopListener = Proxy.newProxyInstance(
parent.getClass().getClassLoader()
, new Class[] { desktopListenerClass }, new InvocationHandler() {
public Object invoke(Object proxy, Method method, Object[] args)
throws Throwable {
if ("functionName".equals(method.getName())) {
// do stuff
}
return null;
}
});
// do something with your new object
Method addDesktopListener = parent.getClass().getMethod("");
addDesktopListener.invoke(parent, desktopListener);
}
}
examples cut down to show general method