In the Eclipse environment I have project A. A has dependencies to projects or libraries B and C. (does not make a difference if they are projects or libraries)
B has dependency on LibX.v1 and C has dependency to LibX.v2.
During runtime, A will need B.jar and C.jar. Also classes in B.jar will need LibX.v1 and classes in C.jar will need LibX.v2 .
Being the different versions of the same library, LibX.v1 and LibX.v2 has same classes, so it is possible that a class may be loaded from the wrong version of the library in the runtime, causing lots of trouble. How do I manage these kind of situations?
Kind regards
Seref
Really, you don't. Most Java classloaders don't give any guarantees about the order in which libraries will be loaded, so you really need to be sure not to duplicate dependencies on your classpath. I would check if B or C could be updated to a newer version, or if LibX is fully backward-compatible.
Edit: actually, I found something that may help you. It's called OSGi, and I haven't used it, but it seems like it might do what you're trying to do. Here is a link to an intro article.
you'll need to use a 'multiple classloader' mechanism (which is what OSGI is capable of). In such a scenario the classloader for class B can only see LibX.v1, while classloader C only sees LibX.v2.
You can it do it with OSGI, but OSGI is much more than this and it's not trivial to get started with.
another plug-in framework is JPF, which also offers more than just using multiple classloaders. i think they both use the principle of 1 classloader per plug-in (module, whatever, ...)
maybe have a look at classworlds. I think it's possible to do it with this library. the benefit is that it focuses on the classloading aspect. documentation is not so good. classworlds uses the notion of class realms. take a look at the API Usage example on the site.
i'll add an example here if i find one.
anyway, no trivial solution for this i think
solving it with dependency management is the best solution of course, but it's not always possible.
Related
I expected it's possible to use i.e. Guava-19 in myModuleA and guava-20 in myModuleB, since jigsaw modules have their own classpath.
Let's say myModuleA uses Iterators.emptyIterator(); - which is removed in guava-20 and myModuleB uses the new static method FluentIterable.of(); - which wasn't available in guava-19. Unfortunately, my test is negative. At compile-time, it looks fine. In contrast to runtime the result is a NoSuchMethodError. Means that, the class which was the first on the classloader decides which one fails.
The encapsulation with the underlying coupling? I found a reason for myself. It couldn't be supported because of transitive dependencies would have the same problem as before. If a guava class which has version conflicts occurred in the signature in ModuleA and ModuleB depends on it. Which class should be used?
But why all over the internet we can read "jigsaw - the module system stops the classpath hell"? We have now multiple smaller "similar-to-classpaths" with the same problems. It's more an uncertainty than a question.
Version Conflicts
First a correction: You say that modules have their own class path, which is not correct. The application's class path remains as it is. Parallel to it the module path was introduced but it essentially works in the same way. Particularly, all application classes are loaded by the same class loader (by default at least).
That there is only a single class loader for all application classes also explains why there can't be two versions of the same class: The entire class loading infrastructure is built on the assumption that a fully qualified class name suffices to identify a class with a class loader.
This also opens the path to the solution for multiple versions. Like before you can achieve that by using different class loaders. The module system native way to do that would be to create additional layers (each layer has its own loader).
Module Hell?
So does the module system replace class path hell with module hell? Well, multiple versions of the same library are still not possible without creating new class loaders, so this fundamental problem remains.
On the other hand, now you at least get an error at compile or launch due to split packages. This prevents the program from subtly misbehaving, which is not that bad, either.
Theoretically it is possible to use different versions of the same library within your application. The concept that enables this: layering!
When you study Jigsaw under the hood you find a whole section dedicated to this topic.
The idea is basically that you can further group modules using these layers. Layers are constructed at runtime; and they have their own classloader. Meaning: it should be absolutely possible to use modules in different versions within one application - they just need to go into different layers. And as shown - this kind of "multiple version support" is actively discussed by the people working on java/jigsaw. It is not an obscure feature - it is meant to support different module versions under one hood.
The only disclaimer at this point: unfortunately there are no "complete" source code examples out there (of which I know), thus I can only link to that Oracle presentation.
In other words: there is some sort of solution to this versioning problem on the horizon - but it will take more time until to make experiences in real world code with this new idea. And to be precise: you can have different layers that are isolated by different class loaders. There is no support that would allow you that "the same object" uses modV1 and modV2 at the same time. You can only have two objects, one using modV1 and the other modV2.
( German readers might want to have a look here - that publication contain another introduction to the topic of layers ).
Java 9 doesn't solve such problems. In a nutshell what was done in java 9 is to extend classic access modifiers (public, protected, package-private, private) to the jar levels.
Prior to java 9, if a module A depends on module B, then all public classes from B will be visible for A.
With Java 9, visibility could be configured, so it could be limited only to a subset of classes, each module could define which packages exports and which packages requires.
Most of those checks are done by the compiler.
From a run time perspective(classloader architecture), there is no big change, all application modules are loaded by the same classloader, so it's not possible to have the same class with different versions in the same jvm unless you use a modular framework like OSGI or manipulate classloaders by yourself.
As others have hinted, JPMS layers can help with that. You can use them just manually, but Layrry might be helpful to you, which is a fluent API and configuration-based launcher for running layered applications. It allows you to define the layer structure by means of configuration and it will fire up the layer graph for you. It also supports the dynamic addition/removal of layers at runtime.
Disclaimer: I'm the initial creator of Layrry
Edit: After reviewing the play, the example I used below is a tad misleading. I am looking for the case where I have two 3rd party jars (not homegrown jars where I have access to the source code) that both depend on different versions of the same jar.
Original:
So I've recently familiarized myself with what OSGi is, and what ("JAR Hell") problems it addresses at its core. And, as intrigued as I am with it (and plan on migrating somewhere down the road), I just don't have it in me to begin learning what it will take to bring my projects over to it.
So, I'm now lamenting: if JAR hell happens to me, how do I solve this sans OSGi?
Obviously, the solution would almost have to involve writing my own ClassLoader, but I'm having a tough time visualizing how that would manifest itself, and more importantly, how that would solve the problem. I did some research and the consensus was that you have to write your own ClassLoader for every JAR you produce, but since I'm already having a tough time seeing that forest through the trees, that statement isn't sinking in with me.
Can someone provide a concrete example of how writing my own ClassLoader would put a band-aid on this gaping wound (I know, I know, the only real solution is OSGi)?
Say I write a new JAR called SuperJar-1.0.jar that does all sorts of amazing stuff. Say my SuperJar-1.0.jar has two other dependencies, Fizz-1.0.jar and Buzz-1.0.jar. Both Fizz and Buzz jars depend on log4j, except Fizz-1.0.jar depends on log4j-1.2.15.jar, whereas Buzz-1.0.jar depends on log4j-1.2.16.jar. Two different versions of the same jar.
How could a ClassLoader-based solution resolve this (in a nutshell)?
If you're asking this question from an "I'm building an app, how do I avoid this" problem rather than a "I need this particular solution" angle, I would strongly prefer the Maven approach - namely, to only resolve a single version of any given dependency. In the case of log4j 1.2.15 -> 1.2.16, this will work fine - you can include only 1.2.16. Since the older version is API compatible (it's just a patch release) it's extremely likely that Fizz 1.0 won't even notice that it's using a newer version than it expected.
You'll find that doing this will probably be way easier to debug issues with (nothing confuses me like having multiple versions of even classes or static fields floating around! Who knows which one you're dealing with!) and doesn't need any clever class loader hacks.
But, this is exactly what all the appservers out there have to deal with. Pretend that your Fizz and Buzz are web applications (WARs), and Super-Jar is you appserver. Super-Jar will arrange a class loader for each web app that "breaks" the normal delegation model, i.e. it will look locally (down) before looking up the hierarchy. Go read about it in any of the appservers's documentation. For example http://download.oracle.com/docs/cd/E19798-01/821-1752/beade/index.html.
Use log4j-1.2.16. It only contains bugfixes wrt 1.2.15.
If Fizz breaks with 1.2.16, fork and patch it, then submit those patches back to the author of Fizz.
The alternative of creating custom classloaders with special delegation logic is very complex and likely to cause you many problems. I don't see why you would want to do this rather than just use OSGi. Have you considered creating an embedded OSGi framework, so you don't have to convert your whole application?
I'm facing the following problem: I have one module in my webapp that needs jaxb 1.x and the other module needs jaxb 2.x. The first module doesn't work with the new version of jaxb, and the opposite. How can I use these two jars in one project?
Thanks.
For a regular application, usually very different versions use different package names. If this is the case, you can use them both at once without problem. However if they are the same, you can use jarjar to rename the package.
However since you are using a web container each application should use the version you deploy and not the other version. i.e. the web container works it out for you.
OSGi is another container which manages the versions much more explicitly and give you more control over these issues (however I believe you need it just for this)
You have got a jar-hell issue. Generally speaking in normal java environment it's impossible to solve this problem. You have to force modularization into your project by using OSGI. Starting point: http://www.osgi.org/About/HowOSGi
If you are using the JAXB reference implementation, then you can use your JAXB 1 models with the JAXB 2 runtime by including the jaxb1-impl.jar.
http://jaxb.java.net/faq/index.html#running1Apps
As Shaman said is imposible to resolve this issue.
Let's see: the servlet container JRE has only one classloader, and this classloader can load and use one class from jaxb or the other, but not both that will give you a classdefnotfound exception or something similar.
You can not solve this directly:
you can get the code (is opensource) and change the package of one to another name so the classloader can use both. I do not recommend you this solution, is a bad one.
Better is that you migrate the code to use the most modern API (jaxb 2)
I'm writing a server application which makes use of external modules. I would like to make them to be upgradeable without requiring server restart. How do I do that? I've found OSGi but it looks very complicated and big for my task.
Simple *.jar files are ok, but once they are loaded, I suppose, I cannot unload them from VM and replace with another version on-the-fly.
What approach can you suggest?
It seems like OSGi is exactly what you're asking for. It can be complex, but there are ways to deal with that. Some of the complexity can be mitigated by using SpringDM or something similar to handle the boilerplate tasks of registering and consuming services in the runtime. Annotation-driven service registration and dependency injection really reduces the amount of code that needs to be written.
Another way to reduce complexity is to deploy the bulk of your application in a single bundle and only deploy the parts that need to be modular into their own bundles. This reduces your exposure to registering and using services from other bundles in the runtime as well as reducing the complexity of deployment. Code running within a bundle can use other code in the same bundle just as in a standard Java app - no need to interact with the OSGi runtime. The opposite of this approach is to break up your application into lots of discrete bundles that export well-defined services to other bundles in the system. While this is a very modular approach, it does come with extra complexity of managing all those bundles and more interaction with the OSGi runtime.
I would suggest taking a look at the book "OSGi in Action" to get a sense of the issues and to see some decent samples.
It would at least require you to define your custom classloader... I don't see how can this be simpler than just using Felix, Equinox, Knoplerfish or any open source Osgi runtime to do the task.
Maybe SpringDM is simpler...
What you're going for is definitely possible. I believe that you can unload classes from memory by loading them in a separate ClassLoader and then disposing that ClassLoader. If you're not wanting to go all out and use OSGI, I'd recommend something like JBoss Microcontainer (http://www.jboss.org/jbossmc) or ClassWorlds (http://classworlds.codehaus.org/). It's not too terribly difficult to write something like this from scratch if your needs are specialized enough.
Hope this helps,
Nate
If you follow the ClassLoader route (is not that difficult, really), I suggest each module to be packaged in its own jar, and use a different ClassLoader to read each jar. that way, unloading a module is the same as "discarding" the ClassLoader.
OSGi is not so complicated - using PAX runner with maven worked as a breeze.
Or implement your own ClassLoader and set it to JVM :
java -Djava.system.class.loader=com.test.YourClassLoader App.class
Imagine I'm developing module A with two dependencies for B and C. What if B depends on module D version 2.0 but C depends on D version 3.0. To make things worse let D-3.0 is not back compatible with D-2.0 (interfaces were changed for example) and B is not supported already and there is no new version of B which could work with new version of D.
Is there any way I could run A with B and C dependencies?
Is there any way to have some class different versions to be correctly loaded and used from classpath?
Thanks for ideas.
OSGi is the way. : ) I am still learning this, but the least it does it solves this problem known as Jar Hell. See here http://www.osgi.org/About/WhyOSGi
Versioning - OSGi technology solves JAR hell. JAR hell is the problem that library A works with library B;version=2, but library C can only work with B;version=3. In standard Java, you're out of luck. In the OSGi environment, all bundles are carefully versioned and only bundles that can collaborate are wired together in the same class space. This allows both bundle A and C to function with their own library. Though it is not advised to design systems with this versioning issue, it can be a life saver in some cases.
You might find this repackaging tool useful. http://code.google.com/p/jarjar/ This allows you to rename a package. (Assuming the incompatible libraries have the same packages) Once you have renamed one or both you can use the different name to distigush which package/library you want to use.
I have seen this used to create a library which five different version of xerces, selectable by package. :P
NOTE: The JRE places embedded packages under com.sun.* with packages like com.sun.org.apache.*
Whether this can be resolved is very dependent on the type of dependencies you have. If they are direct compile time dependencies, then you have a big problem and may have to do some redesign.
OSGi may help, but only if the dependencies are runtime and accessed via interfaces as OSGi services. In this way, each service used is created using its own defined dependencies (and versions) and only exposed to the user via the interface. The consumer will only have a dependency to that interface. Thus in your case, if you have only references to classes in B and C that do not have direct references to D, you should be able to utilize OSGi.
On the other hand, if you reference classes from both B and C that have direct references to classes from D, this will not solve your problem since your module will be forced to import packages with appropriate versions and your conflict will still exist.