Edit: After reviewing the play, the example I used below is a tad misleading. I am looking for the case where I have two 3rd party jars (not homegrown jars where I have access to the source code) that both depend on different versions of the same jar.
Original:
So I've recently familiarized myself with what OSGi is, and what ("JAR Hell") problems it addresses at its core. And, as intrigued as I am with it (and plan on migrating somewhere down the road), I just don't have it in me to begin learning what it will take to bring my projects over to it.
So, I'm now lamenting: if JAR hell happens to me, how do I solve this sans OSGi?
Obviously, the solution would almost have to involve writing my own ClassLoader, but I'm having a tough time visualizing how that would manifest itself, and more importantly, how that would solve the problem. I did some research and the consensus was that you have to write your own ClassLoader for every JAR you produce, but since I'm already having a tough time seeing that forest through the trees, that statement isn't sinking in with me.
Can someone provide a concrete example of how writing my own ClassLoader would put a band-aid on this gaping wound (I know, I know, the only real solution is OSGi)?
Say I write a new JAR called SuperJar-1.0.jar that does all sorts of amazing stuff. Say my SuperJar-1.0.jar has two other dependencies, Fizz-1.0.jar and Buzz-1.0.jar. Both Fizz and Buzz jars depend on log4j, except Fizz-1.0.jar depends on log4j-1.2.15.jar, whereas Buzz-1.0.jar depends on log4j-1.2.16.jar. Two different versions of the same jar.
How could a ClassLoader-based solution resolve this (in a nutshell)?
If you're asking this question from an "I'm building an app, how do I avoid this" problem rather than a "I need this particular solution" angle, I would strongly prefer the Maven approach - namely, to only resolve a single version of any given dependency. In the case of log4j 1.2.15 -> 1.2.16, this will work fine - you can include only 1.2.16. Since the older version is API compatible (it's just a patch release) it's extremely likely that Fizz 1.0 won't even notice that it's using a newer version than it expected.
You'll find that doing this will probably be way easier to debug issues with (nothing confuses me like having multiple versions of even classes or static fields floating around! Who knows which one you're dealing with!) and doesn't need any clever class loader hacks.
But, this is exactly what all the appservers out there have to deal with. Pretend that your Fizz and Buzz are web applications (WARs), and Super-Jar is you appserver. Super-Jar will arrange a class loader for each web app that "breaks" the normal delegation model, i.e. it will look locally (down) before looking up the hierarchy. Go read about it in any of the appservers's documentation. For example http://download.oracle.com/docs/cd/E19798-01/821-1752/beade/index.html.
Use log4j-1.2.16. It only contains bugfixes wrt 1.2.15.
If Fizz breaks with 1.2.16, fork and patch it, then submit those patches back to the author of Fizz.
The alternative of creating custom classloaders with special delegation logic is very complex and likely to cause you many problems. I don't see why you would want to do this rather than just use OSGi. Have you considered creating an embedded OSGi framework, so you don't have to convert your whole application?
Related
Is there some sort of tool that you can point at a set of Java classes, and it produces output showing the transitive imports of each class?
I understand that imports are not "transitive" from the point of view of the language itself - i.e. if com.acme.X imports com.acme.Y, and com.acme.Y imports com.acme.Z, that does not mean that you can refer to com.acme.Z within com.acme.X. But that's not what I mean:
Rather, I mean that com.acme.X nonetheless depends upon com.acme.Z (at least under the current implementations of X and Y), and I want to know that fact. In fact I want to know it for a large number of classes, and so I'm hoping that there's a tool do determine it automatically.
Either a standalone tool or an Eclipse plugin or feature would be great.
Thanks in advance.
EDIT to hopefully show what I want this for:
I have a huge monolithic jar that contains many features that are (essentially) completely unrelated. I'd like to break it apart into several smaller, more manageable, and more self-coherent jars.
Unfortunately, I can't do it simply by breaking it up based on packages, because many of the packages themselves are not self-coherent either. That is, for example, there's a "com.acme.utils" package. Two things in that package are probably have nothing in common except for the fact that they're both, in some sense, "utilities". One may be a utility for some particular business function, another may be TCP/IP utilities, another may be a set of string utilities, another may be some completely unrelated business function.
And there are a bunch of packages like this. So when you look at the transitivity of imports from the point of view of packages, they snowball without limit, and so more or less everything in the monolithic jar depends on everything else in the monolithic jar.
So I'd like to start by considering transitivity of imports from the class point of view, rather than the package point of view. That way I should be able to more easily determine what classes need to be reorganized from what existing packages into new, more coherent packages, and then after that I can break the monolithic jar apart by packages / sets of packages.
we're using sonar for software metrics. http://www.sonarsource.org/
We are having a discussion at work and an interesting point came up:
Say you are developing a small library, call it somelib. Say that somelib needs to do some logging, but you don't want to reinvent the wheel, so you decide to use a 3rd party logging library.
Additionally, you want to make integration of somelib as painless as possible, so you distribute a single JAR file (somelib.jar), which has the other logging JAR, call it logger.jar, embedded inside of it. Much like what Maven's jar-with-dependencies assembly does.
Now comes the issue. Since your product is a library, what if your customer is using somelib and also happen to be using a different version of the same logging library on their own. Now we have a classpath problem.
This seems to me like it would be a common problem for people that write libraries, so what is the typical solution?
Do they avoid using JAR bundling methods altogether? Even if we do that, there is still an issue with a user's code expecting version X of the logging library, and somelib's code expecting version Y.
Do they somehow insert a dummy package prefix so that the logger classes in somelib won't conflict?
What about dynamic loading of the logger library? (though this still has versioning problems from 1.)
You may consider to use OSGI or wait for JDK 8 and its Jigsaw project.
Sorry for such question since it is more my duty to search for it, but I looked everywhere and didn't find answer for it.
Is it possible to get all apache commons components in one jar or there is no such archive?
No, we don't have a "all-of-commons" jar (nor should we). It'd be large, and the functionality would be a pretty bizarre mix. You could always (attempt) to create one yourself, but you might need to do some shading if there's any version mis-matching.
I highly doubt it. It makes no sense to combine such a broad array of functionality into a single jar. Most of the projects have nothing to do with one another and it is unlikely to require them all for a single application.
You can download the whole of spring in one hit. I have seen project "using" Spring, not as a container but as a way of including lots of libraries which can work together. i.e. no actual Spring class is used.
However I suggest you use a proper dependency management tool such as maven and you shouldn't need to download a jar yourself ever again. ;)
A good web site for finding dependencies is http://mvnrepository.com/ which have the XML you need to add to your pom.xml to get any version of most JAR you need.
The problem scienario is as follows (Note: this is not a cross-jar dependency issue, so tools like JarAnalyzer, ClassDep or Tattletale would not help. Thanks).
I have a big project which is compiled into 10 or more jar artifacts. All jars depend on each other and form a dependency hierarchy.
Whenever I need to modify one of the jars, I would check out the relevant source code and the source code for projects that depend on it. Modify the code, compile, repackage the jars. So far so good.
The problem is: I may forget to check one of the dependent projects, because inter-jar dependencies can be quite long, and may change with time. If this happens some jars may go "out-of-sync" and I will eventually get a NoSuchMethodException or a some other class incompatibility issue at run-time, which is what I want to avoid.
The only solution I can think of, the most straighforward one, is to check out all projects, and recompile the bunch. But this takes time, especially if I re-build it every small change. I do have a continuous integration server, that could do this for me, but it's shared with other developers, so seeing if the build breaks is not an option for me.
However, I do have all the jars so hypothetically it should be possible to verify jars which depend on the code that I modified have an inconsistency in method signature, class names, etc. But how could I perform such check?
Has anyone faced a similar problem before? If so, how did you solve it? Any tools or methodologies would be appreciated.
Let me know if you need clarification. Thanks.
EDIT:
I would like to clarify my question a little bit.
The ultimate goal of this task is to check that the changes that I have made will compile against the whole project. I am looking for a tool/technique that would aid me perform such check.
Consider this example:
You have 2 projects: A and B which are deployed as A.jar and B.jar respectively. A depends on B.
You wish to modify B, so you check it out and modify a method signature that A happens to depend on. You can compile B and run all tests by itself without any problems because B itself does not depend on anything. So you happily commit your changes.
In a few hours the complete project integration fails because A could not be compiled!
How do I avoid this?
The kind of tool I am looking for would retrieve A.jar and check that all dependencies in A on the new modified B are still fine. Like a potential compilation error that would happen if I were to recompile A and B sources together.
Another solution, as was suggested by many of you, is to set up a local continuous integration system that would recompile the whole project locally. I don't mind doing this, but I want to avoid doing it inside my workspace. On the other hand, if I check-out all sources to another temporary workspace, then I need to mirror my local changes to the temporary workspace.
This is quite a big issue in my team, as builds break very often because somebody forgot to check out (or open in Eclipse) the right set of projects. I tried persuading people to check-out source and recompile the bunch before commits, but not only it takes time, it needs running quite a few commands so most people just find it too troublesome to do. If the technique is not easy or automated, then it's unusable.
If you do not want to use your shared continuous integration server you should set up a local one on your developer machine where you perform the rebuild processes on change.
I know Jenkins - it is easy to setup (just start) on a local machine and I would advice to run it locally if no one is provided in the IT infrastructure that fits your needs.
Checking signatures is unfortunately not enough. Having the correct signatures does not mean it'll work. It's all about contracts and not just signatures. I mean what happens if the new version of a library has the same method signature, but accepts an ArrayList parameter now in reversed order? You will run into issues - sooner or later. I guess you maybe consider implementing tools like Ivy or Maven:
http://ant.apache.org/ivy/
http://maven.apache.org/
Yes it can be pain to implement it but once you have it it will "guard" your versions forever. You should never run into such an issue. But even those build tools are not 100% accurate. The only proper way of dealing with incompatible libraries, I know you won't like my answer, is extensive regression testing. For this you need bunch of testing tools. There are plenty of them out there: from very basic unit testing (JUnit) to database testing (JDBC Proxy) and UI testing frameworks like SWTBot (depends if your app is a web app or thick client).
Please note if your project gets really huge and you have large amount of dependencies you always not using all of the code there. Trying to check all interfaces and all signatures is way too much. Its not necessary to test it all when your code use lets say 30 % of the library code. What you need is to test what you really use. And this can be only done with extensive regression testing.
I have finally found a whole treasure box of answers at this post. Thanks for help, everyone!
The bounty goes to K. Claszen for the quickest and most input.
I'm also think that just setup local Jenkins is a best idea. What tool you use for build? Maybe you can improve you situation with switching to Maven as build tool? In more smart and don't recompile full project if you don't ask it directly. But switch to in can be HUGE paint in the neck - it hardly depends on how you project organized now...
And about VCS- exist Mercurial/SVN bridge - so you can use local Mercurial for you development ....
check this link: https://www.mercurial-scm.org/wiki/WorkingWithSubversion
There is a solution, jarjar, which allows to have different versions of the same library to be included multiple times in the dependency graph.
I use IntelliJ, not Eclipse, so maybe my answer is too IDE-specific. But in IntelliJ, I would simply include the modules from B into A, so that when I make changes to A, it breaks B immediately when compiling in the IDE. Modules can belong to multiple projects, so this is not anything like duplication, it's just adding references in the IDE to modules in other projects.
In the Eclipse environment I have project A. A has dependencies to projects or libraries B and C. (does not make a difference if they are projects or libraries)
B has dependency on LibX.v1 and C has dependency to LibX.v2.
During runtime, A will need B.jar and C.jar. Also classes in B.jar will need LibX.v1 and classes in C.jar will need LibX.v2 .
Being the different versions of the same library, LibX.v1 and LibX.v2 has same classes, so it is possible that a class may be loaded from the wrong version of the library in the runtime, causing lots of trouble. How do I manage these kind of situations?
Kind regards
Seref
Really, you don't. Most Java classloaders don't give any guarantees about the order in which libraries will be loaded, so you really need to be sure not to duplicate dependencies on your classpath. I would check if B or C could be updated to a newer version, or if LibX is fully backward-compatible.
Edit: actually, I found something that may help you. It's called OSGi, and I haven't used it, but it seems like it might do what you're trying to do. Here is a link to an intro article.
you'll need to use a 'multiple classloader' mechanism (which is what OSGI is capable of). In such a scenario the classloader for class B can only see LibX.v1, while classloader C only sees LibX.v2.
You can it do it with OSGI, but OSGI is much more than this and it's not trivial to get started with.
another plug-in framework is JPF, which also offers more than just using multiple classloaders. i think they both use the principle of 1 classloader per plug-in (module, whatever, ...)
maybe have a look at classworlds. I think it's possible to do it with this library. the benefit is that it focuses on the classloading aspect. documentation is not so good. classworlds uses the notion of class realms. take a look at the API Usage example on the site.
i'll add an example here if i find one.
anyway, no trivial solution for this i think
solving it with dependency management is the best solution of course, but it's not always possible.