I think most of you will know, programmers often reuse code from other software. I think, most of the time it is a good idea. But if you use code from another project your program depends on the other project.
I my current case I got three java projects A, B and C. Now A uses B and B uses C. I'm using eclipse IDE and added B to the buildpath of A and C to the buildpath of B. Now there is an compiler error that A can't resolve something from C. So I have to add C to the buildpath of B.
So what is the best way, to resolve the dependencies while keeping your programm as independent as possible from other projects?
I would like to know is in general and in reference to my current situation. Are there better ways to do this? I.e. there are classpath settings in the launch / debug configuration view, but I think they won't help at compile time.
Thanks in advance.
This sounds like part of the problem set fixed by Maven. Using Maven and Eclipse, namely m2eclipse, you can have projects use other projects and all the dependency resolution is handled for you.
It sounds to me like you're doing what you have to without incorporating a dependency management tool like Ivy or Maven, which provide you the capability of "transitive dependency management". With either of these tools you can just specify that A depends on B and B depends on C and they will automatically know that A is going to need C as well.
The advantages of Maven (this is what I have experience in) also comes into play when it's time to package your projects for deployment since it can easily gather all of those dependencies (all the way down the hierarchy) and place them together into a distribution folder or a fat JAR that contains all of your dependencies. It takes some reading and set-up time to get into a tool like Maven, but it does make the task of managing your dependencies a whole lot easier, especially as they grow.
We use Maven and it's essential for our projects. It's a good time for you to learn - dependencies on more than 3 projects can be frightening. Maven deals with versions so that if, for whatever reason, you have to depend on Foo.1.2.3 then Maven will ensure you don't get the wrong version.
However it's not trivial. If you use Netbeans it's built in better than Eclipse and may help you learn. (Also projects are fairly switcheable between the two systems).
Maven supports a lot of concept in its POM (pom.xml) file including licence info, contributors, arguments, etc. so you get a lot more than just dependency management. And it supports modularisation of projects.
Don't skip the learning curve - you need to know how it works. But you will also find previous SO questions that will help
Others have mentioned several of the good tools, maven probably being the most common. Ivy is another one that is more targeted at just dependency management. I personally use gradle which has some of the best of all of those features underneath a familiar groovy wrapper... that is still evolving and spottily documented. ;)
One thing to be aware of is how these tools handle transitive dependencies. In your example, C is a transitive dependency of A because A depends on B which depends on C. Some of these build tools will handle this type of dependency differently and it can surprise you when you least expect it.
For example, if A actually refers to code from C, ie: it has a compile-time dependency on C, then your A->B->C setup will work in something like Maven. On the other end, gradle will also make you declare that A depends on C... since it does. Runtime dependencies are fully resolved either way.
The surprise comes when you've been transitively including something for months and some of your code has relied on aspects of C and you decide you no longer need a B dependency. Suddenly your code won't build until you figure out you need a A->C dependency specified. In this example, that's pretty trivial to discover but sometimes it isn't.
And if talk like that makes your head swim a little and you don't plan on your project getting much more complicated... then you can probably just stick with what you are doing for a while. As others mentioned, it's the right way to do it without a tool helping you.
Use maven to manage your dependencies and then use the dependency plugin to see the dependencies.
you can run
mvn dependency:analyze
or
mvn dependency:tree -Dverbose=true
this will help you a lot.
no doubt you should use a dependency management tool as people have noted... manually though, archive B and C in B_C.jar. Test that B's dependence on C is resolved within the Jar.
Then add B_C.jar in the classpath...
Dependency management is a huge topic. Maven, Ivy and other tools have been developed to ease the pain with some success. Both of those tools create a dependency hierarchy so you don't run into the situation you described. They also have Eclipse plugins so that Eclipse will recognize that hierarchy.
To truly use these frameworks, you will have to change your current build process. Maven probably requires more of a commitment than Ivy, but neither is trivial, and understanding how to set it up takes some time. That said, it is very helpful to have your dependencies defined and managed clearly.
Free maven books:
http://www.sonatype.com/documentation/books
Related
Assume the following situation: my Maven project depends on a jar A, which depends on 10 other jars which transitively depend on a lot more other jars. I get a huge classpath and if am building a war/ear, I get a huge artifact.
Actually, I am using only the class foo in jar A. The class foo uses a few other classes, which are contained in three other jars. So I really only need jar A and three other jars to compile, not the whole bunch of dependencies (and their dependencies and so on).
Is there a way to (semi-)automatically analyse dependency trees on the class level? As far as I know Maven has no build-in functionality for this.
Just to make this clear: I know that such situations should not occur in a good software architecture. But if I get a jar A which is really just a collection of classes for different purposes, I potentially get a lot of unnecessary dependencies when I build the dependency tree with Maven. And changing A is not something I can do.
Some (long) time ago I've started Maven plugin for this:
https://github.com/highsource/storyteller-maven-plugin
How to find unneccesary dependencies in a maven multi-project?
It works but in no way finished/documented etc. I also don't want to "sell" it here in any way.
But what you write were exactly my thoughts then. maven-storyteller-plugin basically analyzed dependencies of classes and built a huge graph of them. Then it could tell if you actually need dependencies you've declared in your project or not. It could also export nice graphs of dependencies (using GraphViz).
I never had time to finish it, but maybe someone would be interested? Heavylifting is done already.
I'm starting to fix a java project that has used maven and while I've got the project to build, at runtime it fails with missing dependencies. I've had a look and the errors are missing optional dependencies of included compile time dependencies. I can go through and add these but it seems to me that I can have everything building and running nicely only for some piece of code that I missed to use a missing dependency and the whole thing falls apart.
What I really want to know is whether there is an automated way to find optional dependencies that I have chosen to not include. I have used mvn dependency:tree but this only shows the dependencies I have (not sure of the scope it checks) and I have tried mvn dependency:analyze but this seems to show dependencies it thinks I don't use and those that have been pulled down indirectly. What I cannot see is how to see a list of optionals I don't include.
Currently my method of working around this is to read the poms and try to work it out from there, but I don't see this as particularly robust.
For reference, I am fairly new to maven style dependency management and on the face of it like it, but this optional thing is a bit of a stumbling block for me. I understand that optionals stop me pullin down dependencies I won't be using, but it hasn't clicked for me how I can workout what optionals are available and that I do need.
I am using Eclipse Juno, m2Eclipse (also have maven 3.0.5 cli), java 6/7.
Anyone got any ideas of how I can do this better, or what I am completely overlooking?
No the things are - somewhat - just this way. Maven does not do dependency management, it allows you to do dependency management by offering tools to use and analyze them. So the work still is on the developers side. People often mix that up.
This is mainly caused by the fact that projects often have different deployment targets. As a result sometimes they collect a bunch of jar files which are copied once into tomcat and a different set of files for weblogic. So there might be a readme in your project that states what to copy prior to deployment of the maven artifacts. Or it is implicit knowledge - then you're doomed.
dependency:analyze works on bytecode not on sources. therefore it does not see what maven knows.
Maybe mvn help:effective-pom gives a better basis to analyze the whole thing? Or you could try to modify the dependency plugin to show that information as well. Maven plugins are not so hard to work with.
I'm not aware of a plugin that displays all optional transitive dependencies. But since the pom.xml files of dependencies are downloaded into the local maven repo you could do a text search there.
A while ago there was a discussion on optional dependencies as well: Best strategy for dealing with optional dependencies - it might be helpful too.
I have a project where our CLASSPATH is getting out of hand. We have jars that depend on other jars, which in turn depend on even more jars, as well as a large collection of third-party jars. So each time we use one of these jars we need to construct a rather large CLASSPATH, and if downstream dependencies change it is no longer in sync.
One thought is to maybe write a tool a little like pkg-config to help with all of this. I did some searching and didn't find anything. Are there any tools like this that I've missed or do people have alternate solutions?
No, I don't want to use Maven. We already have a different build system in place.
If you aren't already using a build tool, I recommend starting with Maven. The main thing that turns people off Maven is that it has a standard project structure. Personally, I think that's a good thing.
If you want to be a little more free-form with your projects, there are the Maven Ant Tasks for Ant. I much prefer these to Ivy, but part of that is because I'm used to the Maven artifact naming conventions.
In either case (or if you use Ivy), look into a repository manager. Nexus is written by the people who maintain Maven, and I find it easier to use than Artifactory, which is another widely used system. One benefit for Artifactory is that they provide hosting (for a fee).
You may want to use maven with an IDE plugin like m2eclipse. All you have to do is type the class in your code and it will help you search for and bring in the right packages with their dependancies. It will also automatically configure the classpath for you. Here is a quick tutorial http://www.sonatype.com/people/2010/03/adding-dependencies-using-m2eclipse/
I wonder if its a trivial question, which i am not aware.
In a multi-module maven project, lets say that there is a 'common' module. For example, there are 5 modules out of which 1 module is common. Is there a way to determine if the other 4 modules depend the common module class-wise i.e for each and every class in the common module, i want to know the classes in the other modules which depend on that class in common? (actually maven does not matter here though).
Does eclipse itself has this feature?
It would be great if the tool gives a diagramatic representation.
As far as I know, Maven doesn't work 'class-wise', module is its atomic element in terms of dependencies.
You can use mvn dependency:tree in order to get the dependencies per module, in eclipse/intellij enterprise edition you
have a graphical representation for the results but that's it.
Basically you must differ between compile time and run time dependencies.
Of course if you have an 'unsatisfied' compile time dependency in some class, for example, using Logger but not having log4j/other relevant library in the class path you'll get the error during the compilation of your class, Its compiler's job, not maven's.
Now runtime dependencies are even harder to track, example:
if you're running inside of some container and you define your log4j library dependency in a 'provided' scope, then you're relying on container that it will bring that library to you and will take care of all the class loading stuff.
But How maven can know what's going on inside a container?
So, Bottom line, what's you're asking for is impossible in maven and I've tried to explain why :)
Hope this helps
In Eclipse you can simply do that by selecting the class you want to examine and press CTRL-SHIFT-G which will you search for a reference within the workspace. In this case it means you have to have opened all the modules of the multi-module. The drawback is that you need to do this for every class you would like to know of.
I'm one of the developers, so I'm not unbiased, but I believe that Restructure101 is perfect for what you want. Point RS101 at the root POM and you'll see a dependency map of all the POMs, something like this:
Then you can chase dependencies from one pom to another by double-clicking to expand any item to whatever level you want. In this case I have drilled into Maven-core to discover what is used by code in maven-compat:
You can also use Restructure101 to reorganize classes between poms (like creation/improvement of a common pom as you mention), for example by dragging classes to new poms and seeing the effect on the pom-level dependencies. An action list is exported to your IDE.
The companion product Structure101 has related capabilities, worth checking, but I'd prefer Restructure101 for what you describe.
I am currently dealing with a huge Eclipse project (not written by me). This project doesn't use any dependency management tools. It references hundreds of JARs.
Some of these JARs contain same packages (and classes), but in different versions. Currently, resolving conflicts works by manually (and randomly!) reordering these JARs in Order&Export (in Project Properties).
This was done for a long time now, and there are now lots of packages/classes with different vendors/versions/product-lines.
Reordering causes some parts of the project to fail while other parts start working, and oppositely.
Strangely, lots of orders do not cause build errors, but only runtime errors.
Can this mess be solved by an tool, which would suggest certain automatic order of dependent JARs?
Google for JarAnalyzer, that helps at least to figure how the dependecies are build up. Use the jars, your eclipse project is producing, as well. However you can not really automate this. Imagine one of your eclipse projects in needing bad-1.0.jar and another one uses bad-1.2.jar. Very often you can not replace the 1.0 one with the 1.2 one because your project wont compile any more. So in the long run you have to REMOVE outdated jars, switch to a "common version" amoung all subprojects and fix the compiler errors. And while you do that, switch to ivy or maven.
Do your jar files even have proper names or do you have 3 different versions of bad.jar which look the same in the filesystem but are in fact of different version? If so, start by renaming all relevant jar files to include the version number (can often eb found in the manifest file) ... heck I once did what you do and wrote me with JArAnalyzer, a bit groovy and some shell scripts a small tool that generated all the ivy files for the project.
you can use maven, ivy to clean the mess :) . And that spring doesn't work properly try this:first clean then build the project.
"Strangely, lots of orders do not cause build errors, but only runtime
errors."
This is not strange. As you wrote, classes are present in different versions, which does not necessarily means compilation error, but means different behaviour and different sub dependencies.
Avoid a "random" or "automatic order" approach. I would advise you the usage of Maven for handling your dependencies (in order to know precisely which library depends on which one). You will probably discover that many of the libraries you're including are not required, and that the dependency management tool will handle for you "automatically" all dependencies between dependencies, you will have however to add/force exclusion for specific libraries versions.
Much more, it will help you to simplify the code and eventually remove one line of code and 40 dependencies...(relying on a side framework misused such Spring or any other one).