I have a project where our CLASSPATH is getting out of hand. We have jars that depend on other jars, which in turn depend on even more jars, as well as a large collection of third-party jars. So each time we use one of these jars we need to construct a rather large CLASSPATH, and if downstream dependencies change it is no longer in sync.
One thought is to maybe write a tool a little like pkg-config to help with all of this. I did some searching and didn't find anything. Are there any tools like this that I've missed or do people have alternate solutions?
No, I don't want to use Maven. We already have a different build system in place.
If you aren't already using a build tool, I recommend starting with Maven. The main thing that turns people off Maven is that it has a standard project structure. Personally, I think that's a good thing.
If you want to be a little more free-form with your projects, there are the Maven Ant Tasks for Ant. I much prefer these to Ivy, but part of that is because I'm used to the Maven artifact naming conventions.
In either case (or if you use Ivy), look into a repository manager. Nexus is written by the people who maintain Maven, and I find it easier to use than Artifactory, which is another widely used system. One benefit for Artifactory is that they provide hosting (for a fee).
You may want to use maven with an IDE plugin like m2eclipse. All you have to do is type the class in your code and it will help you search for and bring in the right packages with their dependancies. It will also automatically configure the classpath for you. Here is a quick tutorial http://www.sonatype.com/people/2010/03/adding-dependencies-using-m2eclipse/
Related
Semi-greenthumb here. I'm looking to download some Apache Commons and Google Guava libraries to use in Eclipse. Multiple Q&As (example, example) have said to download the library myself, and then either load it in Eclipse by path as a "User Library" that I can add manually to projects or go through an automated project management plugin like Maven. However, that leaves the question, where should I actually store the library on my system? (Mac OS)
Ideally, I want it in a directory that is common to all Mac/*NIX systems. However, this Q&A seems to suggest that doing so would be a bad idea, and this comment implies that I should keep a separate copy of the library within each project that uses it. This seems like it would be both a waste of space (for projects that use the same library version), as well as make linting Java files in a separate text editor a hassle due to libraries being stored within an Eclipse project's file structure rather than at the system level.
So where should I put 3rd party Java libraries?
I faced the same issue when I was maintaining my project dependencies in a manual way. It is difficult to have control over them, and sometimes updating a library can be a really painful experience if that update breaks a transitive dependency.
All this pain went away when I switched to Maven.
When you configure Maven, you can set the directory where these libraries will reside (common path is {user.dir}/.m2 } and every time a dependency is added to a project (via POM), then Maven will check if that library is already downloaded. If not, it will download it and store it for any future use (of the same version). It also resolves transitive dependencies for you, so you don't have to worry of breaking it when manually replacing a JAR.
This way you don't have to worry where the libraries are, your IDE will reference them automatically using the apropiate Maven plugin
I'm not saying you should use Maven, but if your problem is managing dependencies, then Maven (or any other dependency management system, eg: Gradle) may help you.
The comment you cited is pretty naïve in its approach. There are far too many build management tools to handle dependencies without having to deal with these minutiae.
If you decide on a tool such as Maven, your dependencies will be downloaded into a specified local repository (a directory on your filesystem), and all Mavenized applications can easily be configured to use those (shared) artifacts.
Most Java supported IDEs like Eclipse come with the option to initialize projects with Maven (or Gradle, as another example) and have sleek interfaces to easily edit their configuration files to specify which dependencies your projects will use.
I would strongly recommend either of those as opposed to manual JAR/artifact management, even for basic personal tinkering projects.
I need to put some old java class library code that I have into a repo, from where others can check it out and build it. You know, like any public repo.
But, I'm not sure what the best way to do this is in the java world. In old-fashioned projects, we just used to supply the build scripts and a list of dependencies. You gathered or installed the dependencies separately before running the build scripts.
But these days for many languages, you have package managers and the like that pull from remote locations and your build scripts need to include dependency fetching.
Basically, I'm not familiar with how java libs and programs are packaged.
Should I include the (dependency) libs in the repo? And update them whenever a new version is out?
Does java now have a package manager that will pull in the latest versions of the dependencies?
Do I leave it upto the people checking out to download the libs themselves before they run the build scripts?
I'd prefer it if the solution didn't involve installing a huge package manager. Gradle wants to pull in like 150MB+ of stuff and as far as I am aware, it isn't ubiquitous on java deployments.
Thanks.
Use Maven. I believe these days it's the #1 "package manager" (not a term that's usually used to describe it, but quite apt) by a large margin. It's built into Netbeans, IntelliJ IDEA, and I believe Eclipse.
However, it won't just "pull the latest versions" of your dependencies, since your application may break. Only the versions you specify. Therefore, you should periodically update (and test) your code to reduce incompatibilities when someone tries to use your library in an application which directly or indirectly pulls newer versions of the same libs (and they get into a bit of "dll hell"), or reduce your use of third-party libraries in general.
You should also consider publishing your library in a compiled form to Maven Central so that using your library would be as easy as adding a dependency to the pom.xml. The problem that Maven solves, after all, is not so much making it easy to build your library (since just bundling the dependencies gets you most of the way), but making it easy to use your library.
Perhaps the reason I stalled learning Java until now is because I HATE how Java handles external libraries. I'm stuck keeping them in one place, adding them individually, fixing problems with versioning and every time I move/rename them, and copying and writing the classpath over and over each time I release a Java application.
There has to be an elegant solution to all of this. I keep all of my libraries (regardless of task, platform, or other) in their own little folder inside a "lib" folder in my development folder, kind of like this:
Dev
-lib
+JS-jQuery
+Flex-Degrafa
-Java-Xerces
+Xerces-1.2.3
+More libraries
I can use either Netbeans or Eclipse for Java dev, but none of them provide a very streamlined (and not to mention idiot-proof) way of managing all of these.
A nudge in the right direction or an online article/tutorial on this would be greatly appreciated.
You can either use Ant + Ivy or Maven to manage your library dependencies.
If it is only dependency management you're after and you're happy with the rest of your build process, I would use Ivy, as it can unobtrusively manage your dependencies, leaving your existing build process intact. There is a plugin for Eclipse called IvyIDE that contributes your dependencies via a classpath container.
Maven 2 has a steeper learning curve but provides a much richer set of functionality for building your projects and Eclipse integration through m2eclipse or IAM.
Personally I use Maven as I have a large number of projects to work with and Maven is particularly suited to efficient development across lots of projects.
Have a look at the introductory documentation to see what works for you.
Ivy Tutorial
Maven Getting Started Guide
Netbeans 6.7.1's Maven support is quite good and comes out of the box with the IDE.
The Eclipse addon was frustrating enough that I gave Netbeans another try.
A third choice besides ChssPly76's options is to use Ant with the Maven Ant Tasks. I don't know if I'd call any of these solutions particularly "elegant," but they do spare you the need to manage your own lib/ directory and classpath variables.
If you're working on Linux you can install Java libraries with APT or RPM.
Otherwise, I normally check precompiled JARs into a lib directory in my project's version control repository and make sure the names of the JAR files include full version information. E.g. lib/foo-1.5.6.jar, not lib/foo.jar.
To avoid having to manually set the classpath before running your app, you can set the classpath in the Manifests of the JARs themselves to define the dependencies of each JAR file. The JVM will follow all the dependencies when loading classes.
Maven is often more trouble than it's worth, but the ability to open a maven project directly into IDEs such as IntelliJ is excellent. For example, IntelliJ will download all dependencies and have them available without having to run a build first, or an mvn command and then a project refresh. It also isn't necessary to re-generate the project every time a dependency is added. I work with a number of Eclipse developers who switched to IntelliJ for this alone.
However, one shortfall of Maven is that many libraries (or versions of libraries) are not available on public repositories. Therefore it is often necessary to set up a local repository such as archiva. In ant, it would just be a matter of adding it to the lib directory in the repository.
Maven can also attack when you need to do something that maven doesn't directly support via a plugin. What would normally be a few lines of ant can often turn into a morning's worth of work.
Finally, buildr is an excellent way of using Maven's dependency management and plugins, while also supporting ad-hoc tasks.
I think most of you will know, programmers often reuse code from other software. I think, most of the time it is a good idea. But if you use code from another project your program depends on the other project.
I my current case I got three java projects A, B and C. Now A uses B and B uses C. I'm using eclipse IDE and added B to the buildpath of A and C to the buildpath of B. Now there is an compiler error that A can't resolve something from C. So I have to add C to the buildpath of B.
So what is the best way, to resolve the dependencies while keeping your programm as independent as possible from other projects?
I would like to know is in general and in reference to my current situation. Are there better ways to do this? I.e. there are classpath settings in the launch / debug configuration view, but I think they won't help at compile time.
Thanks in advance.
This sounds like part of the problem set fixed by Maven. Using Maven and Eclipse, namely m2eclipse, you can have projects use other projects and all the dependency resolution is handled for you.
It sounds to me like you're doing what you have to without incorporating a dependency management tool like Ivy or Maven, which provide you the capability of "transitive dependency management". With either of these tools you can just specify that A depends on B and B depends on C and they will automatically know that A is going to need C as well.
The advantages of Maven (this is what I have experience in) also comes into play when it's time to package your projects for deployment since it can easily gather all of those dependencies (all the way down the hierarchy) and place them together into a distribution folder or a fat JAR that contains all of your dependencies. It takes some reading and set-up time to get into a tool like Maven, but it does make the task of managing your dependencies a whole lot easier, especially as they grow.
We use Maven and it's essential for our projects. It's a good time for you to learn - dependencies on more than 3 projects can be frightening. Maven deals with versions so that if, for whatever reason, you have to depend on Foo.1.2.3 then Maven will ensure you don't get the wrong version.
However it's not trivial. If you use Netbeans it's built in better than Eclipse and may help you learn. (Also projects are fairly switcheable between the two systems).
Maven supports a lot of concept in its POM (pom.xml) file including licence info, contributors, arguments, etc. so you get a lot more than just dependency management. And it supports modularisation of projects.
Don't skip the learning curve - you need to know how it works. But you will also find previous SO questions that will help
Others have mentioned several of the good tools, maven probably being the most common. Ivy is another one that is more targeted at just dependency management. I personally use gradle which has some of the best of all of those features underneath a familiar groovy wrapper... that is still evolving and spottily documented. ;)
One thing to be aware of is how these tools handle transitive dependencies. In your example, C is a transitive dependency of A because A depends on B which depends on C. Some of these build tools will handle this type of dependency differently and it can surprise you when you least expect it.
For example, if A actually refers to code from C, ie: it has a compile-time dependency on C, then your A->B->C setup will work in something like Maven. On the other end, gradle will also make you declare that A depends on C... since it does. Runtime dependencies are fully resolved either way.
The surprise comes when you've been transitively including something for months and some of your code has relied on aspects of C and you decide you no longer need a B dependency. Suddenly your code won't build until you figure out you need a A->C dependency specified. In this example, that's pretty trivial to discover but sometimes it isn't.
And if talk like that makes your head swim a little and you don't plan on your project getting much more complicated... then you can probably just stick with what you are doing for a while. As others mentioned, it's the right way to do it without a tool helping you.
Use maven to manage your dependencies and then use the dependency plugin to see the dependencies.
you can run
mvn dependency:analyze
or
mvn dependency:tree -Dverbose=true
this will help you a lot.
no doubt you should use a dependency management tool as people have noted... manually though, archive B and C in B_C.jar. Test that B's dependence on C is resolved within the Jar.
Then add B_C.jar in the classpath...
Dependency management is a huge topic. Maven, Ivy and other tools have been developed to ease the pain with some success. Both of those tools create a dependency hierarchy so you don't run into the situation you described. They also have Eclipse plugins so that Eclipse will recognize that hierarchy.
To truly use these frameworks, you will have to change your current build process. Maven probably requires more of a commitment than Ivy, but neither is trivial, and understanding how to set it up takes some time. That said, it is very helpful to have your dependencies defined and managed clearly.
Free maven books:
http://www.sonatype.com/documentation/books
What would you suggest as a replacement to the Maven Java build toolset? Just plain Ant scripts? SCons?
It depends on what you use anyway. Maven 1? Maven 2? I find Maven 2 decently documented, and quite powerful, when used in combination with Ant tasks and some home baked Java plugins.
There are Ant tasks to use Maven features from Ant: http://maven.apache.org/ant-tasks/index.html . So you don't really need to write your own "library".
And you can call Ant from Maven: http://maven.apache.org/plugins/maven-antrun-plugin/ .
You can also script things using Groovy for example: http://groovy.codehaus.org/GMaven+-+Executing+Groovy+Code .
Or just write Java code for the custom bits you need: http://maven.apache.org/guides/plugin/guide-java-plugin-development.html .
There's no reason not to mix and match, when you can do it ;)
While I like the idea of Maven, I always find myself fighting it for anything but the simplest configurations. Its architecture is plugin-based, and the quality of the plugins varies widely. Documentation is sketchy at best; half the time I can't figure out if the doc I'm reading is for Maven 1 or 2. And how often have you had to resort to using the maven-ant-plugin to do something basic?
I've been using Apache Ivy for about a year now, and I'm pretty happy with it. There is a bit of a learning curve though. But there's an active mailing list, and the author is very responsive.
What I ended up doing was writing a "library" of a few simple Ant targets such as "resolve", "compile", "install", etc. that expect things to be in standard Maven places (such as src/main/java) and then <import> the targets into my various projects. This way you get the best of both worlds- Maven-style configuration, and all the power of Ant.
I'm fond of using Ant with Ivy. Ivy uses Maven repositories to fetch library dependencies without forcing you to change your entire build system to suit Maven.
I've been using buildr for some of my projects. Very terse and readable build scripts (no xml) and it uses maven repos (with trivial effort, maven 1 and maven 2).
I was looking for something post-maven1 to work on for some projects and m2 wasn't all that appealing. Maven had already left all m1 users high and dry with the need to redo all of their build systems (and some of mine cannot be expressed in m2 as far as I can tell).
I really like using ant4eclipse - you set up your project dependencies in eclipse as normal and you can write a single ant script that'll build all the projects in the right order.
For third-party libs, you can either set up a single eclipse project containing (and exporting) all of your jars [I DON'T RECOMMEND THIS], or a separate project for each third-party lib, exporting the jar. [I recommend the latter b/c you can track project->third-party deps and upgrade parts when needed.]
I would go with buildr, if i am expecting complex build targets in my build. The benefit is, you are not dependent on any particular library at all. You have all the APIs from Ruby to do whatever you want; unlike Ivy in which case you are still tied up to ANT. Effort of migration to Ivy is worth if you want to introduce library dependency to an already established ANT based build mechanism which does all the operations you want to do.
There are at least three different parts of Maven: the repository, the implementation independent lifecyle and the conventions for a default project. Using Ivy provides only the repository.
I've not used it but I think that easyant is designed to be a more complete replacement to Maven. It uses Ivy as the repository but then adds conventions and standard modules.