When you start to use a third party library in Java, you add their jars to your project. Do you also add the jars to the repository or do you just keep a local installation. And if the latter, how do you sync between team members to be able to work?
Thanks.
Yes. You should add to the repository whatever is required for a developer on a clean system (aside from having the JDK and ant installed) to check out and build the project.
No, if you use Maven. Put them into Maven repository (if they are not there yet, most open source libraries are in public Maven repositories already).
Yes, if you use Ant.
If you're using ANT and you want maven style dependency management without maven, take a look at Ivy. It can download from maven repositories and can even read maven pom files.
Yes, third-party libraries should be version controlled since you will want everyone to compile against the same version of that library. This way when updates to that third-party library happen you will simply have to change it in one place and everyone will update to the new version. Otherwise you could easily end up with the famous: "It compiles on my machine!"-syndrome.
Assuming you do not use Maven, then you should definitely check in versions of 3rd party jars. You (or more importantly, the maintenance developer coming along after you) should be able to pull a version of your application from the repository and have all the correct versions of all the 3rd party jars required at compile- and runtime.
You also should maintain the versioning history of the 3rd party jars, again to help the maintenance developer out. Checking them in to the repository is the easiest and most effective way to do this.
I would recommend versioning everything you need to build a project. Unless your using a build tool like maven.
Short Answer
No
Longer answer
In the interim, use a folder on your network separate from the repo.
A Long Term Solution
Use maven. It was built for handling your build, configuration and external jar dependencies.
Go here for documentation though. (The official maven documentation is known to be pretty spotty).
Lately I've been keeping JARs out of SVN. I have a separate repository that I use to pull them into projects, manage dependencies, and keep track of different versions.
This is what Maven is born for. I've got to learn how to use it well, but until then I'm keeping my own JAR repository.
Related
Semi-greenthumb here. I'm looking to download some Apache Commons and Google Guava libraries to use in Eclipse. Multiple Q&As (example, example) have said to download the library myself, and then either load it in Eclipse by path as a "User Library" that I can add manually to projects or go through an automated project management plugin like Maven. However, that leaves the question, where should I actually store the library on my system? (Mac OS)
Ideally, I want it in a directory that is common to all Mac/*NIX systems. However, this Q&A seems to suggest that doing so would be a bad idea, and this comment implies that I should keep a separate copy of the library within each project that uses it. This seems like it would be both a waste of space (for projects that use the same library version), as well as make linting Java files in a separate text editor a hassle due to libraries being stored within an Eclipse project's file structure rather than at the system level.
So where should I put 3rd party Java libraries?
I faced the same issue when I was maintaining my project dependencies in a manual way. It is difficult to have control over them, and sometimes updating a library can be a really painful experience if that update breaks a transitive dependency.
All this pain went away when I switched to Maven.
When you configure Maven, you can set the directory where these libraries will reside (common path is {user.dir}/.m2 } and every time a dependency is added to a project (via POM), then Maven will check if that library is already downloaded. If not, it will download it and store it for any future use (of the same version). It also resolves transitive dependencies for you, so you don't have to worry of breaking it when manually replacing a JAR.
This way you don't have to worry where the libraries are, your IDE will reference them automatically using the apropiate Maven plugin
I'm not saying you should use Maven, but if your problem is managing dependencies, then Maven (or any other dependency management system, eg: Gradle) may help you.
The comment you cited is pretty naïve in its approach. There are far too many build management tools to handle dependencies without having to deal with these minutiae.
If you decide on a tool such as Maven, your dependencies will be downloaded into a specified local repository (a directory on your filesystem), and all Mavenized applications can easily be configured to use those (shared) artifacts.
Most Java supported IDEs like Eclipse come with the option to initialize projects with Maven (or Gradle, as another example) and have sleek interfaces to easily edit their configuration files to specify which dependencies your projects will use.
I would strongly recommend either of those as opposed to manual JAR/artifact management, even for basic personal tinkering projects.
I need to put some old java class library code that I have into a repo, from where others can check it out and build it. You know, like any public repo.
But, I'm not sure what the best way to do this is in the java world. In old-fashioned projects, we just used to supply the build scripts and a list of dependencies. You gathered or installed the dependencies separately before running the build scripts.
But these days for many languages, you have package managers and the like that pull from remote locations and your build scripts need to include dependency fetching.
Basically, I'm not familiar with how java libs and programs are packaged.
Should I include the (dependency) libs in the repo? And update them whenever a new version is out?
Does java now have a package manager that will pull in the latest versions of the dependencies?
Do I leave it upto the people checking out to download the libs themselves before they run the build scripts?
I'd prefer it if the solution didn't involve installing a huge package manager. Gradle wants to pull in like 150MB+ of stuff and as far as I am aware, it isn't ubiquitous on java deployments.
Thanks.
Use Maven. I believe these days it's the #1 "package manager" (not a term that's usually used to describe it, but quite apt) by a large margin. It's built into Netbeans, IntelliJ IDEA, and I believe Eclipse.
However, it won't just "pull the latest versions" of your dependencies, since your application may break. Only the versions you specify. Therefore, you should periodically update (and test) your code to reduce incompatibilities when someone tries to use your library in an application which directly or indirectly pulls newer versions of the same libs (and they get into a bit of "dll hell"), or reduce your use of third-party libraries in general.
You should also consider publishing your library in a compiled form to Maven Central so that using your library would be as easy as adding a dependency to the pom.xml. The problem that Maven solves, after all, is not so much making it easy to build your library (since just bundling the dependencies gets you most of the way), but making it easy to use your library.
I have a project where our CLASSPATH is getting out of hand. We have jars that depend on other jars, which in turn depend on even more jars, as well as a large collection of third-party jars. So each time we use one of these jars we need to construct a rather large CLASSPATH, and if downstream dependencies change it is no longer in sync.
One thought is to maybe write a tool a little like pkg-config to help with all of this. I did some searching and didn't find anything. Are there any tools like this that I've missed or do people have alternate solutions?
No, I don't want to use Maven. We already have a different build system in place.
If you aren't already using a build tool, I recommend starting with Maven. The main thing that turns people off Maven is that it has a standard project structure. Personally, I think that's a good thing.
If you want to be a little more free-form with your projects, there are the Maven Ant Tasks for Ant. I much prefer these to Ivy, but part of that is because I'm used to the Maven artifact naming conventions.
In either case (or if you use Ivy), look into a repository manager. Nexus is written by the people who maintain Maven, and I find it easier to use than Artifactory, which is another widely used system. One benefit for Artifactory is that they provide hosting (for a fee).
You may want to use maven with an IDE plugin like m2eclipse. All you have to do is type the class in your code and it will help you search for and bring in the right packages with their dependancies. It will also automatically configure the classpath for you. Here is a quick tutorial http://www.sonatype.com/people/2010/03/adding-dependencies-using-m2eclipse/
Warning: I have just picked up Maven, so things mentioned might be wrong or not best practice.
I have a medium size open source project that I am migrating to Maven from the basic
NetBeans project management. This is not a developer team sharing the same room, this is 1-5 people over the internet sharing a SVN repo. Reading over the how-tos on dependencies, it seems that the only way to get dependencies is to get them from an online repo or install them locally.
This is not what I was looking for. I want to keep all dependencies in the SVN for many reasons including portability (anybody can pass by, check out the repo, build, and use; all that simply without manual adding to local repo's and whatnot), getting newer versions (discussed below), and manual versioning.
The other issue I have with the maven repository is that they are quite behind in versions. Logback for example is 0.9.18 in mvnbrowser but 0.9.24 officially. PircBot is 1.4.6 in mvnbrowser but 1.5.0 officially. Why such old versions?
Issue 3 is that I have dependencies that don't even exist in the repos, like Easier Java Persistence.
So
How can I force all dependencies to come from /lib for example
On a related note, can mvn build from library's SVN repo directly? Just curious
Is there an automatic way to get the newest version directly from a dependencies site/svn repo if they also use Maven? IE libraries like commons-lang or logback
Is there a better way of managing dependencies? (IE Ivy or some weird POM option I'm missing)
FYI, this is a Java project with 3 modules, project global dependencies and module specific dependencies.
Bonus points if it can work with the bundled version of Maven that comes with NetBeans.
Not a duplicate of
Maven: add a dependency to a jar by relative path - Not wanting to install to local repository
maven compile fails because i have a non-maven jar - Don't think a System dependency is the right answer
maven look for new versions of dependencies - Still uses(?) repository, just the latest (old) version
This is not what I was looking for. I want to keep all dependencies in the SVN for many reasons (...)
I will come back on this but the solution I described in Maven: add a dependency to a jar by relative path (using a file-based repository) allows to implement such a solution.
The other issue I have with the maven repository is that they are quite behind in versions. Logback for example is 0.9.18 in mvnbrowser but 0.9.24 officially. PircBot is 1.4.6 in mvnbrowser but 1.5.0 officially. Why such old versions?
It looks like mvnbrowser indices are totally out of date (making it useless as repository search engine) because the maven central repository does have logback-core-0.9.24.jar (the logback project is doing what has to be done to make this happen) but only has an old pircbot-1.4.2.jar. Why? Ask the pircbot team. Anyway, you're right, the central repository might not always have ultimate versions.
Issue 3 is that I have dependencies that don't even exist in the repos, like Easier Java Persistence.
Yeah, this happens too.
How can I force all dependencies to come from /lib for example
As previously hinted, you should re-read carefully the solution suggested in Maven: add a dependency to a jar by relative path. This solution is not about installing libraries to the local repository but is about using a file-based repository (that could thus be stored in SVN). You might have missed the point, this matches your use case. And also check Brett's answer for a variation.
On a related note, can mvn build from library's SVN repo directly? Just curious
Didn't get that one. Can you clarify?
Is there an automatic way to get the newest version directly from a dependencies site/svn repo if they also use Maven? IE libraries like commons-lang or logback
Maven supports version ranges and you could use a syntax allowing to use "any version greater than X". But I do NOT recommend using version ranges at all, for the sake of build reproducibility. You don't want the build to suddenly fail because of some automatic update that happened on your back. Only upgrade if you need bug fixes or new features, but do it explicitly (if it ain't broke, don't fix it).
You might also find mentions of the LATEST and RELEASE version markers. I don't recommend them neither for the same reasons as above and even less since they're removed from Maven 3.x.
Is there a better way of managing dependencies? (IE Ivy or some weird POM option I'm missing)
Can't say for Ivy. But in the Maven land, if you can't host up a "corporate" repository for your project (Nexus, Archiva, Artifactory), then the file-based repository is IMO the best approach.
Setup your own Maven repository.
http://archiva.apache.org/
Perhaps the reason I stalled learning Java until now is because I HATE how Java handles external libraries. I'm stuck keeping them in one place, adding them individually, fixing problems with versioning and every time I move/rename them, and copying and writing the classpath over and over each time I release a Java application.
There has to be an elegant solution to all of this. I keep all of my libraries (regardless of task, platform, or other) in their own little folder inside a "lib" folder in my development folder, kind of like this:
Dev
-lib
+JS-jQuery
+Flex-Degrafa
-Java-Xerces
+Xerces-1.2.3
+More libraries
I can use either Netbeans or Eclipse for Java dev, but none of them provide a very streamlined (and not to mention idiot-proof) way of managing all of these.
A nudge in the right direction or an online article/tutorial on this would be greatly appreciated.
You can either use Ant + Ivy or Maven to manage your library dependencies.
If it is only dependency management you're after and you're happy with the rest of your build process, I would use Ivy, as it can unobtrusively manage your dependencies, leaving your existing build process intact. There is a plugin for Eclipse called IvyIDE that contributes your dependencies via a classpath container.
Maven 2 has a steeper learning curve but provides a much richer set of functionality for building your projects and Eclipse integration through m2eclipse or IAM.
Personally I use Maven as I have a large number of projects to work with and Maven is particularly suited to efficient development across lots of projects.
Have a look at the introductory documentation to see what works for you.
Ivy Tutorial
Maven Getting Started Guide
Netbeans 6.7.1's Maven support is quite good and comes out of the box with the IDE.
The Eclipse addon was frustrating enough that I gave Netbeans another try.
A third choice besides ChssPly76's options is to use Ant with the Maven Ant Tasks. I don't know if I'd call any of these solutions particularly "elegant," but they do spare you the need to manage your own lib/ directory and classpath variables.
If you're working on Linux you can install Java libraries with APT or RPM.
Otherwise, I normally check precompiled JARs into a lib directory in my project's version control repository and make sure the names of the JAR files include full version information. E.g. lib/foo-1.5.6.jar, not lib/foo.jar.
To avoid having to manually set the classpath before running your app, you can set the classpath in the Manifests of the JARs themselves to define the dependencies of each JAR file. The JVM will follow all the dependencies when loading classes.
Maven is often more trouble than it's worth, but the ability to open a maven project directly into IDEs such as IntelliJ is excellent. For example, IntelliJ will download all dependencies and have them available without having to run a build first, or an mvn command and then a project refresh. It also isn't necessary to re-generate the project every time a dependency is added. I work with a number of Eclipse developers who switched to IntelliJ for this alone.
However, one shortfall of Maven is that many libraries (or versions of libraries) are not available on public repositories. Therefore it is often necessary to set up a local repository such as archiva. In ant, it would just be a matter of adding it to the lib directory in the repository.
Maven can also attack when you need to do something that maven doesn't directly support via a plugin. What would normally be a few lines of ant can often turn into a morning's worth of work.
Finally, buildr is an excellent way of using Maven's dependency management and plugins, while also supporting ad-hoc tasks.