Warning: I have just picked up Maven, so things mentioned might be wrong or not best practice.
I have a medium size open source project that I am migrating to Maven from the basic
NetBeans project management. This is not a developer team sharing the same room, this is 1-5 people over the internet sharing a SVN repo. Reading over the how-tos on dependencies, it seems that the only way to get dependencies is to get them from an online repo or install them locally.
This is not what I was looking for. I want to keep all dependencies in the SVN for many reasons including portability (anybody can pass by, check out the repo, build, and use; all that simply without manual adding to local repo's and whatnot), getting newer versions (discussed below), and manual versioning.
The other issue I have with the maven repository is that they are quite behind in versions. Logback for example is 0.9.18 in mvnbrowser but 0.9.24 officially. PircBot is 1.4.6 in mvnbrowser but 1.5.0 officially. Why such old versions?
Issue 3 is that I have dependencies that don't even exist in the repos, like Easier Java Persistence.
So
How can I force all dependencies to come from /lib for example
On a related note, can mvn build from library's SVN repo directly? Just curious
Is there an automatic way to get the newest version directly from a dependencies site/svn repo if they also use Maven? IE libraries like commons-lang or logback
Is there a better way of managing dependencies? (IE Ivy or some weird POM option I'm missing)
FYI, this is a Java project with 3 modules, project global dependencies and module specific dependencies.
Bonus points if it can work with the bundled version of Maven that comes with NetBeans.
Not a duplicate of
Maven: add a dependency to a jar by relative path - Not wanting to install to local repository
maven compile fails because i have a non-maven jar - Don't think a System dependency is the right answer
maven look for new versions of dependencies - Still uses(?) repository, just the latest (old) version
This is not what I was looking for. I want to keep all dependencies in the SVN for many reasons (...)
I will come back on this but the solution I described in Maven: add a dependency to a jar by relative path (using a file-based repository) allows to implement such a solution.
The other issue I have with the maven repository is that they are quite behind in versions. Logback for example is 0.9.18 in mvnbrowser but 0.9.24 officially. PircBot is 1.4.6 in mvnbrowser but 1.5.0 officially. Why such old versions?
It looks like mvnbrowser indices are totally out of date (making it useless as repository search engine) because the maven central repository does have logback-core-0.9.24.jar (the logback project is doing what has to be done to make this happen) but only has an old pircbot-1.4.2.jar. Why? Ask the pircbot team. Anyway, you're right, the central repository might not always have ultimate versions.
Issue 3 is that I have dependencies that don't even exist in the repos, like Easier Java Persistence.
Yeah, this happens too.
How can I force all dependencies to come from /lib for example
As previously hinted, you should re-read carefully the solution suggested in Maven: add a dependency to a jar by relative path. This solution is not about installing libraries to the local repository but is about using a file-based repository (that could thus be stored in SVN). You might have missed the point, this matches your use case. And also check Brett's answer for a variation.
On a related note, can mvn build from library's SVN repo directly? Just curious
Didn't get that one. Can you clarify?
Is there an automatic way to get the newest version directly from a dependencies site/svn repo if they also use Maven? IE libraries like commons-lang or logback
Maven supports version ranges and you could use a syntax allowing to use "any version greater than X". But I do NOT recommend using version ranges at all, for the sake of build reproducibility. You don't want the build to suddenly fail because of some automatic update that happened on your back. Only upgrade if you need bug fixes or new features, but do it explicitly (if it ain't broke, don't fix it).
You might also find mentions of the LATEST and RELEASE version markers. I don't recommend them neither for the same reasons as above and even less since they're removed from Maven 3.x.
Is there a better way of managing dependencies? (IE Ivy or some weird POM option I'm missing)
Can't say for Ivy. But in the Maven land, if you can't host up a "corporate" repository for your project (Nexus, Archiva, Artifactory), then the file-based repository is IMO the best approach.
Setup your own Maven repository.
http://archiva.apache.org/
Related
What's the best way to download remote maven artifact from within a maven plugin while also supporting maven 2.2.x1), 3.0.x, 3.1.x and newer?
Aether seems a good candidate but it changed package namespace between 3.0.5 and 3.1 breaking compatibility.
There's pieces of information around,
some pointing towards maven-dependency-tree (as suggested in this SO post) (how?)
yet another to jcbi,
Another SO post which I think uses Aether underneath and presumably suffers from the same version incompatibility problem
Try to manually force your version of Ather based on the ResolveArtifact example
Try to port the really complex solution reflection solution from Apache Karaf
Some other util from maven-shared ?
So what's the correct, or at least working, way to go on about this?
1) Where Maven 2.2.x is a nice to have as pointed out by #khmarbaise is EoL
If you're in a plugin and want to have another plugin invoked, the best solution - by far - is to use Mojo Executor. This way, you won't rewrite the maven-dependency-plugin ...
Considering the maven version issues, well, I guess you should indeed give up on maven 2.* support.
I hope I can keep this question specific enough, my team at work is currently debating the best way to manage our dependencies for a huge project (150+ dependencies ~300mb).
We have two main problems
Keeping all the developers dependencies the same so we are compiling against the same files
Ensure the project (once compiled) is comliped against the same dependencies
The two ideas that have been suggested are using a BirJar (all dependencies in one file) and just adding a version number to it and using a shared folder and pointing everyone's machines at the same place.
Or making including all the dependencies in the jar when we compile it (a jar, of jars, of jars) and just have a project that "has no dependencies"
Someone also mentioned setting up an internal version of Ivy and pointing all the code to pull dependencies from there.
What are the best practices regarding massive dependency management?
Why don't you use Maven and its dependency management ?
You can specify each dependency, its particular version and its scope (compile-time, for testing, for deployment etc.). You can provide a master pom.xml (the config file) that specifies these, and developers can override if they need (say, to evaluate new versions).
e.g. I specify a pom.xml that details the particular jars I require and their versions (or range). Dependent jars are determined/downloaded automatically. I can nominate which of these jars are used for compilation vs. deployment etc. If I use a centralised repository such as Nexus I can then build my artefact (e.g. a library) and deploy that into Nexus, and it'll become available for other developers to download in exactly the same manner as 3rd party libs etc.
Incase you dont like/want to follow the Maven project structure...
If you already use Ant, then your best bet is to use Ivy for dependency management.
http://ant.apache.org/ivy/
It provides a rich set of ant tasks for dependency manipulation.
from : Ant dependency management
I am developing a web-app and use maven for dependency management (duh). Some of the needed jars are already available in the server lib folder, but do not match the "maven naming scheme", ie missing the version suffix.
I would like to use them for development and deployment, but..
1. i cant point maven to them because maven seem to need a version suffix. I cant omit it in the pom.
2. If i define the dependency outside maven then maven is obviously unable to build.
3. Renaming the files inside the server distribution sounds like a kludge.
What would Brian Boitano do? I mean, there sure is an elegant solution that im not aware of, or at least a good argument for one of the three solutions above.
Thank you
PS. i am using jboss 5.1 and maven 2.2.1 atm, but its subject to change
You can provide those jars as a dependency with a system scope if you want explicitly to identify where they live. For more info have a look here
IF those are not proprietary libs you are using, I'd recommend you use official versions of those from maven repository.
If they are proprietary you can manually install jar to your local repository using maven(you can use your version, suffixes, group names, artifactid etc) and then use them in your pom.
Is there any way to force Maven to use remote artifacts and not those installed on your machine? since I worry about runtime errors and not compilation errors build server is not valid option.
P.S. I know I could delete or rename the .m2 folder, but I bet there is some more clever way of doing this. Maybe some plugin or special command param?
Having no local repository would mean your classpath consisting almost entirely of URLs on remote servers. I can't see why this would be supported as execution would be awful, and any dropped connection would result in classloader issues. Having a local repository ensures the jars are available before compilation/execution begins.
Also consider that WAR and EAR projects (and many using the dependency plugin) rely on downloading the jars to complete their packaging. There would be a huge overhead if these had to be retrieved from a remote repository on every build. I'm pretty sure the managers of central would not be keen on dealing with that load.
Some alternatives for you to consider:
If you want to force a clean local repository on each build, you can use the purge goal of the dependency plugin.
If you want to keep builds isolated, you can use separate Maven settings by passing -Dorg.apache.maven.global-settings=/path/to/global/settings.xml
Alternatively you can override the local repository on a per build basis by passing -Dmaven.repo.local=/some/repo/path
If you want to avoid hitting remote repositories on each build, add <updatePolicy>never</updatePolicy> to your remote repository configurations. This means Maven will only check for updates if you force it to with a "-U" switch on the command line
If you want to take the latest version of a dependency, you can use the LATEST keyword in the version declaration (instead of the version number), though this can be risky if the dependency is incompatible.
If you want to take the current release version of a dependency, you can use the RELEASE keyword in the version declaration (instead of the version number). This is like LATEST, but tends to be the newest stable build, rather than the newest.
If you want to take the latest version of a dependency within a range, use Maven's version range notation, for example [1.0.0,2.0.0) means any version from 1.0.0 inclusive to 2.0.0 exclusive
For more details on LATEST and RELEASE, see section 9.3.1.3 of the Maven book.
If you use an internal repository manager (obligatory Nexus and Artifactory references here), the overhead of purging the local repository is greatly reduced - you'll just have an increased local network traffic load.
I don't think there's really a way to do what you are asking for. You could look into depending on SNAPSHOT releases (but that means changing your version string of the upstream projects to be SNAPSHOT versions).
Incidentally, this was discussed at length in a recent Java Posse episode (#268). I don't think they ended up with a solution, but you may get some good ideas there.
I also like some of Rich Seller's ideas, which I'll be looking into myself.
When you start to use a third party library in Java, you add their jars to your project. Do you also add the jars to the repository or do you just keep a local installation. And if the latter, how do you sync between team members to be able to work?
Thanks.
Yes. You should add to the repository whatever is required for a developer on a clean system (aside from having the JDK and ant installed) to check out and build the project.
No, if you use Maven. Put them into Maven repository (if they are not there yet, most open source libraries are in public Maven repositories already).
Yes, if you use Ant.
If you're using ANT and you want maven style dependency management without maven, take a look at Ivy. It can download from maven repositories and can even read maven pom files.
Yes, third-party libraries should be version controlled since you will want everyone to compile against the same version of that library. This way when updates to that third-party library happen you will simply have to change it in one place and everyone will update to the new version. Otherwise you could easily end up with the famous: "It compiles on my machine!"-syndrome.
Assuming you do not use Maven, then you should definitely check in versions of 3rd party jars. You (or more importantly, the maintenance developer coming along after you) should be able to pull a version of your application from the repository and have all the correct versions of all the 3rd party jars required at compile- and runtime.
You also should maintain the versioning history of the 3rd party jars, again to help the maintenance developer out. Checking them in to the repository is the easiest and most effective way to do this.
I would recommend versioning everything you need to build a project. Unless your using a build tool like maven.
Short Answer
No
Longer answer
In the interim, use a folder on your network separate from the repo.
A Long Term Solution
Use maven. It was built for handling your build, configuration and external jar dependencies.
Go here for documentation though. (The official maven documentation is known to be pretty spotty).
Lately I've been keeping JARs out of SVN. I have a separate repository that I use to pull them into projects, manage dependencies, and keep track of different versions.
This is what Maven is born for. I've got to learn how to use it well, but until then I'm keeping my own JAR repository.