Develop Reference

Best practice for deploying different versions of project for different other projects

We have many Maven projects, and one of these projects is included as a dependency in every other project. The problem is, that when deploying a new version of this dependency used by the others, every project gets this new version, which could lead to problems.
Of course, I could manually change the version every time I deploy the project, but that could lead to problems as well, i.e. when forgetting to change the version before deploying it.
I also saw the solution of using a ${version} placeholder in the "version"-tag, but that would mean, that I have to specify the version every time I'm doing a Maven command.
Is there a solution for such problems, where you have a dependency used in many other projects and need a different version in everyone of these projects?

The first thing I see
The problem is, that when deploying a new version of this dependency
used by the others, every project gets this new version, which could
lead to problems.
This shows a big issue. You seemed to be violating the foundational rule of immutable releases.
In consequence, the version is in the end useless because it's always the same and does not transport any kind of information.
There is first to say you should follow semantic versioning.
Also you should use either maven-release-plugin to increment the numbers automatically (but that will not solve the issue minor or major release), there are tools to identify such things). This should be solved by using a CI/CD setup (Jenkins, etc.).
Tools to check changes (compatibility) are things like RevAPI or
JAPI-Checker etc. Also some useful information here.
Furthermore you can do that via different setups The mentioned ${version} is simply wrong and will not work in several ways.
Upgrading a larger number of projects can be done by using something like Renovate or things like Dependabot or even some existing Maven plugins which can be run via CI/CD automatically (scheduled) which you should even do for security scans etc. That means automation is the keyword here.

tools for patch dependency

I have such a problem：
When customer asked for a new feature of our system,we patched all the changed files into a patch and send it to a workmate who work with customer and do the release.
However,not every patch released in order.So,there is a chance patch A is depending patch B but patch B is releasing in front of patch A.
Due to the workmate is not familiar with programming so he can't figure out the reason.I have to spend time to see what's wrong.
When the number of patch waiting for release is growing up,releasing a patch seem like a nightmare for us.
Is there a tool for such dependency analysis?so we can see the dependency of patch and can reduce the time spending for figure out the dependency.
Thx a lot.

You can do it using any dependency management system like Maven, internal artifact repository for production-ready components (like Nexus) and release branches for hot-fixes (if you have to ship updates in a near realtime way).
Using this approach you get:
Testing of any complete release (including integration tests) which you ship to a customer.
Dependency-broken versions are not to build, because you can switch to production-ready repository for pre-ship build.
You can mark production packages with SCM tags and know what exactly is pushed to a customer.
You can simply make a diff between shipped and current package.
In few words:
Divide development and production releases and you protect yourself to build a dependency broken production package.

the correct way to handle these situation is using a Configuration Management procedure.
a simple one, for example, involves a CVS/SVN and a changelog between revision A and revision B.
every file in the changelog will compose the patch.
a more complex procedure will introduce baselines and intermediate releases.

When should mvn release be used in the project lifecycle?

To clarify the question :
I am looking for established best-practices or a pro/con analysis of known practices
by project lifecycle I mean : deploy to pre-integration, integration, QA, preprod and prod environment.
For some context:
Our project deploys to integration and QA every week, currenlty we create a new release for each integration deployment, but this doesn't feel right. It leads to updating all the poms every week breaking dev level dependencies, forcing every dev to do a refresh of their eclipse configurations. We have large workspaces and eclipse doesn't handle the refreshes so well thus a lot of wasted time.
I am not overly familiar with the maven release conventions and have been unable to find the ones regarding the point of the application lifecycle when mvn release should be used.
If the pattern we use now is accepted/correct/established I will have another question :)

The approach I use to avoid the Eclipse dev level dependency update issue is to leave the relevant trunk or branch version number unchanged until such time as the release becomes significant. This way you can have properly tagged/versioned releases to QA etc so that you can track issues back but not require devs to update dependencies. To achieve this I use the following command but override the version numbers to get the desired release number but re-enter the current snapshot version as the new snapshot version:
mvn release:prepare -DautoVersionSubmodules=true
P.S. I have a diagram that demonstrates this but unfortunately insufficient rights in this forum to attach it. I would happily provide it if someone can facilitate attaching.
P.P.S Maybe now...
Note also the support for early branching (2.1) and late branching (2.2).

In our shop, all of our POMs in SVN have <version>9999-SNAPSHOT</version> (for their own version as well as internal dependencies). This never changes.
During the build, we have a simple ant build.xml that takes the version number (established outside of maven) as a -Dversion=... parameter, and simply does:
<replace includes="**/pom.xml" token="9999-SNAPSHOT" value="${version}"/>
<artifact:mvn ... />
That change is local to the build process's working copy -- it's never checked in to version control.
This way all release builds have a "real" version number, but dev effectively never has to deal with version numbers.
The above is, as you say in your question, emphatically not The Right Way to do this, but it has worked well for us for the ~9 mos since we adopted maven. We have tens of maven modules, all of which move in lock-step through the QA/release process.
One implication of this approach is that you'll need separate eclipse workspaces for each branch you're working on, as otherwise the copies of a project from dif't branches will collide.

[Not really an answer, but the best I have...]
Related to MNG-624.
Depending on how many projects you have, even the burden on your source-control system may be an issue.
Does anyone use an independent numbering scheme with Maven snapshots to avoid version-number churning? In theory, you could do what you'd do without Maven - use an internal numbering system of some kind for the weekly builds. The builds would be deployed to different repositories as dictated by workflow; you'll need separate repositories for dev, QA, maybe one in between for integration test. When you're down to release candidates, start using non-snapshot releases. I'm just evaluating Maven, though - I've no experience with doing this.
Some of the Nexus documentation (for the Professional version) talks about how to do build staging, which may be relevant.

In the past I used a numbering scheme of my own devising: http://wiki.secondlife.com/wiki/Codeticket_Service
I'm now in the situation where I need to think about maven again, and I'm tempted to re-use the codeticket scheme to generate version numbers/build numbers and apply them via the release plugin but without checking the pom files back in. The checked in pom files will keep the SNAPSHOT version numbers.
For those who care about reproducible builds, you can include the modified POM file in your build result. Personally, I care more about tracking the build artifacts and ensuring that the exact same bits that have been tested end up getting released, so my concern about reproducing builds is slightly less religious than with most (See here).

There is a discussion going on in the maven users list (in which I'm participating) that seems relevant. Basically we're discussing how to avoid all that POM editing that has to be done whenever you cut a (release or feature) branch. The release plugin can do the editing for you, when you create a release branch, but it does not help with feature branches that need to be reintegrated later. Also, all that POM editing causes unecessary pain when you do merges, either rebase merges from trunk or reintegration merges to trunk.
The idea being discussed there is based on the notion that the proper location to record artifact version numbers is in the SCM tool and not in the POM. Basically, maven should be able to derive the artifact version number from the actual SCM tag or branch that the working area is associated to.
Note that there is not a complete solution yet due to some issues still pending on Maven's issue tracker (e.g. MNG-2971). But they are issues with many votes already and I'm optimist they will be fixed soon.

Java jar files into a repository (CVS, SVN..)

Why it's a bad idea to commit Java jar files into a repository (CVS, SVN..)

Because you can rebuild them from the source. On the hand if you are talking about third-party JAR files which are required by your project then it is a good idea to commit them into the repository so that the project is self-contained.

So, you have a project that use some external dependencies. This dependencies are well known. They all have
A group (typically, the organization/forge creating them)
An identifier (their name)
A version
In maven terminology, these informations are called the artifact (your Jar) coordinates.
The dependencies I was talking about are either internal (for a web application, it can be your service/domain layer) or external (log4j, jdbc driver, Java EE framework, you name it, ...). All those dependencies (also called artifacts) are in fact, at their lowest level, binary files (JAR/WAR/EAR) that your CVS/SVN/GIT won't be able to store efficently. Indeed, SCM use the hypothesis that versionned content, the one for which diff operations are the most efficient) is text only. As a consequence, when binary data is stored, their is rarely storage optimization (contrary to text, where only versions differences are stored).
As a consequence, what I would tend to recommand you is to use a dependency management build system, like maven, Ivy, or Gradle. using such a tool, you will declare all your dependencies (in fact, in this file, you will declare your dependencies' artifacts coordinates) in a text (or maybe XML) file, which will be in your SCM. BUT your dependencies won't be in SCM. Rather, each developper will download them on its dev machine.
This transfers some network load from the SCM server to the internet (which bandwidth is often more limitated than internal enterpise network), and asks the question of long-term availability of artifacts. Both of these answers are solved (at least in amven work, but I believe both Ivy and gradle are able to connect to such tools - and it seems some questions are been asked on this very subject) using enterprises proxies, like Nexus, Artifactory and others.
The beauty of these tools is that they make available in internal network a view of all required artifacts, going as far as allowing you to deploy your own artifacts in these repositories, making sharing of your code both easy and independant from the source (which may be an advantage).
To sum up this long reply : use Ivy/Maven/Gradle instead of simple Ant build. These tools will allow you to define your dependencies, and do all the work of downloading these dependencies and ensuring you use the declared version.
On a personnal note, the day I discovered those tools, my vision of dependency handling in Java get from nightmare to heaven, as I now only have to say that I use this very version of this tool, and maven (in my case), do all the background job of downloading it and storing at the right location on my computer.

Source control systems are designed for holding the text source code. They can hold binary files, but that isn't really what they are designed for. In some cases it makes sense to put a binary file in source control, but java dependencies are generally better managed in a different way.
The ideal setup is one that lets you manage your dependencies outside of source control. You should be able to manage your dependencies outside of the source and simply "point" to the desired dependency from within the source. This has several advantages:
You can have a number of projects dependent on the same binaries without keeping a separate copy of each binary. It is common for a medium sized project to have hundreds of binaries it depends on. This can result in a great deal of duplication which wastes local and backup resources.
Versions of binaries can be managed centrally within your local environment or within the corporate entity.
In many situations, the source control server is not a local resource. Adding a bunch of binary files will slow things down because it increases the amount of data that needs to be sent across a slower connection.
If you are creating a war, there may be some jars you need for development, but not deployment and vice versa. A good dependency management tool lets you handle these types of issues easily and efficiently.
If you are depending on a binary file that comes from another one of your projects, it may change frequently. This means you could be constantly overwriting the binary with a new version. Since version control is going to keep every copy, it could quickly grow to an unmanageable size--particularly if you have any type of continuous integration or automated build scripts creating these binaries.
A dependency management system offers a certain level of flexibility in how you depend on binaries. For example, on your local machine, you may want to depend on the latest version of a dependency as it sits on your file system. However, when you deploy your application you want the dependency packaged as a jar and included in your file.
Maven's dependency management features solve these issues for you and can help you locate and retrieve binary dependencies as needed. Ivy is another tool that does this as well, but for Ant.

They are binary files:
It's better to reference the source, since that's what you're using source control for.
The system can't tell you which differences between the files
They become a source of merge-conflicts, in case they are compiled from the source in the same repository.
Some systems (e.g. SVN) don't deal quite well with large binary files.
In other words, better reference the source, and adjust your build scripts to make everything work.

The decision to commit jar files to SCM is usually influenced by the build tool being used. If using Maven in a conventional manner then you don't really have the choice. But if your build system allows you the choice, I think it is a good idea to commit your dependencies to SCM alongside the source code that depends on them.
This applies to third-party jars and in-house jars that are on a separate release cycle to your project. For example, if you have an in-house jar file containing common utility classes, I would commit that to SCM under each project that uses it.
If using CVS, be aware that it does not handle binary files efficiently. An SVN repository makes no distinction between binary and text files.
http://svnbook.red-bean.com/en/1.5/svn.forcvs.binary-and-trans.html
Update in response to the answer posted by Mark:
WRT bullet point 1: I would say it is not very common for even a large project to have hundreds of dependencies. In any case, disk usage (by keeping a separate copy of a dependency in each project that uses it) should not be your major concern. Disk space is cheap compared with the amount of time lost dealing with the complexities of a Maven repository. In any case, a local Maven repository will consume far more disk space than just the dependencies you actually use.
Bullet 3: Maven will not save you time waiting for network traffic. The opposite is true. With your dependencies in source control, you do a checkout, then you switch from one branch to another. You will very rarely need to checkout the same jars again. If you do, it will take only minutes. The main reason Maven is a slow build tool is all the network access it does even when there is no need.
Bullet Point 4: Your point here is not an argument against storing jars in SCM and Maven is only easy once you have learned it and it is only efficient up to the point when something goes wrong. Then it becomes difficult and your efficiency gains can disappear quickly. In terms of efficiency, Maven has a small upside when things work correctly and a big downside when they don't.
Bullet Point 5: Version control systems like SVN do not keep a separate copy of every version of every file. It stores them efficiently as deltas. It is very unlikely that your SVN repository will grow to an 'unmanageable' size.
Bullet Point 6: Your point here is not an argument against storing files is SCM. The use case you mention can be handled just as easily by a custom Ant build.

Tips for maintaining an internal Maven Repository?

I'm interested in maintaining a Maven 2 repository for my organization. What are the some of the pointers and pitfalls that would help.
What are guidelines for users to follow when setting up standards for downloading from or publishing their own artifacts to the repository when releasing their code? What kinds of governance/rules do you have in place for this type of thing? What do you include about it in your developer's guide/documentation?
UPDATE: We've stood up Nexus and have been very happy with it - followed most of Sal's guidelines and haven't had any trouble. In addition, we've restricted deploy access and automated build/deployment of snapshot artifacts through a Hudson CI server. Hudson can analyze all of the upstream/downstream project dependencies, so if a compilation problem, test failure, or some other violation causes the build to break, no deployment will occur. Be weary of doing snapshot deployments in Maven2/Maven3, as the metadata has changed between the two versions. The "Hudson only" snapshot deployment strategy will mitigate this. We do not use the Release Plugin, but have written some plumbing around the Versions plugin when going to move a snapshot to release. We also use m2eclipse and it seems to work very well with Nexus, as from the settings file it can see Nexus and knows to index artifact information for lookup from there. (Though I have had to tweak some of those settings to have it fully index our internal snapshots.) I'd also recommend you deploy a source jar with your artifacts as a standard practice if you're interested in doing this. We configure that in a super POM.
I've come across this Sonatype whitepaper which details different stages of adoption/maturity, each with different usage goals for a Maven Repository manager.

I would recommend setting up one nexus server with at least four repositories. I would not recommend artifactory. The free version of nexus is perfectly fine for a dev team of less than 20 in less than three groups. If you have more users than that, do yourself a favor and pay for the Sonatype release. The LDAP integration pays for itself.
Internal Release
Internal Snapshot
Internal 3rd Party for code used in house that comes from outside sources, or for endorsed 3rd party versions. Put the JDBC drivers, javax.* stuff and stuff from clients and partners here.
External Proxies common proxy for all the usual sources like m2, codehaus etc
Configure Nexus to do the following for internal repos
Delete old Snapshots on regular intervals
Delete Snapshots on release
Build index files. This speeds up local builds too
Have a common settings.xml file that uses these four and only these four sources. If you need to customize beyond this try to keep a common part of the settings file and use profiles for the differences. Do not let your clients just roll their own settings or you will end up with code that builds on one machine but not on any other machine.
Provide a common proxy for your clients. In Nexus, you can add a bunch of proxies to the common Maven sources (Apache, JBoss, Codehaus) and have a single proxy exposed to the internal clients. This makes adding and removing sources from your clients much easier.
Don't mix Internal and 3rd party artifacts in the same repository. Nexus allows you to add jars to an internal repository via a web gui. I recommend this as the way of adding your JDBC drivers and other external code to 3rd party. The UI is quite nice to use when compared to most enterprise software.
Define a common parent POM that defines the Internal snapshot and release repos via the distributionManagement tag. I know lots of people tell you not to do this. And while I freely admit that there are all kinds of problems with doing this, it works out OK if the clients will only be building releases and snapshots to be deployed to a single internal repository.
If you have an existing mis-managed Maven repository, create a 5th repos called Legacy and put the whole repos there. Set up a cron task to delete old files from legacy once they are a year old. That gives everyone a year to move off of it and update their poms.
Establish an easy to stick to naming convention for internal artifacts. I prefer GroupID of Department.Function.Project and an ArtifactId for that componentName. For internal repositories, com/org/net and the company name are likely to be irrelevant. And wrong if the company changes its name. It is far less likely that the sales, accounting or inventory department will be renamed.

Definitely use Nexus. :P
I've used both Nexus and Artifactory. The interface for Nexus is a lot more robust, it's a lot more configurable, and of course, written by Sonatype, who repesents pretty much everything Maven well.
That being said, Artifactory is decent and workable.
A review of Nexus vs. Artifactory
Oh my! Of course, here's a SO quesiton about the matter.
Sonatype does a feature comparison
jFrog (maker of Artifactory) does a feature comparison

Use Artifactory.

I am using Artifactory myself, and love the user interface and ease of deployment/maintenance. That said, I have never used Nexus, and cannot really help you with a proper feature comparison.
Here are some things off the top of my head that I really like about Artifactory (keep in mind Nexus may have these features too):
Nice Web 2.0 interface.
The ability to import your local Maven repository to help get you started.
Ease of integration with existing LDAP servers for security (I'm a big fan of a single repository for storing credentials).
Given that there's really only two major Maven Repository implementation out there, if you really want to make sure you've made the right choice, I'd recommend trying both out, and deciding for yourself which you like better.

Perhaps this is obvious, but, for reproducibility, developers should never overwrite artifacts, they should be new versions.
This also applies to upstream repositories. If you download Apache-commons version 1.2.3, you should really never download it again. Fixes come from latter versions, not applied to existing versions.

Something else to consider:
http://archiva.apache.org/

As the the ORIGINAL QUESTION (technical issues to consider when constructing a M2 repository), I would recommend creating read-only user for browsing the repository and administrative user per administrator (that said: one read-only user for all those users that are not administrators).
Moreover, I would recommend generating backup images periodically (once a day perhaps ?). Very important both if your repository is big or you install your own artifacts from time to time.
Last, but not least, when adding new remote repositories, you must add inclusion/exclusion filters so an artifact lookup in the repository would be done more quickly.
There are lots of other issues to consider, but these are the leading issues I've encountered while managing a Maven internal repository.
For the record, I'm using both Nexus and Artifactory; I can clearly state that while Nexus is very simple and operative (though I sometimes have problems with the installation process on Ubuntu), its free version cannot compete with Artifactory's community (free) edition.
Excluding Artifactory's awesome web 2 UI, its main features, such as security management, periodic backups, and accessibility issues are way beyond those of Nexus.