I started reading about good practices related to multi-module maven projects, specifically about the advice to use separate pom aggregators and parents. I found this site which contains the following paragraph:
What you can see in the module POMs example is that it inherits its version from the parent POM. This is quite natural for small to medium sized projects. The team makes code changes on every module for the next release.
You may use different versions for your modules, but this will not take you very far. If you release the project via its aggregate parent POM, all modules will get released, with their individual version. That’s fine – at first thought. If you look closer you will notice that the version will increase, even if some of the modules have not any changes at all. With this approach, having a multi-module aggregate parent POM using different versions for each submodule, you only have the flexibility to decide, if a new version is major, minor or micro. But is this worth the effort?
As far as I know, when you run the release plugin, it will ask for the new versions of each sub-module. If there were no changes made to a particular sub-module, you can decide not to bump its version.
What did the author mean?
By that paragraph, the author intended to highlight the fact that you can not decide not to build a sub-module, the only thing that you have control upon is the new development version.
Related
Currently I'm in the transition of moving ANT projects to Maven and struggling on how to get the project versioning working correctly. Currently I have about 30+ projects/modules that all rely on each other so everything must be at the latest version to work correctly. This was easily done with ANT but when it comes to Maven I would need to make constant changes to all other released project POM's to allow them to pick up these new changes.
I discussed with a few other developers and we decided we might not even need a maven repo with version numbers, we just have everything at the same version number and build locally or through Jenkins to update our .m2 folders. Does this sound like the correct route for our situation? Are we missing anything doing this?
I did suggest having our test Jenkins to deploy to a repo with version numbers like 1.0.Beta-SNAPSHOT. We have Jenkins setup to build when our testing branches are updated. This means I would not have to locally compile every project on that branch to update my .m2, I could just change the POM to pull all these Beta-SNAPSHOT versions in one place. Would there be a good way for me to do this that would not affect the release if it was pushed and released with this version number set? If I wanted to use my local versions I would then just switch this version number to 1.0.0 which isn't within the repo but my local .m2.
Any suggestions on how to properly manage the maven projects/modules with version numbers would be welcome! Something that reduces the need to change every POM when releasing 1 of the projects/modules would be best!
Our developer struggle with this problem a lot. It is a lot of manual work to update all the POMs for a release.
We are going to aim for multi-module projects, which also seems like a good fit for you.
If you say, that everything must be using the latest versions all the time, I would put all the projects into one large multi-module project. This means that you have one (git) repository with a main POM in the root directory and a directory for each module (sub-project) with its own POM references the main POM as parent.
Then you can run mvn clean install on the parent and build all the modules with consistent version numbers. So releasing is then just one large build.
You should note, though, that you tie the projects (modules) closely together in this way, but it I understood you correctly, they are already tightly interrelated.
I stumbled upon the Multi Module Maven Release Plugin which offers the functionality of only building changed modules (and everything that depends on them).
As it only works with Git (we have SVN) and I do not know whether it is stable/well tested, I wonder if this functionality is also achievable without using the plugin (or reverse engineering it).
We are a large company with about 2000 separate Java projects. For historic reasons, we do not have multi-module projects, but we would like to introduce them.
Logically, we already have "groups" of projects, i.e. someone responsible for (say) 50 projects which are closely related. This someone regularly publishes a BOM which contains recent, coherent versions of these 50 projects.
Now it would make a lot of sense to grab these 50 projects and put them into one large multi-module project. Still, it would be necessary to publish a BOM because other projects (outside our group) should have coherent versions.
So, summarised, we need a BOM that contains the versions of all 50 projects that are part of the multi-module project. I wonder what would be the "Maven way" to create such a BOM. What I can think of:
The bom is the 51st project of the multi-module project. The versions of the dependencies are set by properties in the parent pom.
The bom is generated from the information present in the multi-module project and published as side artifact (this probably requires us to write a Maven plugin for this).
What would be advisable?
We are using BOMs as well for our multi-modules projects, but we are not tying their generation or update to the build of those modules.
A BOM is only updated when our release management process completes the delivery of a built module (or group of modules): once delivered, then the BOM is updated and pushed to Nexus (stored as a 1.0-SNAPSHOT version, constantly overridden after each delivery)
The BOM is then included within our POM (for mono or multi-module projects) and use for dependency management only, meaning our projects depends on artifact without the version: the dependency management from the BOM provides with the latest delivered version of other dependent modules.
In other words, we separate the build aspect (done here with maven) from the release part: the "bills of materials" represent what has been delivered, and ensure all projects are building with versions deemed working well together (since they have been delivered into production together).
I've never seen 2K of commercial Java projects, so will base my answer on how open source works:
Libraries shouldn't be grouped by people - they should be grouped by the problems that they solve. Often open source projects have multiple libs e.g. Jackson has jackson-databind, jackson-datatype-jsr310, etc. These libs tightly relate to each and may depend on each other.
Such groups shouldn't be too big. Some projects may have 1, others - 5 or 10. 50 libs in a group sounds way too much.
It's easier if libs in a group are released all at the same time (even if only one is updated). This makes it straightforward to keep track of versions in the apps that use multiple libs from a group.
There should be no dependencies between groups! And this is probably the most important rule. Deep hierarchy of libraries that depend on each other is not acceptable because now you need to keep compatibility between many projects and libs. This just doesn't scale. Which means there will be occasional copy-paste code between libs - this is the lesser evil.
There could be some exceptions to the last rule (maybe a lib that is used everywhere) but those must keep backward compatibility of the public API until there are no projects that depend on the old API. Such libs are very hard to maintain and it's better to opensource them.
Standalone projects now can depend on libraries from the same or different groups, but because the version within the group is the same, it's easy to set it as a property just once. Alternatively:
You can look at <scope>import</scope> which allows importing <dependencyManagement> sections from other POM files like parent POMs within a group (for some reason never worked for me).
Or at xxx-all modules - a module that depends on all other modules within group and thus when you depend on it, you also depend on others transitively.
EDIT: This is about doing Continuous Delivery with Maven and having it orchestrated with Jenkins. Maven is definitively not designed for that, and this question is part of our effort to get an efficient workflow without using Maven releases. Help is appreciated.
We use Maven -SNAPSHOTs within major versions to ensure customers always get the latest code for that given version, which works well. For technical reasons we have two independent Maven jobs - one for compiling sources to jars, and one for combining the appropriate jars to a given deployment. This also works well.
We then have Jenkins orchestrating when to invoke the various steps, and this is where it gets a bit tricky, because if we do the normal mvn clean install in step one, this means that all the snapshot artifacts get recompiled, which in turn makes Jenkins think that all the snapshots changed (as their fingerprint - aka MD5 checksum - changed) even if the sources used to generate the artifacts did not change, triggering all the downstream builds instead of just those which dependencies did change.
I have so far identified these things as varying between builds:
META-INF/maven/.../pom.properties (as it contains a timestamp)
META-INF/MANIFEST.MF (contains JDK and user)
timestamps in jar file
I have found ways around the two first, but the latter is a bit more difficult. It appears that AbstractZipArchiver (which does all the work in zipFile() and zipDir()) is not written to allow any kind of extension to how the archive is being generated.
For now I can imagine four approaches (but more ideas are very welcome):
Create a derivative of the current maven-jar-plugin implementation allowing for a timestamp=<number> attribute which is then used for all entries inserted into the jar file. If not set, the current behavior is kept.
Revise the Jenkins fingerprinting scheme so it knows about jar files and only looks at the entries contents, not their metadata.
Attach a plugin to the prepare-package stage responsible for touching the files with a specific time stamp. This requires all files to be present at that time (meaning that the jar plugin cannot be allowed to touch the MANIFEST.MF file)
Attach an extra plugin to the "package" phase which rewrites the finished jar file, zeroing out all zip entry timestamps in the process.
Again, the goal is to make maven SNAPSHOT artifacts fully time independent so given the same source you get an artifact with the same MD5 checksum. I also believe, however, that this could be beneficial for release builds.
How should I approach this?
As per my comment, I still think the answer is to do none of the things you suggest, and instead use releases in preference to snapshots for artifacts which you are in fact releasing to customers.
The problems you describe are:
you have a multi-module project which takes a long time to build because you have more than 100 modules,
you have two snapshot artifacts which you think ought to be identical (because the source code and metadata were identical at build time), but they have different checksums.
My experience with Maven tells me that if you try and adhere to the "Maven Way", tools will work well for you out-of-the-box, but if you deviate then you'll have a bad time. Unfortunately, the Maven Way is sometimes elusive :-)
Multi-module projects in Maven are very useful when you have families of modules with code that varies in sympathy, e.g. you have a module containing a bunch of interfaces, and some sibling modules providing implementations. It would be unusual to have more than a dozen modules in a multi-module project. All the modules ought to share the version number of the parent (Maven doesn't enforce this, which in my opinion is confusing).
When you build a snapshot version of a multi-module project, snapshots of all modules are built, even if the code in a particular module hasn't changed. Therefore you can look at a family of modules in your repositiory, and know that at compile time the inter-module code references were satisfied.
For example, in a domain model module you might have an interface:
public interface Student {
void study();
}
and in some sibling modules, which would declare compile-scoped dependencies on the domain model in their POMs, you might have implementations.
If you were then to change the interface in the domain model module:
public interface Student {
void study();
void drink(Beer beer);
}
and rebuild the multi-module project, the build will fail. The dependent modules will fail to build, even though their code and POMs have remained the same. In a multi-module project, you only install or deploy artifacts if all the child modules build successfully, so rebuilding snapshots is usually very desirable - it's telling you something about the inter-module dependencies.
If:
you have an excessive number of modules, and/or
those modules can't reasonably share the same version number, and/or
you don't need any guarantees about code references between modules,
then your modularisation is incorrect. Don't use multi-module projects as a build system (you have Jenkins for that), use it instead to express relationships between modules of your code.
In your comment, you say:
RELEASE artifacts behave the same way when being rebuilt by Jenkins.
The point of point of release artifacts is that you do not rebuild them - they are definitive! If you use something like Artifactory, you will find that you cannot deploy a release artifact more than once - your Jenkins job should fail if you attempt it.
This is a fundamental tenet in Maven. One of the aims of Maven is that it if two developers on separate workstations were to attempt the same release, they would build artifacts which were functionally identical. If you are build an artifact which expresses a dependency (maybe for compilation purposes, or because it's being assembled into .war etc.) on another, then:
if the dependency is a snapshot, Maven might seek a newer version from the repository.
if the dependency is a release, the version in your local repository is assumed to be definitive.
If you could rebuild a release artifact, you would create the possibility that two developers have dissimilar versions in their repository, and you'd have dissimilar builds depending on which workstation you used. Don't do it.
Another critical detail is that a release artifact cannot depend on snapshot artifacts, again, you would lose various guarantees.
Releases are definitive, and it sounds like you want your assembly to depend on definitive artifacts. Jenkins makes tagging and releasing multi-module projects very straightforward.
In summary:
Check your modularisation: one enormous multi-module project is not useful.
If you don't want to continually rebuild snapshots, you need to do releases.
Never release snapshots to your customer.
Follow the dependency graph of your assembly project and release any snapshots.
Release the assembly project, bumping your minor version.
Ensure your customer refers to the complete version number of your assembly in communications.
I'm not sure whether the title makes a whole lot of sense and whether this post already answers my question, but here it is:
We have a multi-module project. As you would expect this projects has a combination of internal and third party dependencies. For the third party dependencies we define these in the dependency management section of our parent POM so that we can manage the versions of these dependencies in a single common place.
As for inter-project (internal) dependencies, so far we've just entered the versions within each modules POM where a dependency is required. Then when doing a prepare with the release plugin, these versions are updated appropriately - all very nice.
What we want, like with the third party dependencies, is to be able to specify the internal dependency versions in the parent POM and therefore have a single common place. I see three potential approaches.
We do this by creating a property in the parent POM.
We do it via the dependency management section in the parent POM.
We use the project version property as the dependency version.
The preference would be to use one of the first two approaches, though there isn't really a strong reason for this. This leads me onto the main concern and question: If we use either of the first two approaches, will the release plugin still update the dependency version during the prepare stage?
All thoughts/feedback appreciated.
In the end we used the project.version property to help manage this. However, from what I understand, I believe using the dependency management section in the parent POM would also work.