Currently I'm in the transition of moving ANT projects to Maven and struggling on how to get the project versioning working correctly. Currently I have about 30+ projects/modules that all rely on each other so everything must be at the latest version to work correctly. This was easily done with ANT but when it comes to Maven I would need to make constant changes to all other released project POM's to allow them to pick up these new changes.
I discussed with a few other developers and we decided we might not even need a maven repo with version numbers, we just have everything at the same version number and build locally or through Jenkins to update our .m2 folders. Does this sound like the correct route for our situation? Are we missing anything doing this?
I did suggest having our test Jenkins to deploy to a repo with version numbers like 1.0.Beta-SNAPSHOT. We have Jenkins setup to build when our testing branches are updated. This means I would not have to locally compile every project on that branch to update my .m2, I could just change the POM to pull all these Beta-SNAPSHOT versions in one place. Would there be a good way for me to do this that would not affect the release if it was pushed and released with this version number set? If I wanted to use my local versions I would then just switch this version number to 1.0.0 which isn't within the repo but my local .m2.
Any suggestions on how to properly manage the maven projects/modules with version numbers would be welcome! Something that reduces the need to change every POM when releasing 1 of the projects/modules would be best!
Our developer struggle with this problem a lot. It is a lot of manual work to update all the POMs for a release.
We are going to aim for multi-module projects, which also seems like a good fit for you.
If you say, that everything must be using the latest versions all the time, I would put all the projects into one large multi-module project. This means that you have one (git) repository with a main POM in the root directory and a directory for each module (sub-project) with its own POM references the main POM as parent.
Then you can run mvn clean install on the parent and build all the modules with consistent version numbers. So releasing is then just one large build.
You should note, though, that you tie the projects (modules) closely together in this way, but it I understood you correctly, they are already tightly interrelated.
Related
This may be an easy answer for you but I've been spending lot of my time changing the project versions. Here is my situation:
I have 8 java spring projects in my eclipse workspace. 5 projects depend on the 3 other libraries.If I change the versioning of any of the three projects [which is quite often], I need to change the versions all the 5 project pom.xml files. I found it very cumbersome. I tried changing the versioning of these 3 dependent projects to a constant value [e.g. latest], then my local set up is working fine but CI/CD doesn't change the next incremental version.
I'm pretty sure I'm not the only one who is facing this type of situation. Hoping someone can guide me on how to manage the versioning locally and in the CI/CD process.
Thank you for reading this!
1st solution: Put all eight projects into one multi-module project and build it together with the same version number
2nd solution: Do not change the versions so often, only if you publish everything as productive version to the outside world.
I have a library and a program, both under my control and built using Gradle. What's the best way to develop these two at the same time?
I have set up a private maven repository to distribute the library and that's working, but I don't want to release to that repository every little experiment I make during development. It's slow and disruptive to users of the library.
I tried installing the jar to the local maven repository as explained here: Gradle alternate to mvn install but the project that's using the library is not picking up that newly installed version.
I think, you can try to use multi-project builds for that if it's possible. But you will likely need to restructure both your current projects to become modules of the same new project.
What's the best way to develop these two at the same time?
It depends by how the team is organized and what are your policies.
Of course if the team can use a git repo and access to the source code you can just use git without pushing a new version on the maven server for each commit or push.
Otherwise if other users can only use the final library, you have to push the version on the maven server.
I have set up a private repository to distribute the library and that's working, but I don't want to release to that repository every little experiment I make during development. It's slow and disruptive to users of the library.
Every maven repo has 2 different repositories:
release
snapshot
Usually release repo is used only for stable releases and the snapshot repo is used to publish little change, beta release and so on.
In any case it is not required that every changes in the code is pushed in the maven repo (it is a your choice)
It's slow
The time to upload artifacts usually is not so big, in any case you can evaluate to push the release in the maven repo with a CI server.
The best method seems to be to make one project include the other one when present by adding:
if (file("../libraryproject").exists()) {
includeBuild "../libraryproject"
}
to the settings.gradle file of the project that uses the library. That can be committed to the source code repo because when that directory doesn't exist, the dependency will be included in the traditional way.
This is called Composite Build in the Gradle world, IntelliJ seems to handle properly and theres'a recorded webcast showing the whole setup: https://www.youtube.com/watch?v=grPJanXfRPg
EDIT: This is about doing Continuous Delivery with Maven and having it orchestrated with Jenkins. Maven is definitively not designed for that, and this question is part of our effort to get an efficient workflow without using Maven releases. Help is appreciated.
We use Maven -SNAPSHOTs within major versions to ensure customers always get the latest code for that given version, which works well. For technical reasons we have two independent Maven jobs - one for compiling sources to jars, and one for combining the appropriate jars to a given deployment. This also works well.
We then have Jenkins orchestrating when to invoke the various steps, and this is where it gets a bit tricky, because if we do the normal mvn clean install in step one, this means that all the snapshot artifacts get recompiled, which in turn makes Jenkins think that all the snapshots changed (as their fingerprint - aka MD5 checksum - changed) even if the sources used to generate the artifacts did not change, triggering all the downstream builds instead of just those which dependencies did change.
I have so far identified these things as varying between builds:
META-INF/maven/.../pom.properties (as it contains a timestamp)
META-INF/MANIFEST.MF (contains JDK and user)
timestamps in jar file
I have found ways around the two first, but the latter is a bit more difficult. It appears that AbstractZipArchiver (which does all the work in zipFile() and zipDir()) is not written to allow any kind of extension to how the archive is being generated.
For now I can imagine four approaches (but more ideas are very welcome):
Create a derivative of the current maven-jar-plugin implementation allowing for a timestamp=<number> attribute which is then used for all entries inserted into the jar file. If not set, the current behavior is kept.
Revise the Jenkins fingerprinting scheme so it knows about jar files and only looks at the entries contents, not their metadata.
Attach a plugin to the prepare-package stage responsible for touching the files with a specific time stamp. This requires all files to be present at that time (meaning that the jar plugin cannot be allowed to touch the MANIFEST.MF file)
Attach an extra plugin to the "package" phase which rewrites the finished jar file, zeroing out all zip entry timestamps in the process.
Again, the goal is to make maven SNAPSHOT artifacts fully time independent so given the same source you get an artifact with the same MD5 checksum. I also believe, however, that this could be beneficial for release builds.
How should I approach this?
As per my comment, I still think the answer is to do none of the things you suggest, and instead use releases in preference to snapshots for artifacts which you are in fact releasing to customers.
The problems you describe are:
you have a multi-module project which takes a long time to build because you have more than 100 modules,
you have two snapshot artifacts which you think ought to be identical (because the source code and metadata were identical at build time), but they have different checksums.
My experience with Maven tells me that if you try and adhere to the "Maven Way", tools will work well for you out-of-the-box, but if you deviate then you'll have a bad time. Unfortunately, the Maven Way is sometimes elusive :-)
Multi-module projects in Maven are very useful when you have families of modules with code that varies in sympathy, e.g. you have a module containing a bunch of interfaces, and some sibling modules providing implementations. It would be unusual to have more than a dozen modules in a multi-module project. All the modules ought to share the version number of the parent (Maven doesn't enforce this, which in my opinion is confusing).
When you build a snapshot version of a multi-module project, snapshots of all modules are built, even if the code in a particular module hasn't changed. Therefore you can look at a family of modules in your repositiory, and know that at compile time the inter-module code references were satisfied.
For example, in a domain model module you might have an interface:
public interface Student {
void study();
}
and in some sibling modules, which would declare compile-scoped dependencies on the domain model in their POMs, you might have implementations.
If you were then to change the interface in the domain model module:
public interface Student {
void study();
void drink(Beer beer);
}
and rebuild the multi-module project, the build will fail. The dependent modules will fail to build, even though their code and POMs have remained the same. In a multi-module project, you only install or deploy artifacts if all the child modules build successfully, so rebuilding snapshots is usually very desirable - it's telling you something about the inter-module dependencies.
If:
you have an excessive number of modules, and/or
those modules can't reasonably share the same version number, and/or
you don't need any guarantees about code references between modules,
then your modularisation is incorrect. Don't use multi-module projects as a build system (you have Jenkins for that), use it instead to express relationships between modules of your code.
In your comment, you say:
RELEASE artifacts behave the same way when being rebuilt by Jenkins.
The point of point of release artifacts is that you do not rebuild them - they are definitive! If you use something like Artifactory, you will find that you cannot deploy a release artifact more than once - your Jenkins job should fail if you attempt it.
This is a fundamental tenet in Maven. One of the aims of Maven is that it if two developers on separate workstations were to attempt the same release, they would build artifacts which were functionally identical. If you are build an artifact which expresses a dependency (maybe for compilation purposes, or because it's being assembled into .war etc.) on another, then:
if the dependency is a snapshot, Maven might seek a newer version from the repository.
if the dependency is a release, the version in your local repository is assumed to be definitive.
If you could rebuild a release artifact, you would create the possibility that two developers have dissimilar versions in their repository, and you'd have dissimilar builds depending on which workstation you used. Don't do it.
Another critical detail is that a release artifact cannot depend on snapshot artifacts, again, you would lose various guarantees.
Releases are definitive, and it sounds like you want your assembly to depend on definitive artifacts. Jenkins makes tagging and releasing multi-module projects very straightforward.
In summary:
Check your modularisation: one enormous multi-module project is not useful.
If you don't want to continually rebuild snapshots, you need to do releases.
Never release snapshots to your customer.
Follow the dependency graph of your assembly project and release any snapshots.
Release the assembly project, bumping your minor version.
Ensure your customer refers to the complete version number of your assembly in communications.
I have a Maven project that needs to be versioned. I have chosen to use the versions-maven-plugin as my versioning plugin but am unsure if that's the best option.
I have read the documentation that such plugin actually modifies the POM and I don't really like that approach. I have worked on projects where they had separate build.properties file that got modified manually.
What I want to achieve is to have my CI generating the artifact for me ready to be deployed and update the version number automatically.
So, any suggestions? How have you done before?
Thank you
I'd get the version number from the one source that matters: that's the source code management system (Subverson, Mercurial, or Git), not Maven.
I'd say that Maven might be out of synch unless your Maven plug-in is getting it from SCM.
Use the Release Plugin. You want to perform automatic deployment and batch release. The Versions Plugin is designed for something else.
We have found MAVEN-RELEASE-PLUGIN super useful and can not imagine releasing and managing version with it.
Warning: I have just picked up Maven, so things mentioned might be wrong or not best practice.
I have a medium size open source project that I am migrating to Maven from the basic
NetBeans project management. This is not a developer team sharing the same room, this is 1-5 people over the internet sharing a SVN repo. Reading over the how-tos on dependencies, it seems that the only way to get dependencies is to get them from an online repo or install them locally.
This is not what I was looking for. I want to keep all dependencies in the SVN for many reasons including portability (anybody can pass by, check out the repo, build, and use; all that simply without manual adding to local repo's and whatnot), getting newer versions (discussed below), and manual versioning.
The other issue I have with the maven repository is that they are quite behind in versions. Logback for example is 0.9.18 in mvnbrowser but 0.9.24 officially. PircBot is 1.4.6 in mvnbrowser but 1.5.0 officially. Why such old versions?
Issue 3 is that I have dependencies that don't even exist in the repos, like Easier Java Persistence.
So
How can I force all dependencies to come from /lib for example
On a related note, can mvn build from library's SVN repo directly? Just curious
Is there an automatic way to get the newest version directly from a dependencies site/svn repo if they also use Maven? IE libraries like commons-lang or logback
Is there a better way of managing dependencies? (IE Ivy or some weird POM option I'm missing)
FYI, this is a Java project with 3 modules, project global dependencies and module specific dependencies.
Bonus points if it can work with the bundled version of Maven that comes with NetBeans.
Not a duplicate of
Maven: add a dependency to a jar by relative path - Not wanting to install to local repository
maven compile fails because i have a non-maven jar - Don't think a System dependency is the right answer
maven look for new versions of dependencies - Still uses(?) repository, just the latest (old) version
This is not what I was looking for. I want to keep all dependencies in the SVN for many reasons (...)
I will come back on this but the solution I described in Maven: add a dependency to a jar by relative path (using a file-based repository) allows to implement such a solution.
The other issue I have with the maven repository is that they are quite behind in versions. Logback for example is 0.9.18 in mvnbrowser but 0.9.24 officially. PircBot is 1.4.6 in mvnbrowser but 1.5.0 officially. Why such old versions?
It looks like mvnbrowser indices are totally out of date (making it useless as repository search engine) because the maven central repository does have logback-core-0.9.24.jar (the logback project is doing what has to be done to make this happen) but only has an old pircbot-1.4.2.jar. Why? Ask the pircbot team. Anyway, you're right, the central repository might not always have ultimate versions.
Issue 3 is that I have dependencies that don't even exist in the repos, like Easier Java Persistence.
Yeah, this happens too.
How can I force all dependencies to come from /lib for example
As previously hinted, you should re-read carefully the solution suggested in Maven: add a dependency to a jar by relative path. This solution is not about installing libraries to the local repository but is about using a file-based repository (that could thus be stored in SVN). You might have missed the point, this matches your use case. And also check Brett's answer for a variation.
On a related note, can mvn build from library's SVN repo directly? Just curious
Didn't get that one. Can you clarify?
Is there an automatic way to get the newest version directly from a dependencies site/svn repo if they also use Maven? IE libraries like commons-lang or logback
Maven supports version ranges and you could use a syntax allowing to use "any version greater than X". But I do NOT recommend using version ranges at all, for the sake of build reproducibility. You don't want the build to suddenly fail because of some automatic update that happened on your back. Only upgrade if you need bug fixes or new features, but do it explicitly (if it ain't broke, don't fix it).
You might also find mentions of the LATEST and RELEASE version markers. I don't recommend them neither for the same reasons as above and even less since they're removed from Maven 3.x.
Is there a better way of managing dependencies? (IE Ivy or some weird POM option I'm missing)
Can't say for Ivy. But in the Maven land, if you can't host up a "corporate" repository for your project (Nexus, Archiva, Artifactory), then the file-based repository is IMO the best approach.
Setup your own Maven repository.
http://archiva.apache.org/