We have started to implement Continuos Delivery for our Java Builds using Maven and Teamcity tooling for CI and Build automation.
We have few common jars that are built as standalone jar artefacts and are consumed by web modules.
Frequency of the change to these common modules is high; we have started to adopt the approach discussed in various forums What is the Maven way for automatic project versions when doing continuous delivery? and in this blog
http://blog.xebia.com/2012/09/30/continuous-releasing-of-maven-artifacts/ to use Major.Minor.BugFix-${revision} for all the common jars.
Value for revision is set in Parent POM as SNAPSHOT for local development and in case of Teamcity builds it is set to ${BuildNumberCounter}-${SVNRepoRevisionNumber} e.g. 1.0.0-10-233
For a Web Module that needs to consume the jar and always wants to pick the latest version Dependency range is defined as [1.0.0,2.0.0). This seems to be working fine; however to be honest we have not yet used this in anger, so will see if we hit challenges.
The problem that we have straightaway is that for local desktop development the dependency range in the Webmodule always resolves to the latest numbered release rather than snapshot build that was created by the developer for local testing of the common jar with the Web Module. We believe it is valid for the developer to be able to test the change of common jar with web modules locally. Only way it can be achieved is by committing the change and Teamcity producing new numbered release which is not ideal as it would potentially break the build of all Web Modules that use that common jar.
Wonder if anyone has faced similar problem and would have a solution.
Related
I just started developing a project in Spring MVC and i want to know how important Maven is.
The following are the key features of Maven :
Simple project setup that follows best practices - get a new project or module started in seconds
Consistent usage across all projects - means no ramp-up time for new developers coming onto a project
Superior dependency management including automatic updating, dependency closures (also known as transitive dependencies)
Able to easily work with multiple projects at the same time
A large and growing repository of libraries and metadata to use out of the box, and arrangements in place with the largest Open Source projects for real-time availability of their latest releases
6.Extensible, with the ability to easily write plugins in Java or scripting languages
Instant access to new features with little or no extra configuration
Ant tasks for dependency management and deployment outside of Maven
Model based builds: Maven is able to build any number of projects into predefined output types such as a JAR, WAR, or distribution based on metadata about the project, without the need to do any scripting in most cases.
Coherent site of project information: Using the same metadata as for the build process, Maven is able to generate a web site or PDF including any documentation you care to add, and adds to that standard reports about the state of development of the project. Examples of this information can be seen at the bottom of the left-hand navigation of this site under the "Project Information" and "Project Reports" submenus.
Release management and distribution publication: Without much additional configuration, Maven will integrate with your source control system (such as Subversion or Git) and manage the release of a project based on a certain tag. It can also publish this to a distribution location for use by other projects. Maven is able to publish individual outputs such as a JAR, an archive including other dependencies and documentation, or as a source distribution.
Dependency management: Maven encourages the use of a central repository of JARs and other dependencies. Maven comes with a mechanism that your project's clients can use to download any JARs required for building your project from a central JAR repository much like Perl's CPAN. This allows users of Maven to reuse JARs across projects and encourages communication between projects to ensure that backward compatibility issues are dealt with.
Getting started with maven only takes about 10 minutes. Reasons why you should learn maven:
It helps you manage your dependencies very easily so you don't need to add jars to your project classpath manually
You can run unit tests
Has over 20 useful plugins which you can use. Plugins make up lifecycles like test, package which make your work more efficient
You can use it to build your project
The most important thing about it at the beginning is that you don't need to worry about setting up your project by adding dependencies, maven does it for you automatically.
Read this spring guide for building with maven
Any other guides in this section for spring boot has the same mechanism
I am trying to understand all the implications of switching our java projects from a Snaphot/Release policy to build promotion.
One obvious step is that each build ends up creating an artifact that might be going all the way to the production environment, so there's not Snapshot anymore. But then, how should I manage the link from a project to other artifacts, that may or may not be allowed to go to prod?
I've had a hard time finding valuable information on this particular subject. Of course, build promotion is talked about a lot, but dependency management in the light of a migration to build promotion has less visibility.
I see two choices:
One can only depend on artifacts that have been previously promoted to the production environment
When one depends on another artifacts, the built artifact can only go to the last environment of its dependencies. That is, if I depend from an artifact that was allowed to go to test and not prod, then my build won't be allowed to go to prod
Are there industry standards regarding this topic? Or best practices?
Thanks a lot for your help :)
edits:
We deploy to Artifactory three kind of artifacts:
Libraries
EARs
The modules inside the EARs. Some of those are "public" layers that are needed by any EAR that wants to interact with the currently built EAR
We deploy EARs to JEE servers. Our libraries and public layers are deployed to Artifactory and packaged in the EARs, so they are not directly deployed on the JEE containers.
One project builds several modules, and everything is packaged in an EAR, along with its dependencies. One project can depend on a module of another project and that's where it gets complicated...
We distinguish between "deployable artifacts" and "libraries".
Deployable artifacts (like ears, wars, standalone jars) go through a pipeline, so they are promoted and tested in different steps. They cannot be dependencies for any other artifact.
Libraries, on the other hand, are not promoted. When they are built (as a release version), are immediately available as possible dependency for all other artifacts (the release build includes unit tests and some integration tests). They are tested and promoted indirectly when they are used in deployable artifacts.
Is it true that one of the real benefits of Maven is that its projects structure is clearly defined and therefore it can be opened in any IDE that supports Maven?
We develop in a team consisting of 5 people and we use three different IDEs (IntelliJ, Eclipse, Netbeans). I know that I cannot open an Ant project of Eclipse in Netbeans but with Maven this is possible.
Is this really reason enough to use Maven?
Maven and Ant are two different things.
Ant is a build tool while Maven accumulates the functionality of:
resolving dependencies (you only specify the dependencies, and Maven cares about downloading them in the correct version from the default repositories on the i-net, or the ones you specified explicitly)
a build tool (incorporating Ant),
creating distribution artifacts (containing not only the jar/war/etc. file of the project but also dependencies, resources, documentation),
runtime environment (e.g. starting Jetty and deploying your artifact)
unit testing
integration testing
deployment (including signing, incrementing the version, deploying the artifacts etc.)
and more (look out for plugins on i-net)
Maven uses lifecycles to achieve this. You can trigger certain functionality at certain points in the lifecycle. For example unit tests should be run right after compiling while integration tests (e.g. selenium tests) require more setup, maybe initializing a web server and deploying the WAR file.
We have been using Ivy for a few months and have our own hosted "Ivy Repo" on a web server here in the office. All of our projects are configured to go to this repo to resolve dependencies.
We have several "commons" type JARs that are used by many of our projects. Because of this, and because we only have 1 repo, we're finding a lot of ugly overhead coming from the following scenario:
A developer is given a task to add a feature to Project 1 (which depends on a Common jar)
During the course of developing Project 1, the developer realizes he/she needs to make changes to the Common jar
Common jar changes are made
Common jar has to go through code review and normal code promotion
Build master publishes new Common jar
Project 1 can resume development now that Common jar has been updated
This is becoming ridiculous and painful for our team.
To me, the obvious solution is to provided ant targets in each project that allow the developer to publish/resolve locally (to and from their file system). That way they can break the Common jar 9 ways to Sunday, but without losing 2 - 4 days while waiting for Common to get published. This way, the developer makes local changes to both Project 1 and Common, and the code goes through our promotion process all at once.
I know this is possible with Ivy, but I'm so new to it I wouldn't even know where to begin.
Currently we use a global ivy.settings file for all projects. In the settings file, we use a chain resolve that has 1 url resolver inside of it, which connects to our "ivy repo".
I believe the following is the only change that will be necessary, but I'm not 100% sure:
In ivy.settings we will need to add a local file system resolver before the url resolver gets called; this way we check the local file system for dependencies before moving on to the ivy repo (web server)
Configure each project's ivy.xml with an option somehow that allows local cache publishing
Tweak the Ant builds to have a publish-locally target that exercises the option mentioned above
I believe these changes will allow us to: (1) always look locally for dependencies before looking to the web server, (2) publish locally as a build option (target).
If this is not true, or if I am missing any steps, please advise! Otherwise, I can probably figure out how to add a file system resolver from the Ivy docs, but have no idea how to get the publish-locally target to work. Any ideas? Thanks in advance!
I too would prefer Marks approach.
As to publish-locally you can tell the publish task which resolver(resolver="local") to use. This way it can publish to the local filesystem or to any defined resolver.
<ivy:publish
resolver="local"
overwrite="true"
revision="${project.version}">
<artifacts pattern="dist/[artifact]-[revision].[type]" />
</ivy:publish>
And if you use a chain resolver you should set returnFirst="true" so that resolving will stop when something was found locally.
Ivy supports dynamic revisions:
Stable code would reference the latest approved version of the commons jar:
<dependency org="my-org" name="commons" rev="latest.release"/>
Unstable (in development) code would reference the latest unapproved version of the code
<dependency org="my-org" name="commons" rev="latest.integration"/>
So you need to change the build process for your commons module to have two publishing targets. One for unstable snapshots of your code the other for formal releases.
(See the status attribute on the ivy publish task)
Note:
In Maven you have two types of repository, release and snapshot. Ivy support for this concept is more subtle and more powerful IMHO.
I'm confused about the use of maven in development and production environments - I'm sure it's something simple that I'm missing. Grateful for any help..
I set up maven inside eclipse on my local machine and wrote some software. I really like how it's made things like including dependent jars very easy.
So that's my development environment. But now I want to release the project to production on a remote server. I've searched the documentation, but I can't figure out how it's supposed to work or what the maven best practice is.. Are you supposed to:
a) Also be running maven on your production environment, and upload all your files to your production environment and rebuild your project there? (Something in me baulks at the idea of rebuilding 'released' code on the production server, so I'm fairly sure this isn't right..)
b) use mvn:package to create your jar file and then copy that up to production? (But then what of all those nice dependencies? Isn't there a danger that your tested code is now going to be running against different versions of the dependent jars in the production environment, possibly breaking your code? Or missing a jar..?)
c) Something else that I'm not figuring out..
Thanks in advance for any help!
You're supposed to have your code under version control (and you never "upload" files to another machine, you "download" them from the Version Control System if required).
You're supposed to package your code in a format (a WAR, an EAR, another kind of bundle) that can be deployed on the production environment for execution. Such bundles typically include the dependencies. To build more complex bundles, the Maven Assembly Plugin can help.
Maven generated artifacts (JARs, WARs, whatever) should be shared via a remote repository (and thus deployed - I mean mvn deploy here - to this remote repository). A remote repository can be a simple file system served via a web server or a more advanced solution like Nexus.
Development is usually done using SNAPSHOT dependencies (e.g. 1.0-SNAPSHOT). At release time, you're supposed to change the version into a "fixed" version (e.g. 1.0) and some other bits from your pom.xml, run the build to check that everything is ok, commit the modified pom.xml, create a tag in the VCS, promote the versions to a new SNAPSHOT (e.g. 1.1-SNAPSHOT) in the pom.xml, commit the new pom.xml in the VCS. The entire process require some work but this can be automated using the Maven Release Plugin.
On the production environment, get the artifacts to be deployed from the remote repository and deploy them (some projects automate the deployment to the production server using Maven but that's another story).
Of course, there are variations around this (deployment to production is most of time company specific) but the general idea is there.
You need to look into the Maven Assembly Plugin and the Maven Release Plugin.
When building artifact you usually state what scope the dependency has. In default scope it should be packaged in your archive. If you do not want it, use scope "provided" - in such case you have to prepare runtime environment providing the dependency. It's generaaly a bad idea to rebuild a package only for deployment.
As for deploying, you can use maven's antrun plugin to copy files locally or via scp .