I have a standard Java application that handles both REST and UI calls. What is the best way for me to create and manage an application version (major.minor.release.build)? I'm using Subversion, Maven, Bamboo (continuous build) and Spring in the stack. I would like the version to be tied together with SVN, Bamboo and Maven. And, would like to be able to log version on start-up -- likely using some Spring bean.
There must be a framework/pattern out there to help with this. I'd rather not roll my own.
Thank you!
Why not use Semantic Versioning? It is what most people expect nowadays, it is pretty well defined and it is out there. Good enough for me.
Maven has a release plugin. This is a bear to setup first but once it is working it works well. It does all the nitty gritty of making sure everything is cleanly checked in, tagged properly, and does the magic with the version numbers. It is not a ask to look forward to, but at least now it is properly done. It pays to setup some maven repository. We use Nexus and can recommend that, but I heard good things of artifactory too.
During testing we do not rely on the maven version too much but on the build number, which we put in a discrete place on the web pages and similar artifacts so we can quickly determine which exact build we're talking about. We use hudson which provides the build number in an environment variable, but Bamboo must provide that too. The filter copy functionality makes that pretty straightforward.
Hudson tags the VCS (we use git, but that does not matter) with the build number and the maven release plugin tags the releases.
You can include the SCM revision number in your artifact using the maven build-number plugin (http://mojo.codehaus.org/buildnumber-maven-plugin/), e.g. in a filtered resource, such as a properties file.
If you are using Artifactory as your binary repo then it can also tag your binary artifacts with a build number and have full traceability from your artifact to the CI server build that created it. Currently this is supported with Hudson, TeamCity and Bamboo.
Related
What exactly do Gradle and Maven "do" that eclipse or sts doesn't? From what I've read it builds, runs, etc which can all be done in eclipse easily.
If I have an existing project I've created, built, and currently runs in eclipse via tomcat, what would I use gradle for?
There's not a lot of benefit to using Maven or Gradle on a small project that you never share with anyone; an IDE can do the build just as well. But as the number of developers increases and the complexity of the build increases, it becomes very useful to separate the build instructions from the IDE. Let's drill into these a little bit.
With the increase in developers, you don't want everyone to have to come by and use your IDE to get a build done. That would be really annoying! So they're on their own machines, but then they tend to have different setups (how dare they have different user account names!) and probably have their IDE installations set up a little differently too. So we need some kind of build instructions that people can use to get things going, and it helps if everyone can use the same build instructions repeatably so that you don't get too many instances of “but it works on my machine!”. It's also very helpful if those instructions are simple enough to use that a new programmer to the team can get up to speed rapidly.
But the other thing that often happens as projects grow in scope is that their builds become more complicated. They very often gain additional dependencies (they didn't start out needing a high-performance date parser and MIME-type identifier, buit they've become required since and you don't want to have to write all those from scratch) and that means you've got to make sure that when the build is done, the right version of those dependencies is used. But that's not the only way that complexity increases. It's also very often the case that you find you're using more automatically-generated code. You might find yourself working with XML schemas or WSDL a lot, or maybe your using Hibernate, or Spring, or … well, there's lots of ways in which things can get complicated, OK? Getting the various steps to do all the build right, reliably, in these sorts of scenarios can be a bit tricky, but encoding them as instructions to something like Maven makes life a lot easier once you've taken the jump in the first place. (It gets even more important when you start trying to deal with projects which need many different sub-programs that work in concert; some of those are plain hard to build even with Maven or Gradle or any other tool.)
And then there's the possibility of offloading work to a build server, running tests automatically, managing dependencies cleanly, etc. IDEs don't handle these all that well by themselves; where they do a reasonable job of it, it's usually because they're using a tool like Maven under the covers to do the heavy lifting.
tl;dr
You don't have to make your code work with a build system, but it helps if you do and in many ways.
Maven and Gradle can do many things that Eclipse doesn't. However, the most important thing they do, is to decouple the bulding and testing processes from the IDE you choose (i.e. Eclipse). When you work on a large environment, with many programmers, usually you can not control the IDE they use. So, it's better to use a tool like Maven and Gradle to standardize these tasks. The same happens with the code examples of a book: instead of the authors having to provide the instructions for configuring any IDE to execute them, they provide the Maven or Gradle files, so the reader can build and test them on any IDE he's using.
Another very important feature that Maven and Gradle give you, is the fact that dependencies are managed without the need of having the executable code under source version control. Instead of having the executable code you depend on as part of the project, you declare the dependencies on a text file (which is under source version control), and then get them from a repository.
However, you may only see the real advantages of using tools like Maven or Gradle (and even Jenkins or Hudson), when you think in large scale projects, developed along many months by teams composed of many developers).
Gradle and Maven are build tools. Maven was first and is a bit older, Gradle is newer and has redefined a way of how projects are built and maintained. In my opinion it's also much easier to use, more readable and easier to maintain. I prefer Gradle ;)
You use eclipse or STS (any other IDE) for development. And while You finish this process You need to provide a configured artifact (war, ear, whatever...) to production and deploy it there. These artifacts have well defined format and the application won't be run from eclipse or STS at the production environment. It's tiresome and error prone to prepare such artifacts by hand.
Gradle or Maven can take responsbility of building and preparing these artifacts (in fact such tools can do much more) off Your shoulders, they make this process automated.
I am working on a Spring Web Project that we are using Subversion, Eclipse and JBoss to build it.
The way we have it now is that we check everything into Subversion, the source code, jars that the project needs (Spring, Log4J, etc), so at any time someone can go to Subversion and rebuild the project.
Most of the projects at this firm only have one programmer on each project and the firm only has 3 programmers anyway. We are thinking about using Maven, but I am starting to think that it is overkill... Can I please get some feed back about your views on this?
If you have a system that works for you, then don't change it.
On the other hand Maven helps a lot in keeping up with dependencies changes.
In most cases with Maven you have to update a dependency version in one place and a new version of that dependency is there ready to be tested and used.
If you decided to switch to Maven, you must have some kind of local repository management system like Nexus. Think of it like a source control for your binary dependencies. Another must would be Maven support in your choice of IDE. I think at this point all major Java IDEs have it.
At one point I was exactly in the same predicament and decided that it was worth it to switch (and it was).
However, mastering the whole setup may be daunting. If nobody at your company did it before, I think it may take somebody on your team 4-6 weeks to do a conversion project (learing Maven concepts, setting up POMs, integrating with IDE, etc.). Hence weigh carefully future benefits against current deadlines.
Currently we have a Java Restlet API with dependencies controlled via Maven. When we update the API we run maven assembly:assembly which does the unit tests etc and produces a single jar file. We then upload this to the production server and run it using nohup.
Is there a better or more automated way of doing this? Is this where something like Hudson would come in?
Thanks
My experience goes with webapp-deployment. But same should hold true here. Use Maven, Cargo, Nexus (or Artifactory), Hudson and probably, Jira in conjunction of product release.
Automated release process are more reliable because there is no human factor involved that may forget a step.
We also use Liquibase for database versioning. And, if you are dealing with database changes in your application deployment. You'll realize Liquibase boosts so much confidence while running alter scripts.
I would suggest to go through the following resources
Automated Deployment with Maven - going the whole nine yards If you can, literally follow this pattern.
Maven 2 Effective Implementation -- this book really helped us a lot.
There are several Maven plugins to help deployment. The most general of them is Cargo, but there are also app server specific plugins for some concrete servers like JBoss.
Most companies I have worked for (actually, all) have had some sort of custom in-house built deployment system; even if build was done using a standard framework (like Maven in use at my current company).
Part of this is because there are many aspects that tie closely to company-specific infrastructure, capacity management and monitoring systems; and so even though there are open-source systems, there is usually something that needs to be tweaked.
It sounds like you are running your app on its own--it isn't part of any application server. If you aren't using an application server, there are probably some ways to get cargo and maven to deploy it for you, but you may be better off just using some shell scripts to deploy and run the application.
However, as your application grows, you may find a need for an application server like Jetty, JBoss, Glassfish, Tomcat, etc. When this happens, take a look at the cargo plugin for Maven because it will allow you to do something like:
mvn cargo:redeploy
That will package up your application, send it to the server and restart the app. If you want Hudson to do this for you automatically you can add it as a target to build.
Cargo can save you a lot of time when you have to frequently update an application server.
We are currently using JDeveloper to build our production EARs. One problem with this is that if the developer doesn't add new files to a VCS, then that developer is the only one capable of making EARS, and therefore he can use unversioned files as well.
What would be a good system that seperates this, so that EAR files can be correctly produced without depending on the local developers workspace (this would also ensure that they add their files to a VCS before allowing to make a deployment/check-in).
One problem with this is that if the developer doesn't add new files to a VCS, then that developer is the only one capable of making EARS,
If the developer doesn't use the VCS, this is not your only problem:
You cannot reproduce things in another environment, you're tied to the developer machine (but you're aware of that). What if he is sick?
Not versioning files means you don't have any history of modifications and that you don't know what you put into production ("Hmm, what is in this version? Wait, let's open the EAR to check that.").
And last but not least, in case of hardware failure (e.g. a hard drive crash), you can say good bye to everything that is not in the VCS.
So, the first thing to fix is to ALWAYS version files, even if there is only one developer as working alone doesn't save you from the mentioned problems. These points should be reminded (the developer needs to be aware of them to understand their importance).
To make sure this happens, you should actually not rely on the developer machine to build the EAR but rather rely on an "external" process that would be the reference. You want to avoid this syndrome:
alt text http://img716.imageshack.us/img716/9901/worksonmymachinestarbur.png
To put such a process in place, you need to automate the build (i.e. your whole build can be run in one command) and to break the dependency with your IDE. In other words, do not use the IDE to build your EAR but rather use a tool like Maven or Ant (which are IDE agnostic). That would be the second thing to fix.
Once you'll have automated your build process, you could go one step further and run it continuously: this is called Continuous Integration (CI) and allows to get frequent, ideally immediate, feedback about changes (to avoid big bang integration problems). That would be the third thing to fix.
Given your actual toolset (which is far from ideal, there is not much community support for the tools you are using), my recommendation would be to use Ant (or Maven if you have some knowledge of it) for the build and Hudson for the continuous integration (because it's extremely easy to install and to use, and it has a Dimensions plugin).
Here's my recommendation:
Get a better IDE - IntelliJ, Eclipse, or NetBeans. Nobody uses JDeveloper
Have developers check into a central version control system like Subversion or Git.
Set up a continuous integration facility using Ant and either Cruise Control or Hudson to automate your builds.
What we do is use cruisecontrol. It does two things, it lets us do continuous integration builds, so that we have nightly builds as well as lightweight builds that get built every time a change is checked it.
We also use it to more specifically address your issue. When we want to ship, we use cruisecontrol to kick off a build, that is tagged with the proper production build version. It will grab the code from our version control system (we use SVN) and will build on that, so it is not dependent on developers local environments.
One thing you might also want to consider is creating a production branch to build out of. So, production ears for a particular release are always built from that branch. This way you have even have a bit more control over what goes into the build.
Instead of doing builds from developer workspaces, setup Maven, and have something like Hudson run your Maven build. The artificats of this build (your ear) gets deployed.
I have a large, growing OSGi application with a number of bundles. I am curious to know the best way to manage this type of application. Currently, I am using Eclipse and Maven, but although this is great for building bundles (via maven-bundle-plugin), as of now it has not been easy to manage the entire application.
What I would like to do is either have ONE run configuration or ONE pom.xml that can be launched and the entire application/project be built and launched. Also, I would like to have something that would be good for debugging.
I have heard of PAX Construct and have it installed in Eclipse, but so far it has been of little help (maybe I'm not using it correctly).
I am sure there are people out there with large OSGi applications that are being managed correctly. Any advice that could be shared would help tremendously.
Thank you,
Stephen
A run configuration is possible via Pax Runner. It lets you choose OSGi platform implementation, specify profiles (pre-packaged sets of bundles for some role, e.g. web, log, ds, etc.) and has good provisioning support, for instance it can load bundles from Maven repository. As a result, you can have a run configuration like
--platform=felix
--log=INFO
--profiles=scalamodules,ds,config,log
mvn:com.my/bundle/1.0.1-SNAPSHOT#update
# other bundles
In case your application is very large or you have different applications, there a way to create own profiles as well.
Well...
It all deopends on what do You mean by "managing" the application.
For dev time launching, building and debugging - Eclipse IDE should fit the bill just perfectly.
Maven... I can't speak for it, as I've never used it myself.
We have a pretty large eclipse based application (several, actually) and on the dev side of things we are not using anything special besides the Eclipse and it's integrated SCM.
In the cc build server, we also use headless eclipse to do the building and packaging.
Now the setup of the workspace has gone a bit out of hand of late with all the dependencies and intermediate build steps, so we are investigating Buckminster for managing the materialization of target platform and workspace resources.
If that works out, we'll probably move to building with Bucky as well - it sure looks promising.
(I do not have any experience with PAX, but at a glance, it looks promising as well...)
i'm quite new to OSGi but,
wouldn't it be possible to use OBR-service in such a way that
you would have one OBR repository file which needs the bundles
and let the OBR-service figure out the dependencies and populate your OSGIhost for you?
This area I think has very poor support at the moment. OSGI doesn't really define anything about deployment or packaging so its up to other frameworks (e.g. Eclipse) to come up with their own way of doing it.
If you are building an RCP (Eclipse base) application, then the eclipse systems do all this stuff, right down to creating exes etc. However builds are mainly done on the Eclipse workspace, headless builds are trickier. The Tycho project is trying to make this more sensible by joining the Maven and Eclipse build cycles, however it is still focussed on RCP applications rather than generic OSGI.
If you not doing RCP, which is my situation as well, then you probably have to roll your own solution, as I haven't found any general solution. Here's an outline of what we do:
We define one POM project that lists all the bundles that are contained in your application. All this project does is list the references - lets call it the 'bundle-list' project.
Then, we use pax provision to run the project in development mode. This is achieved by making the 'bundle-list' pom the parent of the provisioning pom of the pax project (usually in the 'provision' folder). Then, when you start pax, it uses the list of bundles from that project to start OSGI. The bundle references in the 'bundle-list' project have to be marked as 'provided' scope for this to work.
Then, to create a distribution, we have another project. This project also has the 'bundle-list' project as its parent. This project uses various plugins to create a distribution, including downloading the bundle jars. The distribution includes scripts that start up OSGI, but these are hand written, there's no pax systems here.
This works well for us to keep the list of bundles in one place, but there's still a lot of hand written scripts, and there are issues sharing configuration between the two systems - e.g. config files, bundle start levels etc.