I'm trying to figure out the best way to automate the execution of and result recording of load tests. We currently use Maven; ideally the solution would be executed as part of the Maven project life-cycle, so that people do not need so spend time on effort setting up the tests.
It should save the output of the tests (e.g. as some files that could be committed into version control), and should have the sense to not compare tests run on a dusty x486 with a 8 core Sparc.
JMeter is another Apache project which is very well adapted for automation (you can control most things from the command line); there are also several plugins that you can use to integrate it into Maven. Personally, I think this is by far the best tool for this sort of requirement.
You still need to create the actual tests to run and decide which environment to use - but this will always be the case no matter which tool you choose.
Have a look here, here and here.
Related
I'm trying to understand what is provided by ci.gitlab.com. I would like to create a build using Gradle for my Java project and have it all run on ci.gitlab.com servers.
The process for adding a runner seems to involve leveraging a localized installation of GitLab CI. There are shared runners available, but they seem to be geared toward Ruby, Node, etc.
Is it possible to use ci.gitlab.com for a fully hosted CI solution?
Its not important if there are only Ruby, Node or other Runners. If they are shared runners, you can specify the image in your .gitlab-ci.yml which the runners use to run you tests. See here (i wrote a example .gitlab-ci.yml):
The runners will pull the right image and will run your tests in the specified image.
More information here.
What exactly do Gradle and Maven "do" that eclipse or sts doesn't? From what I've read it builds, runs, etc which can all be done in eclipse easily.
If I have an existing project I've created, built, and currently runs in eclipse via tomcat, what would I use gradle for?
There's not a lot of benefit to using Maven or Gradle on a small project that you never share with anyone; an IDE can do the build just as well. But as the number of developers increases and the complexity of the build increases, it becomes very useful to separate the build instructions from the IDE. Let's drill into these a little bit.
With the increase in developers, you don't want everyone to have to come by and use your IDE to get a build done. That would be really annoying! So they're on their own machines, but then they tend to have different setups (how dare they have different user account names!) and probably have their IDE installations set up a little differently too. So we need some kind of build instructions that people can use to get things going, and it helps if everyone can use the same build instructions repeatably so that you don't get too many instances of “but it works on my machine!”. It's also very helpful if those instructions are simple enough to use that a new programmer to the team can get up to speed rapidly.
But the other thing that often happens as projects grow in scope is that their builds become more complicated. They very often gain additional dependencies (they didn't start out needing a high-performance date parser and MIME-type identifier, buit they've become required since and you don't want to have to write all those from scratch) and that means you've got to make sure that when the build is done, the right version of those dependencies is used. But that's not the only way that complexity increases. It's also very often the case that you find you're using more automatically-generated code. You might find yourself working with XML schemas or WSDL a lot, or maybe your using Hibernate, or Spring, or … well, there's lots of ways in which things can get complicated, OK? Getting the various steps to do all the build right, reliably, in these sorts of scenarios can be a bit tricky, but encoding them as instructions to something like Maven makes life a lot easier once you've taken the jump in the first place. (It gets even more important when you start trying to deal with projects which need many different sub-programs that work in concert; some of those are plain hard to build even with Maven or Gradle or any other tool.)
And then there's the possibility of offloading work to a build server, running tests automatically, managing dependencies cleanly, etc. IDEs don't handle these all that well by themselves; where they do a reasonable job of it, it's usually because they're using a tool like Maven under the covers to do the heavy lifting.
tl;dr
You don't have to make your code work with a build system, but it helps if you do and in many ways.
Maven and Gradle can do many things that Eclipse doesn't. However, the most important thing they do, is to decouple the bulding and testing processes from the IDE you choose (i.e. Eclipse). When you work on a large environment, with many programmers, usually you can not control the IDE they use. So, it's better to use a tool like Maven and Gradle to standardize these tasks. The same happens with the code examples of a book: instead of the authors having to provide the instructions for configuring any IDE to execute them, they provide the Maven or Gradle files, so the reader can build and test them on any IDE he's using.
Another very important feature that Maven and Gradle give you, is the fact that dependencies are managed without the need of having the executable code under source version control. Instead of having the executable code you depend on as part of the project, you declare the dependencies on a text file (which is under source version control), and then get them from a repository.
However, you may only see the real advantages of using tools like Maven or Gradle (and even Jenkins or Hudson), when you think in large scale projects, developed along many months by teams composed of many developers).
Gradle and Maven are build tools. Maven was first and is a bit older, Gradle is newer and has redefined a way of how projects are built and maintained. In my opinion it's also much easier to use, more readable and easier to maintain. I prefer Gradle ;)
You use eclipse or STS (any other IDE) for development. And while You finish this process You need to provide a configured artifact (war, ear, whatever...) to production and deploy it there. These artifacts have well defined format and the application won't be run from eclipse or STS at the production environment. It's tiresome and error prone to prepare such artifacts by hand.
Gradle or Maven can take responsbility of building and preparing these artifacts (in fact such tools can do much more) off Your shoulders, they make this process automated.
How are people writing builds for their Play applications?
I understand that you can run play auto-test on the command-line, however, I would like to do a bit more than just execute the tests.
So for example, if I wanted to add checkstyle and clover to my build, I could easily write an Ant script that would check those two things and fail if there are any checkstyle errors or the clover coverage is not 90%.
I was just wondering whether there already was a "Play" way of doing what I have just described.
not, not as far as I know, although there are addons to use ant/maven in Play, you could try to integrate them. But users may skip then and use the normal play.
An idea would be to modify the Play python script to run these validations before any action, but that would require all developers to use that modified script.
There isn't a "Play" way that I know of, either. I've used Gradle in the past with some Play apps to do that kind of stuff.
At the moment I use SVN to manage java source code. Is there a solution out there whereby I can check in code and have the new code automatically compiled into a JAR file? Somehow the check in would need to trigger the compile process.
You need a continuous integration tool. Hudson would be a good choice (I've been using it the past year and it works really well)
You could do this with a CI server; the server can sit on another machine (the build server) and get the latest commit, build it, and compile it to a JAR. (Of course, this isn't really ideal, on large projects [due to lots of changes being submitted, and so on], but on personal things it should work to a reasonable degree).
There are a number of continuous integration programs which do this very thing. Two I've used are Bamboo (commercial) and CruiseControl (open source). Bamboo is about 1000x easier to setup then CruiseControl, and really pays for itself after you've created a hundred build plans or so. With CruiseControl you will have killed yourself long before you get to your hundred build plan.
Is it possible to set up continuous build of projects written in .NET and Java on single build server with single set of applications?
I've seen CruiseControl has support for both world but as far as I know these are two separate applications. Should I go with a separate machine to build Java projects (a machine for .NET projects is already set up)?
Hudson has support for both Ant (out of the box IIRC) and NAnt (through plugin) so that's an option for you.
CruiseControl supports several different build options include Ant, Maven, NAnt, Phing (php), Rake, XCode, and then the generic "exec" which you can use to invoke any command-line script/tool you want. Not a problem mixing Java and .NET on the same server.
Of course this is not unique to CruiseControl. There are lots of CI tools that support multiple build technologies as you can see on this matrix of features.
Disclaimer: I work on CruiseControl. OTOH since I don't make money on it I don't much care which tool people choose. I care more about advancing the state of CI practices which is why I organize the Continuous Integration and Testing Conference (CITCON).
Ant and NAnt can reasonably easily execute arbitrary processes, including each other, so the actual build part shouldn't be too hard.
I'd expect the tricky bit to be getting the reports (including unit test results) into an appropriate format. How's your XSLT? ;)
EDIT: Now that I think about it, my first agile project had a continuous build server (just CruiseControl, I believe) which must have been doing some of this... I suspect at the time we directly invoked Visual Studio to build the code and NUnit to test it. If I were at the same company I'd check, but that was two jobs ago :(
You could checkout Atlassian Bamboo. Unfortunately, its not free, unless you are applying for an opensource/community license for use with opensource software.
You can use two different products on the same machine. Or you can run a single builds system across multiple machines. It is really up to the load you place on your CI.