How to setup a sophisticated java development infrastructure? - java

I'm looking for a complete java development infrastructure with an integration of:
an IDE like eclipse,
a build system like Maven,
a version control system like subversion,
a continuous integration server like Hudson,
a repository manager like Nexus,
an automated release plugin like Maven Release Plugin.
Further I'd like to have:
a predefined multi component project structure
...and optionally:
an issue manager like Jira,
an integration with an open source hoster like sourceforge.
Evaluating all these systems could take a long time, making the setup of a running infrastructure a job of a month or longer.
At work I'm ready to setup each system individually, but for my private development at home I'd like to have something like Devware - A development environment virtual appliance (unfortunately I didn't find a download link) where everything is already installed and functional.
So, could you please give me some advice which combinations create a working infrastructure? Or even better where to find a preconfigured development infrastructure?
PS: I'm not committed to any of the named products, so feel free to suggest alternatives if they match better.

If you're looking to avoid some integration pains, check out SecureCI, which is a free download of various open-source products in an integrated stack.
Blurb from the site:
SecureCI™ provides organizations with an integrated suite of open source tools that assist in the automation of the software build, test, and deploy process. By integrating secure code analysis and web security testing technologies, SecureCI extends continuous integration to incorporate automated security analysis and security testing of applications as well.
The current version bundles these tools:
Project Management
Trac (0.11.5)
Sonar (1.10.1)
Security Scanning
ratproxy (1.58)
Development
Subversion (1.4.6)
Hudson (1.322)
Apache Ant (1.7.1)
Apache Maven (2.2.1)
Sonatype Nexus (1.3.6)
Testing
JUnit (4.6)
Cobertura (1.9.2)
Selenium Remote Control (1.0.1)
Code Analysis
Checkstyle (5.0)
FindBugs (1.3.8)
PMD (4.2.5)
JDepend (2.9)
As far as IDE integrations are concerned, later versions of Eclipse now come integrated with Mylyn, this provides a task focused API for connecting to various tools.
The Trac integration can be installed with the Connector Install Wizard.
Other Eclipse integrations to use:
Subversion integration: Subclipse and the Subclipse Mylyn integration)
Hudson plugin
Maven integration: M2Eclipse (also has some Mylyn connectors)
Code Coverage: ECLEmma (I find the Cobertura plugin doesn't work too well myself)
Selenium plugin
Checkstyle plugin
Findbugs plugin
PMD plugin
JDepend plugin or CAP

Or just use IntelliJ from JetBrains.

You might want to look into http://www.poweredbypulse.com. It provides easy to install pre-configured Eclipse instances. You can also build/configure your own.
It has been a while since I actually used this so it may have changed since then.

Related

Why exactly did you choose MAVEN for your selenium project

I was wondering what could be the good answer to "Why was the need to use Maven and Not Jenkins or other framework"
This is with respect to an automation project created in selenium and testNG with Maven framework
There are a couple of fundamental and functional difference between the usability of Maven and Jenkins
Maven :
Apache Maven popularly known as Maven is a Software Project Management tool. Based on the concept of a Project Object Model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information dynamically which is configured through a file pom.xml.
From Selenium Framework perspective, Maven enables you to to get rid of the manual configuration of Selenium JARS and WebDrivers. You can easily switch over from one version of the underlying software to another through a simple update in pom.xml.
Jenkins :
Jenkins is a popular tool for Continous Integration (CI) which can be used as a simple CI server or turned into the continuous delivery hub for any project. Jenkins being a self-contained Java-based Program is always ready to run out-of-the-box with packages for Windows, Mac OSX and other Unix-like operating systems. Jenkins can be easily configured through its web interface to integrate on-the-fly check-ins and built-in help.
From Selenium Test Automation perspective, through Jenkins you would be able to schedule the Test Execution whenever you require and as many times you may require.

Anything I can build upon for continuous delivery in java?

If I were using python I would probably like to use pip as a nice installer for continuous delivery with its nice repository integration and scripting capabilities.
Do I have anything similar in java which would be useful for me in continuous deployment?
Can someone recommend me how they do full continuous deployment in java?
I'm going to have multiple servers with complex configurations and huge multiple clusters with databases, NOSQL's (and using maven for the some of the projects while others are just downloaded pacakges) etc etc... anyone has recommendation for that?
Again I think pip is a very nice installer and could help me, anyone has experience maybe with ubuntu juju?
However if I use ubuntu juju that would mean I would have to use ubuntu based servers and not centos.
There's a kind of bright line between Java app build and Java app deployment. Build CI in Java is pretty straightforward with a variety of tools available - build scripting (Ant, Maven, Gradle, etc), continuous builds (Jenkins, Go, Anthill, etc), and repositories (Nexus, Artifactory, etc). Dependency management for libraries is a hairball for Java, so definitely use Maven or Ivy for it.
Deployment is a much wilder and less mature world. The environments are potentially far more complex, and often include messy non-Java things like relational databases. You can hand-roll scripts, or use ControlTier or Capistrano or something like that (which will still involve some hand-rolling).
I'm not completely clear what pip does, but here is my toolchain for CI/CD
You need a build tool:
Maven (does a lot of stuff, including downloading dependencies and driving you crazy)
ANT (will poke you until you die with xml brackets)
Gradle and others (pretty much everybody including ANT uses/can use Ivy for downloading dependencies from repositories)
You need an CI server
jenkins
various commercial options (Teamcity, Bamboo ...)
For the deploying part you need something to deploy your apps.
This really depends on the build tool you use (which should be able to do the deployment). Maven has some plugins for this afaik, but I think you will have to google for your app server and the build tool to find a solution for your specific need.
Probably what you are looking for is building a deployment pipeline. Check a video example here: http://www.youtube.com/watch?v=6CEQOuHM86Y
There are multiple ways to achieve it. Ill tell you my preferred one.
Components you will need:
VCS server (SVN, Git)
CI Server (Jenkins, Hudson, TeamCity)
Build Tool (Maven, Ant, Gradle)
Artifact Repository (Artifactory, Nexus)
Deployment Tool (Rundeck, Puppet, Deployinator, Capistrano)
Target Environment/s (Application Server like Tomcat, JBoss)
Workflow:
1) CI Server polls VCS Server for changes
2) When a change is found (i.e., a commit), starts job execution, getting an artifact (CI Server will compile and run tests). CI Server internally will use a Build tool like Maven.
3) CI Server uploads artifact to an Artifact Repository
4) Deployment Tool reads Artifact Repository and provides a list of artifacts to be deployed, plus the list of Target Environments where Developer/Ops can select a combination of both and deploy the artifact in the selected server.
Take into consideration some criteria at the moment of picking the tools. If you have a really big infrastructure (like 200+ Target Environments), robust solutions like Puppet make sense. If you have a modest one (let say, 10 Target Environments) then Rundeck may be enough. Also, take into consideration that some of the listed tools are not free (Puppet Enterprise is not free beyond ten nodes, for example).

Confusion in choosing between JavaGit, JGit and EGit

I am making a Java application that uses Git. I found that there is something called
JavaGit, EGit and JGit.
I know that JavaGit and EGit/JGit are different. What I don't understand is the difference between EGit and JGit. Both are hosted on Eclipse projects but one seems to be Eclipse related and the other not.
I don't use Eclipse and I don't plan to, so I really don't care much about "Eclipse integration".
Is JGit somehow connected to Eclipse? (It is hosted on www.eclipse.org, and the documentation of JGit also has a lot of "eclipse" keywords inside)
Out of the three, what will I need to connect my Java program (not written in Eclipse) to work with Git repositories?
I would prefer a standalone Git library (i.e., without depending on a local Git installation)
EDIT: One more question:
Does JGit support Git bundle command?
EDIT: I found the answer -- it does (haven't tried it yet). It is in transport package in a class called BundleWriter
You can use JGit as standalone library, but, as mentioned in the JGit Homepage:
JGit can be found within:
* EGit, Eclipse team provider for Git
In that sense, JGit is connected to Eclipse in that EGit uses JGit features to develop the Eclipse Git plugin.
Considering all Eclipse projects are now versioned in Git repos (instead of previously CVS repos), I would consider JGit for your Java project, as it seems that:
JavaGit isn't updated for quite some time (2008)
Eclipse is committed to offer a good support for Git through EGit, which should ensure by extension the development of JGit (used by EGit).
Thorbjørn Ravn Andersen comments:
Even though you say that Egit uses JGit, it is not quite clear in the above that EGit is an Eclipse plugin for git giving git support in the same way that CVS is already supported in Eclipse
The migration of Eclipse projects from CVS to Git is in progress, and has been reported here (by Chris Aniszczyk).
I’ve monitored conversations concerning the migration of project code from CVS to Git in the dev-lists of several projects.
Most developers, it seems, understand the value proposition of migrating, despite their concerns that with migration comes a whole lot of new learnin’ as they figure out how to say productive in the new environment.
Inevitably, discussion about the team provider for Git being developed by the EGit project becomes an prominent part of the conversation. “Is EGit ready for primetime?” is often the question that bounced around
The official bug illustrated Eclipse commitment to provide "adequate tooling support" for all Eclipse projects now on Git repo is the bug 293192.
In order to deprecate old VCS in favour of git (bug 270854), adequate tooling
should be available. This bug is created to track that.
[...] I opened/linked relevant EGit bugs in order to have a better visibility/tracking
of what's missing.
So Egit is the only project referenced in that bug to provide said adequate tooling support for the new (D)VCS chosen for all Eclipse projects.
(bug 270854 is about Deprecate old VCS tools, which includes essentially CVS.)
Just in (March 17th, 2001): The State of Git at Eclipse – Early 2011 (Chris Aniszczyk):
I’m happy that things have come quite a long way since last year based on Eclipse Marketplace stats and the community contributions the projects have received.
So what’s next? In terms of tooling, we’re getting closer to our 1.0 release (which is planned for the Indigo release).
The git tooling at Eclipse is much better these days, if you need help or have questions, please start with the most excellent EGit User Guide.
If you’re an eclipse.org project, please consider starting the process to move your project repository to Git. There are a lot of eclipse.org projects already on Git.
It would be nice to see the all of the eclipse.org projects have plans to move to Git by the Indigo release.
As one of the authors of JGit and EGit, sorry for the confusion.
JGit is an EDL (new-style BSD) licensed, lightweight, pure Java library implementing the Git version control system. It can be used in a standalone fashion and is embedded in applications such as Gerrit, Eclipse, Netbeans and IntelliJ. EGit embeds JGit to use it to work with Git repositories and expose Git constructs from within Eclipse. At the moment, JGit doesn't support the git-bundle command but you're welcome to contribute support, it shouldn't be that hard, please see our contributor guide.
Just to chime in, as I just stumbled on this this question just now.
There is a difference between the Eclipse Foundation and the Eclipse IDE. The Foundation is the umbrella organization, while the IDE is one of the projects developed through the foundation. So, the important thing to know is that just as Apache started with the web server and branched out to become a Foundation hosting a number of projects, so has the Eclipse Foundation now expanded beyond its "roots" in the IDE.
So in this context the JGit code provides a Java API for git commands, while EGit uses JGit to provide git support for the Eclipse IDE.
Personally, I also find this a little confusing at times, and sometimes it almost seems that having so much going on has in some ways degraded the focus of these organizations; however I can also see the utility in having one umbrella organization instead of 20 similar but different organizations.
JGit is git implemented from scratch as a Java library whereas JavaGit is (was ?) a Java API wrapping the native git implementation (i.e. it's calling the native git command line).
EGit is the Git Eclipse Team Provider integrating JGit into the Eclipse IDE and is providing the UI to work with Git from inside Eclipse.
In the meantime a large part of the Eclipse community and also a large number of corporate developers is using EGit/JGit (that's the reason why the companies backing these projects put money into that). See the project pages ([1], [2]) and ohloh if you want to get some insight who is contributing.
JGit is a library to manipulate git repository. You do not use JGit alone, unless you write a program to manipulate git repository in the way you like.
If you use Eclipse as the IDE, you install EGit as a version control plugin (defaulted in Indigo). EGit calls JGit so you also have JGit installed if you install EGit.
Note, for Netbeans, you install NBGit as a version control plugin. NBGit also calls JGit.
JavaGit: not actively maintained?
JGit is the Java implementation of Git. It is a library, that also can be used in your own applications. It also provides some sort of CLI operations.
EGit on the other side is the Eclipse team provider plugin for Git, which uses JGit as Git implementation. Simplified you could say EGit is the UI part, and JGit the background part.
JGit doesn't depend on EGit, but EGit does depend on JGit.
From: https://www.eclipse.org/forums/index.php/t/273443/

Eclipse Spring IDE 2.3.2 plug-in vs. SpringSource Tool Suite 2.3.2

I'd like to play around with Spring for an upcoming Java EE project. Is there any difference between the Eclipse Spring IDE 2.3.2 plug-in (in the new Eclipse Marketplace) and the SpringSource Tool Suite (available at SpringSource).
More precisely: In my understanding, the Tool Suite is basically Eclipse 3.5 with a set of plug-ins. So the question is, are the plug-ins provided by the Eclipse Spring IDE 2.3.2 the same or is there any difference in functionality?
Thanks for your help, I hope you understand that the Eclipse platform can be overwhelming for beginners.
SpringSource ToolSuite is Eclipse + Spring IDE + others. The others are things like support for Roo, dmServer, tcServer, Grails, etc.
If all you need is the Spring IDE plugin (which gives support for Spring Framework development), then vanilla Eclipse + Spring IDE is going to be enough.
I quite like the pre-bundled STS versions, though, it's one less thing to worry about.
Feature Comparison: http://download.springsource.com/release/STS/doc/STS-feature_comparison.pdf
With the Eclipse Marketplace Client build in newer Eclipse versions (AFAIK starting with Helios) it is also possible to install Spring Tool Suite into an existing Eclipse. I have just done that.
Even though you can use just Eclipse + Spring IDE plugin, I would suggest going for SpringSource Tool Suite simply because it is stable set of very useful plugins. Since we switched to STS we never looked back :)
If you are already using Eclipse, you are likely using other plugins for your existing projects. Also you are like to use tool supports other than Spring for your future projects.
Spring IDE plugin with your existing Eclipse makes more sense if Spring is not your only requirement.

Automatic Hudson CI setup and plugin updates through apt?

We've used Hudson for quite a while to implement a CI server with all the bells and whistles. The setup is quite straight forward, when installing from the provided RPMs and Debs, but through googling I haven't been able to figure out whether the plugins are installable using apt/rpm or some other package manager?
The reason is that I would like to create a (meta)package for Ubuntu which would install and also update both Hudson and all the plugins through the normal upgrade mechanism. At the same time I could create a template setup for other projects, say JavaEE project needs git, cobertura and Chuck Norris plugins, while my Python project needs plugins XXX and YYY.
Anybody got such a setup? As a workaround I figured setting up a number of Maven POMs, which would do the init, and later upgrades, but I feel this would require more scripting on the side, which I'm not very eager to do.
Any other suggestions for this would also be appreciated.
I haven't been able to figure out whether the plugins are installable using apt/rpm or some other package manager?
They are installed "manually".
The reason is that I would like to create a (meta)package for Ubuntu which would install and also update both Hudson and all the plugins through the normal upgrade mechanism. (...)
I'm not aware of an existing solution for this (very likely because packaging plugins is a big mistake in general: it's a lot of work for the linux distros and plugins go much faster than the release cycles). So I guess you'll have to package them yourself.
And be careful with upgrades of Hudson, they often break plugins (in other words, I recommend to test the upgrade of Hudson in a test environment, like a virtual machine).
Anybody got such a setup? (...)
I bundle plugins in an archive and unzip them in <HUDSON_HOME>/plugins/.

Categories

Resources