Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I'm a junior developer and recently started working for a very small office where they do a lot of in-house development. I have never worked in a project that involved more than one developer or were as big and complex as these ones.
The problem is that they don't use all the tools available (version control, automated building, continuous integration, etc) to their full extent: mainly a project is one big project in eclipse/netbeans using cvs for version control and everything checked in (including library jars), they started using branching for the first time when I started doing branches for small tasks and merging them back. As the projects get bigger and more complex, problems start to arise with dependencies, project structure tied to an IDE, building can be a PITA sometimes, etc. It's hectic at best.
What I want is to set up a development environment where most of these problems will go away and I will save time and effort. I would like to set up projects in a manner independent of IDE used using version control (I'm leaning towards SVN right now), avoid dependency messes and automate building as much as possible.
I know that there are multiple approaches and tools for this and do not want to see a holy war started, I would really appreciate practical recommendations based on experience and what you have found to be useful when facing similar problems. All projects are Java projects and range from web applications to "generic" ones, and I use Eclipse most of the time, but also use Netbeans if needed. Thanks in advance.
You seem to be almost exactly in the point where the place I worked at was when I started there 1,5 years ago, only difference being that you've started toying with branches which is actually something we still don't do at my work but more about that later on in this answer.
Anyway, you're listing a very good set of tools which can help a small company and those work really nicely as subtopics so without further ado,
Version control systems
Most commonly small companies currently use CVS or SVN and there's nothing bad in that, in fact I'd be really worried if no version control was really used at all. However you have to use version control right, just having one won't make your life easier. We currently use CVS and are looking into Mercurial, but we've found that the following works as a good set of conventions when working with CVS (and I'd suspect SVN too):
Have separate users for all commiters. It's beyond valuable to know who commited what.
Don't allow empty commit messages. In fact if possible, configure the repository to reject any commits without comments and/or default comment. Initial commit for FooBarizer is better than Empty log message
Use tags to mark milestones, prototypes, alphas, betas, release candidates and final versions. Don't use tags for experimental work or as footnotes/Post-It notes.
Don't use branches since they really don't work if you're continuing on developing the application. This is mainly because in CVS and SVN branching just doesn't work as expected and it becomes an exercise in futility to maintain any more than two living branches ( head and any secondary branch ) over time.
Always remember that for the software company the source code is your source of income and contains all your business value, so treat it that way. Also if you have extra 70 minutes, I really recommend that you watch through this talk Linus Thorvalds gave at Google about git and (d)VCS in general, it's really insightful.
Automated builds and Continuous Integration environments
These are about the same actually. Daily builds is a PR joke and has little no resemblance to the state of the actual software beyond some very rudimentary "Does it compile?" issues. You can compile a lot of awful code noise that doesn't do anything, keeping the software quality up has nothing to do with getting the code to compile.
On the other hand unit tests is a great way to maintain software quality and I can with a bit of personal pride say that rigorous unit testing helps even the worst of the programmers to improve a lot and catch stupid errors. In fact there has so far only been a total of three bugs that code I have written has reached production environments and I'd say that in 18 months that's a pretty damn good achievement. In our new production code we usually have a instruction code coverage of +80%, mostly +90% and in one special case reaching all the way to 98%. This part is very lively field and you're better of Googling for the following: TDD, BDD, unit tests, integration tests, acceptance tests, xUnit, mock objects.
That's a bit of a lengthy preface, I know. The actual meat for all the above is this: If you want to have automated builds, have them occur every time someone commits and make sure there's a constantly increasing and improving amount of unit tests for production code. Have the continuous integration system of your choice (we use Hudson CI) run all the unit tests related to project and only accept builds if all the tests pass. Do not make any compromises! If unit tests show that the software is broken, fix the software.
Additionally, Continuous Integration systems aren't just for compiling code but instead they should be used for tracking the state of the software project's metrics. For Hudson CI I can recommend all these plugins:
Checkstyle - Checks if the actual source code is written in a way you define. Big part of writing maintainable code is to use common conventions.
Cobertura - Code coverage metrics, very useful to see how the coverage develops over time. Also keeping in line with the "source is God" mentality, allows you to discard builds if coverage falls below a certain level.
Task Scanner - Simple but sweet: Scans for specific tags such as BUG, TODO, NOTE etc. in your code and creates a list from them for everyone to read. Simple way to track short notes or known bugs which needs fixing or whatever you can come up with.
Project structure and Dependency Management
This is a controversial one. Basically everyone agrees that having an unified structure is great but since there's several camps with different requirements, habits and views to issue they tend to disagree. For example Maven people really believe that there's only one way - the Maven way - to do things and that's it while Ivy supporters believe that the project structure shouldn't be hammered down your throat by external parties, only the dependencies need to be managed properly and in an unified manner. Just that it's not left unclear, our company simply loves Ivy.
So since we don't use project structure imposed by external parties, I'm going to tell you a bit about how we got into what we got into our current project structure.
In the beginning we used individual projects for actual software and related tests (usually named Product and Product_TEST). This is very close to what you have, one huge directory for everything with the dependencies as JARs directly included in the directory. What we did was that we checked out both projects from CVS and then linked the actual project to the test software project in Eclipse as runtime dependency. A bit clunky but it worked.
We soon came to realize that these extra steps are completely useless since by using Ant - by the way, you can invoke Ant tasks directly in Hudson - we could tell the JAR/WAR building step to ignore everything by either file name (say, everything that ends with Test or TestCase) or by source folder. Pretty soon we converted our software project to use a simple structure two root folders, src and test. We haven't looked back ever since. The only debate we currently have is if we should allow for a third folder called spikes to exist in our standard project structure and that's not a very heated debate at all.
This has worked tremendously well and doesn't require any additional support or plugins from any of IDEs out there which is a great plus - number two reason we didn't choose Maven was seeing how M2Eclipse basically took over Eclipse. And since you must be wondering, number one reason for rejecting Maven was the clunkiness of Maven itself, endless amount of lengthy XML declarations for configuration and the related learning curve was considered a too big cost as to what we would get from using it.
Rather interestingly later on commiting to Ivy instead of Maven has allowed us to a smooth shift to do some Grails development which uses folder and class names as conventions for just about everything when structuring the web application.
Also a final note about Maven, while it claims to promote convention over configuration, if you don't want to do things exactly the way the Maven's structure says you should do things, you're in a world of pain for the aforementioned reasons. Certainly that's an expected side effect of having conventions but no convention shouldn't be final, there always has to be at least some room for changes, bending the rules or choosing the appropriate from a certain set.
In short, my opinion is that Maven is a bazooka, you work in a house and you ultimate goal is to have it bug free. Each of these are good on it's own and work even if you pick any two of them, but the three together just doesn't work.
Final words
As long as you have less than 10 code-centric people, you have all the flexibility needed to do the important decisions. When you go beyond that, you have to live with whatever choices you've made, no matter how good or bad they are. Don't just believe things you hear on the Internet, sit down and test everything rigorously - heck, our senior tech guy even wrote his bachelor's thesis about Java web frameworks just to figure out which one we should use - and really figure out what you really need. Don't commit to anything just because you may need some of the functionality it provides in distant future, pick those things that has the lowest possible negative impact to the whole company. Being the 10th person hired to the company I work at I can undersign everything in this paragraph with my own blood, we currently have 16+ people working and changing certain conventions would actually be a bit scary at this point.
Our development stack (team of 10+ developers)
Eclipse IDE with M2Eclipse and Subclipse/Subversive
Subversion for source control, some developers also use TortoiseSVN where Subclipse fails
Maven 2 for project configuration (dependencies, build plugins) and release mgmt (automatic tagging of releases)
Hudson for Continuous Integration (creates also snapshot releases with source attachments and reports)
Archiva for artifact repository (multiple repositories, e.g. releases and snapshots are separated)
Sonar for code quality tracking (e.g. hotspots, coverage, coding guidelines adherence)
JIRA for bug tracking
Confluence for developer wiki and communication of tech docs with other departments
Docbook for manuals (integrated into build)
JMeter for stress testing and long-term performance monitoring
Selenium/WebDriver for automated browser integration tests
Jetty, Tomcat, Weblogic and Websphere as test environments for web apps. Products are deployed every night and automated tests are run on distributed Hudsons.
Mailinglist with all developers for announcements, general info mails
Daily stand up meetings where everbody tells about what he's currently doing
This setup is considered standard for our company as many departments are using those tools and there is a lot of experience and community support for those.
You are absolutely right about trying to automate as much as possible. If your collegues start to see the benefits when aspects of the development phases are automated, they will be encouraged to improve on their own. Of course, every new technology gimmick ("tool") is a new burden and has to be managed and maintained. This is where the effort is moved. You save time e.g. when maven automatically performs your releases, but you will waste time on managing maven itself. My experience is that every time I introduced a new tool (one of the aboves), it takes time to be adopted and cared about, but in the end it will bring advantages to the whole team when real value is experienced - esp. in times of stress when the tools take over much of the work you would have to do manually.
A fine, admirable instinct. Kudos to you.
Part of your problem might not be solved using tools. I'd say that source code management needs some work, because it doesn't sound like branching, tagging, and merging is done properly. You'll need some training and communication to solve that.
I haven't used CVS myself, so I can't say how well it supports those practices. I will point out that Subversion and Git would be better choices. At worst, you should be reading the Subversion "red bean" book to get some generic advice on how to manage source code.
Personally, I'm not a Maven fan. I believe it's too heavyweight, especially when compared to Ant and Ivy. I'd say that using those with Cruise Control could be the solution to a lot of your problems.
You didn't mention unit testing. Start building TestNG and Fit tests into your build cycle.
Look into IntelliJ - I think its a better IDE than either Eclipse or NetBeans, but that's just me.
Best of luck.
Maven is great, however, it can have a fair bit of a learning curve, and it requires that the project fits a very specific file structure. If you have a big legacy project, it may be difficult to mavenize it. In that case, Ant+Ivy would do the same without the stringent requirements that maven has.
For build automation, Hudson is beyond awesome. I've used a couple different systems, but that is unquestionably the easiest to get set up and administer.
I recommend to use Maven for building your projects. Using Maven brigns value to the project, because:
Maven promotes convention over configuration what equals a good project structure
thanks Maven plugins eases generating projects for IDE's (Eclipse, Netbeans, Idea)
handles all dependecies and complete build lifecycle
faciliates projects modularization (via mulitimodule projects)
helps with releases/versions burden
improve code quality - easy integration with continous integration servers and lot of code quality plugins
Maven can be a bit daunting given its initial learning curve, but it would nicely address many of your concerns. I also recommend you take a look at Git for version control.
For project and repository management, I use trac with subversion.
Here's what i'm using right now, but i will probably switch a few parts (see the end of this post).
Eclipse as IDE with a few plugins : JADClipse (to decompile .class on the fly, pretty useful), DBViewer for a quick access to database through Eclipse, WTP (Web Tools Platform) integrated into Eclipse for running Tomcat 6 as a developement web server (pretty fast), Mylyn (linked with JIRA bug-tracker).
I'm too wondering about "IDE independant projects", right now we are all sticked on Eclipse - Eclipse project files (.project, .classpath, .settings) are even commited in the CVS repository (in order to have a project fully ready once checked out) - but with Netbeans, supported by Sun and running faster and faster with each release (and each new JRE version), the question isn't closed.
CVS for storing projects, with nearly no branches (only for patches).
I'm working on environment production with Oracle SGBDR but I'm using HSQLDB on my developement computer to make test and build and development process way faster (with the help of the open-source DDLUtils tool to ease database creation and data injections). Otherwise i use SQLWorkbench for quick BD tasks (including schemas comparison) or the Oracle (free) SQLDeveloper for some Oracle specific tasks (like investating sessions locks and so on).
Tests are only JUnit tests (either simple unit test cases or more complex test cases (nearly "integrations" ones), nearly always runing on HSQLDB to run faster.
My build system is Ant (launched from Eclipse) for various small tasks (uploading a war on a remote server for example) and (mainly) Maven 2 for :
the build process
the publishing of the released artefacts
the publishing of the project's web site (including reports)
launching tests campaigns (launched every night)
The continuous integration front-end is Luntbuild, and the front-end for the Maven repository is Archiva.
All this works. But I'm pretty disappointed by a few elements of this ecosystem.
Mainly Maven, it's just too time-consuming and i have a lot of griefs versus this tool. Conflicts dependencies resolution is a joke. Lot of XML lines in every POM.xml, redundant in every project (even with the help of a few POM roots). Plugins are way too inconsistent, buggy, and it's really difficult to find clear documentation explaining what has to be configured, and so on.
So i'm wondering about switching from Maven to ANT+Ivy. For what i've seen so far, it's seems pretty cool (there are various Conflict manager for the conflicts dependencies resolutions and you can even write your own conflict manager), there is no need to have an additionnal tool installed and configured (as ANT is running natively under Eclipse, whereas Maven needs a separate plugin - i've tried the 3 Mavens plugins by the way, and have found all the three of them buggy).
However Maven 3 is on its way, i'll give it a try but i don't expect it to be fundamentaly different from Maven 2.
Hudson would seem a better choice than Luntbuild, too, but this part won't be changed for the now.
And Subversion will probably replace CVS in a near future (even if i nearly don't have any trouble with CVS).
Lots of good advice here. I have just a few additions:
I think that, unlike the rest, an IDE is a personal tool, and each developer should have some freedom to select the one that works best for him. (For example, many love Eclipse, while I ditched it for NetBeans because Maven integration was, uh, problematic in Eclipse.)
I thought I was going to hate Maven, but now I get along with it fairly well. The main problem I have these days is finding out where the conventions are documented.
I would advise introducing tools one at a time. If you try to mechanize all aspects of software development at a by-hand shop in one stroke, there will likely be massive resistance. Make your business case and get agreement on one good common tool, or just get permission to set it up for your use but in a commonly-accessible way and let people see what it does for you. After a few of these, people will develop a habit of wondering how aspect X could be automated, so additional tools should be easier to introduce.
The single most best thing you can do without disrupting other people and their way of working is setting up hudson to watch the CVS repository for each of your project. Just doing that will give a central place to see cvs commit messages.
Next step is getting these projects to compile under Hudson. For Eclipse this typically means either switching to ant or - as we did - use ant4eclipse to model the existing eclipse build process. Not easy but very worthwhile. Remember to send out mails when the build breaks - this is extremely important. Ant4eclipse requires team project sets - introducing them in your organization Will make your colleagues happy the next time they need to set up a fresh workspace.
When you have a situation where your stuff builds properly whenever somebody commits changes then consider making that automatically built code the code to actually go to the customer. As it was built on the build server and not on a developers machine, you know that you can reproduce the build. That is invaluable in a "hey fix this ancient version" situation.
Related
I've to audit the code-architecture quality and maintainability (in the end to be sure we have what we paid for) a Java EE web project based on JSF/CDI/EJB3.0/JPA (just to name some of the technologies involved).
This may not be the right place to ask but how do you deal with this kind of task?
Basically, I would proceed from coarse-grained to fine-grained, i.e. from the whole architecture to the java code.
Is it better to deal with each layer completely?
Should I spend more time on the low-level layers?
Do you assess the whole thing (build, deployment, test)?
At the lower physical/implementation level, what I like to do is adopt maven as a build tool, and then configure the extensive maven reporting, to produce a website full of various code metrics.
For starters there is the maven checkstyle plugin which can report on code conformance to a specified standard, consistency in coding standards has many obvious benefits, most projects would simply adopt one of the pre-configured standards e.g. sun or apache.
The findbugs plugin lists potential programming errors
There are a choice of code coverage reports, I've used cobertura. These show line for line in an application which parts are covered by unit tests. Maven supports unit tests in the build life-cycle, running them as part of the build. This has saved me a few times.
The PMD plugin identifies duplicated code, and highlights areas that may need refactoring.
Once this is setup and becomes part of the normal build cycle, it basically takes care of itself, and you won't have to worry about doing large bi-annual audits/catch-up.
Many of the reports have threshold limits that can be configured to fail the build if breached, i.e. more than n% checkstyle errors, cause a build failure.
Maven also promotes a modular approach to building applications, this results in smaller more understandable and re-usable modules, as well as separation of concerns, i.e. separate modules for presentation and persistence layers. The main benefit that maven provides is managing the inter-dependencies between the modules.
This doesn't help you very much at the higher-level architecture layers though, so a complementary approach will be required to cover that dimension.
See some sample reports at this link
http://maven.apache.org/plugins/maven-dependency-plugin/project-reports.html
To help in the code level audit and probably in project health too one software that can help is SONAR... it's very simple to setup just some maven commands, comes with a lot of proven code standards like code quality, reusability, bad practices measurements and so on...
it Runs on your project SVN or CVS and generate a website with graphics represent past and current status of the metrics it's creating, so you can navigate the project data and keep track of the improvements or faults.
It also uses all those maven and maven plugins listed in the other answer like cobertura, find bugs etc...
http://www.sonarsource.org/
Just download and point to your Repo.
In addition to the lower-level code metrics and static analysis already mentioned, I'd add a tool like Structure101 to help analyze higher-level structure and dependencies. It can also help in refactoring the same.
Identifying clusters of dependencies can help determine if the app was written with separation of concerns and modularity in mind, and can help identify potential pain points when considering extension or modification.
Be sure to break it down into areas of concern and address them individually. Areas I can think of to consider are:
Conformance to specified requirements (hopefully these are specific)
Performance/scalability
Code quality (including conventions you want followed)
Test coverage
Plugin/library licenses
It looks like others have addressed items 3 and 4. Since you're asking the question now (presumably after you've received the product) 1 and 2 will probably have to be manual process unless you have automated functionality tests already written (or want to automate tests so you can check future versions of what you bought). 5 is an item that is sometimes overlooked but can be VERY important. You probably don't want GPL code hooked in if you're going to be reselling this software. You need to review the licenses of every library included and decide which ones are compatible with your goals.
To Understand your architecture, you can try JavaDepend it gives the possibility to query code with CQL, like SQL for database, with more than 82 metrics and many interactive views to go deep inside your design, architecture and implementation.
I'm working on a growing java project and I'm probably going to cooperate with somebody else to improve some features.
I'd like to use some tools to improve the quality of my work keeping in mind that:
I don't have too much time to spend on this project
it's a small project but it's really important for me
I don't want to buy software/hardware for it
I'm already using SVN
what do you think about maven and junit? is it worth spending time for them?
Do you know any other good tool?
Maven and JUnit are good for enforcing good habits (unit testing, uniform structure) and together with good SCM habits, I would say those are amongst the most important things for collaborative development.
Since you are not using JUnit my guess is you don't have any unit tests yet. This would seem to me the most important step for you to take, if other people will start working on your code. Without unit tests, someone can easily break functionality without knowing it.
Create a suite of unit tests that cover at least 80% of the code. You can use Cobertura to measure code coverage. This might seem like a lot of work (it is) but will save you far more time in the future.
Maven is the de-facto standard for building and deployment at the moment, but it has its drawbacks too. If you have a well documented build procedure in place (either using Ant or custom scripts) I would suggest it is less important to introduce Maven than to add unit tests.
JUnit is good for helping verify your code on any project.
Maven has a learning curve that can be hard to get over. If you have one module and a relatively simple set of build steps you may find it simpler to use Ant.
On the other hand with a Maven build you can simply add additional reports to your code to check various parameters on your code, and it is much harder to migrate to Maven than if you've conformed to its conventions from the start.
Examples of Maven plugins that can help check your code:
Findbugs (static analysis of possible bugs)
Checkstyle (enforce coding standards)
PMD (more static analysis)
PMD CPD (copy paste detection)
JDepend (cyclic dependency checking and package coupling)
Cobertura (code coverage)
If you're interested in the code quality plugins, also consider Sonar, it wraps these plugins up and gives you some funky reports.
If you're interested in best practice, also consider a Continuous Integration server, Hudson is free and integrates well with Maven.
We use Maven and JUnit on a fairly large project and find it very helpful.
For project planning, I highly recommend FogBugz. It's the best issue tracking system I have seen to date with good support for project management as well, and free for teams of up to 2 people.
If you use Eclipse and SVN I would recommend you to take a look at Mylyn. Its underlying concepts are very simple but it helps a lot when working on a team.
In my modest opinion, Maven is too annoying for the real benefit of it. Maybe ant is just enough for your deployment tasks.
I've used maven on one project and miss it. There is a fairly large upfront investment though in getting it setup and configured. The XML documentation was out of date when i was working with it (perhaps this has improved). Once you get past this initial setup though it's a wonderful time saver.
As for JUnit, it's great. Use it.
Both of these tools should be treated as an investment. At first it may seem like a lot of unrelated stuff, but the project will grow more predictably with less problems over the long haul.
I'm setting up a Java development shop, currently just for myself as the only developer, but with hopes of needing to hire others as the business grows. Obviously I'm hoping to set it up right so that as more people come in, they can be productive right away. Please help suggest things I want to do, and tools to do them.
Here's what I think I need:
Distributed source code/revision control (Subversion?)
Bug tracking (does Trac do this?)
documentation (both internal and customer facing)
team communication
frequent automated building
maybe something to make sure automatic tests pass as part of the check-in process?
I like Hudson for Continuous Integration builds, and I like JIRA for issue tracking. Eclipse has plugins for both.
Hudson can watch software repositories and rebuild those projects that use the changed resources.
If you need more documentation than javadoc can cover (which is quite a lot) then consider a Wiki. Easy to use, and with a bit of structure you can massage it into a PDF.
Source control is a bugger. Too many to choose from. For a small development team start with either subversion or CVS (which is old but has supreme IDE support) and when you outgrow that and know your needs, then migrate to a better one. Most have migration tools from svn or cvs. It is harder to move from e.g. git to Mercurial, and you defintively want one with more than one implementation. Remember to have good backups of the source control repository - it IS your business. Frequent rsyncs, often tapes.
EDIT: You also want decent hardware. For the Continuous Integration server, the fastest build machine you can afford. For yourself the largest monitor you can afford (not in size, in resolution) for your primary monitor and as many extra monitors as you can afford to have (including adapters to your computer). I have found that Mac's use the pixels better than Windows, so that might also be a point.
My primary monitor is pivoted 90 degrees. This allows me to see many lines at once instead of a few long lines. (For some reason tradition says that editing areas should be wide and short, which may work in word but not in code where lines should not be wider than 72 characters)
Note on Eclipse: Use the source repository to have a single workspace per project! Use the Java Editor Save Function to reformat your code everytime you save - this makes it more readable up front, and goes better with the source repository as changes are marked in the correct version.
Edit: The reason for the CI server needing to be better than your development machine is because it will run all your tests every time you check stuff into your source repository. After a while, that WILL take time.
Personally I have found tests working well for library routines. They specify what works and what doesn't. It is harder to write good tests for whole applications, but you may want to look into that from the beginning, as it allows you to ensure that everything works for every check in. Write a comment if you are not familiar with the concept.
Whatever you choose for the individual parts, you will be glad if they can work together. Hudson knows how to talk to JIRA for instance. JIRA knows how to look in CVS.
To see what a big amount of people thinks to make a good software development environment read this article on the 12 points of joels test
This list does not include the things more important for you. Getting clients getting them to pay and manage legal stuff and taxes.
Most important, the right staff:
get great people who find work and handle customers (aka sales)
get software engineers who are smart and get things done (http://www.joelonsoftware.com/items/2007/06/05.html)
get someone who knows about accounting and the local legal and tax regulations, so you don't get any surprises
Tools / Processes:
use a distributed version control system like git or mercurial
jira or the like for bug/issue tracking
continuous integration with hudson or cruisecontrol
wiki system to share the teams tangible knowledge
unit tests, clover, checkstyle, findbugs, ...
From a managerial point of view i would try
daily standup meetings (checkout scrum) to keep the team updated members to commit by saying what the did, do, and will do
timeboxed meetings, everything else sucks.
plan iterations/sprints
let team do task time estimations
pair programming (gets you better code)
code reviews (builds trust)
weekly in house "techtalks" to build a strong sense for the team
twitter like communication tool to keep all insync and informed with minimal distraction
develop team towards dynamic languages (groovy, scala, ...)
yourself, listen to what the guys at http://manager-tools.com/ have to say...
good luck!
We have been using:
version control:
subversion - Its not distributed but it is accessible over a few different protocols if firewalls are an issue. I'm not sure if distributed version control is necessary for us and reading Eric Sink's take is entertaining at least
issue tracking :
Fogbugz - You get some team discussion and communication for free with it because of the built-in wiki and discussion boards.
continuous integration:
CruiseControl - we had been talking about switching to Hudson, but Cruise is working really well right now - it runs our unit tests.
dev environment:
Netbeans and Eclipse - There is really no reason to pay for a Java IDE. An import point for getting going fast is that Netbeans and Eclipse both store all of their project data as text files which version control nicely. See this question. We had giant headaches when using an IDE which used binary project files.
profiler:
JDK VisualVM - Its free and it works. I used to really like YourKit, but VisualVM does so much now.
documention:
Combination of javadoc and fogbugz wiki pages plus the cruise dashboard for internal. For external we are using RoboHelp and we dislike it.
other tools:
Findbugs - huge help in catching things that are sometimes really stupid and sometimes amazing quirks that you'd have never realized. PMD is good for some of this as well.
We find chat tools to be really helpful for communication. We used to have access to Sametime and it had a giant conferencing feature that was really great. That was taken away for an unknown reason by the overlords though.
Here is what development stack our team of five developers is using over a year now:
Eclipse IDE (worked better for us than NetBeans)
Maven as a project comprehension and build tool -- simply a must tool!
Nexus: The Maven Repository Manager -- serves as a local Maven repo for proxying, and for managing internal and 3rd party libs; simple in use and is really necessary if you're going to use Maven
Subversion for source versioning -- was chosen mainly due to very good IDE support (Subclipse for Eclipse IDE)
Trac as a bug tracking and requirement management tool -- it nicely integrates with Subversion, has very useful plugins including blog and discussion plugins; also it can be integrated with Eclipse Mylyn.
Hudson as continuous integration, which nicely integrates with Subversion, Maven and Trac -- very valuable even for a small team.
Sonar code quality management platform -- a tool which integrates a large number of code quality matrix with intuitive web interface supporting code review and drill-down facility for analysis of the problems; integrates with Maven.
In our case this development stack is running under Ubuntu (workstation components: Eclipse IDE, Maven) and CentOS (server components: Maven, Nexus, Subversion, Trac, Hudson, Sonar).
As for the documentation, LaTeX (TexLive and Kile under Ubuntu) works just great supporting high quality PDF generation. The documentation source can be managed by Subversion the same way as the application source. Allows making of simple several page document and large multi-chapter books.
Hope this helps.
Distributed source code/revision control (Subversion?)
Subversion is good enough unless you want to use this as an opportunity to learn Git or Mercurial.
Bug tracking (does Trac do this?)
Trac works fine if you don't mind CamelCaseTurningIntoWikiWords. JIRA is fancier but (as noted) not free, and I find its data-ink ratio annoyingly low. Good once you learn which parts of the UI to ignore, though.
documentation (both internal and customer facing)
Internally, I would prefer communication over documentation unless you're really willing to commit to spending time keeping internal documentation updated. Javadoc your integration points (internal/external APIs), but try to keep your code and build scripts as self-documenting as you can.
That said, requirements docs are useful. I'd suggest using one tracking system for bugs and requirements. JIRA lets you create hierarchical issues, but I wouldn't bother with that till you need it.
Trac includes a Wiki, which can be handy although too much content will get stale.
Customer-facing docs are all over the map -- depends a lot on what you're doing. If you need big fat manuals (either printed or HTML or on-line help) FrameMaker + the DITA Open Toolkit is a nice combo, but a FrameMaker license is going to set you back $1000.
At a certain point this shades into corporate communications (marcom and PR), which are kind of a different animal.
team communication
Google Talk (or the instant messaging framework of your choice), email, and the occasional Skype teleconference are probably all you need.
An internal wiki and/or internal blogs might be nice but you probably don't need them right away, even once you've added a few more developers. Trac includes a wiki.
frequent automated building
Switching from CruiseControl to Hudson has made everyone here much happier -- two teams made the same decision independently and both are pleased with it. Hudson's flexible, easy to configure, and shiny.
I have yet to be convinced by Maven. I suspect its value depends on how much you want automated downloads of the latest versions of all your open-source libraries.
Edited (Feb. 2010) to add: Several months of Maven 2 experience now and I'm still not convinced. I think (like many frameworks, e.g. Rails) it may work well if you're starting from scratch and structuring your project according to its conventions, but if you already have your own structure in place it's a lot less advantageous.
maybe something to make sure automatic tests pass as part of the check-in process?
I'd advise against it. Most of the time the tests will be passing (or should be, anyway) As long as continuous integration notifies you quickly of failures, and you also have stable automated builds (a couple of days' worth of nightly builds, tagged iteration builds, etc.), you want more and faster checkins, not fewer and slower.
Subversion is not a distributed approach -- go for Mercurial, Bazaar or git instead!
Yes, Trac does do bug tracking (among other things -- check it out!).
Documentation is indeed a must but I'm not sure what you're asking -- tools for it? Why not just javadoc?
For communication you can have many tools, such as skype, email, IM of many kinds, and so forth -- you need to express your specific issues better to get specific advice, I think. Google Wave once it matures may be just great, but it's not production-ready yet.
For continuous build check out CruiseControl -- of course it also run tests &c.
You can write "triggers" for any of the build systems I've mentioned (and even good old svn;-) to run some test suite and reject the commit if it fails.
A few more items:
IDE
Billing Software - will you be charging by the
hour? If so you might want to track
what your time is going towards.
offsite backup of some kind.
Each one of your bullet points is probably worthy of a community wiki by themselves. Though in the end you might not care so much about best of breed in each area, but care more about how well they all integrate with your IDE or with each other.
Also, if you really want to get new teammates up and running quickly, consider putting as much of your dev environment into source control as you can, so you can just checkout your "dev-env" project onto a new computer and be up and running instantly!
One of your specs (in your question) says:
maybe something to make sure automatic
tests pass as part of the check-in
process?
I would suggest this is essential. Check out this matrix of continuous integration servers to see which one fits your requirements.
If you're starting off with just yourself as the only developer but scaling up at some point, depending on what it is you're developing, it might be worth to check out a Platform as a Service (PaaS) offering, like https://www.openshift.com or https://www.heroku.com/, both which offer developer accounts, but you can scale up to paid accounts as and when needed.
OpenShift have CI cartridges for Jenkins so you can spin up DVCS (git), CI using Jenkins, and an app server like JBoss AS or WildFly all in one go in a matter of minutes... depending on your needs and how much time you want to dedicate to setting this up yourself locally, I'd be more inclined to look at using a PaaS.
Rather than use Trac, consider Redmine. It allows you to have multiple projects stored in the one issue tracking system. You can then use the same setup for each project, rather than having to have N instances of Trac.
For starters, SVN will be enough for you. Definitely get a bugtracker. JIRA is good but isn't free. Enforce a rule "No commits without a bugtrecker ticket". This way you will be able to track the development. Do a cruise control and run a build + unit tests after every checkin into the main branch. Bigger changes should be made on a separate branch and then merged into the main branch.
Invest in a good IDE (I recommend intelliJ IDEA) and a good profiler (I recommend JProfiler). They're not free, but they are definitely worth their price.
I'm the sole developer for an academic consortium headquartered at a university in the northeast. All of my development work involves internal tools, mostly in Java, so nothing that is released to the public. Right now, I feel like my development workflow is very "hobbyist" and is nothing like you would see at an experienced software development firm. I would be inclined to say that it doesn't really matter since I'm the only developer anyway, but it can't hurt to make some changes, if for no other reason than to make my job a little easier and get a few more technologies on my resume. Right now my workflow is something like this:
I do most of my development work in Eclipse on my laptop. Everything is saved locally on my laptop, and I don't use a VCS, nor do I really backup my code (except for occasionally emailing it to myself so I can see it on a different computer - yeah, I told you my development environment needs work).
When I'm done with a project and want to deploy it or if I just want to test it, I use the built-in Jar tool in Eclipse to make an executable .jar of my project. If I use external .jar libraries, I use the Fat-Jar plugin to include those .jars in my executable .jar.
After I create the .jar, I manually upload it to the server via SFTP and test it with something like java -jar MyProject.jar.
Oh yeah, did I mention that I don't unit test?
The most glaringly obvious problem that I'd like to fix first is my lack of source control. I like git because of it's distributed nature, but it doesn't seem to integrate with Eclipse well and I've heard that it doesn't work very well on Windows, which is my primary development OS. So, I'm leaning toward SVN, which I do have some experience with. I do have my own personal server, and I think I'll use that for my source control, because I'd rather be my own admin than have to deal with university bureaucracy. I had some trouble setting up SVN once before, but I'll give it another shot. Maybe I'll also install something like Trac or Redmine for bug-tracking, todo list, etc?
What about building and deployment? There has to be a better way than using Fat-Jar and manually uploading my jar to the server. I've heard about tools like Ant and Maven - do these apply to what I want to do? How can I get started using those?
I suppose I'd eventually like to integrate unit testing with JUnit too. Even though it probably should be, that is not my primary concern right now, because so far my applications aren't terribly complex. I'd really like to work on simplifying and streamlining my workflow right now, and then I'll ease into unit testing.
Sorry for the long question. I guess my question boils down to, for a sole developer, what tools and methodologies can/should I be using to not only make my job easier, but also just to expose myself to some technologies that would be expected requisite knowledge at a dedicated development house?
edit: Thanks for the great answers so far. I didn't mean to suggest that I wanted to make my workflow "enterprisey" just for the sake of doing it, but to make my job simpler and to get a few technologies under my belt that are typically used in enterprise development environments. That's all I meant by that.
It seems to me like you actually have a pretty good idea of what you need to do.
Using Subversion (or other VCS) is a must. Although it might be wise to setup a separate SVN repository for your work-related code rather than using a personal one.
You can integrate Subversion with Eclipse using a plugin like Subclipse, which I've found works pretty well.
I'd definitely use Ant or Maven - my preference is Ant because it's more flexible, and I think it would suit your development style more than Maven as well. But you might also want to look into Apache Ivy, which handles dependency-management.
Basically you set up an ant task which runs your compile, build and deployment steps - so that when you create a final JAR package you can be sure that it's been unit tested as that is part of your ant script. The best way to get started with ant is to look at some examples, and read through the manual.
As for unit testing - you can gradually build up with unit testing. I would recommend using JUnit in conjunction with a code coverage tool such as Cobertura (which is easy to set up) - it will help you to understand how much code your tests are covering and is an indicator about how effective your tests are.
It may also be worth your while setting up something like Trac - it's important to be able to keep track of bugs, and a wiki is surprisingly useful for documentation.
In other words, all of this sounds like you're on the right lines, you just need to get started using some of these tools!
If you're really set on distributed source control, I'd recommend you look at Bazaar. Its GIT-like distributed source control that's designed around performing very high quality merges. Out of the box it works on all platforms including Windows and they have a TortoiseBZR client.
Really though, any source control is better than none. If you're the sole developer then there's no need for anything more complex than SVN. Large companies and projects use SVN all the time with little issue.
As far as unit testing goes, you should make yourself familiar with JUnit. The fact that you're aware of unit testing and know you should be doing it is still several steps ahead of most ad-hoc developers.
Use version control. Period. SVN has great intergration with Eclipse and Windows. Get the TourtisSVN client for windows and use the subclipse plugin with Eclipse.
I would recomend getting an external HD or use one of your companies' servers for putting your repository on and do backups often. Subversion works great with deployment and upgrading as well. Just learn how to do it and you will never look back :)
As far as Unit Testing, some people would say that is the way to go but I have not found enough evidence to start the practice myself. If sombody on this question can convince me otherwise then please do!
Also, don't look to "enterprise" your workflow - look to make it better. Practices that work well with huge teams and corperations may not work well for you. I am pretty much an only developer myself and know the situation you are in. Just try everything and only keep what feels natural after a while.
But make sure to just try SVN! If your company has a LINUX server with apache see if you can set up your server there using DAV-SVN.
:)
I think you answered most of your own questions.
Source control: pick SVN - easy installation, great integration with Eclipse (subclipse).
Use Ant to build your project and deploy it (SCP/SFTP task)
Keep all your settings (Eclipse project settings, build xmls, etc.) in SVN.
Use Bugzilla to keep track of your bugs/issues/requests/ideas.
It would be VERY beneficial to start working with version control. Start now, don't delay! Git is moving REALLY fast, and there's already a TortoiseGit being developed. SVN is still a great standard to work with. And I have not worked with Mercurial, but that's another VCS that's worth looking into.
Other than that, I don't see why your workflow has to be enterprisey. It just has to be efficient and comfortable. That said, I think you should try working with a simple text editor and compiling from the command line. Most of the world's best programmers still use that instead of an IDE, and it will help you understand the processes underneath your favorite IDE.
Check out the Pragmatic Programmers' Pragmatic Starter Kit.
It schools you on the important basics of software development that universities/etc. seem to pass up, like version control, unit testing, and project automation (in that order), and does it in a very approachable manner.
It will give you a solid base to keep going from there.
Take a look # Matt Raible's Appfuse.
It incorporates Maven and Unit Testing.
http://raibledesigns.com/rd/tags/appfuse
Although you put it as the last thing, I think you should start using jUnit without delay.
The reason is, it's probably the easiest of the ambitions you've identified, and the tools are almost certainly already built into your Eclipse build.
Make a new folder in your project called 'jUnit'.
Let's say you have an Employee class, with setAnnualSalary() and getMonthlySalary() methods.
Right click on the jUunit folder, new -> "jUnit test case". This will make a new class. Call it TestEmployee. Eclipse generates the boilerplate stuff for you, as usual.
Add a void method with a name beginning with 'test':
public void testSalaryCalc() {
Employee emp = new Employee("John Doe");
emp.setAnnualSalary(12000);
assertEquals(1000,emp.getMonthlySalary());
}
Right click, "run as" -> "jUnit test". (the first time Eclipse may prompt you to do some setup for the project. Just do what it says.)
If Employee works properly, you'll see a green bar.
Sabotage the Employee class, run the test again, and you'll see a red bar, as well as output telling you what the failure was.
Congratulations: you are unit testing!
Right-clicking on the parent directory and choosing "Run as jUnit test" will run every Testcase class in the directory. Later you can incorporate jUnit into your build process, but don't worry about that for now.
Autocomplete will show you all the variations on assert() you can use. You can read up on it, and aim towards practices where you write the test cases before the implementation that passes them. But just doing the simple stuff above gets you big benefits.
Once you have a set up version control and a few unit-tests, I would consider a continous integration server (you wanted to be enterprisey, right?).
Even if you are and stay the sole developer, this might help you to uncover a few errors. Things you forgot to check in or the likes. A CI-server regularly checks out all your sources, does a clean build an runs all your tests. It also contacts you in the case of errors.
This gives you the guarantee that you (or any other person) is able to check out your code and build/run your projects.
I would recommend to take a look at Hudson
Like others have said, you already clearly know what you need to do. A VCS is a must, CI or bug-tracking may be overkill (for a single developer a spreadsheet might suffice for bug-tracking).
One thing that might benefit you greatly is keeping an organized product backlog. In solo development, I find keeping focused on the high-priority features and avoiding feature creep to be one of my biggest challenges. Keeping a backlog helps immensely. It doesn't have to be much more than a prioritized list of features with some notes about the scope of each. At my workplace, we keep this info in Trac, but here again, a spreadsheet may be all you need.
And I want to put in a plug for unit testing, particularly Test Driven Development (TDD). Kent Beck's book is a good place to start. I find that TDD helps keep me honest and focused on what I really need to do, particularly on a single-developer project without QA. Sometimes it seems like the code writes itself.
You got some really solid answers so short post adding a link to an article about Test Driven Development which is an agile practice that will look good on your CV.
TDD
If you are running a Windows Server where you want to put your SVN server, use Visual SVN as the server. It is super easy to setup and use, it supports both basic authentication and windows authentication. It is also free to use.
Eclipse has quite a few modules to integrate with a SVN server, so use one of those, or the already suggested Tortoise SVN.
All the previous comments covered almost everything you might ever need :-)
I want to add another approach on how to develop (the development workflow).
I suggest that you read the following article and although it's a git workflow you can use the same idea for any other tool you might be using.
http://nvie.com/posts/a-successful-git-branching-model/
I think the title sums it up. I just want to know why one or the other is better for continous integration builds of Java projects from Svn.
I agree with this answer, but wanted to add a few points.
In short, Hudson (update: Jenkins) is likely the better choice now. First and foremost because creating and configuring jobs ("projects" in CC vocabulary) is just so much faster through Hudson's web UI, compared to editing CruiseControl's XML configuration file (which we used to keep in version control just to keep track of it better). The latter is not especially difficult - it simply is slower and more tedious.
CruiseControl has been great, but as noted in Dan Dyer's aptly-named blog post, Why are you still not using Hudson?, it suffers from being first. (Um, like Britain, if you will, later into the industrial revolution, when others started overtaking it with newer technologies.)
We used CruiseControl heavily and have gradually switched over to Hudson, finally using it exclusively. And even more heavily: in the process we've started using the CI server for many other things than before, because setting up and managing Hudson jobs is so handy. (We now have some 40+ jobs in Hudson: the usual build & test jobs for stable and development branches; jobs related to releasing (building installers etc); jobs that run some (experimental) metrics against the codebase; ones that run (slow) UI or integration tests against a specific database version; and so on.)
From this experience I'd argue that even if you have lots of builds, including complicated ones, Hudson is a pretty safe choice because, like CC, you can use it to do anything, basically. Just configure your job to run whatever Ant or Maven targets, Unix shell scripts, or Windows .bat scripts, in the order you wish.
As for the 3rd party stuff (mentioned here by Jeffrey Fredrick) - that is a good point, but my impression is that Hudson is quickly catching up, and that there's already a very large number of plugins available for it.
For me, the two things I can name that I miss about CruiseControl are:
Its warning emails about broken builds were more informative than those of Hudson. In most cases the root cause was evident from CC's nicely formatted HTML mail itself, whereas with Hudson I usually need to follow the link to Hudson web UI, and click around a little to get the details.
The CruiseControl dashboard is better suited, out of the box, as an "information radiator" (shown on a public monitor, or projected on a wall, so that you can always quickly see the status of all projects). With Hudson's front page, we needed some Greasemonkey tricks to get job rows all nicely green/red.
Minor disclaimer: I haven't been following the CC project closely for the last year or so. (But from a quick look, it has not changed in any dramatic way.)
Note (2011-02-03): Hudson has been renamed/forked as Jenkins (by Hudson creator Kohsuke Kawaguchi and others). It looks as if Oracle—which controls the Hudson name—will keep "Hudson" around too, but my personal recommendation is to go with Jenkins, no matter what Oracle says.
As a long time CruiseControl committer and someone who has never used Hudson I'm pretty biased, but my take on it is:
Hudson is much easier to get up and running (in large part from a nice web interface) and has a very active plugin development community.
CruiseControl has support from lots of 3rd party stuff and has the benefit of doing some neat tricks with the xml configuration like plugin preconfiguration and include.projects which lets you version the configuration information with the project.
If you're only going to have a few builds I think Hudson is the clear winner. If you're going to have lots -- and don't mind the xml -- then I think CruiseControl's xml configuration tricks become a real strength.
My last project, we started off on CruiseControl. Which rocked. Then we moved to Hudson, which rocked even more. The things I liked about Hudson:
The upstream and downstream projects. So a commit to your data access code will eventually also trigger a build of the presentation layer.
Easily use an existing project as the starting point of a new one - so if you are in the habit of creating development branches, then making sure these are under continuous integration is a snap.
One difference is that Hudson is the product of a single genius intellect—Kohsuke Kawaguchi. Because of that, it's consistent, coherent, and rock solid. The downside could be some limitation on the rate of progress. However, Kohsuke is incredibly prolific, so I wouldn't be too worried about that. And, it's extensible, so if there's something Kohsuke doesn't have time for (or doesn't want), you can probably do it yourself.
I looked at both Cruise Control and Hudson but choose Hudson as it was much easier to setup and configure. Hudson seems very widely used these days with regular releases and lots of extensiblity through plugins. I would highly recommend it.
Hudson is the more user-friendly alternative in my opinion. It can be set up and maintained completely via the web interface (apart from the initial installation of the webapp, of course).
The only way this could be said about CruiseControl is if you count the built-in XML file editor.
Still, having used both, I'd still prefer any one over having no automated build.
I tried Cruise control...Its good...But documents are fragmented. Dashboard is confusing. Widget creation is also confusing. Never tried hudson. Will try on weekend.
I recently setup Jenkins for building Borland BDS 2006 projects making use of Subversion and I am very happy with it. I never used CruiseControl yet, so I can't compare. Read my blog post for more information.
Continuous integration of Delphi project with Jenkins