I'm using eclipse in a Java environment. I have to introduce testing to an already underway project. How to start? Unit tests on the classes, and what else? What are the best tools for the job (considering I'm using eclipse)? TestNG, JUnit, JTiger?
How do I make others adapt themselves to use the tests?
Thanks in advance!
Eclipse has a great support of JUnit. This looks like a great starting point. I would add a new source directory test and create a package structure mirroring your src folder. Then you can add your unit tests one by one. Good luck!
Unless your team already is used to writing tests, testing all old components is not really feasible (sometimes not even then), as writing testable software requires a specific mentality.
I think, the best time to start writing tests is when you can define some new component that is critical enough to warrant the extra effort, but somehow new, so the already existing code base would not be tested as much.
This way you could find a testing approach, identify the benefits and learning the mentality without putting too much effort in something that might not work for your team.
About tooling: I sadly cannot really compare the different tools, as I only have experience with JUnit.
JUnit is easy to start with, as the corresponding tooling is already included in Eclipse (wizards to create templates, running and evaluation options...), and there are plenty of documentation and examples available on the net.
A lot of good questions.
If your project does not tests at all and already underway you should introduce the tests incrementally. When you have to develop a new feature write test for this feature, classes that you are going to change and features that could be broken. The same is when you are fixing bugs. First write test that reproduces the bug, then fix it.
I used JUnit and TestNG. Both are almost the same. TestNG has groups, i.e. you can mark test to be belong to group like development, build, integration, etc. This is mostly relevant if you have a lot of tests and it takes significant time to run them all.
Do you already have automatic build? If not start from this. If you prefer to use maven it is relatively simple. When your build is ready write a couple of unit tests (just to have something to fail...)
Then install Hudson/Jenkins and define your project there. People will see how cool is it that once you commit your new code the build runs almost immediately and you see all failed tests. Probably try to show the strength of TDD to your boss and try to explain him that he should force all team members to write tests.
If you have enough energy introduce Sonar to your company. People will see how awful the code that they are writing and how poor the test coverage is. The they will see how quickly the test coverage is growing up and will probably invest more into unit testing.
Shortly, good luck. You are on the right way.
JUnit and TestNG are both fine. TestNG has more capabilities and can be helpful with integration tests, JUnit is more focused on unit tests.
You may want to get acquainted with some kind of mocking library like Mockito. Code coverage tools like Cobertura and continuous integration tools like Jenkins are great too.
Using a DI framework like Spring or Guice is helpful for writing more easily-testable code. Whether you use a DI framework or not, the more loosely-coupled your code the easier it is to test. Since your project is already under way it is probably too late for this part, which will make your task harder.
It can be very hard to get co-workers to cooperate with testing. You may want to introduce it selectively on pieces where it can make the most difference. Writing tests for already-finished functionality is usually painful and a waste of time. Tests should be small, have few dependencies, be understandable, and should run quickly. The more painful it is to write tests, run the tests, and fix broken tests, the more resistance you will get.
Related
I am currently working on a Java library - that is, a bunch of classes that are exclusively intended to be used in other projects. Naturally, it has no main() function.
Now, I want to test my progress. And by "test" I don't mean some professional standardized system; I mean I have a very simple function that I want to run to gather information, which will be modified as the project becomes more complete.
I was hoping I could drop an executable class into the Test Packages folder, and just click Run. Unfortunately, NetBeans complains that there are no main classes found.
So, how do I test a library project, without adding an executable class to my distributable source?
You should absolutely look into unit testing frameworks, such as JUnit. IDEs typically have support for running tests easily, and it looks like Netbeans does too. (I don't use Netbeans myself, but I'd have been shocked if it didn't support JUnit.) It's a lot simpler to do this than to have main methods everywhere. After all, a main method will only test one route through your code - with unit tests, you can have lots of tests, each testing one small piece of your code.
Even if you don't want to go into unit testing in a fully-fledged way (which I'd strongly urge you to, by the way), unit tests can be a very straightforward way of just running some code and experimenting with it. I sometimes use it when developing against a 3rd party library for the first time - leaving unit tests to show and document my understanding of the library's behaviour. (Obviously the better the library and its documentation, the less need there is for this, but it's still useful...)
I have used both JUnit and CPPUnit in Netbeans extensively and find that it is fairly easy to get test coverage for libraries with those tools. IntelliJ IDEA does a decent job with JUnit as well so that is an option if you don't like the Netbeans interface. The xUnit frameworks have gotten me out of a jam many times since they are very good at isolating errors quickly. As Jon said they also help to capture the requirements/behavior of your system so that is an added bonus.
How to write a unit test framework?
Can anyone suggest some good reading?
I wish to work on basic building blocks that we use as programmers, so I am thinking of working on developing a unit test framework for Java.
I don't intend to write a framework that will replace junit;
my intention is to gain some experience by doing a worthy project.
There are several books that describe how to build a unit test framework. One of those is Test-Driven Development: By Example (TDD) by Kent Beck. Another book you might look at is xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros.
Why do you want to build your own unit test framework?
Which ones have you tried and what did you find that was missing?
If (as your comments suggest) your objective is to learn about the factors that go into making a good unit test framework by doing it yourself, then chapters 18-24 (Part II: The xUnit Example) of the TDD book show how it can be done in Python. Adapting that to Java would probably teach you quite a lot about Python, unit testing frameworks and possibly Java too.
It will still be valuable to you to have some experience with some unit test framework so that you can compare what you produce with what others have produced. Who knows, you might have some fundamental insight that they've missed and you may improve things for everyone. (It isn't very likely, I'm sorry to say, but it is possible.)
Note that the TDD people are quite adamant that TDD does not work well with databases. That is a nuisance to me as my work is centred on DBMS development; it means I have to adapt the techniques usually espoused in the literature to accommodate the realities of 'testing whether the DBMS works does mean testing against a DBMS'. I believe that the primary reason for their concern is that setting up a database to a known state takes time, and therefore makes testing slower. I can understand that concern - it is a practical problem.
Basically, it consists of three parts:
preparing set of tests
running tests
making reports
Preparing set of tests means that your framework should collect all tests which you want to run. You can specify these tests (usually classes with test methods which satisfy some convention or marked with certain annotation or implement marker interface) in a separate file (java or xml), or you can find them dynamically (making a search over classpath).
If you choose the dynamic searching, then you'll probably have to use some libraries which can analyse java bytecode. Otherwise you'll have to load all the classes in your classpath, and this a) requires much time and b) will execute all static initializers of loaded classes and can cause unexpected tests results.
Running tests can vary significantly depending on features of your framework. The simplest way is just calling test methods inside a try/catch block, analysing and saving results (you have to analyze 2 situations - when the assertion exception was thrown and when it was not thrown).
Making reports is all about printing analyzed results in xml/html/wiki or whatever else format.
The Cook's Tour is written by Kent Beck (I believe; it's not attributed), and describes the thought process that went into writing JUnit. I would suggest reading it and considering how you might choose an alternate line of development.
Our project contains 2600 class files and we have decided to start using automated tests.
We know we have should have started this 2599 class files ago, but how and where should large projects start to write tests?
Pick a random class and just go?
What's important to know? Are there any good tools to use?
Write a unit test before you change something, and for every bug you encounter.
In other words, test the functionality you are currently working on. Otherwise, it is going to take a lot of time and effort to write tests for all classes.
Start writing tests for each bug that is filed (Write the test, watch it fail, fix the bug, test again). Also test new features first (they are more likely to have errors). It will be slow in the beginning, but as your test infrastructure grows, it will become easier.
If you are using java 5 or higher, use junit 4.
Learn about the difference of unit tests, integration tests and acceptance tests. Also have a look at mocking.
Other answers have given useful advice, but I miss a clear articulation of the underlying principle: strive to maximize the benefit from your efforts. Covering a large legacy codebase with unit tests to a significant extent takes a lot of time and effort. You want to maximize the outcome of your effort from the start. This not only gives valuable feedback early on, but helps convincing / keeping up the support of both management and fellow developers that the effort is worth it.
So
start with the easiest way to test the broadest functionality, which is typically system/integration tests,
identify the critical core functionality of the system and focus on this,
identify the fastest changing/most unstable part(s) of the system and focus on these.
Don't try unit tests first. Do system tests (end-to-end-tests) that cover large areas of code. Write unit tests for all new code.
This way you stabilize the old code with your system regression tests. As more and more new code comes in the fraction of code without unit tests begin to fade away. Writing unit tests for old code without the system tests in place will likly break the code and will be to much work to be justified as the code is not written with testability in mind.
You may find Michael Feathers' book Working Effectively with Legacy Code useful.
You're fairly dorked now, but write tests that bolster the most critical code you have. For example if you have code that allows functionality based upon users' rights, then that's a biggy - test that. The routine that camelcases a name and writes it to a log file? Not so much.
"If this code broke, how much would it suck" is a good litmus test.
"Our internal maintenance screens would look bad on IE6" is one answer.
"We'd send 10,000,000 emails to each of our customers" is another answer.
Which classes would you test first, hehe.
You might find this book relevant and interesting. The author explains how to do exactly what you ask for here.
http://my.safaribooksonline.com/0131177052
Oh, and one more thing - having an insufficient number of unit tests is far better than having none. Add a few at a time if that's all you can do. Don't give up.
What is the most commonly used approach used for testing in Java projects (iterative development) ?
My suggestion is that you should have a healthy mix of automated and manual testing.
AUTOMATED TESTING
Unit Testing
Use NUnit to test your classes, functions and interaction between them.
http://www.nunit.org/index.php
Automated Functional Testing
If it's possible you should automate a lot of the functional testing. Some frame works have functional testing built into them. Otherwise you have to use a tool for it. If you are developing web sites/applications you might want to look at Selenium.
http://www.peterkrantz.com/2005/selenium-for-aspnet/
Continuous Integration
Use CI to make sure all your automated tests run every time someone in your team makes a commit to the project.
http://martinfowler.com/articles/continuousIntegration.html
MANUAL TESTING
As much as I love automated testing it is, IMHO, not a substitute for manual testing. The main reason being that an automated can only do what it is told and only verify what it has been informed to view as pass/fail. A human can use it's intelligence to find faults and raise questions that appear while testing something else.
Exploratory Testing
ET is a very low cost and effective way to find defects in a project. It take advantage of the intelligence of a human being and a teaches the testers/developers more about the project than any other testing technique i know of. Doing an ET session aimed at every feature deployed in the test environment is not only an effective way to find problems fast, but also a good way to learn and fun!
http://www.satisfice.com/articles/et-article.pdf
Personal experience would suggest the most popular approach is none at all.
I've worked with TDD (Test Driven Development) before, and my feelings towards it are mixed. Essentially, you write your tests before you write your code, and write your code to satisfy the requirements of the test. TDD forces you to have an extremely clear idea about your requirements prior to starting. An added benefit is that, once you are done development, assuming you followed closely to TDD procedures, you'd have a complete set of test suites to go with the code. The down side is that it takes an extremely long time, and at times, you'd just want to skip a couple of steps (e.g. maybe writing code before tests like a sane person would prefer to do).
More can be read here (wiki link)
Unit testing?
Contract-based programming, a la Eiffel?
Waterfall model?
Different shops do different things. If there were one method to rule them all, you wouldn't be asking this question.
On the premise of doing testing at all, I would say that testing with JUnit is the common approach to do testing in Java.
Although most tests are written with JUnit mostly tests tend to be more integration tests than unit tests. (meaning not testing one thing in isolation but some things together)
Additionally test are mostly not written in a test first approach but in parallel or after a specific feature has been implemented.
If you go to a team that makes more advanced use of testing you might probably find some CI Server (Cruise Control, Hudson) running the tests at least once a day during a nightly build.
In the order of most commonly used approach:
no tests at all
manual tests: running the app,
clicking or providing input, check
results
try to write some JUnits, forget
about them, slide to 2 and 1
Start with TDD, see that it's hard
then slide to 3, 2 and 1
on the theoretical side there are loads of ways to properly test the code.
If you are looking for something practical take a look at
Clean Code Talk. Take a look at the whole series, about 5 talks (can't post more than one link).
My Suggestion for testing of the java project is to keep it simple.
Steps :-
Manual Testing :-Achieve a stable product.
Automation Testing :- Maintain the quality of the product.
Report Generation and reporting :- Let people know the quality of the product.
Continuous Integration :-Make it a complete automated,continuous tool.
When developer will commit the Functionality then Start testing the it module by module.Try to compare the Actual Output with the expected output and against that Log the issues.
When developer resolved with the issues,Start with the Integration Testing and also start testing the resolved state issues and check whether any regression occur due to issue Fixing.
At last when product become the stable one,Then start for automating the the modules.
You can also follow automation step by step like:-
1.Automating the modules.
2.Report generation and send mail for product HealthCheck.
3.Continuous Integration and Automation testing on private server on local machine.
I'm working on a growing java project and I'm probably going to cooperate with somebody else to improve some features.
I'd like to use some tools to improve the quality of my work keeping in mind that:
I don't have too much time to spend on this project
it's a small project but it's really important for me
I don't want to buy software/hardware for it
I'm already using SVN
what do you think about maven and junit? is it worth spending time for them?
Do you know any other good tool?
Maven and JUnit are good for enforcing good habits (unit testing, uniform structure) and together with good SCM habits, I would say those are amongst the most important things for collaborative development.
Since you are not using JUnit my guess is you don't have any unit tests yet. This would seem to me the most important step for you to take, if other people will start working on your code. Without unit tests, someone can easily break functionality without knowing it.
Create a suite of unit tests that cover at least 80% of the code. You can use Cobertura to measure code coverage. This might seem like a lot of work (it is) but will save you far more time in the future.
Maven is the de-facto standard for building and deployment at the moment, but it has its drawbacks too. If you have a well documented build procedure in place (either using Ant or custom scripts) I would suggest it is less important to introduce Maven than to add unit tests.
JUnit is good for helping verify your code on any project.
Maven has a learning curve that can be hard to get over. If you have one module and a relatively simple set of build steps you may find it simpler to use Ant.
On the other hand with a Maven build you can simply add additional reports to your code to check various parameters on your code, and it is much harder to migrate to Maven than if you've conformed to its conventions from the start.
Examples of Maven plugins that can help check your code:
Findbugs (static analysis of possible bugs)
Checkstyle (enforce coding standards)
PMD (more static analysis)
PMD CPD (copy paste detection)
JDepend (cyclic dependency checking and package coupling)
Cobertura (code coverage)
If you're interested in the code quality plugins, also consider Sonar, it wraps these plugins up and gives you some funky reports.
If you're interested in best practice, also consider a Continuous Integration server, Hudson is free and integrates well with Maven.
We use Maven and JUnit on a fairly large project and find it very helpful.
For project planning, I highly recommend FogBugz. It's the best issue tracking system I have seen to date with good support for project management as well, and free for teams of up to 2 people.
If you use Eclipse and SVN I would recommend you to take a look at Mylyn. Its underlying concepts are very simple but it helps a lot when working on a team.
In my modest opinion, Maven is too annoying for the real benefit of it. Maybe ant is just enough for your deployment tasks.
I've used maven on one project and miss it. There is a fairly large upfront investment though in getting it setup and configured. The XML documentation was out of date when i was working with it (perhaps this has improved). Once you get past this initial setup though it's a wonderful time saver.
As for JUnit, it's great. Use it.
Both of these tools should be treated as an investment. At first it may seem like a lot of unrelated stuff, but the project will grow more predictably with less problems over the long haul.