Develop Reference

Unit testing instance of an Iterable

I am developing a Junit 4 test case (black box) for the following method:
/**
* Returns all points in the symbol table.
*/
public Iterable<Point2D> points();
The method is supposed to return all points inside the KdTree symbol table. I don't make any assumptions regarding a particular implementation of an Iterable being returned from the method, therefore I develop the following unit test:
#Test
public void emptySTPointsTest() {
KdTreeST<Integer> kd = new KdTreeST<>(); // initializing empty ST
Iterable<Point2D> pts = kd.points(); // arbitrary decision
assertFalse(pts.iterator().hasNext()); // ?!
}
Are there other ways to check if the instance of an Iterable is (not) empty? What is the "right" (or most common) way of implementing these kinds of unit tests, when you can't explicitly check number of elements and you don't know which order they come in (not just empty case, but in general)?
P.S I have tried to utilize Hamcrest's Matchers, however Iterable interface doesn't specify size() method, so I guess it wouldn't be that useful in this case.
Edit: Instead of
Iterable<Point2D> pts = new HashSet<>();
I should have written Iterable<Point2D> pts = kd.points();

Your method is fine; there's nothing wrong with it. As the other answer points out, you're testing the iterator you got from the HashSet and not from the KdTreeST.
Also, there is a Hamcrest Matcher for your situation.
assertThat(pts.iterator(), is(emptyIterable()));
(assuming you statically import from org.hamcrest.Matchers)

The unit test that you've written is unit-testing the HashSet class (in particular, the behavior of HashSet#iterator() when the set is empty). It's not testing your points method.
To answer your question about empty Iterables, however... it looks like that's certainly one way to test for an empty Iterable. The Iterable interface only has one method: iterator.
Perhaps declaring points to return an Iterable is a bit too abstract for your needs. Why not bump it up to Collection<Point2D>? This way, you can use Collection#isEmpty(), which is easier to read.

One assert statement tests several conditions

I found many information about bad practices to use several asserts in one method (one test condition). It's clear. Each method should test one and only one condition. So it should be one assert in most cases.
But I can't find any information about best practices to situation, if my one assert tests several conditions.
Example:
#Test
public void findAllPerformers_returnsPerformersInAlphabeticalOrder() {
List<String> performers = reportRepository.findAllPerformers();
assertThat("Should return performers in alphabetical order",
performers, contains("Bart Simpson", "Homer Simpson",
"Ned Flanders", "Xena Warrior Princess"));
}
First, I had to have 2 more test methods (conditions): findAllPerformers_returnsAllUniquePerformers and findAllPerformers_returnsEachPerformerOnlyOnce. But then I realized that my first method tests this two conditions too!
So, what should I do?
Leave all methods as is.
Remove 2 other test methods and rename my findAllPerformers_returnsPerformersInAlphabeticalOrder method to very long name with all three conditions included (because it really tests all three conditions!)
Just remove 2 other test methods.
Leave all methods but invoke findAllPerformers_returnsPerformersInAlphabeticalOrder method from two other methods.
Copy-paste findAllPerformers_returnsPerformersInAlphabeticalOrder method contents to two other methods.
Anything else...

If this test fails then the person looking at the failed test would initially see findAllPerformers_returnsPerformersInAlphabeticalOrder().
Looking at that alone it appears that the method under test returned a list that wasn't in alphabetical order, not that the list didn't contain 'Bart Simpson'.
Avoid options 4 and 5, they don't really solve anything and will increase duplication/maintenance without any added value (since tests will fail for reasons not mentioned in the test name). Instead try to find a way to actually test what you want with the assert.

Having trouble with a mock object unit test

I'm toying with mock objects and I'm having trouble setting up a unit test that checks if a list is being properly sorted. The conceit of this is that I'm mocking a clock, but simply using the times() method isn't working. The list should be in ascending order, but getting the first index simply returns whatever value was set last.
For reference, here is a working mock test that tests the clock being set at midnight:
#Test
public void shouldSetAtMidnight() {
expect(mock.instant()).andReturn(Instant.from(this.midnight));
expect(mock.getZone()).andReturn(this.timeZone);
replay(mock);
this.st.setDesiredValue(72);
SetPoint[] sched = this.st.getSchedule();
verify(mock);
assertEquals(LocalTime.MIDNIGHT, sched[0].getScheduledTime());
}
Here, setting the desired value is only important in that it adds a SetPoint to the array (setDesiredValue creates a SetPoint which has an int and a LocalTime). This is where I'm having difficulty:
#Test
public void shouldOrderTwoSetPointsAddedOutOfOrder() {
expect(mock.instant()).andReturn(Instant.from(this.midnight)).times(2);
expect(mock.getZone()).andReturn(this.timeZone).times(2);
replay(mock);
this.st.setDesiredValue(73);
this.st.setDesiredValue(71);
SetPoint[] schedule = this.st.getSchedule();
verify(mock);
assertEquals(71, schedule[0].getTemp());
}
setDesiredValue is supposed to associate an int value with a LocalTime, which in this case should be midnight. It then adds a SetPoint with those characteristics to the list and calls Collection's sort() method to sort them from lowest to highest. Now, I'm assuming that my problem here is that I'm associating two values with the exact same time despite calling times(2) for both expects, but I just recently started with mock objects and just don't have the knowledge of them to know where to go from here. Running this test returns 73 instead of 71.

Your assumption is correct. When you say .andReturn(Instant.from(this.midnight)).times(2), the exact value passed to .andReturn is repeated twice.
You could do .andReturn(Instant.from(this.midnight)).andReturn(Instant.from(this.midnight)), which will give you two instants. But as a general rule, this is very bad form for a unit test. Instant.from(this.midnight) comes from outside the test, so you can't predict if calling this function twice will result in the same value or different values, making the test non-deterministic. Okay for an integration test, not okay for a unit tests, where you would be using mocks. It is better to use a real number, like .andReturn(Instant.fromEpochSecond(10)).andReturn(Instant.fromEpochSecond(15)).
You're not providing a whole lot of details about st and any reasons why you think the test should not return 73 instead of 71.

How can I simplify testing of side-effect free methods in Java?

Functions (side-effect free ones) are such a fundamental building block, but I don't know of a satisfying way of testing them in Java.
I'm looking for pointers to tricks that make testing them easier. Here's an example of what I want:
public void setUp() {
myObj = new MyObject(...);
}
// This is sooo 2009 and not what I want to write:
public void testThatSomeInputGivesExpectedOutput () {
assertEquals(expectedOutput, myObj.myFunction(someInput);
assertEquals(expectedOtherOutput, myObj.myFunction(someOtherInput);
// I don't want to repeat/write the following checks to see
// that myFunction is behaving functionally.
assertEquals(expectedOutput, myObj.myFunction(someInput);
assertEquals(expectedOtherOutput, myObj.myFunction(someOtherInput);
}
// The following two tests are more in spirit of what I'd like
// to write, but they don't test that myFunction is functional:
public void testThatSomeInputGivesExpectedOutput () {
assertEquals(expectedOutput, myObj.myFunction(someInput);
}
public void testThatSomeOtherInputGivesExpectedOutput () {
assertEquals(expectedOtherOutput, myObj.myFunction(someOtherInput);
}
I'm looking for some annotation I can put on the test(s), MyObject or myFunction to make the test framework automatically repeat invocations to myFunction in all possible permutations for the given input/output combinations I've given, or some subset of the possible permutations in order to prove that the function is functional.
For example, above the (only) two possible permutations are:
myObj = new MyObject();
myObj.myFunction(someInput);
myObj.myFunction(someOtherInput);
and:
myObj = new MyObject();
myObj.myFunction(someOtherInput);
myObj.myFunction(someInput);
I should be able to only provide the input/output pairs (someInput, expectedOutput), and (someOtherInput, someOtherOutput), and the framework should do the rest.
I haven't used QuickCheck, but it seems like a non-solution. It is documented as a generator. I'm not looking for a way to generate inputs to my function, but rather a framework that lets me declaratively specify what part of my object is side-effect free and invoke my input/output specification using some permutation based on that declaration.
Update: I'm not looking to verify that nothing changes in the object, a memoizing function is a typical use-case for this kind of testing, and a memoizer actually changes its internal state. However, the output given some input always stays the same.

If you are trying to test that the functions are side-effect free, then calling with random arguments isn't really going to cut it. The same applies for a random sequence of calls with known arguments. Or pseudo-random, with random or fixed seeds. There's a good chance are that a (harmful) side-effect will only occur with any of the sequence of calls that your randomizer selects.
There is also a chance that the side-effects won't actually be visible in the outputs of any of the calls that you are making ... no matter what the inputs are. They side-effects could be on some other related objects that you didn't think to examine.
If you want to test this kind of thing, you really need to implement a "white-box" test where you look at the code and try and figure out what might cause (unwanted) side-effects and create test cases based on that knowledge. But I think that a better approach is careful manual code inspection, or using an automated static code analyser ... if you can find one that would do the job for you.
OTOH, if you already know that the functions are side-effect free, implementing randomized tests "just in case" is a bit of a waste of time, IMO.

I'm not quite sure I understand what you are asking, but it seems like Junit Theories (http://junit.sourceforge.net/doc/ReleaseNotes4.4.html#theories) could be an answer.

In this example, you could create a Map of key/value pairs (input/output) and call the method under test several times with values picked from the map. This will not prove, that the method is functional, but will increase the probability - which might be sufficient.
Here's a quick example of such an additional probably-functional test:
#Test public probablyFunctionalTestForMethodX() {
Map<Object, Object> inputOutputMap = initMap(); // this loads the input/output values
for (int i = 0; i < maxIterations; i++) {
Map.Entry test = pickAtRandom(inputOutputMap); // this picks a map enty randomly
assertEquals(test.getValue(), myObj.myFunction(test.getKey());
}
}
Problems with a higher complexity could be solved based on the Command pattern: You could wrap the test methods in command objects, add the command object to a list, shuffle the list and execute the commands (= the embedded tests) according to that list.

It sounds like you're attempting to test that invoking a particular method on a class doesn't modify any of its fields. This is a somewhat odd test case, but it's entirely possible to write a clear test for it. For other "side effects", like invoking other external methods, it's a bit harder. You could replace local references with test stubs and verify that they weren't invoked, but you still won't catch static method calls this way. Still, it's trivial to verify by inspection that you're not doing anything like that in your code, and sometimes that has to be good enough.
Here's one way to test that there are no side effects in a call:
public void test_MyFunction_hasNoSideEffects() {
MyClass systemUnderTest = makeMyClass();
MyClass copyOfOriginalState = systemUnderTest.clone();
systemUnderTest.myFunction();
assertEquals(systemUnderTest, copyOfOriginalState); //Test equals() method elsewhere
}
It's somewhat unusual to try to prove that a method is truly side effect free. Unit tests generally attempt to prove that a method behaves correctly and according to contract, but they're not meant to replace examining the code. It's generally a pretty easy exercise to check whether a method has any possible side effects. If your method never sets a field's value and never calls any non-functional methods, then it's functional.
Testing this at runtime is tricky. What might be more useful would be some sort of static analysis. Perhaps you could create a #Functional annotation, then write a program that would examine the classes of your program for such methods and check that they only invoke other #Functional methods and never assign to fields.
Randomly googling around, I found somebody's master's thesis on exactly this topic. Perhaps he has working code available.
Still, I will repeat that it is my advice that you focus your attention elsewhere. While you CAN mostly prove that a method has no side effects at all, it may be better in many cases to quickly verify this by visual inspection and focus the remainder of your time on other, more basic tests.

have a look at http://fitnesse.org/: it is used often for Acceptance Test but I found it is a easy way to run the same tests against huge amount of data

In junit you can write your own test runner. This code is not tested (I'm not sure if methods which get arguments will be recognized as test methods, maybe some more runner setup is needed?):
public class MyRunner extends BlockJUnit4ClassRunner {
#Override
protected Statement methodInvoker(final FrameworkMethod method, final Object test) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
Iterable<Object[]> permutations = getPermutations();
for (Object[] permutation : permutations) {
method.invokeExplosively(test, permutation[0], permutation[1]);
}
}
};
}
}
It should be only a matter of providing getPermutations() implementation. For example it can take data from some List<Object[]> field annotated with some custom annotation and produce all the permutations.

I think the term you're missing is "Parametrized Tests". However it seems to be more tedious in jUnit that in the .Net flavor. In NUnit, the following test executes 6 times with all combinations.
[Test]
public void MyTest(
[Values(1,2,3)] int x,
[Values("A","B")] string s)
{
...
}
For Java, your options seem to be:
JUnit supports this with version 4. However it's a lot of code (it seems, jUnit is adamant about test methods not taking parameters). This is the least invasive.
DDSteps, a jUnit plugin. See this video that takes values from appropriately named excel spreadsheet. You also need to write a mapper/fixture class that maps values from the spreadsheet into members of the fixture class, that are then used to invoke the SUT.
Finally, you have Fit/Fitnesse. It's as good as DDSteps, except for the fact that the input data is in HTML/Wiki form. You can paste from an excel sheet into Fitnesse and it formats it correctly at the push of a button. You need to write a fixture class here too.

Im afraid that I dont find the link anymore, but Junit 4 has some help functions to generate testdata. Its like:
public void testData() {
data = {2, 3, 4};
data = {3,4,5 };
...
return data;
}
Junit will then thest your methods will this data. But as I said, I cant' find the link anymore (forgot the keywords) for a detailed (and correct) example.

How to test void method with Junit testing tools?

I just happen to implement a method void followlink(obj page,obj link) which simply adds page and link to queue. I have unsuccessfully tried to test this kind of method.
All I want is to test that in the queue contains page and link received from followlink method. My test class already extends TestCase. So what is the best way to test such a method?

The JUnit FAQ has a section on testing methods that return void. In your case, you want to test a side effect of the method called.
The example given in the FAQ tests that the size of a Collection changes after an item is added.
#Test
public void testCollectionAdd() {
Collection collection = new ArrayList();
assertEquals(0, collection.size());
collection.add("itemA");
assertEquals(1, collection.size());
collection.add("itemB");
assertEquals(2, collection.size());
}

You could test the size if the queue before and after calling your method, something like:
int size = queue.length();
followLink(page, link);
assertEquals(size+1, queue.length()); // or maybe size+2?
another thing you might do is start off with an empty queue, call followLink, then dequeue the first item and test its values.

Most likely, your queue is private, so checking the size isn't going to work. The "design for testing" solution I've seen is to make use of package private methods and members instead. Since your junit tests are likely to be in the same package, they'll have access.
That by itself lets the "queue.length()" side effect test work.
But I'd go further: really, you should consider checking that your method has inserted the correct page and link to your queue. The details for that require more knowledge about how you're representing (and combining) page and link.
The jMock solution is also very good, though in truth I'm much more familiar with writing my own test harnesses.

So - check after the call of the method the queue, if the values passed to the method are added to the queue. You need access to the queue (something like getQueue()) for this.

Use jMock library and mock a storage that holds your queue, jMock allows to assert whether mocked method was called and with what parameters.

We have two ways to check the following :
Verify using the size of the Queue/Collection as in the program below :
(As pointed out by Patrick)
#Test
public void testfollowlinkAdd() {
int size = queue.length();
queue.add(pageLink); //Considering an Object of PageLink
Assert.assertTrue(size+1, queue.length());
}
OR
Make use of MockObjects. Frameworks like Mockito can be helpful here.