Develop Reference

Writing unit test for #Nonnull annotated parameter

I have a method like this one:
public void foo(#Nonnull String value) {...}
I would like to write a unit test to make sure foo() throws an NPE when value is null but I can't since the compiler refuses to compile the unit test when static null pointer flow analysis is enabled in IDE.
How do I make this test compile (in Eclipse with "Enable annotation-based null analysis" enabled):
#Test(expected = NullPointerException.class)
public void test() {
T inst = ...
inst.foo(null);
}
Note: In theory the static null pointer of the compiler should prevent cases like that. But there is nothing stopping someone from writing another module with the static flow analysis turned off and calling the method with null.
Common case: Big messy old project without flow analysis. I start with annotating some utility module. In that case, I'll have existing or new unit tests which check how the code behaves for all the modules which don't use flow analysis yet.
My guess is that I have to move those tests into an unchecked module and move them around as I spread flow analysis. That would work and fit well into the philosophy but it would be a lot of manual work.
To put it another way: I can't easily write a test which says "success when code doesn't compile" (I'd have to put code pieces into files, invoke the compiler from unit tests, check the output for errors ... not pretty). So how can I test easily that the code fails as it should when callers ignore #Nonnull?

Hiding null within a method does the trick:
public void foo(#NonNull String bar) {
Objects.requireNonNull(bar);
}
/** Trick the Java flow analysis to allow passing <code>null</code>
* for #Nonnull parameters.
*/
#SuppressWarnings("null")
public static <T> T giveNull() {
return null;
}
#Test(expected = NullPointerException.class)
public void testFoo() {
foo(giveNull());
}
The above compiles fine (and yes, double-checked - when using foo(null) my IDE gives me a compile error - so "null checking" is enabled).
In contrast to the solution given via comments, the above has the nice side effect to work for any kind of parameter type (but might probably require Java8 to get the type inference correct always).
And yes, the test passes (as written above), and fails when commenting out the Objects.requireNonNull() line.

Why not just use plain old reflection?
try {
YourClass.getMethod("foo", String.class).invoke(someInstance, null);
fail("Expected InvocationException with nested NPE");
} catch(InvocationException e) {
if (e.getCause() instanceof NullPointerException) {
return; // success
}
throw e; // let the test fail
}
Note that this can break unexpectedly when refactoring (you rename the method, change the order of method parameters, move method to new type).

Using assertThrows from Jupiter assertions I was able to test this:
public MethodName(#NonNull final param1 dao) {....
assertThrows(IllegalArgumentException.class, () -> new MethodName(null));

Here design by contract comes to picture. You can not provide null value parameter to a method annotated with notNull argument.

You can use a field which you initialize and then set to null in a set up method:
private String nullValue = ""; // set to null in clearNullValue()
#Before
public void clearNullValue() {
nullValue = null;
}
#Test(expected = NullPointerException.class)
public void test() {
T inst = ...
inst.foo(nullValue);
}
As in GhostCat's answer, the compiler is unable to know whether and when clearNullValue() is called and has to assume that the field is not null.

how to use mockito spy for lazy evaluation?

I want to use mockito spy.
When I set a return value in both following ways:
when(imagesSorterSpy.sortImages(imagesAsInsertionOrder, user)).thenReturn(imagesSorterSpy.sortImages(imagesAsInsertionOrder, user, fakeNowDate));
doReturn(imagesSorterSpy.sortImages(imagesAsInsertionOrder, user, fakeNowDate)).when(imagesSorterSpy).sortImages(imagesAsInsertionOrder, user);
I see the return value is being evaluated eagerly
meaning when this "setting" line is executed.
how can i force the spy to evaluate the return value only on demand?
meaning when the "when" condition is met.
update
Thanks to #RobbyCornelissen I have tried this code:
when(imagesSorterSpy.sortImages(imagesAsInsertionOrder, user)).thenAnswer(new Answer() {
public Object answer(InvocationOnMock invocation) {
Object[] args = invocation.getArguments();
ImagesSorter mock = (ImagesSorter)invocation.getMock();
return mock.sortImages((List<Image>)args[0], (UserInfo)args[1], fakeNowDate);
}
});
But it didn't help:
1) the "when" expression was invoked immediately. (not wanted)
2) eventually the callback wasn't call.

First let me warn you on partial mocks, because that is what the code is actually doing, it's wrong design wise. It may be more relevant to use a strategy pattern to compose behavior of the tested subject. Mockito team (including me) strongly advises to stay away of partial mocks whenever possible.
EDIT : I don't know the code and I don't know exactly which component under test but from what I gather there's a type responsible to sort images, let's call it ImagesSorter.
So first case ImagesSorter is a dependency of a test subject, so in this case just stubbing the mock of ImagesSorter will do.
If however it is ImagesSorter itself under test, and stubbing a special method of this class is called a partial mock and it is plain wrong. It exposes internal of the production code in the test. So there's several solutions.
As the code snippet showed in the answer shows a fakeDate, one of the solution is to not use things like new Date() and code a simple class TimeSource whose sole responsibility is to provide a date. And in tests the bwhavior of this TimeSOurce could be overriden.
A simplier solution would be to use JodaTime as it provides this functionality built in.
If the scope of test goes beyond changing the date, then maybe ImagesSorter needs a way to be configured with other objects. Inspiration on how to do it can be found with the cache builder of guava. If the configuration is dead simple then a simple constructor coud do it.
That could look like :
class ImagesSorter {
ImagesSorterAlso algo;
ImagesSorter(ImagesSorterAlgo algo) { this.algo = algo; }
Iterable sortImages(...) {
algo.sort(...);
}
}
interface ImagesSorterAlgo {
Iterable sort(...);
}
Now about your questions :
1) the "when" expression was invoked immediately. (not wanted)
It is expected imagesSorterSpy is a spy so by default it calls the real code. Instead you should use the alternate API, the same that #RobbyCornelissen showed. i.e.
doAnswer(sortWithFakeDate()).when(imagesSorterSpy).sortImages(imagesAsInsertionOrder, user);
// with BDD aliases (BDDMockito) which I personnaly finds better
willAnswer(sortWithFakeDate()).given(imagesSorterSpy).sortImages(imagesAsInsertionOrder, user);
will(sortWithFakeDate()).given(imagesSorterSpy).sortImages(imagesAsInsertionOrder, user);
sortWithFakeDate() would a static factory method that returns the answer, so the code reads well, and maybe reused elsewhere.
2) eventually the callback wasn't call.
This issue is most probably due to non equal arguments. You may need to check the equals method. Or relax the stub using the any() matcher.

I don't know the types of the arguments and classes you're using, so I can't provide a complete example, but you can stub using callbacks with the Answer<T> interface:
Mockito.doAnswer(new Answer() {
Object answer(InvocationOnMock invocation) {
ImagesSorter mock = (ImagesSorter) invocation.getMock();
Object[] args = invocation.getArguments();
return mock.sortImages((List<Image>) args[0], (UserInfo) args[1],
fakeNowDate);
}
}).when(imagesSorterSpy).sortImages(imagesAsInsertionOrder, user);

Stop Testng test from beforeInvocation method

Can I "stop" running test from beforeInvocation method of IInvokedMethodListener2?
I do some kind of soft skip with "throw new SkipException" in beforeInvocation, but I need to do "hard skip" - stop test without any trace of execution.
I first need to check some test method's annotations and then run or not run that specific test.
Thanks ...

Throwing new SkipException in IInvokedMethodListener2.beforeInvocation() yields the same result (and report) as for a test depending on a failed test: test method makes into the report and is duly marked as skipped. So the test method does not get executed.
If it's not hard enough, I'd recommend test exclusion, not skipping. There is
BeanShell expression feature to run custom inclusions and exclusions.
Also, IAnnotationTransformer would allow you to disable (equivalent of "enabled=false" attribute) a test method at runtime, the simplest example being:
public class SkippingTransformer implements IAnnotationTransformer {
#Override
public void transform(ITestAnnotation ita, Class type, Constructor c, Method method) {
if(method.getName().startsWith("skipMe")) {
ita.setEnabled(false);
}
}
}
This will opt out test methods completely (that is, test count reduces).
But you'll most likely need to take care of disabling chains of dependent methods. There may be other aspects to consider.

Mockito acts strangely when I assign multiple custom matchers to a single method

I'm wanting to use two custom matchers for a single method. Basically, if I pass the method VALUE_A, I want it to return RESULT_A, and if I pass it VALUE_B, I want it to return RESULT_B. So here's a code excerpt :
class IsNonEmpty extends ArgumentMatcher<Get> {
public boolean matches(Object get) {
//For some reason, this method is called when I assign the IsEmpty matcher to MockHtable.get()
//When this happens, the value of the get argument is null, so this method throws an NPE
return Arrays.equals(((Get) get).getRow(), SERIALIZATION_HELPER.getValidBytes(key));
}
}
class IsEmpty extends ArgumentMatcher<Get> {
public boolean matches(Object get) {
return !(Arrays.equals(((Get) get).getRow(), SERIALIZATION_HELPER.getValidBytes(key)));
}
}
[...]
//This line executes just fine
Mockito.when(mockHTable.get(Mockito.argThat(new IsNonEmpty()))).thenReturn(dbResult);
[...]
//This line calls IsNonEmpty.matches() for some reason. IsNonEmpty.matches() throws an NPE
Mockito.when(mockHTable.get(Mockito.argThat(new IsEmpty()))).thenReturn(emptyResult);
When I assign the IsEmpty custom matcher to mockHTable.get() method, it calls the IsNonEmpty.matches() function. No idea why it's doing this. So I change the IsNonEmpty class to this :
class IsNonEmpty extends ArgumentMatcher<Get> {
public boolean matches(Object get) {
//For some reason, this method is called when I assign the IsEmpty matcher. Weird, no?
if(get == null) {
return false;
}
return Arrays.equals(((Get) get).getRow(), SERIALIZATION_HELPER.getValidBytes(key));
}
}
and then everything works just fine! IsNonEmpty.matches() is still called when I assign the IsEmpty matcher to the mockHTable.get() function, but my matchers work exactly how they should.
So what's the deal? Why does this happen? Is my work-around an adequate way to compensate for this quirky behavior, or am I Doing It Wrong?

The reason why IsNonEmpty.matches() gets called on the second line of stubbing is that the Mockito.argThat(new IsEmpty()) returns null, which is then passed to mockHTable.get(). This call has to be checked against the earlier stubbing, to see whether it's a match; and that means calling IsNonEmpty.matches().
I'm not sure why this makes your test fail - it's hard to tell without seeing all of the code.
But, I would seriously recommend using doReturn...when instead of when...thenReturn whenever you have to stub the same mock more than once. You won't encounter issues like this if you do. In fact, I prefer to use doReturn...when in preference to when...thenReturn always (and similarly doThrow and doAnswer), although most people prefer when...thenReturn.
Re-writing one of your stubbing lines with the doReturn...when syntax looks like the following. The other is similar.
Mockito.doReturn(dbResult).when(mockHTable).get(Mockito.argThat(new IsNonEmpty()));
Lastly, a plea, on behalf of the Mockito development team (of which I am a member). If you think there is a bug in Mockito here - and from your description, I think there may well be - please EITHER
send a message to the Mockito mailing group (mockito#googlegroups.com) OR
raise an issue on the Mockito issues list (http://code.google.com/p/mockito/issues/list).
It's useful to the Mockito team if you can actually post a complete example, rather than just what you think the key lines are - sometimes the cause of a Mockito problem is in quite an unexpected place.

JUnit4 fail() is here, but where is pass()?

There is a fail() method in JUnit4 library. I like it, but experiencing a lack of pass() method which is not present in the library. Why is it so?
I've found out that I can use assertTrue(true) instead but still looks unlogical.
#Test
public void testSetterForeignWord(){
try {
card.setForeignWord("");
fail();
} catch (IncorrectArgumentForSetter ex){
}
// assertTrue(true);
}

Call return statement anytime your test is finished and passed.

As long as the test doesn't throw an exception, it passes, unless your #Test annotation specifies an expected exception. I suppose a pass() could throw a special exception that JUnit always interprets as passing, so as to short circuit the test, but that would go against the usual design of tests (i.e. assume success and only fail if an assertion fails) and, if people got the idea that it was preferable to use pass(), it would significantly slow down a large suite of passing tests (due to the overhead of exception creation). Failing tests should not be the norm, so it's not a big deal if they have that overhead.
Note that your example could be rewritten like this:
#Test(expected=IncorrectArgumentForSetter.class)
public void testSetterForeignWord("") throws Exception {
card.setForeignWord("");
}
Also, you should favor the use of standard Java exceptions. Your IncorrectArgumentForSetter should probably be an IllegalArgumentException.

I think this question needs an updated answer, since most of the answers here are fairly outdated.
Firstly to the OP's question:
I think its pretty well accepted that introducing the "expected excepetion" concept into JUnit was a bad move, since that exception could be raised anywhere, and it will pass the test. It works if your throwing (and asserting on) very domain specific exceptions, but I only throw those kinds of exceptions when I'm working on code that needs to be absolutely immaculate, --most APIS will simply throw the built in exceptions like IllegalArgumentException or IllegalStateException. If two calls your making could potentitally throw these exceptions, then the #ExpectedException annotation will green-bar your test even if its the wrong line that throws the exception!
For this situation I've written a class that I'm sure many others here have written, that's an assertThrows method:
public class Exceptions {
private Exceptions(){}
public static void assertThrows(Class<? extends Exception> expectedException, Runnable actionThatShouldThrow){
try{
actionThatShouldThrow.run();
fail("expected action to throw " + expectedException.getSimpleName() + " but it did not.");
}
catch(Exception e){
if ( ! expectedException.isInstance(e)) {
throw e;
}
}
}
}
this method simply returns if the exception is thrown, allowing you to do further assertions/verification in your test.
with java 8 syntax your test looks really nice. Below is one of the simpler tests on our model that uses the method:
#Test
public void when_input_lower_bound_is_greater_than_upper_bound_axis_should_throw_illegal_arg() {
//setup
AxisRange range = new AxisRange(0,100);
//act
Runnable act = () -> range.setLowerBound(200);
//assert
assertThrows(IllegalArgumentException.class, act);
}
these tests are a little wonky because the "act" step doesn't actually perform any action, but I think the meaning is still fairly clear.
there's also a tiny little library on maven called catch-exception that uses the mockito-style syntax to verify that exceptions get thrown. It looks pretty, but I'm not a fan of dynamic proxies. That said, there syntax is so slick it remains tempting:
// given: an empty list
List myList = new ArrayList();
// when: we try to get the first element of the list
// then: catch the exception if any is thrown
catchException(myList).get(1);
// then: we expect an IndexOutOfBoundsException
assert caughtException() instanceof IndexOutOfBoundsException;
Lastly, for the situation that I ran into to get to this thread, there is a way to ignore tests if some conidition is met.
Right now I'm working on getting some DLLs called through a java native-library-loading-library called JNA, but our build server is in ubuntu. I like to try to drive this kind of development with JUnit tests --even though they're far from "units" at this point--. What I want to do is run the test if I'm on a local machine, but ignore the test if we're on ubuntu. JUnit 4 does have a provision for this, called Assume:
#Test
public void when_asking_JNA_to_load_a_dll() throws URISyntaxException {
//this line will cause the test to be branded as "ignored" when "isCircleCI"
//(the machine running ubuntu is running this test) is true.
Assume.assumeFalse(BootstrappingUtilities.isCircleCI());
//an ignored test will typically result in some qualifier being put on the results,
//but will also not typically prevent a green-ton most platforms.
//setup
URL url = DLLTestFixture.class.getResource("USERDLL.dll");
String path = url.toURI().getPath();
path = path.substring(0, path.lastIndexOf("/"));
//act
NativeLibrary.addSearchPath("USERDLL", path);
Object dll = Native.loadLibrary("USERDLL", NativeCallbacks.EmptyInterface.class);
//assert
assertThat(dll).isNotNull();
}

I was looking for pass method for JUnit as well, so that I could short-circuit some tests that were not applicable in some scenarios (there are integration tests, rather than pure unit tests). So too bad it is not there.
Fortunately, there is a way to have a test ignored conditionally, which actually fits even better in my case using assumeTrue method:
Assume.assumeTrue(isTestApplicable);
So here the test will be executed only if isTestApplicable is true, otherwise test will be ignored.

There is no need for the pass method because when no AssertionFailedException is thrown from the test code the unit test case will pass.
The fail() method actually throws an AssertionFailedException to fail the testCase if control comes to that point.

I think that this question is a result of a little misunderstanding of the test execution process. In JUnit (and other testing tools) results are counted per method, not per assert call. There is not a counter, which keeps track of how many passed/failured assertX was executed.
JUnit executes each test method separately. If the method returns successfully, then the test registered as "passed". If an exception occurs, then the test registered as "failed". In the latter case two subcase are possible: 1) a JUnit assertion exception, 2) any other kind of exceptions. Status will be "failed" in the first case, and "error" in the second case.
In the Assert class many shorthand methods are avaiable for throwing assertion exceptions. In other words, Assert is an abstraction layer over JUnit's exceptions.
For example, this is the source code of assertEquals on GitHub:
/**
* Asserts that two Strings are equal.
*/
static public void assertEquals(String message, String expected, String actual) {
if (expected == null && actual == null) {
return;
}
if (expected != null && expected.equals(actual)) {
return;
}
String cleanMessage = message == null ? "" : message;
throw new ComparisonFailure(cleanMessage, expected, actual);
}
As you can see, in case of equality nothing happens, otherwise an excepion will be thrown.
So:
assertEqual("Oh!", "Some string", "Another string!");
simply throws a ComparisonFailure exception, which will be catched by JUnit, and
assertEqual("Oh?", "Same string", "Same string");
does NOTHING.
In sum, something like pass() would not make any sense, because it did not do anything.