Develop Reference

Should all of my unit tests always pass when method is given bad input?

I am somewhat new to test driven development and am trying to determine whether I have a problem with my approach to unit testing.
I have several unit tests (let's call them group A) that test whether my method's return is as expected.
I also have a unit test "B" whose passing condition is that an IllegalArgumentException is thrown when my method is given invalid input.
The unit tests in group A fail when the method is given invalid input since the method needs valid input to return correctly.
If I catch the exception, unit test "B" will fail, but if I don't catch the exception, the tests in group A will fail.
Is it OK to have unit tests fail in this way, or can I modify the code in some way so that all tests always pass?
Am I doing TDD all wrong?
Here's a notion of my code for more clarity:
public class Example{
public static String method(String inputString, int value){
if(badInput){
throw new IllegalArgumentException();
}
//do some things to inputString
return modifiedInputString;
}
}
public class ExampleTests{
#Test
public void methodReturnsIllegalArgumentExceptionForBadInput(){
assertThrows(IllegalArgumentException.class, ()->{Example.method(badInput,badValue);})
}
//"Group A" tests only pass with valid input. Bad input causes IllegalArgumentException
#Test
public void methodReturnsExpectedType(){
assertTrue(actual == expected);
}
#Test
public void methodReturnsExpectedValue(){
assertTrue(actual == expected);
}
#Test
public void methodReturnsExpectedStringFormat(){
assertTrue(actual == expected);
}
}

As you've correctly noted in the comment the problem is with too broad test setup and too implicit tests.
Tests are much more readable when they are self-contained on the business level. Common test setup should be focused on setting up technical details, but all business setup should be within each test itself.
Example for your case (a conceptual example, must be reworked to match the details of your implementation):
#Test
public void givenWhatever_whenDoingSomething_methodReturnsExpectedType(){
given(someInputs);
Type result = executeSut(); //rename executeSut to actual function under test name
assertTrue(result == expected);
}
This way by just looking at a test a reader knows what scenario is being tested. Common test setup and helper functions such as given abstract away technical details, so the reader does not get distracted at the first inspection. If they are interested, the details are always available - but are usually less important, such can be hidden.

Java JUNIT ERROR

I am using JUNIT to produce a test suite on a triangle project named (TriangleClassifier.java) and I am trying to read the below method that is to test the a,b,c sides and the expected output based on those values and then apply this to JUNIT although does not accept my test code and produces the following error. Any suggestions? thanks
JUNIT Error
expected: java.lang.String<ISOSCELES> but was: TriangleClassifier$TriangleType<ISOSCELES>
junit.framework.AssertionFailedError
at TestSuite.test1(TestSuite.java:9)
Method trying to test
public static TriangleType classify (int a, int b, int c) {
if (isInRange(a) & isInRange(b) & isInRange(c)) {
if (isTriangle(a, b, c)) {
if (a == b & b == c) {
return TriangleType.EQUILITERAL;
} else if (a !=b & a != c & b != c) {
return TriangleType.SCALENE;
} else {
return TriangleType.ISOSCELES;
}
} else {
return TriangleType.NOT_A_TRIANGLE;
}
} else {
return TriangleType.OUT_OF_RANGE;
}
}
TestSuite.java
import static org.junit.Assert.*;
import org.junit.Test;
public class TestSuite {
#Test
public void test1() {
assertEquals("ISOSCELES", TriangleClassifier.classify(100, 100, 1));
}
}
UPDATE
Have tried your suggestions of changing
assertEquals("ISOSCELES", TriangleClassifier.classify(100, 100, 1));
to
assertEquals(TriangleType.ISOCELES,TriangleClassifier.classify(100,100,1));
ALTHOUGH my JUNIT now produces a new error - Uncompilable source code - Erroneous tree type:
java.lang.RuntimeException
at TestSuite.test1(TestSuite.java:9)
AND
the below line where TriangleType is flaging an error with cannot find symbol - assertEquals(TriangleType.ISOCELES,TriangleClassifier.classify(100,100,1));
ERROR Screenshot
public enum TriangleType

By writing
assertEquals("ISOSCELES", TriangleClassifier.classify(100, 100, 1));
you tell JUNIT, that you expect the function to return the String "ISOSCELES". What your function actually returns is the constant TriangleType.ISOSCELES.
Change your assertion to
assertEquals(TriangleType.ISOSCELES, TriangleClassifier.classify(100, 100, 1));

Your assertion is comparing a String to a TriangleType, so naturally, these won't be equal. You want to write
assertEquals(TriangleType.ISOSCELES, TriangleClassifier.classify(100,100,1));
which compares a TriangleType to a TriangleType.
Update
Since it appears that you've defined TriangleType as an inner type of TriangleClassifier, you will either need to add an import statement for it to your class, or refer to it as
assertEquals(TriangleClassifier.TriangleType.ISOSCELES, TriangleClassifier.classify(100,100,1));

Without going too deeply into your algorithms, the error is most obviously highlighted by the return type of your classify method contrasted against your unit test. classify returns a TriangleType enumeration, but your JUnit test is expecting an object of type String to be returned. Since the two types are not equivalent, the expected and actual values cannot be equal. Hence the test fails.
Either change your test to expect a TriangleType, or your classify method to return a String.

Two things here:
You really have not much of a clue what you are doing. When you are on such a level that simple compiler error messages prevent you from making progress, you probably have to step back and focus on learning the java basics again before overburdening yourself with unit-testing and such things (although unit tests are a great thing, and one should start using them as early as possible).
Then I would suggest that you stop using assertEquals.
JUnit has another assert, called assertThat. You can use that like:
assertThat(TriangleClassifier.classify(100,100,1), is(TriangleType.ISOSCELES));
... and here the compiler would already give you a hint that
assertThat(TriangleClassifier.classify(100,100,1), is("some string"));
is invalid.
Further reading on assertThat/is: here

Using a `when` as verification

I have a utility method used in hundreds of tests to mock the return value from a customised randomiser. Here's a (highly artificial) model of my code:
interface CardRandomiser{
Card getCard(Suit suit);
}
void mockCard(Suit suit, Face face) {
CardRandomiser rand = mock(CardRandomiser.class);
when(rand.getCard(suit)).thenReturn(new Card(suit, face));
Game.setCardRandomiser(rand);
}
This can then be used as:
mockCard(Suit.CLUBS, Face.QUEEN);
Card card = pickACardAnyCard();
assertThat(card.getFace(), is(Face.QUEEN));
However this makes some bugs a bit hard to pick up. If the method under test incorrectly asks for Suit.HEARTS then the mock returns null and the assertion correctly fails. But it's impossible to tell through the error message what was passed to the mock.
Clearly I could handle this with a verify. I could pass the mock back out of mockCard function and then separately verify that is was called with the correct value. But that really clutters up the tests assertions that are not really related to what's being tested. Given every time this method is called I am specifying an expected argument value I'd prefer to put the assertion in one place. Note that this all occurs before the method under test is called.
Ideally I'd like the when statement to throw an exception if it's called with an unexpected value. Something like:
when(rand.getCard(suit)).thenReturn(new Card(suit, face));
when(rand.getCard(intThat(not(is(suit))))).thenThrow(new IllegalArgumentException());
This works and stops the test when getCard is called which is better. But it still doesn't allow me to show what the incorrect argument was.
I also tried it using an ArgumentCaptor and then checking the captured value. But it's clear they are for verify statements rather than when statements.
Is there a standard Mockito way of doing this, or do I need to clutter my tests with verify statements?

You can configure mockito answer using thenAnswer, e.g.
private CardRandomiser mockCard(final Suit suit, final Face face) {
CardRandomiser rand = mock(CardRandomiser.class);
when(rand.getCard(any(Suit.class))).thenAnswer(new Answer<Card>() {
#Override
public Card answer(InvocationOnMock invocation) throws Throwable {
if(!suit.equals(invocation.getArguments()[0])) {
throw new IllegalArgumentException(
String.format("Value %s passed, but mocked for %s", invocation.getArguments()[0], suit));
}
return new Card(suit, face);
}
});
return rand;
}

Test method that returns void

I have a void method and I want to test it. How do I do that?
Here's the method:
public void updateCustomerTagCount() {
List<String> fileList = ImportTagJob.fetchData();
try {
for (String tag : fileList) {
Long tagNo = Long.parseLong(tag);
Customer customer = DatabaseInterface.getCustomer(tagNo);
customer.incrementNoOfTimesRecycled();
DatabaseInterface.UpdateCustomer(customer);
}
} catch(IllegalArgumentException ex) {
ex.printStackTrace();
}
}

when the method returns void, you can't test the method output. Instead, you must test what are the expected consequences of that method. For example:
public class Echo {
String x;
public static void main(String[] args){
testVoidMethod();
}
private static void testVoidMethod() {
Echo e = new Echo();
//x == null
e.voidMethod("xyz");
System.out.println("xyz".equals(e.x)); //true expected
}
private void voidMethod(String s) {
x = s;
}
}

It might not be always true, but basic concept of unit test is to check if function works as expected and properly handling errors when unexpected parameters/situation is given.
So basically unit test is against the functions that takes input parameters and return some output so we can write those unit test.
The code like yours, however, includes some other dependency (database call) and that's something you can't execute unless you write integration-test code or real database connection related one and actually that's not recommended for unit test.
So what you need to do might be introducing unit test framework, especially Mockto/Powermock or some other stuff that provides object mocking feature. With those test framework, you can simulate database operation or other function call that is going to be happening outside of your test unit code.
Also, about how do I test void function, there is nothing you can with Assert feature to compare output since it returns nothing as you mentioned.
But still, there is a way for unit test.
Just call updateCustomerTagCount() to make sure function works. Even with just calling the function, those unit test can raise your unit test coverage.
Of course for your case, you need to mock
ImportTagJob.fetchData();
and
DatabaseInterface.getCustomer(tagNo);
and have to.
Let mocked
ImportTagJob.fetchData();
throw empty list as well as non-empty list and check if your code works as you expected. Add exception handling if necessary. In your code, there are two condition depends on whether fieList are null or non-null, you need to test it.
Also, mock those objects and let them throw IllegalArgumentException where you expect it to be thrown, and write an unit test if the function throws a exception. In Junit, it should be like
#Test(expected = IllegalArgumentException.class)
public void updateCustomerTagCountTest(){
// mock the objects
xxxxx.updateCustomerTagCount();
}
That way, you can ensure that function will throw exception properly when it has to.

Simulate first call fails, second call succeeds

I want to use Mockito to test the (simplified) code below. I don't know how to tell Mockito to fail the first time, then succeed the second time.
for(int i = 1; i < 3; i++) {
String ret = myMock.doTheCall();
if("Success".equals(ret)) {
log.write("success");
} else if ( i < 3 ) {
log.write("failed, but I'll try again. attempt: " + i);
} else {
throw new FailedThreeTimesException();
}
}
I can setup the success test with:
Mockito.when(myMock).doTheCall().thenReturn("Success");
And the failure test with:
Mockito.when(myMock).doTheCall().thenReturn("you failed");
But how can I test that if it fails once (or twice) then succeeds, it's fine?

From the docs:
Sometimes we need to stub with different return value/exception for the same method call. Typical use case could be mocking iterators. Original version of Mockito did not have this feature to promote simple mocking. For example, instead of iterators one could use Iterable or simply collections. Those offer natural ways of stubbing (e.g. using real collections). In rare scenarios stubbing consecutive calls could be useful, though:
when(mock.someMethod("some arg"))
.thenThrow(new RuntimeException())
.thenReturn("foo");
//First call: throws runtime exception:
mock.someMethod("some arg");
//Second call: prints "foo"
System.out.println(mock.someMethod("some arg"));
So in your case, you'd want:
when(myMock.doTheCall())
.thenReturn("You failed")
.thenReturn("Success");

The shortest way to write what you want is
when(myMock.doTheCall()).thenReturn("Success", "you failed");
When you supply mutiple arguments to thenReturn like this, each argument will be used at most once, except for the very last argument, which is used as many times as necessary. For example, in this case, if you make the call 4 times, you'll get "Success", "you failed", "you failed", "you failed".

Since the comment that relates to this is hard to read, I'll add a formatted answer.
If you are trying to do this with a void function that just throws an exception, followed by a no behavior step, then you would do something like this:
Mockito.doThrow(new Exception("MESSAGE"))
.doNothing()
.when(mockService).method(eq());

I have a different situation, I wanted to mock a void function for the first call and run it normally at the second call.
This works for me:
Mockito.doThrow(new RuntimeException("random runtime exception"))
.doCallRealMethod()
.when(spy).someMethod(Mockito.any());

To add on to this and this answer, you can also use a loop to chain the mocked calls. This is useful if you need to mock the same thing several times, or mock in some pattern.
Eg (albeit a farfetched one):
import org.mockito.stubbing.Stubber;
Stubber stubber = doThrow(new Exception("Exception!"));
for (int i=0; i<10; i++) {
if (i%2 == 0) {
stubber.doNothing();
} else {
stubber.doThrow(new Exception("Exception"));
}
}
stubber.when(myMockObject).someMethod(anyString());

The shortest would be
doReturn("Fail", "Success").when(myMock).doTheCall();