Problem
I'm using Test-Driven Development and having trouble making my tests define my code well enough. A simple example of my problem is as follows.
I have MyObject from which I want to call either methodA() or methodB() belonging to OtherObject depending on what argument MyObject receives in its own callMethod(int).
Anticipated code (and desired functionality)
This is essentially what I want the code to do - but I want to do it test first:
public class MyObject {
private final OtherObject otherObject;
public MyObject(OtherObject otherObject) {
this.otherObject = otherObject;
}
public void callMethod(int i) {
switch (i) {
case 0:
otherObject.methodA();
break;
case 1:
otherObject.methodB();
break;
}
}
}
Writing it test first
To achieve this I start by writing a test - check that methodA() is called when calling callMethod(0). I use JUnit and Mockito.
public class MyObjectTest {
private final OtherObject mockOtherObject = mock(OtherObject.class);
private final MyObject myObject = new MyObject(mockOtherObject);
#Test
public void callsMethodA_WhenArgumentIs0() {
myObject.callMethod(0);
verify(mockOtherObject).methodA();
}
}
I create the classes/methods needed to get rid of errors and make the test pass by implementing MyObject's method like this:
public void callMethod(int i) {
otherObject.methodA();
}
Next a test for the other option - calling callMethod(1)
#Test
public void callsMethodB_WhenArgumentIs1() {
myObject.callMethod(1);
verify(mockOtherObject).methodB();
}
And I get a final solution of:
public void callMethod(int i) {
otherObject.methodA();
otherObject.methodB();
}
The issue
This works but is clearly not what I want. How do I progress to the code I want using tests? Here I have tested for the behaviour I would like. The only solution I can think of is to write some more tests for behaviour I would not like to see.
In this example it would be okay to write 2 more tests to check that the other method is not being called but surely doing it like that is more of an issue in the general case. When there are more options, more complexity in which methods and how many different methods are called depending on the circumstances.
Say there were 3 methods in my example - would I have to write 3 tests to check the right method is called - then 6 more if I'm checking the 2 other methods aren't called for each of the 3 cases? (Whether you try and stick with one assertion per test or not you still have to write them all.)
It looks like the number of tests will be factorial to how many options the code has.
Another option is to just write the if or switch statements but techically it wouldn't have been driven by the tests.
I think you need to take a slightly bigger-picture view of your code. Don't think about what methods it should call, but think about what the overall effect of those methods should be.
What should the outputs and the side-effects be of calling callMethod(0)?
What should the outputs and the side-effects be of calling callMethod(1)?
And don't answer in terms of calls to methodA or methodB, but in terms of what can be seen from outside. What (if anything) should be returned from callMethod? What additional behaviour can the caller of callMethod see?
If methodA does something special that the caller of callMethod can observe, then include it in your test. If it's important to observe that behaviour when callMethod(0) happens, then test for it. And if it's important NOT to observe that behaviour when callMethod(1) happens, then test for that too.
In regard to your specific example, I'd say you are doing it exactly right. Your tests should specify the behavior of your class under test. If you need to specify that your class doesn't do something under some circumstance, so be it. In a different example, this would not bother you. For instance, checking both conditions in this method probably would not raise any objections:
public void save(){
if(isDirty)
persistence.write(this);
}
In the general case, you are right again. Adding complexity to a method makes TDD harder. The unexpected result is that this is one of the greatest benefits of TDD. If your tests are hide to write, then your code is also too complex. It will be hard to reason about and hard to maintain. If you listen to your tests, you'll consider changing your design in a way that simplifies the tests.
In your example, I might leave it alone (it's pretty simple as is). But, if the number of case grows, I'd consider a change like this:
public class MyObject {
private final OtherObjectFactory factory;
public MyObject(OtherObjectFactory factory) {
this.factory = factory;
}
public void callMethod(int i) {
factory.createOtherObject(i).doSomething();
}
}
public abstract class OtherObject{
public abstract void doSomething();
}
public class OtherObjectFactory {
public OtherObject createOtherObject(int i){
switch (i) {
case 0:
return new MethodAImpl();
case 1:
return new MethodBImpl();
}
}
}
Note that this change adds some overhead to the problem you are trying to solve; I would not bother with it for two cases. But as cases grow, this scales very nicely: you add a new test for OtherObjectFactory and a new implementation of OtherObject. You never change MyObject, or it's tests; it only has one simple test. It's also not the only way to make the tests simpler, it's just the first thing that occurred to me.
The overall point is that if your tests are complex, it doesn't mean testing isn't effective. Good tests and good design are two sides of the same coin. Tests need to bite off small chunks of a problem at a time to be effective, just like code needs to solve small chunks of a problem at a time to be maintainable and cohesive. Two hands wash each other.
Great question. Applying TDD to the letter (especially using the Devil's Advocate technique as you did) reveals some interesting problems indeed.
Mark Seemann has a recent article about a similar issue where he proves that using a different, slightly more strict kind of mock solves the problem. I don't know if Mockito can do that but with frameworks such as Moq, making mockOtherObject a strict mock would in your example result in the exception we want, because a call to the non-prepared method methodB() would be made.
That being said, this still sorts of falls into "testing what your code shouldn't do" and I'm not a big fan of verifying that things don't happen - it rigidifies your tests a lot. The only exception I see is if a method is critical/dangerous enough to your system to justify using defensive means to ensure it isn't called, but this shouldn't happen often.
Now, something might trump the whole conundrum - the Refactor part of the TDD cycle.
During that step, you should realize the switch statement smells a bit. How about a more modular, decoupled way ? If we think about it, the action to be taken in callMethod() is really decided by
MyObject's instantiator (which passes the appropriate OtherObject upon construction)
callMethod()'s caller (which passes the appropriate i parameter from which will depend the method call)
So an alternative solution could be to somehow combine the passed i with one method in the object that was injected at construction to trigger the expected action (#tallseth 's Factory example is exactly about that).
If you do this, OtherObject doesn't have to have 2 methods any more - the switch statement and the Devil's Advocate bug disappear altogether.
Related
I want to test a function in my code. This function calls another function in the same class but in my test I don't want to call it (I don't need it). Somehow, my test always goes into that inner function and makes errors. Is there any means to "skip" the call to that inner function ?
Here's an example :
void function1() {
if(condition == true) {
variable1 = function2()
}
}
Object function2() {
//Do something
return Object;
}
Is there a way to avoid calling function 2 ?
Thank you.
First you should consider fixing those errors thrown from function2().
If however you want to test the function1() isolated then the behavior you are describing is called Test Doubles. One kind of those test doubles is Mocking where you can drive the behavior of a class or a method.
There are frameworks doing this such as Mockito, but of course you can solve that problem on your own not depending on frameworks.
For example if you use Mockito you should end up mocking the function2() method like this
YouClassName mockedClass = mock(YouClassName.class);
when(mockedClass.function2()).thenReturn(new Object()); //you can of course return anything here
Its pretty common to test functions in isolation but in times where are dependencies between objects. If there are no dependencies you should probably consider not using test doubles (unless you are sure what you are doing).
Frameworks such as Mockito offer concepts such as spies. Using a Mockito spy, you can gain full control over which methods get invoked.
But: you only do that for very specific cases.
When you have a hard time testing your production code, then most likely: because you have written hard to test code.
Thus: you could try to use a spy here, but I would rather advice to step back and re-think what exactly you intend to do here.
class Elephant extends Animal {
public Elephant(String name) {
super(name);
}
void makeNoise() {
logger.info(" Elephant make Sound");
}
void perform(String day) {
if (day.equals("thursday") || day.equals("friday")) {
makeNoise();
}
}
}
Now i want to test the perform method. How can I unit test this method using JUnit?
Solution with Mockito Spy
import org.junit.Test;
import static org.mockito.Mockito.*;
public class ElephantTest {
#Test
public void shouldMakeNoise() throws Exception {
//given
Elephant elephant = spy(new Elephant("foo"));
//when
elephant.perform("friday");
//then
verify(elephant).makeNoise();
}
}
Negative tests:
#Test
public void elephantShouldDontMakeNoisesOnMonday() {
//given
Elephant elephant = spy(new Elephant("foo"));
//when
elephant.perform("monday");
//then
verify(elephant, never()).makeNoise();
}
or
#Test
public void shouldDoNotMakeNoisesOnMonday() {
//given
Elephant elephant = spy(new Elephant("foo"));
//when
elephant.perform("monday");
then(elephant).should(never()).makeNoise();
}
Dependency
org.mockito:mockito-core:2.21.0
Read about
Mockito#doNothing()
Mockito#spy(T)
void() functions change the state of a program. This can be done by modifying a variable, a file, a database, etc.
In your case you're writing to a logger. If this results in writing " Elephant make Sound" to a file then you can read that file and see if the data in the file includes your noisy elephant.
If it however doesn't involve anything you can check (i.e.: it simply displays the output on the console) then you might want to look at some form of dependency injection (DI) where you can set the output to a file or something else you can easily read.
It should be noted that you can bypass DI by mocking the object and checking the appropriate methods are getting called.
To test any method, the responsibility to be tested must be visible from the out side of the method by changing state of any variable.
Typically it is done by returning value from the method. But without that, it can be done in many ways by modifying something from outside of the method scope, in case you have any "problem" to return something from the method!
In your case, you only log some message. And your code is not really testable in a sense that it does not do something that is directly related to changing the state of any variable (Because you change the state of other resource other than variable, that is not directly accessible by your code. You have to write some code to read the changes from that external resource, hence makes your testing code dependent to the reading also. If you have some problem with reading, your test case will not pass and that does not go with the spirit of the unit testing. The main idea is to reduce the dependency on external codes or libraries as much as possible). But your code can be testable by doing a slight refactoring / shifting responsiblity like below:
String makeNoise() {
return "Elephant make Sound";
}
String perform(String day) {
if (day.equals("thursday") || day.equals("friday")) {
return makeNoise();
}
}
And then you shift the responsibility of logging the value returned from perform method to the one using it like below:
logger.info(perform(day));
You have various options depending on the tools you are willing to use and the depth your tests should have.
Partial Mocking with plain Java
Create a class (MockElephant) that extends from elephant, overwrite makeNoise so it counts the number of invocations. Use that class in your test to check that makeNoise was called the correct number of times
Partial Mocking with a Framework
You basically do the same as above but instead of manually coding the MockElephant you create it using some mocking framework. Makes the test much simpler, since you need less code. And it is easier to read. But if strange things happen it makes it harder to understand what is going on. In case of Mocking frameworks I think they are worth it.
The reasons why this is called Partial Mocking is that you mock only parts of a class (a single method in this case).
The alternative is to use Normal Mocks, which in your case seems feasible (but can become tough in legacy code).
Here you would inject the Logger as a dependency. For example you could create an additional constructor which allows you to provide the Logger. In your test you would then use a mocked Logger, which again counts it's invocations, probably along with the parameter it received and check that it has the expected values.
Again you can do that with a Mocking Framework or with plain old Java.
Is it a code smell to spy on an object that is being unit tested? For example say I have a LineCounter class whose job is to simply count the number of lines in a string. --
class LineCounter {
public int getNumLines(String string) {
String metadata = getStringMetadata(string);
// count lines in file
return numLines;
}
/** Expensive operation */
protected String getStringMetadata(String string) {
// do stuff with string
}
}
Now I want to write a JUnit 4 test for this to test the getNumLines method while mocking out the expensive getStringMetadata call. I decide to use Mockito's spy mechanism to have getStringMetadata return a dummy value.
class LineCounterTests {
#Test public void testGetNumLines() {
LineCounter lineCounterSpy = Mockito.spy(new LineCounter());
// Mock out expensive call to return dummy value.
Mockito.when(lineCounterSpy.getStringMetadata(Mockito.anyString()).thenReturn("foo");
assertEquals(2, lineCounterSpy.getNumLines("hello\nworld");
}
}
Is this a reasonable thing to do? I feel pretty weird testing a Spy object rather than the actual class, but I can't really think of a reason against it.
I will answer the question in two parts. First, yes it is code smell to mock or spy the class under test. That does not mean that it cannot be done correctly but that it is risk prone and should be avoided whenever possible.
WRT your specific example, I would see how the spy could be correctly used but that would be predicated on the assertion that you have elsewhere fully unit tested getStringMetadata. This then begs the question, if you have fully unit tested getStringMetadata elsewhere then you must know how to test it and therefore why not test getNumLines without the spy.
All this being said, millhouse makes a good point but either way you have to unit test the expensive code somewhere. His suggestion goes a long way to help isolate the expensive code and ensure that you only have to test / exercise it once.
In this situation, it is perfectly legitimate to stub the method that is called by the method under test. It is even the only way I can think of to test it in isolation. You just don't want to extract a single method into it's own class for the sole purpose of testing.
Beware of the side effects in the stubbed method though. It might not be sufficient to stub the returned value, if the stubbed method has side effects then you have to stub the side effects as well. It might even be a reason against it in some situations where the side effects are very complex, but that would most likely be an indication of a code smell in the implementation of the class under test itself.
To answer your question, I find it easy to find reasons for it, but hard to find reasons against it. It's the technique I use every day, it helps me split my implementation in small methods that are tested individually in complete isolation, and I haven't seen any limitation to it yet.
Perhaps I have completely fallen short in my search, but I cannot locate any documentation or discussions related to how to write a unit test for a Java class/method that in turn calls other non-private methods. Seemingly, Mockito takes the position that there is perhaps something wrong with the design (not truly OO) if a spy has to be used in order to test a method where mocking internal method calls is necessary. I'm not certain this is always true. But using a spy seems to be the only way to accomplish this. For example, why could you not have a "wrapper" style method that in turn relies on other methods for primitive functionality but additionally provides functionality, error handling, logging, or different branches dependent on results of the other methods, etc.?
So my question is two-fold:
Is it poorly designed and implemented code to have a method that internally calls other methods?
What is the best practice and/or approach in writing a unit test for such a method (assuming it is itself a good idea) if one has chosen Mockito as their mocking framework?
This might be a difficult request, but I would prefer for those who decide to answer to not merely re-publish the Mockito verbiage and/or stance on spies as I already am aware of that approach and ideology. Also, I've used Powermockito as well. To me, the issue here is that Mockito developed this framework where additional workarounds had to be created to support this need. So I suppose the question I am wanting an answer to is if spies are "bad", and Powermockito were not available, how is one supposed to unit test a method that calls other non-private methods?
Is it poorly designed and implemented code to have a method that internally calls other methods?
Not really. But I'd say that, in this situation, the method that calls the others should be tested as if the others where not already tested separately.
That is, it protects you from situations where your public methods stops calling the other ones without you noticing it.
Yes, it makes for (sometimes) a lot of test code. I believe that this is the point: the pain in writing the tests is a good clue that you might want to consider extracting those sub-methods into a separate class.
If I can live with those tests, then I consider that the sub-methods are not to be extracted yet.
What is the best practice and/or approach in writing a unit test for such a method (assuming it is itself a good idea) if one has chosen Mockito as their mocking framework?
I'd do something like that:
public class Blah {
public int publicMethod() {
return innerMethod();
}
int innerMethod() {
return 0;
}
}
public class BlahTest {
#Test
public void blah() throws Exception {
Blah spy = spy(new Blah());
doReturn(1).when(spy).innerMethod();
assertThat(spy.publicMethod()).isEqualTo(1);
}
}
To me, this question relates strongly to the concept of cohesion.
My answer would be:
It is ok to have methods (public) that call other methods (private) in a class, in fact very often that is what I think of as good code. There is a caveat to this however in that your class should still be strongly cohesive. To me that means the 'state' of your class should be well defined, and the methods (think behaviours) of your class should be involved in changing your classes state in predictable ways.
Is this the case with what you are trying to test? If not, you may be looking at one class when you should be looking at two (or more).
What are the state variables of the class you're trying to test?
You might find that after considering the answers to these types of questions, your code becomes much easier to test in the way you think it should be.
If you really need (or want) to avoid calling the lower-level methods again, you can stub them out instead of mocking them. For example, if method A calls B and C, you can do this:
MyClass classUnderTest = new MyClass() {
#Override
public boolean B() {return true;}
#Override
public int C() {return 0;}
};
doOtherCommonSetUp(classUnderTest);
String result = classUnderTest.A("whatever");
assertEquals("whatIWant", result);
I've used this quite a quite a bit with legacy code where extensive refactoring could easily lead to the software version of shipwright's disease: Isolate something difficult to test into a small method, and then stub that out.
But if the methods being called are fairly innocuous and don't requiring mocking, I just let them be called again without worrying that I am covering every path within them.
The real question should be:
What do I really want to test?
And actually the answer should be:
The behaviour of my object in response to outside changes
That is, depending on the way one can interact with your object, you want to test every possible single scenario in a single test. This way, you can make sure that your class reacts according to your expectations depending on the scenario you're providing your test with.
Is it poorly designed and implemented code to have a method that internally calls other methods?
Not really, and really not! These so called private methods that are called from public members are namely helper methods. It is totally correct to have helper methods!
Helper methods are there to help break some more complex behaviours into smaller pieces of reusable code from within the class itself. Only it knows how it should behave and return the state accordingly through the public members of your class.
It is unrare to see a class with helper methods and normally they are necessary to adopt an internal behaviour for which the class shouldn't react from the outside world.
What is the best practice and/or approach in writing a unit test for such a method (assuming it is itself a good idea) if one has chosen Mockito as their mocking framework?
In my humble opinion, you don't test those methods. They get tested when the public members are tested through the state that you expect out of your object upon a public member call. For example, using the MVP pattern, if you want to test user authentication, you shall not test every private methods, since private methods might as well call other public methods from an object on which depend the object under test and so forth. Instead, testing your view:
#TestFixture
public class TestView {
#Test
public void test() {
// arrange
string expected = "Invalid login or password";
string login = "SomeLogin";
string password = "SomePassword";
// act
viewUnderTest.Connect(login, password);
string actual = viewUnderTest.getErrorMessage;
// assert
assertEqual(expected, actual);
}
}
This test method describes the expected behaviour of your view once the, let's say, connectButton is clicked. If the ErrorMessage property doesn't contain the expected value, this means that either your view or presenter doesn't behave as expected. You might check whether the presenter subscribed to your view's Connect event, or if your presenter sets the right error message, etc.
The fact is that you never need to test whatever is going on in your private methods, as you shall adjust and bring corrections on debug, which in turn causes you to test the behaviour of your internal methods simultaneously, but no special test method should be written expressly for those helper method.
Lets say we have method to test in class A that calls method from class B. To test it we created mock for B and then verify if it was called. Is verify(...) enough for unit test or I need assert actual result of tested method?
Below is simplified example to clarify my concern:
public class StringWriterATest {
StringWriterB b = mock(StringWriterB.class);
#Test
public void stringWriterATest() {
StringBuffer sb = new StringBuffer();
StringWriterA a = new StringWriterA();
a.stringWriterB=b;
a.append(sb);
ArgumentCaptor<StringBuffer> argument = ArgumentCaptor.forClass(StringBuffer.class);
verify(b).append(argument.capture());
assertEquals("StringWriterA", ((StringBuffer)argument.getValue()).toString());
//do we really need this or above is enough for proper unit test of method a.append(sb);
//assertEquals("StringWriterA_StringWriterB", sb);
}
}
public class StringWriterA {
public StringWriterB stringWriterB;
public void append(StringBuffer sb) {
sb.append("StringWriterA");
stringWriterB.append(sb);
}
}
class StringWriterB {
public void append(StringBuffer sb) {
sb.append("StringWriterB");
}
}
Regards,
Max
There is never a need to mock a return value and verify an object at the same time.
Consider this:
StringWriterA is the class under test. Therefore you'll definitely want to use assertions to verify the behavior of this class. In order to do this, you mock out a dependency, StringWriterB.
You do not want to test StringWriterB in your test of StringWriterA, therefore any assertions of StringWriterB interactions in your test are in the wrong place.
You must assume that StringWriterB is behaving as expected. You either want to verify that StringWriterA called StringWriterB correctly (using verify()) or you want to mock its expected behavior and mock the return values.
If you mock, then the verify is implicit since the mocked return value will not be returned if the method is not called.
In your case, StringWriterA.append() does not return any value, so only a verify is even possible. That StringWriterB.append() also works should have a similar verify test in a stringWriterBTest of its own.
Note: It's nice to be explicit with tests. Since test methods are never called outside of a framework, there is never a need to type them out, so you can have much longer method names than in production code methods. A nice convention is:
<underTest>Should<Expected>[When]<Condition>()
i.e.
stringWriterAShouldAppendConstantAndDelegateToStringWriterB()
stringWriterAShouldThrowNullPointerExceptionWhenNullArgument()
When you have test failures in your build (continuous integration), then you don't have to hunt down what went wrong, the method name appears right by the failure and you can read it to know exactly what behavior must be fixed.
In your example, StringWriterB stores no state and the append method could easily be static. In that case then the call is purely a side effect and does not need to be tested.
However, I suspect your real code is much more complex. If there is a of another object accessing StringWriterB then you maye want to mock it out in case there are unexpected calls to it. You also may want to add the verify of B if you expect it to be expanded in the future -- possibly storing state from the append call and having accessors.
One thing to consider is what the purpose of the call to StringWriterA.append() is. If it's job is to append the string StringWriterAStringWriterB then that is what you should be testing and a mock is not necessary. How StringWriterA accomplishes that task is immaterial. If, however, part of its job is to also call the StringWriterB.append() method then a mock may will be necessary unless you want to test StringWriterB in A's test.
My rule of thumb WRT mocks is to use real objects until the wiring for the objects I'm not directly testing gets too hairy or too brittle. If I feel like a good portion of my tests are in actuality testing other objects then mocks would be a good idea.
First of all try to show somebody the test you wrote. It is hard to read in my opinion. You can't really tell from it what behaviour you are testing. A brief intro for you how to make it a bit more readable can be found here How to Write Clean, Testable Code .
Using argument captors is also a smell. Some examples how to avoid it can be found on this tdd blog.
To answer you question, verify is used to verify interactions between classes. It is used to drive the design of your code. The result (if needed) should be specified by a when or given at the beginning of your test.
Further information how to drive your design with mocks (when, given, verify, ...) and how mocks are different to stubs can be found here: Mocks are not stubs. That example uses JMock not Mockito for defining mocks, but it is very similar (it is about the concepts, not the details of implementation and libraries you use).