It is good practise to match mock objects widely but verify them precisely.
for example:
Using this:
when(myMock.has(any())).thenReturn(myValue);
Rather than:
when(myMock.has(eq("blah")).thenReturn(myValue);
Along with:
var result = myMethod();
assertThat(result, is(myValue));
Because it is making sure that it is always returned myValue regardless of the has method input.
There was a good explanation for this rule but I can not find it.
something along the lines: match widely and verify precisely.
It would be great if you can advise me about the name of the rule or some reference to it?
The explanation is quite simple: It will make your live easier.
Imagine the case that a caller will not call your method with "blah". In this case you rely on the mocking framework what will be returned, most likely null, zero or false. Your test will then run into a different direction or even fail with a NullpointerException. For other developers it will be hard to understand what went wrong here.
If you match widely, your test will continue as expected, but you should place a verification afterwards that makes the test fail with a clean reason. Developers tend to omit the verification step, wich renders the test useless quite often.
Usually there is no reason to match on a precise parameter value, except for the case when you want your mock to act differently on two values.
Most frameworks provide methods for the method call verification, e.g. Mockito:
#Mock
private Repository repository;
#Test
private void testReadData() {
Mockito.when(repository.findById(any())).thenReturn(yourEntity);
// run your test
Mockito.verify(repository).findById("foo");
}
Related
I'm in a java context and am using Mockito (but I'm not bound to it) for basic mocking needs.
I have code like this
public class AuditInfoSerializer {
[..]
public Map<String, Object> doStuff(Object a) {
doOtherStuff("hello", new TempClass(someField, <someParams>));
doOtherStuff("world", new TempClass(someField, <otherParams>));
return getResult();
}
}
and in a test I want to verify that there are two instances of TempClass created with the correct set of parameters when I call the doStuff method.
Is this possible somehow?
You don't want to verify temporary data on the object under test. You want to mock dependencies and assert the object under test behavior : that is with this input you have this output.
Mock verifying is a trade off for methods to mock that return nothing but only produces side effect.
So use it only as you don't have the choice.
In your unit test, what you want is asserting what the method to test returns that is getResult().
Do that with Assert.assertEquals(...) not with Mockito.verify(...).
For the most part I agree with #davidxxx's point about the mock verifying tradeoff. If you have a setup that allows you to make assertions about an outcome like a map that is created as a result, go for it!
From an API perspective doStuff is a simple straight-forward method: Throw something at it, get something back. The information you are interested in will be contained in the map (this would be your assertion).
There is a lot going on under the hood before doStuff returns something. Many people tend to want to break up encapsulation when testing stuff. They are constantly looking for ways to uncover what is going on behind the curtains. I believe that's totally natural. But of course, it's also an anti pattern. It doesn't matter what tool you (mis)use to break natural boundaries (mocking frameworks, custom reflection, "back doors" in your code base, etc). It is always wrong. As #Michael already pointed out, the call to doOtherStuff is indeed an implementation detail. Take the perspective of client code that makes a call to doStuff. Is it interested in how the map is created? I doubt it. This should also be your testing perspective.
One last thing about using verification in tests. I would like to mitigate the trade off statement. I really don't like the generalization here. Verification is not always the less attractive choice compared to real assertions:
// Valid test without any verifaction
#Test
void method_foo_returns_gibberish (#Mock SomeInput someInput) {
// Maybe this is just to prevent an NPE ...
when(someInput.readStuff()).thenReturn("bla");
assertEquals("gibberish", Foo.foo(someInput));
}
// Test made possible by verification
#Test
void method_foo_is_readonly (#Mock SomeInput someInput) {
Foo.foo(someInput);
verify(someInput.readStuff());
verifyNoMoreInteractions(mockedList);
}
This is just the most obvious example that I could think of. There is a fraction of BDD geniuses who strive to build their whole architecture around verification driven tests! Here is an excellent article by Martin Fowler
When talking about testing, most of the time, there is no black and white. Using mocks and verification means writing different tests.
As always, it's about picking the right tool.
I'm having doubts about if I should create tests that have many mock objects or not.
I recently read When should I mock? and I'm feeling confused.
Let's take a look at a method I have (it is just to illustrate the problem)
#Override
protected void validate() throws WTException {
Either<ImportError, RootFinderResult> rootPart = getDataValidator().getRootPart();
if (rootPart.isLeft()) {
addValidationMessage(ROOT_PART_NOT_FOUND);
} else if (rootPart.isRight()) {
getObjectsToValidate().forEach(Lambda.uncheckedBiConsumer((part, epmDocuments) -> {
LocalizableMessage rootRevision = getRevision(part);
Optional<EPMDocument> wrongRevisionEPM = epmDocuments.stream()
.filter(epmDocument -> !isSameRevision(rootRevision, epmDocument))
.findAny();
wrongRevisionEPM.ifPresent(epmDocument -> addValidationMessage("blabla"));
}));
}
}
All of the following methods need to have a connection to a server in order to work, otherwise they will throw errors
getDataValidator().getRootPart();
getRevision(part)
!isSameRevision(rootRevision, epmDocument))
In addition I can't create 'real' objects of part or epm documents. This also requires to have a connection to a server.
So at this point, what I really want to test is actually logic of this part of code
Optional<EPMDocument> wrongRevisionEPM = epmDocuments.stream()
.filter(epmDocument -> !isSameRevision(rootRevision, epmDocument))
.findAny();
wrongRevisionEPM.ifPresent(epmDocument -> addValidationMessage("blabla"));
But to test it I need to mock really many objects
#Spy
#InjectMocks
private SameRevision sameRevision;
#Mock
private WTPartRelatedObjectDataValidator wTPartRelatedObjectDataValidator;
#Mock
private ValidationEntry validationEntry;
#Mock
private WTPart rootPart1, rootPart2;
#Mock
private EPMDocument epmDocument1, epmDocument2, epmDocument3;
#Mock
private Either<ImportError, RootFinderResult> rootPart;
#Mock
private LocalizableMessage rootPartRevisionOne, rootPartRevisionTwo;
so finally I can test the logic:
#Test
#DisplayName("Should contain error message when part -> epms revisions are not the same")
void shoulHaveErrorMessagesWhenDifferentRevisions() throws Exception {
doReturn(getMockObjectsToValidate()).when(sameRevision).getObjectsToValidate();
doReturn(rootPart).when(liebherrWTPartRelatedObjectDataValidator).getRootPart();
doReturn(false).when(rootPart).isLeft();
doReturn(true).when(rootPart).isRight();
doReturn(rootPartRevisionOne).when(sameRevision).getRevision(rootPart1);
doReturn(rootPartRevisionTwo).when(sameRevision).getRevision(rootPart2);
doReturn(true).when(sameRevision).isSameRevision(rootPartRevisionOne, epmDocument1);
doReturn(false).when(sameRevision).isSameRevision(rootPartRevisionOne, epmDocument2);
doReturn(true).when(sameRevision).isSameRevision(rootPartRevisionTwo, epmDocument3);
validationEntry = sameRevision.call();
assertEquals(1, validationEntry.getValidationMessageSet().size());
}
where
doReturn(rootPart).when(liebherrWTPartRelatedObjectDataValidator).getRootPart();
doReturn(false).when(rootPart).isLeft();
doReturn(true).when(rootPart).isRight();
doReturn(rootPartRevisionOne).when(sameRevision).getRevision(rootPart1);
doReturn(rootPartRevisionTwo).when(sameRevision).getRevision(rootPart2);
can be moved to #BeforeEach.
At last, I have my test and it works. It validates what I wanted to be validated but in order to come to this point I had to put a lot of effort to come through the whole API which needs interactions with a server.
What do you guys think, is it worth it to create tests like this? I guess this is a wide-open topic 'cause many newbies that try to come into the 'test world' will have similar a problem, so please do not close the topic because of opinion-based judgement and give your feedback on this topic.
You are right. It is a big effort to mock all those dependencies. Let me get over a few points that may make things clearer:
Treat writing tests like an investment: So yes, sometimes it is more effort to write the test than to write the actual code. However, you will thank yourself later when you introduce a bug and the tests can catch it. Having good tests gives you confidence when modifying your code that you didn't break anything, and if you did, your tests will find the issue. It pays off over time.
Keep your test focused on a specific class. Mock the rest: When you mock everything except the class under test, you can be sure that when a problem occurs that it is from the class under test, and not from one of its dependencies. This makes troubleshooting a lot easier.
Think of testability when writing new code: Sometimes it may not be avoidable to have a complicated piece of code which is hard to test. However, generally, this situation can be avoided by keeping the number of dependencies you have to a minimum and writing testable code. For example, if a method needs 5 or 6 more dependencies to do its job, then probably that method is doing too much and could be broken down. Same thing can be said on a class level, modules, etc..
You should mock other dependencies on which your class which going to be tested relies and set up behavior which you need .
This needs to be done to test your method isolated and not dependent on thrirdparty classes
You can write private void methods which can contain your mock behavior and use them in tests ,
In #BeforeEach annotated method you could mock behavior which wil be same in all tests or mock the same mocking behavior for across all tests
In your method which is void you can have spy objects which can be veereified if they were called like Mockito.verify()
Yes, it is quite time investing when you have to mock so many things.
In my opinion, if you add some value when testing something, it is worth testing, the problem could be of course how much time would you consume.
In your specific case, I would test on different "layers".
For example, the methods:
getDataValidator().getRootPart();
getRevision(part)
!isSameRevision(rootRevision, epmDocument))
They can be independently tested and in your case just mock their result, meaning that you don't really care about the parameters there, you just care what happens in case of a certain return value.
So, on one layer you really test the functionality, on the next layer you just mock the result you need, in order to test the other functionality.
I hope it is more clear now...
In my understanding, code testing is to test whether results are right, like a calculator, I need to write a test case to verify if the result of 1+1 is 2.
But I have read many test cases about verifying the number of times a method is called. I'm very confused about that. The best example is what I just saw in Spring in Action:
public class BraveKnight implements Knight {
private Quest quest;
public BraveKnight(Quest quest) {
this.quest = quest;
}
public void embarkOnQuest() {
quest.embark();
}
}
public class BraveKnightTest {
#Test
public void knightShouldEmbarkOnQuest() {
Quest mockQuest = mock(Quest.class);
BraveKnight knight = new BraveKnight(mockQuest);
knight.embarkOnQuest();
verify(mockQuest, times(1)).embark();
}
}
I really have no idea about why they need to verify the embark() function is called one time. Don't you think that embark() will certainly be invoked after embarkOnQuest() is called? Or some errors will occur, and I will notice error messages in the logs, which show the error line number, that can help me quickly locate the wrong code.
So what's the point of verifying like above?
The need is simple: to verify that the correct number of invocations were made. There are scenarios in which method calls should not happen, and others in which they should happen more or less than the default.
Consider the following modified version of embarkOnQuest:
public void embarkOnQuest() {
quest.embark();
quest.embarkAgain();
}
And suppose you are testing error cases for quest.embark():
#Test
public void knightShouldEmbarkOnQuest() {
Quest mockQuest = mock(Quest.class);
Mockito.doThrow(RuntimeException.class).when(mockQuest).embark();
...
}
In this case you want to make sure that quest.embarkAgain is NOT invoked (or is invoked 0 times):
verify(mockQuest, times(0)).embarkAgain(); //or verifyZeroInteractions
Of course this is one other simple example. There are many other examples that could be added:
A database connector that should cache entries on first fetch, one can make multiple calls and verify that the connection to the database was called just once (per test query)
A singleton object that does initialization on load (or lazily), one can test that initialization-related calls are made just once.
Consider the following code:
public void saveFooIfFlagTrue(Foo foo, boolean flag) {
if (flag) {
fooRepository.save(foo);
}
}
If you don't check the number of times that fooRepository.save() is invoked , then how can you know whether this method is doing what you want it to?
This applies to other void methods. If there is no return to a method, and therefore no response to validate, checking which other methods are called is a good way of validating that the method is behaving correctly.
Good question. You raise a good point that mocking can be overly circuitous when you can just check the results. However, there are contexts where this does lead to more robust tests.
For example, if a method needs to make a call to an external API, there are several problems with simply testing the result:
Network I/O is slow. If you have many checks like this, it will slow down your test case
Any round-trip like this would have to rely on the code making the request, the API, and the code interpreting the API's response all to work correctly. This is a lot of failure points for a single test.
If something stupid happens and you accidentally make multiple requests, this could cause performance issues with your program.
To address your sub-questions:
Don't you think that embark() will certainly be invoked after embarkOnQuest() called?
Tests also have value in letting you refactor without worry about breaking things. This is obvious now, yes. Will it be obvious in 6 months?
I really have no idea about why they need to verify the embark()
function is called one time
Verifying an invocation on a mock for a specific number of times is the standard way how Mockito works as you invoke Mockito.verify().
In fact this :
verify(mockQuest, times(1)).embark();
is just a verbose way to write :
verify(mockQuest).embark();
In a general way, the verification for a single call on the mock is what you need.
In some uncommon scenarios you may want to verify that a method was invoked a specific number of times (more than one).
But you want to avoid using so specific verifications.
In fact you even want to use verifying as few as possible.
If you need to use verifying and besides the number of invocation on the mock, it generally means two things : the mocked dependency is too much coupled to the class under
test and or the method under test performs too many unitary tasks that produce only side effects.
The test is so not necessary straight readable and maintainable. It is like if you coded the mock flow in the verifying invocations.
And as a consequence it also makes the tests more brittle as it checks invocation details not the overall logic and states.
In most of cases, a refactoring is the remedy and cancel the requirement to specify a number of invocation.
I don't tell that it is never required but use it only as it happens to be the single decent choice for the class under test.
I have to deal with a legacy application that has no tests. So before I begin refactoring I want to make sure everything works as it is.
Now imagine the following situation:
public SomeObject doSomething(final OtherObject x, final String something) {
if(x == null) throw new RuntimeException("x may not be null!");
...
}
Now I want to test that null check, so to be sure it works and I don't lose it once I refactor.
So I did this
#Test(expected = RuntimeException.class)
public void ifOtherObjectIsNullExpectRuntimeException() {
myTestObject.doSomething(null, "testString");
}
Now, this works of course.
But instead of "testString" I'd like to pass in a random String.
So I tried with:
#Test(expected = RuntimeException.class)
public void ifOtherObjectIsNullExpectRuntimeException() {
myTestObject.doSomething(null, Mockito.anyString());
}
But this is not allowed., as I get org.mockito.exceptions.misusing.InvalidUseOfMatchersException:
... You cannot use argument matchers outside of verifications or stubbing
I do understand the meaning of this, but I wonder whether I can still manage to do what I want without parameterizing my test or the like.
The only libraries I may use are Junit, AssertJ, Mockito and Powermock.
Any ideas?
Tests should be deterministic. Using random values in a test makes it difficult to reproduce behavior when debuging a failed test. I suggest that you just create a String constant for the test such as "abcdefg".
Well, like Mockito is trying to tell you via that exception, that's not really how you'd use anyString. Such methods are only to be used by mocks.
So, why not try testing with an actual random string? My personal favorite in such a scenario: java.util.UUID.randomUUID().toString(). This will virtually always generate a brand new string that has never been used for your test before.
I'd also like to add that if you are writing tests for your SomeObject class that you should avoid mocking SomeObject's behavior. From your example, you weren't exactly doing that, but it looked like you might be going down that route. Mock the dependencies of the implementation you're trying to test, not the implementation itself! This is very important; otherwise you aren't actually testing anything.
You are mixing up concepts here.
All those "mocking" helpers like anyString() are meant to be used when configuring a mock object.
But when you check your testing code:
#Test(expected = RuntimeException.class)
public void ifOtherObjectIsNullExpectRuntimeException() {
myTestObject.doSomething(null, "testString");
}
you will find: there is absolutely no mocking involved for this test. You simply can't use those Mockito calls in that place; because "there is no Mockito" in that place.
And just for the record - no need to go overboard here anyway. Your logic is very clear here: when the first argument is null, then you throw that exception. Thus it really doesn't matter at all what comes in as second argument. So thinking for an hour how to test null with any second argument is, well, in my eyes: waste of your time.
Final hint: there is java.lang.Objects
And that class has a nice check for null, so my production code only looks like
public SomeObject doSomething(final OtherObject x, final String something) {
Objects.requireNonNull(otherObject, "otherObject must not be null");
Objects.requireNonNull(something, "something must not be null");
Only difference there: requires... throws NullPointerExceptions
Final finally: some people suggest to put final on every parameter; but I wouldn't do that. It adds no value in 99% of all cases. It just means that you have more code to read; for no good reasons. But that is a question of style.
EDIT on the comment about having a test to check for potential future changes: you shouldn't do that:
To a certain degree, how your input is verified is an implementation detail. You don't test for implementation details. In other words:
Your method has a certain contract (that you, for example specify informally by writing a javadoc that says "throws NPE on null input"). Your tests should verify exactly that current contract. And the contract is: throws if first argument is null.
And maybe another point of view; as I still think you are wasting your time here! You should make sure that all your interfaces are clear, easy to understand, and easy to use. That they allow users of your code to do the right thing easily; and prevent him from doing wrong things. That is what you should focus on - the quality of your interfaces as a whole!
So instead of worrying how you could write a test for potential future changes; just make sure that your code base is overall consistent.
Well i do not have much knowledge of mockito but you can always create your own random string generator. maybe that can work and u can modify more types of inputs in it
Suppose that you have the following logic in place:
processMissing(masterKey, masterValue, p.getPropertiesData().get(i).getDuplicates());
public StringBuffer processMissing(String keyA, String valueA, Set<String> dupes) {
// do some magic
}
I would like to write a jUnit test for processMissing, testing its behavior in event dupes is null.
Am i doing the right thing here? Should I check how method handles under null, or perhaps test method call to make sure null is never sent?
Generally speaking, what is the approach here? We can't test everything for everything. We also can't handle every possible case.
How should one think when deciding what tests to write?
I was thinking about it as this:
I have a certain expectation with the method
Test should confirm define my expectation and confirm method works under that condition
Is this the right way to think about it?
Thanks and please let me know
First, define whether null is a valid value for the parameter or not.
If it is, then yes, definitely test the behavior of the method with null.
If it is not, then:
Specify that constraint via parameter documentation.
Annotate that constraint on the parameter itself (using an annotation compatible with the tool below).
Use a static analysis tool to verify that null is never passed.
No unit test is required for the invalid value unless you're writing code to check for it.
The static analysis tool FindBugs supports annotations such as #NonNull, with some limited data-flow analysis.
I personally think it would be unnecessarily expensive within large Java codebases to always write and maintain explicit checks for NULL and corresponding, non-local unit tests.
If you want to ensure that people don't call your API with a null argument you may want to consider using annotations to make this explicit, JSR 305 covers this, and its used in Guava. Otherwise you're relying on users reading javadoc.
As for testing, you're spot on in that you can't handle every possible case, assuming you don't want to support null values, I'd say that you may want to throw an IllegalArguemntException rather than a NullPointerException so you can be explicit about what is null, then you can just test for that exception being thrown - see JUnit docs.