While unit-testing, is there a way to enable code-coverage analysis only during some steps?
I wish to measure the code-coverage of the assert part of a test.
The tools that I currently use don't make the difference if the line is executed during the Action-part of the test, or during the assert-part.
Accordingly, I cannot check if all the getter of my beans are read by the assert method.
Ideally I want to activate the covering measure only during the execution of some of my methods.
A sample of code:
//method to test
void runToTest(Bean myBean){
myBean.setA(1);
myBean.setB(2);
aString=myBean.getA()+myBean.getB();
}
#Test
void should_check_all_field(){
myBean=new Bean()
myService.runToTest(myBean);
assertMethode();
}
void assertMethod(){
Assert.assertNotNull(myBean.getA())
}
Currently, the tools I use report than getA and getB are covered. I want a way to detect that getB hasn't been read by the assertMethod.
I use Java and IntelliJ and my test class follows the Arrange-Act-Assert syntax.
No, I am not aware of any tool that supports this functionality.
The code coverage expresses which portion of your source code was run through by the tests. By checking the result of a get method you are testing also the correctness of the getter. Thus, a test checks more than what is directly covered by assertion statements.
What you are probably looking for is mutation testing. You want to see if the code is not just executed by the test but as well if the test is capable of detecting an incorrect behaviour of the code. Mutation testing introduces faults into the source code and analyzes whether the tests can reveal these.
See also: https://stackoverflow.com/a/27400613/584532
Correct me if I misunderstood something, you are doing JUnit test and then you are checking coverage as part of the test? It seems like manual step to me, which is something you do not want in JUnit testing, right?
In the following example I used EasyMock, but you can use whatever you prefer ;-)
#Test
public void testGetters() {
Bean bean = EasyMock.createMock(Bean.class);
// set your expectations
EasyMock.expect(bean.getA()).andReturn(0).times(1);
EasyMock.replay(bean);
// use the bean
bean.getA(); // if this is commented, test fails
// verify your expectations
EasyMock.verify(bean);
}
Is this similar to what you want to achieve?
I used maven project and my dependencies are
<dependencies>
<dependency>
<groupId>org.easymock</groupId>
<artifactId>easymock</artifactId>
<version>3.3.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
</dependencies>
Related
I want to mock a static method in JUnit 5. But unfortunately, JUnit 5 doesn’t support Mockito. Is there another method to achieve the same other than reverting back to JUnit 4?
From Mockito 3.4.0 (2020-07-10), it is possible to mock static methods out of the box even in JUnit 5, without any extension.
In the documentation, you can find an example: 48. Mocking static methods (since 3.4.0)
Important note: You need to use inline mock maker. So the dependency to use is not the core one:
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
<version>3.4.6</version>
<scope>test</scope>
</dependency>
Example:
Class under test:
package teststatics;
public class FooWithStatics {
public static Long noParameters() {
return System.currentTimeMillis();
}
public static String oneParameter(String param1) {
return param1.toUpperCase();
}
}
Test class:
package teststatics;
import org.junit.jupiter.api.Test;
import org.mockito.MockedStatic;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*;
public class FooWithStaticsTest {
#Test
void testStatic() {
// Before mock scope, usual behavior.
assertNotEquals(0L, FooWithStatics.noParameters());
assertNotEquals("yyy", FooWithStatics.oneParameter("xxx"));
// Mock scope
try (MockedStatic mocked = mockStatic(FooWithStatics.class)) {
// Mocking
mocked.when(FooWithStatics::noParameters).thenReturn(0L);
mocked.when(() -> FooWithStatics.oneParameter("xxx")).thenReturn("yyy");
// Mocked behavior
assertEquals(0L, FooWithStatics.noParameters());
assertEquals("yyy", FooWithStatics.oneParameter("xxx"));
// Verifying mocks.
mocked.verify(times(1), FooWithStatics::noParameters);
mocked.verify(times(1), () -> FooWithStatics.oneParameter("xxx"));
}
// After mock scope returns to usual behavior.
assertNotEquals(0L, FooWithStatics.noParameters());
assertNotEquals("yyy", FooWithStatics.oneParameter("xxx"));
}
}
The short answer is no, as the Mockito team is done with their work and is waiting for the JUnit team for an extension and are discussing here a lot.
With some overhead you can: As JUnit 5 provides support for running legacy JUnit 4, and there you can use Mockito. So you can create tests in Junit4 for these cases:
A sample project for migration setup with gradle and with mvn. From there I am using PowerMock 2.0 beta with Mockito 2.
The reason why Mockito doesn't provide static methods mocking at the moment is because of the common belief that static method shouldn't need to be mocked.
However, there is an open item for Mockito here that discusses the issue.
While this doesn't answer your question, in general it tells you why you shouldn't need the feature at all or will allow you to join the conversation with your ideas.
Make sure to have mockito-inline dependency in your POM file
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
<version>3.6.28</version>
<scope>test</scope>
</dependency>
In my case I had to test scenario where exception thrown static method encode() of URLEncoder Class, so for that
try (MockedStatic theMock = mockStatic(URLEncoder.class)) {
theMock.when(() -> URLEncoder.encode("Test/11", StandardCharsets.UTF_8.toString()))
.thenThrow(UnsupportedEncodingException.class);
when(restClient.retrieveByName("Test%2F11")).thenReturn(null);
Assertions.assertThrows(ResponseStatusException.class, ()->service.retrieveByName("Test/11"));
}
We can mock a static method by JMockit.
JMockit is used for mocking the external dependencies outside the test boundary, similar to Mockito and other such mocking libraries.
The most important feature of JMockit is that it lets us mock anything, even the things that are hard to mock with other libraries such as constructors, static and final methods. It even allows mocking the member fields and initialization blocks as well.
Follow the below steps to enable JMockit:
The JMockit artifact is located in the central Maven repository, add the JMockit dependency in pom.xml
<!-- https://mvnrepository.com/artifact/org.jmockit/jmockit -->
<dependency>
<groupId>org.jmockit</groupId>
<artifactId>jmockit</artifactId>
<version>1.49</version>
<scope>test</scope>
</dependency>
Mock the Class method in TestClass:
public class TestClass{
#Test
public void testMethod() {
new MockUp<ClassName>(){
#Mock
//mock the method here
};
}
}
Follow the tutorial to know more about how to use the JMockit.
I have a problem with using mockito-junit-jupiter and mockito-inline together in my groovy code, for testing static classes and methods. Hopefully someone can help me.
My current situation is: I am using Groovy, Junit5 and Mockito.
The important parts in my pom.xml are the following:
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>2.5.8</version>
<type>pom</type>
<exclusions>
<exclusion>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-testng</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Test dependencies -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>${junit5.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>3.15.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<version>3.8.0</version>
<scope>test</scope>
</dependency>
My Tests are working fine. Now I heard, that I can write tests for static classes and methods. Therefor I read a few posts and in all of them they tell me, that I need to add following dependency on top in my pom and i can test statics:
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-inline</artifactId>
<version>3.8.0</version>
<scope>test</scope>
</dependency>
When I do that, I can use Mockito.mockStatic in new tests, but now all my other tests, where I use Mockito are broken, as following error message appears: org.mockito.exceptions.misusing.InvalidUseOfMatchersException
But these lines in which I allegedly use wrong argument matchers, I didnt change and they still look totally fine for me. For example: Mockito.when(pipelineConfigLoaderMock.loadResource(Mockito.any(ResourceDefinition.class))).thenThrow(new LoadingResourceFailedException("Mocked Exception"))
Can someone tell me how to use mockito-junit-jupiter along with mockito-inline in groovy? Do i use wrong dependencies or wrong versions which are not compatible together? The exception InvalidUseOfMatchersException, that I receive here, doesn't seem to show the real problem. I already tried groovy 2.5.14 as well, but nothing changed.
Thx in advance
Edit:
Here is a example of an unit-test, which works before I added 'mockito-inline' to my pom:
#Test
#DisplayName("getMyself - handle 404 NOT_FOUND")
void testGetMyself_HandleNotFoundResponse() {
when(simpleRestClientMock.execute(any(RequestEntity.class))).thenAnswer(new Answer<ResponseEntity>() {
#Override
ResponseEntity answer(InvocationOnMock invocation) {
return new ResponseEntity("AnyResponse", HttpStatus.NOT_FOUND)
}
})
assertThatThrownBy({ ->
underTest.getMyself()
}).isInstanceOf(RuntimeException.class).hasMessage("Unknown status: 404 NOT_FOUND")
}
After adding the dependency to the pom I receive following error:
org.mockito.exceptions.misusing.InvalidUseOfMatchersException:
Invalid use of argument matchers!
0 matchers expected, 1 recorded:
-> at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:55)
This exception may occur if matchers are combined with raw values:
//incorrect:
someMethod(anyObject(), "raw String");
When using matchers, all arguments have to be provided by matchers.
For example:
//correct:
someMethod(anyObject(), eq("String by matcher"));
at com.xxx.xxx.cda.core.http.client.SimpleRestClient.getMetaClass(SimpleRestClient.groovy)
at com.xxx.xxx.cda.pipeline.adapter.JiraAdapterTest.testGetMyself_HandleNotFoundResponse
But adding mockito-inline to my pom, enables me to write an unit-test for static classes, that works, for example:
#Test
void approvalHelperThrowsTimeoutExceptionWithout() {
ApprovalHelper underTest = new ApprovalHelper()
underTest = new ApprovalHelper()
OffsetDateTime startDate = OffsetDateTime.now()
OffsetDateTime stopDate = startDate.minusHours(1)
long incrementer = 0
def closure = { Approval approval ->
incrementer++
return false
}
Mockito.mockStatic(ApprovalHelper).with {theMock ->
theMock.when({ ->
ApprovalHelper.createAndExecTimeoutCommand(Mockito.any(OffsetDateTime.class),
Mockito.any(OffsetDateTime), Mockito.any(Closure.class)) })
.thenThrow(new TimeoutException("Mocked Timeout Exception"))
underTest.aroundApproval(TimeDependent.EMPTY, startDate, stopDate, closure)
assertThat(incrementer).isEqualTo(0L)
}
}
Without 'mockito-inline' dependency I would receive following error:
org.mockito.exceptions.base.MockitoException:
The used MockMaker SubclassByteBuddyMockMaker does not support the creation of static mocks
Mockito's inline mock maker supports static mocks based on the Instrumentation API.
You can simply enable this mock mode, by placing the 'mockito-inline' artifact where you are currently using 'mockito-core'.
Note that Mockito's inline mock maker is not supported on Android.
I hope these examples are helpful to explain my problem.
Best regards
It is too late for your problem but still wanted to put this answer here so that someone else like me might find an answer for their problem.
According to Mockito documentation, Mockito-core and Mockito-inline cannot co-exist in a project. Your mockito-junit-jupiter dependency already depends on mockito-core, thus indirectly causing Mockito-core and Mockito-inline co-exist. If you can eliminate mockito-junit-jupiter dependency and leave only mockito-inline, hopefully it will solve the problem.
I am working with JUnit 5.1.0 in a Maven project. I have IntelliJ IDEA 2018.3.2 (Community Edition). I would like to use the #Tag annotation to describe my tests and be able to filter by those.
The point is that in edit configurations | Configuration | Test Kind there is no "Tag (JUnit 5)" option like I have seen in many examples. Only the option "Tags" is provided.
When I use the option mentioned, I set the field Tag expression to "slow" (without the quotes) (I have one test already tagged like #Tag("slow")).
The point is that no test is found (the test is marked with #Test).
#Test
#Tag("slow")
public void testNotBlank() {
// Given
String excectedText = "";
// When
String text = Checks.notBlank(excectedText, "error");
// Then
assertEquals("", Checks.notBlank(text, "error"));
}
The pom dependency
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.1.0</version>
<scope>test</scope>
</dependency>
First of all, I expected to have the option "Tags (JUnit 5)" and I don't. Then, I expected that selecting "Tags" and giving the expression "slow", the test testNotBlank would be executed.
You can execute tagged tests using the following Test Configuration:
This can be created using the following steps:
Run/Debug configurations > Add new configuration > Junit
Select test kind Tags and varied configurations as to how you want to execute. e.g. I am executing the test tagged with "fast" N times.
Where the test class includes:
public class TaggedTests {
#Test
#Tag(value = "slow")
public void sample() {
System.out.println("slow-test");
}
#Test
#Tag(value = "fast")
public void fastSample() {
System.out.println("fast-test");
}
}
and execution outputs:
fast-test
fast-test... 10 times
The reason why it is not working is because I was using the #Test annotation from JUnit 4 instead of using the one from JUnit 5. Because of that, filtering wasn't working even if JUnit Vintage was imported in the pom.
1)Configuration in Intellij Junit Runner
Please choose the Tags in the highlighted select box and add your tag to test the tagged test methods alone
2)Configuration using maven pom.xml
Add the above configuration at the level and your desire tags inside groups tag separated by a comma as shown in the picture with two tags Add and Multiply.
How can I use injection with Mockito and JUnit 5?
In JUnit4 I can just use the #RunWith(MockitoJUnitRunner.class) Annotation. In JUnit5 is no #RunWith Annotation?
There are different ways to use Mockito - I'll go through them one by one.
Manually
Creating mocks manually with Mockito::mock works regardless of the JUnit version (or test framework for that matter).
Annotation Based
Using the #Mock-annotation and the corresponding call to MockitoAnnotations::initMocks
to create mocks works regardless of the JUnit version (or test framework for that matter but Java 9 could interfere here, depending on whether the test code ends up in a module or not).
Mockito Extension
JUnit 5 has a powerful extension model and Mockito recently published one under the group / artifact ID org.mockito : mockito-junit-jupiter.
You can apply the extension by adding #ExtendWith(MockitoExtension.class) to the test class and annotating mocked fields with #Mock. From MockitoExtension's JavaDoc:
#ExtendWith(MockitoExtension.class)
public class ExampleTest {
#Mock
private List list;
#Test
public void shouldDoSomething() {
list.add(100);
}
}
The MockitoExtension documentation describes other ways to instantiate mocks, for example with constructor injection (if you rpefer final fields in test classes).
No Rules, No Runners
JUnit 4 rules and runners don't work in JUnit 5, so the MockitoRule and the Mockito runner can not be used.
Use Mockito's MockitoExtension. The extension is contained in a new artifact mockito-junit-jupiter:
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<version>4.5.1</version>
<scope>test</scope>
</dependency>
It allows you to write tests as you would have with JUnit 4:
import org.mockito.junit.jupiter.MockitoExtension;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
#ExtendWith(MockitoExtension.class)
class MyTest {
#Mock
private Foo foo;
#InjectMocks
private Bar bar; // constructor injection
...
}
There are different ways to do but the cleaner way and that also respects the JUnit 5 philosophy is creating a org.junit.jupiter.api.extension.Extension for Mockito.
1) Creating mocks manually makes lose the benefit of additional Mockito checks to ensure you use correctly the framework.
2) Calling MockitoAnnotations.initMocks(this) in every test classes is boiler plate code that we could avoid.
And making this setup in an abstract class is not a good solution either.
It couples every test classes to a base class.
If then you need a new base test class for good reasons, you finish with a 3- level class hierarchy. Please avoid that.
3) Test Rules is a JUnit 4 specificity.
Don't even think of that.
And the documentation is clear about that :
However, if you intend to develop a new extension for JUnit 5 please
use the new extension model of JUnit Jupiter instead of the rule-based
model of JUnit 4.
4) Test Runner is really not the way to extend the JUnit 5 framework.
JUnit 5 simplified the hell of the Runners of JUnit 4 by providing an extension model for writing tests thanks to JUnit 5 Extensions.
Don't even think of that.
So favor the org.junit.jupiter.api.extension.Extension way.
EDIT : Actually, Mockito bundles a jupiter extension : mockito-junit-jupiter
Then, very simple to use :
import org.mockito.junit.jupiter.MockitoExtension;
#ExtendWith(MockitoExtension.class)
public class FooTest {
...
}
Here is an addition to the excellent answer of Jonathan.
By adding as dependency the mockito-junit-jupiter artifact, the use of #ExtendWith(MockitoExtension.class) produced the following exception as the test is executed :
java.lang.NoSuchMethodError:
org.junit.platform.commons.support.AnnotationSupport.findAnnotation(Ljava/util/Optional;Ljava/lang/Class;)Ljava/util/Optional;
THe problem is that mockito-junit-jupiter depends on two independent libraries.
For example for mockito-junit-jupiter:2.19.0 :
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>2.19.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.1.0</version>
<scope>runtime</scope>
</dependency>
The problem was I used junit-jupiter-api:5.0.1.
So as junit-jupiter-api moves still often in terms of API, make sure you depend on the same version of junit-jupiter-api that mockito-junit-jupiter depends on.
You have to use the new #ExtendWith annotation.
Unfortunately there is no extension released yet.
On github you can see a beta implementation for the extension. as a example demo test.
How can I use injection with Mockito and JUnit 5?
In JUnit4 I can just use the #RunWith(MockitoJUnitRunner.class) Annotation. In JUnit5 is no #RunWith Annotation?
There are different ways to use Mockito - I'll go through them one by one.
Manually
Creating mocks manually with Mockito::mock works regardless of the JUnit version (or test framework for that matter).
Annotation Based
Using the #Mock-annotation and the corresponding call to MockitoAnnotations::initMocks
to create mocks works regardless of the JUnit version (or test framework for that matter but Java 9 could interfere here, depending on whether the test code ends up in a module or not).
Mockito Extension
JUnit 5 has a powerful extension model and Mockito recently published one under the group / artifact ID org.mockito : mockito-junit-jupiter.
You can apply the extension by adding #ExtendWith(MockitoExtension.class) to the test class and annotating mocked fields with #Mock. From MockitoExtension's JavaDoc:
#ExtendWith(MockitoExtension.class)
public class ExampleTest {
#Mock
private List list;
#Test
public void shouldDoSomething() {
list.add(100);
}
}
The MockitoExtension documentation describes other ways to instantiate mocks, for example with constructor injection (if you rpefer final fields in test classes).
No Rules, No Runners
JUnit 4 rules and runners don't work in JUnit 5, so the MockitoRule and the Mockito runner can not be used.
Use Mockito's MockitoExtension. The extension is contained in a new artifact mockito-junit-jupiter:
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<version>4.5.1</version>
<scope>test</scope>
</dependency>
It allows you to write tests as you would have with JUnit 4:
import org.mockito.junit.jupiter.MockitoExtension;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
#ExtendWith(MockitoExtension.class)
class MyTest {
#Mock
private Foo foo;
#InjectMocks
private Bar bar; // constructor injection
...
}
There are different ways to do but the cleaner way and that also respects the JUnit 5 philosophy is creating a org.junit.jupiter.api.extension.Extension for Mockito.
1) Creating mocks manually makes lose the benefit of additional Mockito checks to ensure you use correctly the framework.
2) Calling MockitoAnnotations.initMocks(this) in every test classes is boiler plate code that we could avoid.
And making this setup in an abstract class is not a good solution either.
It couples every test classes to a base class.
If then you need a new base test class for good reasons, you finish with a 3- level class hierarchy. Please avoid that.
3) Test Rules is a JUnit 4 specificity.
Don't even think of that.
And the documentation is clear about that :
However, if you intend to develop a new extension for JUnit 5 please
use the new extension model of JUnit Jupiter instead of the rule-based
model of JUnit 4.
4) Test Runner is really not the way to extend the JUnit 5 framework.
JUnit 5 simplified the hell of the Runners of JUnit 4 by providing an extension model for writing tests thanks to JUnit 5 Extensions.
Don't even think of that.
So favor the org.junit.jupiter.api.extension.Extension way.
EDIT : Actually, Mockito bundles a jupiter extension : mockito-junit-jupiter
Then, very simple to use :
import org.mockito.junit.jupiter.MockitoExtension;
#ExtendWith(MockitoExtension.class)
public class FooTest {
...
}
Here is an addition to the excellent answer of Jonathan.
By adding as dependency the mockito-junit-jupiter artifact, the use of #ExtendWith(MockitoExtension.class) produced the following exception as the test is executed :
java.lang.NoSuchMethodError:
org.junit.platform.commons.support.AnnotationSupport.findAnnotation(Ljava/util/Optional;Ljava/lang/Class;)Ljava/util/Optional;
THe problem is that mockito-junit-jupiter depends on two independent libraries.
For example for mockito-junit-jupiter:2.19.0 :
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>2.19.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.1.0</version>
<scope>runtime</scope>
</dependency>
The problem was I used junit-jupiter-api:5.0.1.
So as junit-jupiter-api moves still often in terms of API, make sure you depend on the same version of junit-jupiter-api that mockito-junit-jupiter depends on.
You have to use the new #ExtendWith annotation.
Unfortunately there is no extension released yet.
On github you can see a beta implementation for the extension. as a example demo test.