I have two types of tests in my code, ending with UnitTest and IntegrationTest. Of course there are some legacy JUnit 4 tests and new ones supposed to be written with JUnit 5.
What I want:
UnitTestSuite and IntegrationTestSuit classes that could be run from IDE (IntelliJ IDEA) and each of them have filters by the class name ending of tests. Also I want two different gradle tasks each to run their own set of tests (based on suits ideally, or also on the class names at least).
What I've tried:
This test suite works well from IDE, and as I understand it should run both JUnit 4 and JUnit 5 tests. However, it seems that this approach is more like a workaround and not actual suites support.
#RunWith(JUnitPlatform.class)
#IncludeClassNamePatterns({ "^.*UnitTest$" })
public class UnitTestSuite {
}
Also I created this Gradle task, but it doesn't run any tests saying to me:
WARNING: Ignoring test class using JUnitPlatform runner
test { Test t ->
useJUnitPlatform()
include "UnitTestSuite.class"
}
So is there a solution to run both JUnit 4 and JUnit 5 tests, filtered by name (gathered into suits) from the IDE and from the Gradle task?
In Gradle you can configure multiple test tasks, one for JUnit 4 and one for JUnit 5.
I did exactly that in the Spring Framework build. See the testJUnitJupiter and test tasks in spring-test.gradle.
task testJUnitJupiter(type: Test) {
description = "Runs JUnit Jupiter tests."
useJUnitPlatform {
includeEngines "junit-jupiter"
excludeTags "failing-test-case"
}
filter {
includeTestsMatching "org.springframework.test.context.junit.jupiter.*"
}
reports.junitXml.destination = file("$buildDir/test-results")
// Java Util Logging for the JUnit Platform.
// systemProperty("java.util.logging.manager", "org.apache.logging.log4j.jul.LogManager")
}
test {
description = "Runs JUnit 4 tests."
dependsOn testJUnitJupiter, testNG
useJUnit()
scanForTestClasses = false
include(["**/*Tests.class", "**/*Test.class"])
exclude(["**/testng/**/*.*", "**/jupiter/**/*.*"])
reports.junitXml.destination = file("$buildDir/test-results")
}
You can of course name them and configure them however you want.
Alternative option with some caveats is to run JUnit 4 (and even 3) tests using JUnit 5. To do so you will need vintage engine on your runtime classpath, like so:
def junit5Version = "5.7.0"
dependencies {
// other deps
testImplementation "org.junit.jupiter:junit-jupiter:${junit5Version}"
testRuntimeOnly "org.junit.vintage:junit-vintage-engine:${junit5Version}"
}
This also works from IntelliJ IDEA.
Source and further instructions on how to make some other JUnit 4 features work with JUnit 5 are here: https://junit.org/junit5/docs/current/user-guide/#migrating-from-junit4-running
Related
I created 2 Test classes (1 with 30 testcases, the other one with 10 testcases) in Apache Camel with Spring and Maven.
One of the Test class works fine, and runs if I build with Maven.
mvn test <- Works with one Test class
mvn package <- Works also with one Test class
But the other test class don't run. Both are in src/test/java
Also both test classes are in target/test-classes/...
One different is in target/test-classes/... the not working class have an second file with ...Test$1.class in the same folder. But I cannot open it.
And one second different have the classes: The not working class extends from CamelTestSupport and is a junit4. The working test-class is from junit5.
I think the maven-surefire-plugin don't find the junit4 test class. But how can I make maven-surefire-plugin finding junit4 test classes?
The JUnit 4 tests have to be executed with the Vintage engine of JUnit and the JUnit 5 tests with the Jupiter engine.
See this answer how to configure the POM to use both engines depending on the version of the surefire plugin.
I'd like to create dedicated task for integration test in gradle.
I have multimodule project. In project build.gradle I've task:
integrationTest(type:Test){
useJUnit {
includeCategories("examplePackage.IntegrationTest")
}
}
It should run tests marked with:
#Category(IntegrationTest.class)
but when I run this, it says Test events were not received.
What am I doing wrong?
JUnit Platform introduced tagging to replace categories.
Gradle adopted to this. See user guide test grouping
I have a bunch of JUnit tests that extend my base test class called BaseTest which in turn extends Assert. Some of my tests have a #Category(SlowTests.class) annotation.
My BaseTest class is annotated with the following annotation #RunWith(MyJUnitRunner.class).
I've set up a Gradle task that is expected to run only SlowTests. Here's my Gradle task:
task integrationTests(type: Test) {
minHeapSize = "768m"
maxHeapSize = "1024m"
testLogging {
events "passed", "skipped", "failed"
outputs.upToDateWhen {false}
}
reports.junitXml.destination = "$buildDir/test-result"
useJUnit {
includeCategories 'testutils.SlowTests'
}
}
When I run the task, my tests aren't run. I've pinpointed this issue to be related to the custom runner MyJUnitRunner on the BaseTest. How can I set up my Gradle or test structure so that I can use a custom runner while using the Suite.
The solution to this turned out to smaller and trickier than I thought. Gradle was using my custom test runner and correctly invoking the filter method. However, my runner reloads all test classes through its own classloader for Javaassist enhancements.
This lead to the issue that SlowTest annotation was loaded through the Gradle classloader but when passed to my custom runner, the runner checked if the class was annotated with that annotation. This check never resolved correctly as the equality of the SlowTest annotation loaded through two different classloaders was different.
--
Since I've already done the research, I'll just leave this here. After days of digging through the Gradle and the (cryptic) JUnit sources, here's what I got.
Gradle simply doesn't handle any advanced JUnit functionality except the test categorization. When you create a Gradle task with the include-categories or the exclude-categories conditions, it builds a CategoryFilter. If you don't know, a Filter is what JUnit gives to the test-runner to decide whether a test or a test method should be filtered out. The test runner must implement the Filterable interface.
JUnit comes with multiple runners, the Categories is just another one of them. It extends a family of test runners called Suite. These suite based runners are designed to run a "suite" of tests. A suite of tests could be built by annotation introspection, by explicitly defining tests in a suite or any other method that builds a suite of tests.
In the case of the Categories runner, JUnit has it's own CategoryFilter but Gradle doesn't use that, it uses it's own CategoryFilter. Both provide more or less the same functionality and are JUnit filters so that can be used by any suite that implements Filterable.
The actual class in the Gradle responsible for running the JUnit tests is called JUnitTestClassExecuter. Once it has parsed the command line options it requests JUnit to check the runner should be used for a test. This method is invoked for every test as seen here.
The rest is simply up to JUnit. Gradle just created a custom RunNotifier to generate the standard XML files representing test results.
I hope someone finds this useful and saved themselves countless hours of debugging.
TLDR: You can use any runner in Gradle. Gradle has no specifics pertaining to runners. It is JUnit that decided the runners. If you'd like to know what runner will be used for your test, you can debug this by calling
Request.aClass(testClass).getRunner(). Hack this somewhere into your codebase and print it to the console. (I wasn't very successful in attaching a debugger to Gradle.)
I have a gradle project with "unit test" and "integration tests" tasks defined as follows:
test {
include '**/*Test.class'
}
task integrationTest(type: Test) {
include '**/*IT.class'
}
I created a run configuration in IntelliJ to run all unit tests like image shows:
And did the same with the task 'integrationTest':
IntelliJ 'understands' the test task and run it showing graphical results, like in this image:
The same doesn't happen when it runs the 'integrationTest' task. The results are shown in text, like when I run the task by command line.
Answering my own question...
As far as I know you can't make IntelliJ to run tests of a specific task and the Pattern solution doesn't work so well.
So, the only way I found to effectively separate integration tests in IntelliJ was with the use of a JUnit Category.
Create an interface to represent integration tests.
For example:
public interface IntegrationTest {
}
You have to annotate every integration test class with the category annotation and the created interface:
import org.junit.experimental.categories.Category;
import mycompany.mypackage.IntegrationTest;
#Category(IntegrationTest.class)
public class DbfFileProcessorIT {
...
}
Create a build configuration filtering with Category:
Just add idea plugin to gradle works for me
plugins {
idea
}
I have some projects with both unit tests and integration tests. I have them separated so that unit tests run as part of the regular build, and integration tests are only run by a specific "integTest" task. This is all working fine.
I have another project that I didn't write, and can't refactor, which has a single unit test that isn't really a unit test. I have a stock Gradle build script for this project, but I'd like to add the pieces that I put in other projects to run this test as an integration test instead. I haven't done this yet, but I think what I've done in other projects will only half work. I'm certain it will let me run that test as an integration test, but I don't yet know how to make it NOT run as a unit test.
The one test is in "src/test/java", and I'm now going to associate that with my "integTest" task (I used "src/integTest/groovy" before, and I imagine I could add "src/integTest/java" also). How do I REMOVE that directory from being considered by the default "test" task, so "test" never runs any tests?
Update:
Although the title of this posting is about running the unit test as an integration test, I really only needed to know how to exclude the existing test from the unit test pass, which was answered.
Someone seeing the title of this might want to know how to do that, so I'll add detail of how I did this.
The following shows everything that I added to the build script to make this happen:
// Without refactoring the source code of the project, this excludes the one test that looks like a unit test, but is
// actually an integration test.
test { exclude '**/*.*' }
sourceSets {
integTest {
java.srcDir file("src/test/java")
resources.srcDir file("src/test/resources")
runtimeClasspath = output + compileClasspath
}
}
dependencies {
integTestCompile sourceSets.main.output
integTestCompile configurations.testCompile
integTestCompile sourceSets.test.output
integTestRuntime configurations.testRuntime
}
task integTest(type: Test) {
testClassesDir = sourceSets.integTest.output.classesDir
classpath = sourceSets.integTest.runtimeClasspath
// This forces integration tests to always run if the task is run.
outputs.upToDateWhen { false }
}
In your build.gradle you can do:
test {
exclude '**/*.*'
}
or just disable test task by adding the line test.enabled = false