Develop Reference

Simple modern Gradle configuration to mimic default Maven Failsafe configuration

What is a simple (but modern best-practices) way to configure Gradle so that it allows me to run integration tests similar to how a default Maven Failsafe configuration would work? I've read the Gradle docs for tests—in particular the Gradle docs for integration tests—but they seem pretty complicated (in comparison with Maven Failsafe), and moreover I don't believe the examples work in the same way.
What I'm looking for is pretty straightforward:
There would be some separate task—let's call it integrationTest for the sake of discussion.
The integrationTest task would not run when I invoke gradle test.
The integrationTest task would run (after the test task) when I invoke gradle integrationTest.
The integrationTest task would run (after the test task) when I invoke gradle check.
The integrationTest task would have identical dependencies and classpath configurations as the test task.
The integrationTest task would use the same source path (i.e. src/test/java) as the test task, but would only run tests ending in *IT (just to simplify the default Failsafe inclusion pattern for this discussion).
The test task would ignore all tests ending in *IT.
That's actually a pretty simple use case. (I just went into details so there would be no ambiguity.) I can turn that on in Failsafe using two lines to indicate goals, and three lines to indicate a dependency. Since Gradle takes the XML verbosity out of the equation, I should be able to configure that in two lines, right?
Just to be clear, I don't want to use Maven Failsafe in Gradle. I just want to configure Gradle to behave in a similar way as the default Failsafe configuration, as I detailed above.

in particular the Gradle docs for integration tests—but they seem pretty complicated (in comparison with Maven Failsafe)
You're comparing a plugin (Maven Failsafe) to standard/vanilla Gradle configuration. A plugin abstracts away all the underlying details so it's not fair to compare the two. If you look at the source of Maven Failsafe, you'll see it is equally 'complicated' compared to Gradle.
(...) and moreover I don't believe the examples work in the same way.
The docs for configuring integration test work for typical applications. However since you have specific needs, so you'll need to adapt them.
Since Gradle takes the XML verbosity out of the equation, I should be able to configure that in two lines, right?
No because again, you're comparing a plugin to standard/vanilla Gradle configuration. XML vs Groovy/Kotlin DSL is up for debate which is better or worse and is a matter of opinion.
For your requirements, I believe I've captured them below (untested):
sourceSets {
create("integrationTest") {
compileClasspath += sourceSets.test.get().output
runtimeClasspath += sourceSets.test.get().output
}
}
val integrationTestImplementation by configurations.getting {
extendsFrom(configurations.testImplementation.get())
}
configurations["integrationTestRuntimeOnly"].extendsFrom(configurations.runtimeOnly.get())
val integrationTest by tasks.registering(Test::class) {
description = "Runs integration tests."
group = "verification"
testClassesDirs = sourceSets["integrationTest"].output.classesDirs
classpath = sourceSets["integrationTest"].runtimeClasspath
filter {
includeTestsMatching("IT*")
includeTestsMatching("*IT")
includeTestsMatching("*ITCase")
}
shouldRunAfter("test")
}
tasks {
test {
filter {
excludeTestsMatching("IT*")
excludeTestsMatching("*IT")
excludeTestsMatching("*ITCase")
}
}
register("integrationTest", Test::class) {
description = "Runs integration tests."
group = "verification"
testClassesDirs = sourceSets["integrationTest"].output.classesDirs
classpath = sourceSets["integrationTest"].runtimeClasspath
filter {
includeTestsMatching("IT*")
includeTestsMatching("*IT")
includeTestsMatching("*ITCase")
}
shouldRunAfter(test)
}
}
Although this is a bit verbose/awkward since you want integration tests to be within the same source set as test. So the above can be simplified to just:
tasks {
test {
filter {
excludeTestsMatching("IT*")
excludeTestsMatching("*IT")
excludeTestsMatching("*ITCase")
}
}
register("integrationTest", Test::class) {
description = "Runs integration tests."
group = "verification"
testClassesDirs = sourceSets.test.get().output.classesDirs
classpath = sourceSets.test.get().runtimeClasspath
filter {
includeTestsMatching("IT*")
includeTestsMatching("*IT")
includeTestsMatching("*ITCase")
}
shouldRunAfter(test)
}
}

How to run JUnit 5 and JUnit 4 test suites in Gradle?

I have two types of tests in my code, ending with UnitTest and IntegrationTest. Of course there are some legacy JUnit 4 tests and new ones supposed to be written with JUnit 5.
What I want:
UnitTestSuite and IntegrationTestSuit classes that could be run from IDE (IntelliJ IDEA) and each of them have filters by the class name ending of tests. Also I want two different gradle tasks each to run their own set of tests (based on suits ideally, or also on the class names at least).
What I've tried:
This test suite works well from IDE, and as I understand it should run both JUnit 4 and JUnit 5 tests. However, it seems that this approach is more like a workaround and not actual suites support.
#RunWith(JUnitPlatform.class)
#IncludeClassNamePatterns({ "^.*UnitTest$" })
public class UnitTestSuite {
}
Also I created this Gradle task, but it doesn't run any tests saying to me:
WARNING: Ignoring test class using JUnitPlatform runner
test { Test t ->
useJUnitPlatform()
include "UnitTestSuite.class"
}
So is there a solution to run both JUnit 4 and JUnit 5 tests, filtered by name (gathered into suits) from the IDE and from the Gradle task?

In Gradle you can configure multiple test tasks, one for JUnit 4 and one for JUnit 5.
I did exactly that in the Spring Framework build. See the testJUnitJupiter and test tasks in spring-test.gradle.
task testJUnitJupiter(type: Test) {
description = "Runs JUnit Jupiter tests."
useJUnitPlatform {
includeEngines "junit-jupiter"
excludeTags "failing-test-case"
}
filter {
includeTestsMatching "org.springframework.test.context.junit.jupiter.*"
}
reports.junitXml.destination = file("$buildDir/test-results")
// Java Util Logging for the JUnit Platform.
// systemProperty("java.util.logging.manager", "org.apache.logging.log4j.jul.LogManager")
}
test {
description = "Runs JUnit 4 tests."
dependsOn testJUnitJupiter, testNG
useJUnit()
scanForTestClasses = false
include(["**/*Tests.class", "**/*Test.class"])
exclude(["**/testng/**/*.*", "**/jupiter/**/*.*"])
reports.junitXml.destination = file("$buildDir/test-results")
}
You can of course name them and configure them however you want.

Alternative option with some caveats is to run JUnit 4 (and even 3) tests using JUnit 5. To do so you will need vintage engine on your runtime classpath, like so:
def junit5Version = "5.7.0"
dependencies {
// other deps
testImplementation "org.junit.jupiter:junit-jupiter:${junit5Version}"
testRuntimeOnly "org.junit.vintage:junit-vintage-engine:${junit5Version}"
}
This also works from IntelliJ IDEA.
Source and further instructions on how to make some other JUnit 4 features work with JUnit 5 are here: https://junit.org/junit5/docs/current/user-guide/#migrating-from-junit4-running

Spring/JUnit - running Unit Tests that aren't really "tests"

This question is more of a best practices approach. The application we have is Spring Boot 1.5.4 and builds using Gradle. I'm in the process of creating nightly builds with Jenkins, and want to make sure all the unit tests pass in the project.
The project has a number of "tests" like this however:
#SpringBootTest(classes = {Application.class})
#RunWith(SpringRunner.class)
public class DatabaseCreationProcessImplTest {
This particular class creates a sample database image for developers to work off of. Granted we could make straight SQL scripts, but a Java process is useful since there's code that also queries for data from outside sources (e.g. Liferay.)
The reason we're using a unit test for this is because developers can easily run it in IntelliJ to load a new database image. However this isn't really a "test", it's using the test runner as a quick way to run a Java process.
I'm working on setting up nightly builds and I don't want this test to be included in the builds. I can do something like the following in the build script:
test {
exclude 'com/mydomain/service/util/impl/DatabaseCreationProcessImplTest.class'
}
However by doing this, if running the unit test individually in the IDE with the Spring test runner, it is unable to find any tests. I thought about passing in a Boolean value in the Jenkins task for doing this, e.g.
test {
systemProperties 'property': 'value'
testLogging {
events "passed", "skipped", "failed", "standardOut", "standardError"
}
// Exclusions go here
if (Boolean.getBoolean('exclude.tests')) {
exclude 'com/mydomain/service/util/impl/DatabaseCreationProcessImplTest.class'
}
}
However this seems like a hack/kludge... any ways looking for some "best practices" approach for handling this. Is JUnit the right way for quickly running Java processes? Are there other alternatives? Is it possible to create a Gradle script which developers can use to invoke common Java (Spring Boot) process as well?

I think you could group your not-really-tests in a test suite with JUnit's #SuiteClasses annotation:
#Suite.SuiteClasses(DatabaseCreationProcessImplTest.class)
public class NotReallyTests {}
And then use a condition that you pass from your Jenkins command line to exclude the not-really-tests suite:
test {
if (project.hasProperty('excludeNotReallyTests')) {
useJunit {
excludeCategories 'fully.qualified.name.of.your.NotReallyTests'
}
}
}
Your Jenkins command line would then be
$ gradle -PexcludeNotReallyTests=true
It's a little less hacky than your solution in that it keeps track of the grouping of tests that are not really tests in the codebase instead of the build.gradle file.

The Testing API for Android provides several annotations that are used to group tests together. Then you can specify which tests to run by giving one of the annotations on the command line. I do not know the details of how to implement this. It is just a suggestion for you to explore on your own, if you are interested.

How can intelliJ run gradle task as test

I have a gradle project with "unit test" and "integration tests" tasks defined as follows:
test {
include '**/*Test.class'
}
task integrationTest(type: Test) {
include '**/*IT.class'
}
I created a run configuration in IntelliJ to run all unit tests like image shows:
And did the same with the task 'integrationTest':
IntelliJ 'understands' the test task and run it showing graphical results, like in this image:
The same doesn't happen when it runs the 'integrationTest' task. The results are shown in text, like when I run the task by command line.

Answering my own question...
As far as I know you can't make IntelliJ to run tests of a specific task and the Pattern solution doesn't work so well.
So, the only way I found to effectively separate integration tests in IntelliJ was with the use of a JUnit Category.
Create an interface to represent integration tests.
For example:
public interface IntegrationTest {
}
You have to annotate every integration test class with the category annotation and the created interface:
import org.junit.experimental.categories.Category;
import mycompany.mypackage.IntegrationTest;
#Category(IntegrationTest.class)
public class DbfFileProcessorIT {
...
}
Create a build configuration filtering with Category:

Just add idea plugin to gradle works for me
plugins {
idea
}

JUnit & Integration tests - Is it possible to run one ahead of any test that is run

I have extracted all my integration tests out of my multi-module setup and put them all into a separate project. These integration tests are based on spring and a use a real database. I am using dbmaintain which is a database versioning tool, it automatically tracks which SQL files need to be applied and keeps the database in a correct state.
What I would like is to be able to run the code that ensures the database is up to date before any test is run. So if you run all the tests (from Eclipse or Maven in my case) that it will first perform the db check once, or if you run a single test it will first perform the db check. No matter how many tests are run, it should always run the db check.
Right now I am thinking that I will use #BeforeClass in the base test class (all tests ultimately extend from this class) which will instantiate a singleton to do it's work. That singleton will control everything to make sure things only get run once.
I am hoping there is a cleaner way.

By default, the Maven runner for JUnit reserves the right to reorder tests. This is actually a Good Thing(tm), because you can tell the Maven JUnit plugin to run tests in parallel, which means you wouldn't know the order anyways. In addition, other tools (like TeamCity) can be set to run failing tests first.
I think your best bet would be to add your DB update code as part of the test suite setup (not part of your JUnit framework). Use the Exec Maven Plugin to call your DB code, binding it to the generate-test-resources phase. You'll want to make sure that when you run your tests, you actually call Maven to run the test.
JUnit does have the concept of an ExternalResource, which is a more explicit way of declaring the database dependency. It would be a few more lines of code than the base class, but depending on your perpective it may be more elegant.

Within Maven:
(1) Add the dbmaintain plugin: http://source.mysema.com/display/maven/Maven+Plugins
(2a) Call the appropriate goal (e.g. updateDatabase) explicitly before calling test
(2b) Or, if you want the dependency to be executed during a specific phase, then maven supports this, too: http://maven.apache.org/plugins/maven-dependency-plugin/usage.html
Then, you can connect Eclipse to these Maven changes:
How do I start Maven "compile" goal on save in Eclipse?

JUnit doesn't support test ordering. You will need to use TestNG for this. For example:
#Test(groups = "init")
public void initDatabase() { ... }
#Test(dependsOnGroups = "init")
public void test1() { ... }
#Test(dependsOnGroups = "init")
public void test2() { ... }
In this example, initDatabase() will be run first, and only if it succeeds will test1() and test2() be run. If initDatabase() fails, test1() and test2() will not run and they will be marked as "skipped" in the report.
Note also that you can add methods to any group at any time and the dependencies will keep working the way you expect them.