What is a simple (but modern best-practices) way to configure Gradle so that it allows me to run integration tests similar to how a default Maven Failsafe configuration would work? I've read the Gradle docs for tests—in particular the Gradle docs for integration tests—but they seem pretty complicated (in comparison with Maven Failsafe), and moreover I don't believe the examples work in the same way.
What I'm looking for is pretty straightforward:
There would be some separate task—let's call it integrationTest for the sake of discussion.
The integrationTest task would not run when I invoke gradle test.
The integrationTest task would run (after the test task) when I invoke gradle integrationTest.
The integrationTest task would run (after the test task) when I invoke gradle check.
The integrationTest task would have identical dependencies and classpath configurations as the test task.
The integrationTest task would use the same source path (i.e. src/test/java) as the test task, but would only run tests ending in *IT (just to simplify the default Failsafe inclusion pattern for this discussion).
The test task would ignore all tests ending in *IT.
That's actually a pretty simple use case. (I just went into details so there would be no ambiguity.) I can turn that on in Failsafe using two lines to indicate goals, and three lines to indicate a dependency. Since Gradle takes the XML verbosity out of the equation, I should be able to configure that in two lines, right?
Just to be clear, I don't want to use Maven Failsafe in Gradle. I just want to configure Gradle to behave in a similar way as the default Failsafe configuration, as I detailed above.
in particular the Gradle docs for integration tests—but they seem pretty complicated (in comparison with Maven Failsafe)
You're comparing a plugin (Maven Failsafe) to standard/vanilla Gradle configuration. A plugin abstracts away all the underlying details so it's not fair to compare the two. If you look at the source of Maven Failsafe, you'll see it is equally 'complicated' compared to Gradle.
(...) and moreover I don't believe the examples work in the same way.
The docs for configuring integration test work for typical applications. However since you have specific needs, so you'll need to adapt them.
Since Gradle takes the XML verbosity out of the equation, I should be able to configure that in two lines, right?
No because again, you're comparing a plugin to standard/vanilla Gradle configuration. XML vs Groovy/Kotlin DSL is up for debate which is better or worse and is a matter of opinion.
For your requirements, I believe I've captured them below (untested):
sourceSets {
create("integrationTest") {
compileClasspath += sourceSets.test.get().output
runtimeClasspath += sourceSets.test.get().output
}
}
val integrationTestImplementation by configurations.getting {
extendsFrom(configurations.testImplementation.get())
}
configurations["integrationTestRuntimeOnly"].extendsFrom(configurations.runtimeOnly.get())
val integrationTest by tasks.registering(Test::class) {
description = "Runs integration tests."
group = "verification"
testClassesDirs = sourceSets["integrationTest"].output.classesDirs
classpath = sourceSets["integrationTest"].runtimeClasspath
filter {
includeTestsMatching("IT*")
includeTestsMatching("*IT")
includeTestsMatching("*ITCase")
}
shouldRunAfter("test")
}
tasks {
test {
filter {
excludeTestsMatching("IT*")
excludeTestsMatching("*IT")
excludeTestsMatching("*ITCase")
}
}
register("integrationTest", Test::class) {
description = "Runs integration tests."
group = "verification"
testClassesDirs = sourceSets["integrationTest"].output.classesDirs
classpath = sourceSets["integrationTest"].runtimeClasspath
filter {
includeTestsMatching("IT*")
includeTestsMatching("*IT")
includeTestsMatching("*ITCase")
}
shouldRunAfter(test)
}
}
Although this is a bit verbose/awkward since you want integration tests to be within the same source set as test. So the above can be simplified to just:
tasks {
test {
filter {
excludeTestsMatching("IT*")
excludeTestsMatching("*IT")
excludeTestsMatching("*ITCase")
}
}
register("integrationTest", Test::class) {
description = "Runs integration tests."
group = "verification"
testClassesDirs = sourceSets.test.get().output.classesDirs
classpath = sourceSets.test.get().runtimeClasspath
filter {
includeTestsMatching("IT*")
includeTestsMatching("*IT")
includeTestsMatching("*ITCase")
}
shouldRunAfter(test)
}
}
Related
I've defined my sourceSets as
sourceSets {
// Configuring SourceSets for all connector source files.
main {
java {
srcDirs = ['src']
}
}
test {
// integrationTests
}
unitTests {
//unitTests
}
}
test {
// integrationTests
}
task('unitTest', type: Test) {
//unitTests
}
When I kick off ./gradlew build it's check task basically calls the test target.
Can I override the default gradle build tasks's 'check' mechanism to call unitTest here instead of test?
First of all, please consider just using the task test for unit tests and using another source set for integration tests. Gradle and its plugins follow an approach called convention over configuration and the convention of the Java plugin is to use the task test for unit tests.
To actually answer your question, you can access and modify the dependencies of each task using getDependsOn() and setDependsOn(Iterable<?>), so a simple solution to your problem could be the following code:
check {
dependsOn.clear()
dependsOn unitTest
}
This removes all task dependencies from the task check and then adds a dependency on the task unitTest. However, this may cause new problems, because other plugins may automatically register task dependencies for check that will be removed, too.
An obvious improvement to the code above would be to just remove the dependency on the test task instead of all task dependencies. Sadly, this is not as simple as one would think. You may have noticed that the method getDependsOn() returns a Set<Object> and not a Set<Task>. For convenience, task dependencies may be expressed not only using tasks but also by passing other objects that will resolve to tasks later on (e.g. a string containing the name of a task). The current version of the Java plugin uses a Provider<Task> to register the dependency between the task check and the task test. To just remove the dependency on the task test, you would need to iterate over all dependencies in dependsOn and find the one with the type Provider that returns the task test via its get() method. You could then remove that Provider from dependsOn.
As you can see, it is possible to remove task dependencies once they have been registered, but it is not that easy, so you should probably just follow the convention and use the task test for unit tests.
I'm introducing a Github actions pipeline to an existing project to run ./gradlew test. Unsurprisingly, I've run into cases where tests pass locally but not on the build machine, due to various things like mismatching timezones.
By default, gradle doesn't print the stdout for these tests. I am aware that it will do so if passed --info, however the test suite is some 1500 tests in size which makes the pipeline output extremely verbose (it actually makes my browser lag if I turn it on for the full suite and try to view the resulting output in Github).
To fix the initial teething problems, I've resorted to also targeting the suites that are failing (e.g. ./gradlew test --tests "foo.bar.AppTest" --info). This is a bit of a faff, though. Is there a way to tell gradle to print the stdout contents just for tests that have failed? This would put me in a much better position going forward!
This page contains what you are looking for.
It boils down to configuring the test task like so:
test {
testLogging {
// set options for log level LIFECYCLE
events "failed"
}
}
There are more options to finely control logging if you read that page.
Since you probably only need this for github actions, you can use the CI environmental variable to enable your configurations on CI environments only:
test {
doFirst {
if (System.getenv('CI')) {
testLogging {
// set options for log level LIFECYCLE
events "failed"
}
}
}
}
Other CI providers also set this environmental variable
As mentioned in this related answer when dealing with a multi-module android app the following can be used (root build.gradle)
// Call from root build.gradle
setupTestLogging()
fun Project.setupTestLogging() {
for (sub in subprojects) {
sub.tasks.withType<Test> {
testLogging {
exceptionFormat = TestExceptionFormat.FULL
}
}
}
}
(note that while exceptionFormat alone should be enough to get the wanted outcome, the events("standardOut" ...) mentioned above can be specified in the same way).
For mono-module android projects the same solution will work by dropping the part that iterates on the submodules
Tl;dr
Is there a way to detach specific plugin behavior (such as checkstyle's check behavior) from existing gradle lifecycle tasks (gradle check, in this particular case)?
Longer version
In our current gradle Java project setup, we've included checkstyle as one of our plugins for static code checking. It currently runs as a part of Jenkins pipeline through gradle's build task. While this has mostly worked out for what we've needed - namely running our tests and making sure we're sticking to code standards - I've also noticed that we could make our feedback loop a little faster if we could run just the checkstyle's plugin's checks before build kicks in the tests.
To do so, as far as I understand, we'd have to create a custom task that runs only the checkstyle functions checkstyleMain and checkstyleTest and decouple the default checkstyle behavior from gradle's build lifecycle task. I've been looking through both gradle and the checkstyle plugin's docs, but quickly found I'm out of my depth.
Code:
plugins {
id "checkstyle"
}
checkstyle {
toolVersion "8.24"
configFile file("config/checkstyle/checkstyle.xml")
}
checkstyleMain {
source = "src/main/java"
}
checkstyleTest {
source = "src/test/java"
}
That is everything checkstyle related inside of build.gradle, the check task itself isn't customized.
This question is more of a best practices approach. The application we have is Spring Boot 1.5.4 and builds using Gradle. I'm in the process of creating nightly builds with Jenkins, and want to make sure all the unit tests pass in the project.
The project has a number of "tests" like this however:
#SpringBootTest(classes = {Application.class})
#RunWith(SpringRunner.class)
public class DatabaseCreationProcessImplTest {
This particular class creates a sample database image for developers to work off of. Granted we could make straight SQL scripts, but a Java process is useful since there's code that also queries for data from outside sources (e.g. Liferay.)
The reason we're using a unit test for this is because developers can easily run it in IntelliJ to load a new database image. However this isn't really a "test", it's using the test runner as a quick way to run a Java process.
I'm working on setting up nightly builds and I don't want this test to be included in the builds. I can do something like the following in the build script:
test {
exclude 'com/mydomain/service/util/impl/DatabaseCreationProcessImplTest.class'
}
However by doing this, if running the unit test individually in the IDE with the Spring test runner, it is unable to find any tests. I thought about passing in a Boolean value in the Jenkins task for doing this, e.g.
test {
systemProperties 'property': 'value'
testLogging {
events "passed", "skipped", "failed", "standardOut", "standardError"
}
// Exclusions go here
if (Boolean.getBoolean('exclude.tests')) {
exclude 'com/mydomain/service/util/impl/DatabaseCreationProcessImplTest.class'
}
}
However this seems like a hack/kludge... any ways looking for some "best practices" approach for handling this. Is JUnit the right way for quickly running Java processes? Are there other alternatives? Is it possible to create a Gradle script which developers can use to invoke common Java (Spring Boot) process as well?
I think you could group your not-really-tests in a test suite with JUnit's #SuiteClasses annotation:
#Suite.SuiteClasses(DatabaseCreationProcessImplTest.class)
public class NotReallyTests {}
And then use a condition that you pass from your Jenkins command line to exclude the not-really-tests suite:
test {
if (project.hasProperty('excludeNotReallyTests')) {
useJunit {
excludeCategories 'fully.qualified.name.of.your.NotReallyTests'
}
}
}
Your Jenkins command line would then be
$ gradle -PexcludeNotReallyTests=true
It's a little less hacky than your solution in that it keeps track of the grouping of tests that are not really tests in the codebase instead of the build.gradle file.
The Testing API for Android provides several annotations that are used to group tests together. Then you can specify which tests to run by giving one of the annotations on the command line. I do not know the details of how to implement this. It is just a suggestion for you to explore on your own, if you are interested.
I have some projects with both unit tests and integration tests. I have them separated so that unit tests run as part of the regular build, and integration tests are only run by a specific "integTest" task. This is all working fine.
I have another project that I didn't write, and can't refactor, which has a single unit test that isn't really a unit test. I have a stock Gradle build script for this project, but I'd like to add the pieces that I put in other projects to run this test as an integration test instead. I haven't done this yet, but I think what I've done in other projects will only half work. I'm certain it will let me run that test as an integration test, but I don't yet know how to make it NOT run as a unit test.
The one test is in "src/test/java", and I'm now going to associate that with my "integTest" task (I used "src/integTest/groovy" before, and I imagine I could add "src/integTest/java" also). How do I REMOVE that directory from being considered by the default "test" task, so "test" never runs any tests?
Update:
Although the title of this posting is about running the unit test as an integration test, I really only needed to know how to exclude the existing test from the unit test pass, which was answered.
Someone seeing the title of this might want to know how to do that, so I'll add detail of how I did this.
The following shows everything that I added to the build script to make this happen:
// Without refactoring the source code of the project, this excludes the one test that looks like a unit test, but is
// actually an integration test.
test { exclude '**/*.*' }
sourceSets {
integTest {
java.srcDir file("src/test/java")
resources.srcDir file("src/test/resources")
runtimeClasspath = output + compileClasspath
}
}
dependencies {
integTestCompile sourceSets.main.output
integTestCompile configurations.testCompile
integTestCompile sourceSets.test.output
integTestRuntime configurations.testRuntime
}
task integTest(type: Test) {
testClassesDir = sourceSets.integTest.output.classesDir
classpath = sourceSets.integTest.runtimeClasspath
// This forces integration tests to always run if the task is run.
outputs.upToDateWhen { false }
}
In your build.gradle you can do:
test {
exclude '**/*.*'
}
or just disable test task by adding the line test.enabled = false