Run Gradle tests with multiple Java toolchains - java

I've got a Gradle project which uses a Java version specified with the toolchain API:
val minimumJava = JavaLanguageVersion.of(8)
val maximumJava = JavaLanguageVersion.of(16)
java {
toolchain {
languageVersion.set(minimumJava)
vendor.set(JvmVendorSpec.ADOPTOPENJDK)
}
}
I would like to be able to compile with the minimum supported Java version, then run the tests with all the JDKs the project supports.
I tried the following, but apparently only the original tests get executed, all other tests don't, even though the required JDKs get correctly downloaded and set up:
for (javaVersion in JavaLanguageVersion.of(minimumJava.asInt() + 1)..maximumJava) {
val base = tasks.test.get()
val testTask = tasks.register<Test>("testUnderJava${javaVersion.asInt()}") {
javaLauncher.set(
javaToolchains.launcherFor {
languageVersion.set(javaVersion)
}
)
classpath = base.classpath
testClassesDirs = base.testClassesDirs
isScanForTestClasses = true
}
tasks.test.configure { finalizedBy(testTask) }
}
Here is a run in a dumb terminal:
❯ TERM=dumb ./gradlew test testUnderJava10 --rerun-tasks --scan
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on
<<<SNIP>>>
> Task :testClasses
> Task :test
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on
Gradle Test Executor 4 STANDARD_OUT
~~~ Kotest Configuration ~~~
-> Parallelization factor: 1
-> Concurrent specs: null
-> Global concurrent tests: 1
-> Dispatcher affinity: true
-> Default test timeout: 600000ms
-> Default test order: Sequential
-> Default isolation mode: SingleInstance
-> Global soft assertions: false
-> Write spec failure file: false
-> Fail on ignored tests: false
-> Spec execution order: SpecExecutionOrder
-> Remove test name whitespace: false
-> Append tags to test names: false
-> Extensions
- io.kotest.engine.extensions.SystemPropertyTagExtension
- io.kotest.core.extensions.RuntimeTagExtension
- io.kotest.engine.extensions.RuntimeTagExpressionExtension
org.danilopianini.template.test.Tests > A greeting should get printed STARTED
org.danilopianini.template.test.Tests > A greeting should get printed STANDARD_OUT
[:hello=SUCCESS]
> Task :hello
Hello from Danilo Pianini
BUILD SUCCESSFUL in 2s
1 actionable task: 1 executed
org.danilopianini.template.test.Tests > A greeting should get printed PASSED
<<<Other tests have no output!>>>
> Task :testUnderJava9
> Task :testUnderJava8
> Task :testUnderJava16
> Task :testUnderJava15
> Task :testUnderJava14
> Task :testUnderJava13
> Task :testUnderJava12
> Task :testUnderJava11
> Task :testUnderJava10
BUILD SUCCESSFUL in 23s
36 actionable tasks: 36 executed
<<<SNIP>>>
From the build scan, it appears that tests are not executed but those with JDK8. I'm puzzled, the docs say that this should be straightforward:
tasks.register<Test>("testsOn14") {
javaLauncher.set(javaToolchains.launcherFor {
languageVersion.set(JavaLanguageVersion.of(14))
})
}

I think I worked out the root cause of the issues I was experiencing, I'm posting the solution in case someone else runs into similar issues.
I had the following tests configuration:
tasks.test {
useJUnitPlatform()
testLogging {
showStandardStreams = true
showCauses = true
showStackTraces = true
events(*org.gradle.api.tasks.testing.logging.TestLogEvent.values())
exceptionFormat = org.gradle.api.tasks.testing.logging.TestExceptionFormat.FULL
}
}
Which was instructing the task called test to useJunitPlatform(). This setting does not get automatically propagated to all subsequent Test tasks (of course). So, in this case, the solution is simply to use instead:
tasks.withType<Test> {
// Same configuration as above
}
Update 2022-03-16
I decided to create a multi-JVM testing plugin for Gradle, so that all the test tasks get created and much less boilerplate is required across projects.
Update 2023-01-17
Gradle recommends using the task configuration avoidance API.
tasks.withType<Test>().configureEach {
// Same configuration as above
}

Related

upgrading gradle to 6.x error Could not set unknown property 'testClassesDir' for task ':systemtestRun' of type org.gradle.api.tasks.testing.Test

I’m updating the gradle version of my application, it was with version 2.3 and I’m going to 6.7.1
I changed some things, of course, but this error I’m not able to resolve:
> Could not set unknown property 'testClassesDir' for task ':systemtestRun' of type org.gradle.api.tasks.testing.Test.
build.gradle:
task systemtestRun(type: Test) {
description 'run tests'
testClassesDir = sourceSets.systemtest.output.classesDirs
classpath = sourceSets.systemtest.runtimeClasspath + sourceSets.test.runtimeClasspath }
It should be testClassesDirs not testClassesDir:
tasks.register('systemtestRun', Test) {
description = 'Runs system tests.'
group = 'verification'
testClassesDirs = sourceSets.systemtest.output.classesDirs
classpath = sourceSets.systemtest.runtimeClasspath
}

Kotlin JaCoCo - IllegalClassFormatException. Please supply original non-instrumented classes

I'm trying to get test coverage report of our Android application module and executing the testVariantBuildTypeUnitTest task.
Although all tests are passed, TEST-classNameTest.xml file contains error message below. This error is only given for methods contain calls Java from Kotlin. Is there any solution for this issue ?
**Caused by: java.lang.IllegalStateException: Cannot process instrumented class... Please supply original non-instrumented classes.
at org.jacoco.agent.rt.internal_f3994fa.core.internal.instr.InstrSupport.assertNotInstrumented(InstrSupport.java:238)
at org.jacoco.agent.rt.internal_f3994fa.core.internal.instr.ClassInstrumenter.visitField(ClassInstrumenter.java:56)
at org.jacoco.agent.rt.internal_f3994fa.asm.ClassVisitor.visitField(ClassVisitor.java:339)
at org.jacoco.agent.rt.internal_f3994fa.asm.ClassReader.readField(ClassReader.java:1111)
at org.jacoco.agent.rt.internal_f3994fa.asm.ClassReader.accept(ClassReader.java:713)
at org.jacoco.agent.rt.internal_f3994fa.asm.ClassReader.accept(ClassReader.java:401)
at org.jacoco.agent.rt.internal_f3994fa.core.instr.Instrumenter.instrument(Instrumenter.java:90)
at org.jacoco.agent.rt.internal_f3994fa.core.instr.Instrumenter.instrument(Instrumenter.java:108)
We have multimodule android project, because of this we're using this custom Jacoco task to get coverage report:
project.afterEvaluate {
(android.hasProperty('applicationVariants')
? android.'applicationVariants'
: android.'libraryVariants')
.all { variant ->
def variantName = variant.name
def unitTestTask = "test${variantName.capitalize()}UnitTest"
def jacocoReportName = unitTestTask + project.name
tasks.create(name: jacocoReportName, type: JacocoReport, dependsOn: [
"$unitTestTask"
]) {
group = "Reporting"
description = "Generate Jacoco coverage reports for the ${variantName.capitalize()} build"
reports {
html.enabled = true
xml.enabled = true
}
def fileFilter = [
// data binding
'android/databinding/**/*.class',
'**/android/databinding/*Binding.class',
'**/android/databinding/*',
'**/androidx/databinding/*',
'**/databinding',
'**/BR.*',
// android
'**/R.class',
'**/R$*.class',
'**/BuildConfig.*',
'**/Manifest*.*',
'**/*Test*.*'
]
def javaClasses = fileTree(dir: variant.javaCompileProvider.get().destinationDir,
excludes: fileFilter)
def kotlinClasses = fileTree(dir: "${buildDir}/tmp/kotlin-classes/${variantName}",
excludes: fileFilter)
classDirectories.setFrom(files([
javaClasses,
kotlinClasses
]))
def variantSourceSets = variant.sourceSets.java.srcDirs.collect { it.path }.flatten()
sourceDirectories.setFrom(project.files(variantSourceSets))
if (isAndroidLibrary(project)) {
executionData(files([
"${projectDir}/jacoco.exec"
]))
}else{
executionData(files([
"$project.buildDir/jacoco/${project.name}.exec"
]))
}
}
}
}
Regards,
Solution
The problem most likely lies in your module level build.gradle file. You need to remove testCoverageEnabled or set it to false.
android {
...
buildTypes {
release {
minifyEnabled true
}
debug {
// testCoverageEnabled true
minifyEnabled false
}
}
Explanation and References
Firstly I've been struggling with unit test coverage on android with JaCoCo for years and repeatedly encountered the issues multiple times as the AGP (Android Gradle Plugin) team enhanced the AGP (e.g. Similar issue) therefore this solution is subject to change in future releases of the AGP.
Come the release of AGP 4.1 the AGP team changed the way the plugin deals with unit tests. I raised an enter link description here issue with the AGP team and they explained that the plugin (AGP 4.1+) does it's own form of instrumentation to get the coverage which is undoubtedly incompatible with JaCoCo. It was in this issue where the solution was highlighted to me.
Note 1: This issue is not documented in the DSL documentation!
Note 2: It should also be noted that as of when I wrote this AGP 7 is in development where the same issue occurs.
Considerations
This solution will cause a problem generating coverage for instrumented tests as the AGP generates coverage for instrumented tests (in the form of a .ec file) meaning to get instrumented test coverage you must have testCoverageEnabled true.
As a workaround you could:
Put your unit tests in the release build variant
Put your instrumented tests in the debug build variant.
It's still possible to combine the coverage into one report: for example:
task jacocoCombinedUnitTestAndroidTestReport(type: JacocoReport) {
group = "Reporting"
reports {
xml.enabled = true
html.enabled = true
}
def mainSrc = "$project.projectDir/src/main/java"
def releaseKotlinClasses = fileTree(dir: "${buildDir}/tmp/kotlin-classes/release", excludes:["com/example/ui**"])
def debugKotlinClasses = fileTree(dir: "${buildDir}/tmp/kotlin-classes/debug/com/example/ui/")
def releaseJavaClasses = fileTree(dir: "${buildDir}/intermediates/javac/release", excludes: ["com/example/ui**"])
def debugJavaClasses = fileTree(dir: "${buildDir}/intermediates/javac/debug/com/example/ui/" )
def mainClasses = releaseKotlinClasses + debugKotlinClasses
def javaClasses = releaseJavaClasses + debugJavaClasses
sourceDirectories.from = files([mainSrc])
classDirectories.from = files([mainClasses, javaClasses])
executionData.from = fileTree(dir: buildDir, include: ["jacoco/testReleaseUnitTest.exec",
"outputs/code_coverage/debugAndroidTest/connected/**.ec"])
}
This assumes you have any UI code in a ui directory.
You include all release classes excluding the ui directory
You only include your ui debug classes.
Please note this is only valid as of when this answer was written and hopefully Google will fix the issue in the future.

Detekt | finished with non-zero exit value 255

I am trying to run detektBaseline gradle task but facing following Exception
Process ‘command ‘/Applications/Android
Studio.app/Contents/jre/jdk/Contents/Home/bin/java’’ finished with
non-zero exit value 255
my project level gradle configuration for detekt
plugins {
id “io.gitlab.arturbosch.detekt” version “1.0.0.RC6-4”
}
detekt {
version = “1.0.0.RC6-4”
defaultProfile {
input = “$projectDir/rlm/src/main/java”
config = “$projectDir/default-detekt-config.yml”
filters = “./resources/.,./build/.”
}
Any kind of help or guideline would be appreciated.

Cannot add task ':jacocoTestReport' as a task with that name already exists

I am trying to add the following task so that I can get some coverage data in my java + kotlin project (for what it is worth, this is a gradle project)... but I get the following error :
"Cannot add task ':jacocoTestReport' as a task with that name already exists"
Here is the actual task I am trying to add :
task jacocoTestReport(type: JacocoReport, dependsOn: "testDebugUnitTest") {
group = "Reporting"
description = "Generate Jacoco coverage reports for Debug build"
reports {
xml.enabled = true
html.enabled = true
}
// what to exclude from coverage report
// UI, "noise", generated classes, platform classes, etc.
def excludes = [
'**/R.class',
'**/R$*.class',
'**/*$ViewInjector*.*',
'**/BuildConfig.*',
'**/Manifest*.*',
'**/*Test*.*',
'android/**/*.*',
'**/*Fragment.*',
'**/*Activity.*'
]
// generated classes
classDirectories = fileTree(
dir: "$buildDir/intermediates/classes/debug",
excludes: excludes
) + fileTree(
dir: "$buildDir/tmp/kotlin-classes/debug",
excludes: excludes
)
// sources
sourceDirectories = files([
android.sourceSets.main.java.srcDirs,
"src/main/kotlin"
])
executionData = files("$buildDir/jacoco/testDebugUnitTest.exec")
}
Now, the issue I am confused about here, is that I can't find another class of this name anywhere... so perhaps there is something funky going on? I have tried googling this, but haven't really been able to find anything which truly helps me solve the problem.
All help greatly appreciated. I realize this is not a java or kotlin specific problem - but since it is a joint java + kotlin project, I thought I would tag both in this question, in case there is some nuanced issue that somebody else has seen.
Assuming you're already applying the Jacoco Gradle plugin, then yes, it already defines a task called jacocoTestReport, hence the error.
All you need to do is define your specific settings as per the documentation https://docs.gradle.org/current/userguide/jacoco_plugin.html#sec:jacoco_report_configuration
an example is below:
jacocoTestReport {
dependsOn "testDebugUnitTest"
reports {
xml.enabled = true
html.enabled = true
}
}
Most of the other configuration items you've listed belong in the 'jacoco' configuration block.
https://docs.gradle.org/current/userguide/jacoco_plugin.html#sec:jacoco_specific_task_configuration

Gradle: How to partition a task into sequentially executed actions

Is it possible to nest tasks in gradle, such as
task foo(dependsOn: jar){
// task 1
// task 2
// task 3
.
.
.
// task n
}
where the order of execution is jar > foo > task 1 > task 2 > task 3 > ... > task n? I don't want the nested tasks (i.e. task 1, task 2, and etc.) to be exposed to the user. I only want the foo task to be exposed.
It looks like you can simply do the following,
task foo(dependsOn: ['clean', 'jar']){
foo << {
println "First"
}
foo << {
println "Second"
}
foo << {
println "Third"
}
.
.
.
}
where << is shorthand for doLast. I think the neat thing about this approach is that only foo is exposed..the nested tasks remain hidden from the end user. And if you execute foo, you'll get
First
Second
Third
There is no way in Gradle to only expose selected tasks (in the strict sense). There is, however, a way to only show selected tasks in gradle tasks. Unless the --all flag is used, gradle tasks will only show "root" tasks (i.e. tasks that no other task depends on) and tasks that have their group property set.

Categories

Resources