I'm testing an app with JUnit5 and using Jacoco for coverage report.
Tests are executed Ok and test reports are present.
However, Jacoco report has the following logs if service contains methods, annotated with #Transactional
[ant:jacocoReport] Classes in bundle 'my-service' do no match with execution data. For report generation the same class files must be used as at runtime.
[ant:jacocoReport] Execution data for class mypackage/SampleService does not match.
This error occurres for all #Service classes methods, annotated with #Transactional, plain classes coverage is calculated ok.
Here's a sample test:
#SpringBootTest
#ExtendWith(SpringExtension.class)
public class MyServiceTest {
#Autowired
private SampleService sampleService;
#Test
public void doWork(){
sampleService.doWork();
}
}
Works fine. Coverage is non-zero:
public class SampleService {
public void doWork(){
System.out.println("HEY");
}
}
0% coverage:
public class SampleService {
#Transactional
public void doWork(){
System.out.println("HEY");
}
}
Transactional creates a proxy around actuall class. But, isn't there an out-of-box way for Jacoco to handle such a common situation?
I've tried #EnableAspectJAutoProxy annotaion with different flag variations, checked that up-to-date Jupiter engine and Jacoco plugin are used
Here's gradle config:
subprojects {
test {
useJUnitPlatform()
}
jacocoTestReport {
afterEvaluate {
classDirectories.from = files(classDirectories.files.collect {
fileTree(dir: it, exclude: '*Test.java')
})
}
reports {
html.enabled = true
xml.enabled = true
csv.enabled = false
}
}
}
Any help appreciated
I tried it with a test project, similar to what you've described, however I couldn't reproduce the issue. The only difference that I see between your project and mine, is that I've used maven instead of gradle.
Here is the test project:
https://github.com/gybandi/jacocotest
And here is the jacoco result for it (using org.springframework.transaction.annotation.Transactional annotation):
If this doesn't help you, could you upload your test project to github or some other place?
Edit:
#MikaelF posted a link to another answer, which shows how to add offline instrumentation for jacoco.
The solution that was described there worked for me, after I added the following block to build.gradle:
task instrument(dependsOn: [classes, project.configurations.jacocoAnt]) {
inputs.files classes.outputs.files
File outputDir = new File(project.buildDir, 'instrumentedClasses')
outputs.dir outputDir
doFirst {
project.delete(outputDir)
ant.taskdef(
resource: 'org/jacoco/ant/antlib.xml',
classpath: project.configurations.jacocoAnt.asPath,
uri: 'jacoco'
)
def instrumented = false
if (file(sourceSets.main.java.outputDir).exists()) {
def instrumentedClassedDir = "${outputDir}/${sourceSets.main.java}"
ant.'jacoco:instrument'(destdir: instrumentedClassedDir) {
fileset(dir: sourceSets.main.java.outputDir, includes: '**/*.class')
}
//Replace the classes dir in the test classpath with the instrumented one
sourceSets.test.runtimeClasspath -= files(sourceSets.main.java.outputDir)
sourceSets.test.runtimeClasspath += files(instrumentedClassedDir)
instrumented = true
}
if (instrumented) {
//Disable class verification based on https://github.com/jayway/powermock/issues/375
test.jvmArgs += '-noverify'
}
}
}
test.dependsOn instrument
There seems to be an open ticket on the jacoco plugin's github about this as well:
https://github.com/gradle/gradle/issues/2429
Based on single module instrumentation example https://stackoverflow.com/a/31916686/7096763 (and updated version for gradle 5+ by #MikaelF) here's example for multimodule instrumentation:
subprojects { subproject ->
subproject.ext.jacocoOfflineSourceSets = [ 'main' ]
task doJacocoOfflineInstrumentation(dependsOn: [ classes, subproject.configurations.jacocoAnt ]) {
inputs.files classes.outputs.files
File outputDir = new File(subproject.buildDir, 'instrumentedClasses')
outputs.dir outputDir
doFirst {
project.delete(outputDir)
ant.taskdef(
resource: 'org/jacoco/ant/antlib.xml',
classpath: subproject.configurations.jacocoAnt.asPath,
uri: 'jacoco'
)
def instrumented = false
jacocoOfflineSourceSets.each { sourceSetName ->
if (file(sourceSets[sourceSetName].output.classesDirs[0]).exists()) {
def instrumentedClassedDir = "${outputDir}/${sourceSetName}"
ant.'jacoco:instrument'(destdir: instrumentedClassedDir) {
fileset(dir: sourceSets[sourceSetName].output.classesDirs[0], includes: '**/*.class')
}
//Replace the classes dir in the test classpath with the instrumented one
sourceSets.test.runtimeClasspath -= files(sourceSets[sourceSetName].output.classesDirs[0])
sourceSets.test.runtimeClasspath += files(instrumentedClassedDir)
instrumented = true
}
}
if (instrumented) {
//Disable class verification based on https://github.com/jayway/powermock/issues/375
test.jvmArgs += '-noverify'
}
}
}
test.dependsOn doJacocoOfflineInstrumentation
}
full example here:
https://github.com/lizardeye/jacocomultimodulesample
Still, I think this is a durty hack, which can be easily broken with gradle or jacoco updates
Related
I have a cucumber test :
#RunWith(CucumberWithSerenity.class)
#UsePersistantStepLibraries
#CucumberOptions(
features = "src/integrationTest/resources/features/",
glue = {"com/myorg/proj/integrationtest/stepdefinitions"},
plugin = {"pretty", "summary"}
)
public class integrationTestRunner extends ParentClassWithAllMethods {
}
This is test is not defined inside the test folder , but as a seprate folder (as source set, and some one else had created this structure).
The gradle configuration for the sourceset (where the test is ) is as below:
sourceSets {
integrationTest {
java {
compileClasspath += main.output + test.output
runtimeClasspath += main.output + test.output
srcDir file('src/integrationTest/java')
}
resources.srcDir file('src/integrationTest/resources')
}
}
// Dependencies for integrationTest
dependencies {
integrationTestCompileOnly 'org.projectlombok:lombok:1.18.12'
integrationTestAnnotationProcessor 'org.projectlombok:lombok:1.18.12'
integrationTestCompile group: 'org.junit.jupiter', name: 'junit-jupiter-api', version: '5.4.2'
integrationTestImplementation 'net.serenity-bdd:serenity-core:2.2.13'
integrationTestImplementation 'net.serenity-bdd:serenity-cucumber5:2.2.5'
integrationTestImplementation 'net.serenity-bdd:serenity-rest-assured:2.2.13'
integrationTestImplementation 'org.assertj:assertj-core:3.8.0'
integrationTestImplementation 'org.slf4j:slf4j-simple:1.7.7'
}
task integrationTest(type: Test) {
doFirst {
systemProperty 'param1',
System.getProperty('param1')
param1 = System.getProperty('param1')
if (param1 == null)
throw new GradleException("Parameter not passed ./gradlew -DParam<VALUE> ")
}
testClassesDirs = sourceSets.integrationTest.output.classesDirs
classpath = sourceSets.integrationTest.runtimeClasspath
outputs.upToDateWhen { false }
finalizedBy aggregate
}
Looking at the gradle file, there 3 things going on ,
There is a source set defined as integrationTest
The compileCLassPath , runTimeClassPath, srcDir has been defined for the same
Even the resources.srcDir is also being set (where I have application ymls)
I have create application.yml, application-integrationTest.yml inside the resources folder, but still unable to read any proporties using #Value.
Ex:
public class ParentClassWithAllMethods {
#Value("${custom.parameter}")
private String customParameter;
public ResponseOptions<Response> getMethod() {
String debugVal = customParameter;
}
.....
}
When , running the test, I was expecting to get the value defined in my yml file, but its null.
YML file looks like this,
custom:
parameter: https://something.com
Why am I not able to read the application configuration? It's always null !!
Your test class is not starting Spring, that is why you get null. Have a look here https://www.baeldung.com/cucumber-spring-integration, it shows how to connect Spring and Cucumber in JUnit tests.
The test class would then have annotations for Spring and Cucumber like this:
#CucumberContextConfiguration
#SpringBootTest
public class SpringIntegrationTest {
// executeGet implementation
}
I have into the same problem quite a lot of people have here, which is getting proper code coverage information when using Jacoco/Gradle and Powermock.
I have read all the various threads here and in other places and I have successfully managed to create a task (for Gradle 6.4) that does offline instrumentation of my project's classes. For reference the code that does this is the following:
task instrumentClasses(dependsOn: [ classes, project.configurations.jacocoAnt ]) {
inputs.files classes.outputs.files
File outputDir = new File(project.buildDir, 'instrumented')
outputs.dir outputDir
doFirst {
project.delete(outputDir)
ant.taskdef(
resource: 'org/jacoco/ant/antlib.xml',
classpath: project.configurations.jacocoAnt.asPath,
uri: 'jacoco'
)
def instrumented = false
jacocoOfflineSourceSets.each { sourceSetName ->
if (file(sourceSets[sourceSetName as String].output.classesDirs.singleFile.absolutePath).exists()) {
def instrumentedClassedDir = "${outputDir}/${sourceSetName}"
ant.'jacoco:instrument'(destdir: instrumentedClassedDir) {
fileset(dir: sourceSets[sourceSetName as String].output.classesDirs.singleFile, includes: '**/*.class')
}
//Replace the classes dir in the test classpath with the instrumented one
sourceSets.test.runtimeClasspath -= sourceSets[sourceSetName as String].output.classesDirs
sourceSets.test.runtimeClasspath += files(instrumentedClassedDir)
instrumented = true
}
}
if (instrumented) {
//Disable class verification based on https://github.com/jayway/powermock/issues/375
test.jvmArgs += '-noverify'
}
}
}
Now, for the most part this seems to work alright. I have successfully verified that my classes are now properly instrumented and I'm seeing a Jacoco produced report which has the correct information. Problem is though that my SonarQube server still lists the classes in question as non covered. Regarding this I have no idea as to what I need to do to resolve it.
For reference I am using the following version of the sonarqube plugin:
"org.sonarqube" version "2.7"
And my CI runs the Gradle task in the following manner:
- ./gradlew jacocoTestReport sonarqube ${SONAR_GRADLE_EXTRA_PARAMS} -Dsonar.projectKey=${CI_PROJECT_ID} -Dsonar.host.url=${SONAR_URL} -Dsonar.login=${SONAR_LOGIN} -Dsonar.branch.name=${CI_COMMIT_REF_NAME};
I do get that it must be some configuration issue with either SonarQube or the way I run the Gradle task but I am not really sure as to what is the culprit.
If you are able to generate the aggregated jacoco report (aggregated from all source sets), then you can simply specify that in your sonarqube task while running (and sonar will just pick the exact coverage info that jacoco calculated)
./gradlew sonarqube -Dsonar.host.url=https://sonarcloud.io -Dsonar.login=XXXX -Dsonar.organization=XXXXX -Dsonar.coverage.jacoco.xmlReportPaths=build/jacoco-report.xml
FYI I am creating the aggregated report at build/jacoco-report.xml
Below is my gradle configuration (might be useful for you)
plugins {
id 'org.springframework.boot' version '2.3.1.RELEASE'
id 'io.spring.dependency-management' version '1.0.9.RELEASE'
id 'java'
id 'jacoco'
id "org.sonarqube" version "2.8"
}
group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = JavaVersion.VERSION_11
repositories {
mavenCentral()
}
sourceSets {
intTest {
compileClasspath += sourceSets.main.output + sourceSets.test.output
runtimeClasspath += sourceSets.main.output + sourceSets.test.output
}
}
configurations {
intTestImplementation.extendsFrom testImplementation
intTestRuntimeOnly.extendsFrom testRuntimeOnly
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
test {
useJUnitPlatform()
testLogging.showStandardStreams = true //To print logs
}
task integrationTest(type: Test) {
testClassesDirs = sourceSets.intTest.output.classesDirs
classpath = sourceSets.intTest.runtimeClasspath
shouldRunAfter test
testLogging.showStandardStreams = true //To print logs
}
jacocoTestReport {
executionData(file("${project.buildDir}/jacoco/test.exec"), file("${project.buildDir}/jacoco/integrationTest.exec"))
reports {
xml.enabled true
csv.enabled false
xml.destination file("${buildDir}/jacoco-report.xml")
html.destination file("${buildDir}/jacocoHtml")
}
mustRunAfter(test, integrationTest) // integration tests are required to run before generating the report
}
jacocoTestCoverageVerification {
executionData(file("${project.buildDir}/jacoco/test.exec"), file("${project.buildDir}/jacoco/integrationTest.exec"))
violationRules {
rule {
limit {
counter = 'INSTRUCTION'
minimum = 0.94
}
limit {
counter = 'BRANCH'
minimum = 1.0
}
}
}
}
check.dependsOn(integrationTest, jacocoTestCoverageVerification)
tasks.withType(Test) {
finalizedBy jacocoTestReport
}
I have a Spring Boot Java project that builds using Gradle (v6.2.2).
build.gradle.kts
plugins {
base
java
id("org.springframework.boot") apply false
}
val gradleVersionProperty: String by project
val javaVersion: String by project
val springBootVersion: String by project
val springCloudStarterParentBomVersion: String by project
if (JavaVersion.current() != JavaVersion.VERSION_11) {
throw GradleException("This build must be run with JDK 11")
} else {
println("Building with JDK " + JavaVersion.current())
}
tasks.withType<Wrapper> {
gradleVersion = gradleVersionProperty
distributionType = Wrapper.DistributionType.ALL
}
allprojects {
group = "com.meanwhile.in.hell"
version = "$version"
// Repos used in dependencyManagement section of settings.gradle
repositories {
mavenLocal()
mavenCentral()
maven("https://repo.spring.io/snapshot")
maven("https://repo.spring.io/milestone")
}
}
subprojects {
if (!project.name.startsWith("platform")) {
apply {
plugin("java-library")
}
java.sourceCompatibility = JavaVersion.VERSION_11
java.targetCompatibility = JavaVersion.VERSION_11
// Change the default test logging settings
tasks.withType<Test>() {
useJUnitPlatform()
testLogging {
events = setOf(
org.gradle.api.tasks.testing.logging.TestLogEvent.FAILED,
org.gradle.api.tasks.testing.logging.TestLogEvent.PASSED,
org.gradle.api.tasks.testing.logging.TestLogEvent.SKIPPED
)
exceptionFormat = org.gradle.api.tasks.testing.logging.TestExceptionFormat.FULL
}
enableAssertions = false
ignoreFailures = gradle.startParameter.isContinueOnFailure
maxParallelForks =
(Runtime.getRuntime().availableProcessors() / 2).takeIf { it > 0 } ?: 1
}
dependencies {
"api"(enforcedPlatform("org.springframework.boot:spring-boot-dependencies:$springBootVersion"))
"api"(enforcedPlatform("org.springframework.cloud:spring-cloud-dependencies:$springCloudStarterParentBomVersion"))
"implementation"(enforcedPlatform(project(":platform-dependencies")))
"testCompile"("org.springframework.boot:spring-boot-starter-test")
}
}
}
However, I would like to add support for Spring Boot Kotlin sub-projects within it. I have used a very simple sample project from a Kotlin-only project I have that builds fine within it. Without any changes to my root build.gradle.kts file, my current build error is:
* What went wrong:
Execution failed for task ':kotlin-sample-project:bootJar'.
> Main class name has not been configured and it could not be resolved
I have not configured the main class for any of the Java sub-projects and neither have I in my Kotlin-only other project.
My build.gradle.kts in kotlin-sample-project is very simple:
plugins {
id("org.springframework.boot")
}
dependencies {
implementation("org.springframework.cloud:spring-cloud-starter-gateway")
}
And my main class looks like:
src/main/kotlin/sample/KotlinSampleApplication.kts
package com.meanwhile.in.hell.kotlinsample
import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
#SpringBootApplication
open class KotlinSampleApplication
fun main(args: Array<String>) {
runApplication<KotlinSampleApplication>(*args)
}
I have tried to add the kotlin plugin, but the build fails instantly not knowing what it is.
plugins {
base
java
kotlin
}
Error:
Line 9: kotlin
^ Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
public val <T : Any> Class<TypeVariable(T)>.kotlin: KClass<TypeVariable(T)> defined in kotlin.jvm
I have tried to add the kotlin plugin, but the build fails instantly not knowing what it is.
It should be:
build.gradle.kts:
plugins {
kotlin("jvm").version("1.3.72")
}
dependencies {
implementation(kotlin("stdlib-jdk8"))
}
Bot Kotlin JVM plugin should be applied and Kotlin stdlib be present on compile classpath.
We use in our java project jacoco with the gradle-plugin to calculate the coverage.
The problem is we put our classes which were generated from an XML in an extra project and resolved it as a dependency. We want the codecoverage of these model classes too to analyze it. To check if we used all setter methods of the datacontainers in our mapper classes. Do the code coverage in the model project is not an option.
Currently jacoco only shows in the report (html/xml/csv) only our classes which is in the main project but not the classes of external jars. The jacoco session contains the coverage data when I load it with eclipse or intellij.
buildscript {
...
dependencies {
...
classpath 'externalpackage:externalpackage-model'
...
}
}
compile('externalpackage:externalpackage-model:0.0.8')
testCompile('externalpackage:externalpackage-model:tests#jar')
testCompile('externalpackage:externalpackage-model:0.0.8:sources#jar')
jacoco {
toolVersion = "0.7.6.201602180812"
reportsDir = file("$buildDir/customJacocoReportDir")
}
jacocoTestReport {
reports {
xml.enabled true
csv.enabled true
html.enabled true
html.destination "${buildDir}/jacocoHtml"
}
additionalSourceDirs files('externalpackage:externalpagage:0.0.8:sources#jar')
//Doesn't work either
//additionalSourceDirs files('C:/Users/sero/Downloads/test/externalpackage-0.0.8-sources')
//additionalSourceDirs = files('C:/Users/sero/Downloads/test/externalpackage-0.0.8-sources/de/mycompany/.../MyModelClasses.java')
}
The jar source package is like this:
(root)/de/mycompany/.../MyModelClasses.java
Maybe someone has an idea
Found it by myself. Problem was, you need to specify the path the the classes too.
I unzipped the jars into the build folder and added additionalClassDirs and additionalSourceDirs to the report job.
This is the buildfile.
configurations {
externalClasses
externalSources
}
dependencies {
externalClasses "externalpackage:externalpackage-model:0.0.8#jar"
externalSources "externalpackage:externalpackage-model:0.0.8:sources#jar"
...
}
buildscript {
...
dependencies {
...
classpath 'externalpackage:externalpackage-model'
...
}
}
compile('externalpackage:externalpackage-model:0.0.8')
testCompile('externalpackage:externalpackage-model:tests#jar')
testCompile('externalpackage:externalpackage-model:0.0.8:sources#jar')
jacoco {
toolVersion = "0.7.6.201602180812"
reportsDir = file("$buildDir/customJacocoReportDir")
}
task unzipExternalModel(type: Copy){
from zipTree(configurations.externalSources.files.first())
into "$buildDir/tmp/externalSources"
from zipTree(configurations.externalClasses.files.first())
into "$buildDir/tmp/externalClasses"
}
jacocoTestReport {
dependsOn unzipExternalModel
reports {
xml.enabled true
csv.enabled true
html.enabled true
html.destination "${buildDir}/jacocoHtml"
}
additionalSourceDirs = files("$buildDir/tmp/externalSources")
additionalClassDirs = files("$buildDir/tmp/externalClasses")
}
Just for the record, using 7.1.1, this code (inpired on the Vincent's) works:
test {
useJUnitPlatform()
finalizedBy jacocoTestReport
jacoco {
includes = ["com.mycompany.**"]
includeNoLocationClasses true
}
}
task unzipLibrariesJar(type: Copy){
into "$buildDir/tmp/libClasses"
from {
configurations.runtimeClasspath
.filter { it.path.contains("mycompany") }
.collect {zipTree(it) }
}
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
task unzipLibrariesSrc(type: Copy){
into "$buildDir/tmp/libSources"
from {
configurations.runtimeClasspath
.filter { it.path.contains("mycompany") && new File(it.getPath().replace(".jar", "-sources.jar")).exists() }
.collect { new File(it.getPath().replace(".jar", "-sources.jar")) }
.collect { zipTree(it) }
}
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
jacocoTestReport {
dependsOn test
dependsOn unzipLibrariesJar
dependsOn unzipLibrariesSrc
reports {
html.enabled true
}
additionalClassDirs.from = files("$buildDir/tmp/libClasses")
additionalSourceDirs.from = files("$buildDir/tmp/libSources")
}
Note that, I am only interested on the logs from all classes of "com.mycompany" package. You should change this according to your requirements.
My gradle pitest is not able to give me the right results. It looks like it is not able to locate my test files.
I have the following build.gradle file:
apply plugin: "java" apply plugin: "maven" apply plugin: "info.solidsoft.pitest"
group = "myorg" version = 1.0
repositories {
mavenCentral() }
sourceSets.all { set ->
def jarTask = task("${set.name}Jar", type: Jar) {
baseName = baseName + "-$set.name"
from set.output
}
artifacts {
archives jarTask
} }
sourceSets {
api
impl main{ java { srcDir 'src/api/java' srcDir 'src/impl/java' } } test { java { srcDir 'src/test/java' } } }
buildscript {
repositories {
mavenCentral()
//Needed only for SNAPSHOT versions
//maven { url "http://oss.sonatype.org/content/repositories/snapshots/" }
}
dependencies {
classpath 'info.solidsoft.gradle.pitest:gradle-pitest-plugin:1.1.6'
} }
dependencies {
apiCompile 'commons-codec:commons-codec:1.5'
implCompile sourceSets.api.output
implCompile 'commons-lang:commons-lang:2.6'
testCompile 'junit:junit:4.9'
testCompile sourceSets.api.output
testCompile sourceSets.impl.output
runtime configurations.apiRuntime
runtime configurations.implRuntime }
jar {
from sourceSets.api.output
from sourceSets.impl.output }
pitest { println sourceSets.main
targetClasses = ['doubler.*'] targetTests = ['doubler.*'] verbose="on" }
THe output is stored in the correct folder. And when I run gradle test, it also runs fine.
Some additional information about this issue was supplied in the pitest user group.
https://groups.google.com/forum/#!topic/pitusers/8C7BHh-Vb6Y
The tests being run look like this.
#Test
public void testIt2() {
assert new DoublerImpl().testIt(1) == 2;
}
Pitest is correctly reporting that these tests provide 0% coverage of the class. There is no coverage because the assert keyword has been used.
Unless the -ea flag is set in the JVM running the tests assertions are disabled. There is basically hidden if block around this code generated by the compiler
#Test
public void testIt2() {
if (assertionsEnabled) {
assert new DoublerImpl().testIt(1) == 2;
}
}
As assertions are not enabled no code is executed.
To fix the issue use the built in JUnit assertions instead.
http://junit.sourceforge.net/javadoc/org/junit/Assert.html