We use in our java project jacoco with the gradle-plugin to calculate the coverage.
The problem is we put our classes which were generated from an XML in an extra project and resolved it as a dependency. We want the codecoverage of these model classes too to analyze it. To check if we used all setter methods of the datacontainers in our mapper classes. Do the code coverage in the model project is not an option.
Currently jacoco only shows in the report (html/xml/csv) only our classes which is in the main project but not the classes of external jars. The jacoco session contains the coverage data when I load it with eclipse or intellij.
buildscript {
...
dependencies {
...
classpath 'externalpackage:externalpackage-model'
...
}
}
compile('externalpackage:externalpackage-model:0.0.8')
testCompile('externalpackage:externalpackage-model:tests#jar')
testCompile('externalpackage:externalpackage-model:0.0.8:sources#jar')
jacoco {
toolVersion = "0.7.6.201602180812"
reportsDir = file("$buildDir/customJacocoReportDir")
}
jacocoTestReport {
reports {
xml.enabled true
csv.enabled true
html.enabled true
html.destination "${buildDir}/jacocoHtml"
}
additionalSourceDirs files('externalpackage:externalpagage:0.0.8:sources#jar')
//Doesn't work either
//additionalSourceDirs files('C:/Users/sero/Downloads/test/externalpackage-0.0.8-sources')
//additionalSourceDirs = files('C:/Users/sero/Downloads/test/externalpackage-0.0.8-sources/de/mycompany/.../MyModelClasses.java')
}
The jar source package is like this:
(root)/de/mycompany/.../MyModelClasses.java
Maybe someone has an idea
Found it by myself. Problem was, you need to specify the path the the classes too.
I unzipped the jars into the build folder and added additionalClassDirs and additionalSourceDirs to the report job.
This is the buildfile.
configurations {
externalClasses
externalSources
}
dependencies {
externalClasses "externalpackage:externalpackage-model:0.0.8#jar"
externalSources "externalpackage:externalpackage-model:0.0.8:sources#jar"
...
}
buildscript {
...
dependencies {
...
classpath 'externalpackage:externalpackage-model'
...
}
}
compile('externalpackage:externalpackage-model:0.0.8')
testCompile('externalpackage:externalpackage-model:tests#jar')
testCompile('externalpackage:externalpackage-model:0.0.8:sources#jar')
jacoco {
toolVersion = "0.7.6.201602180812"
reportsDir = file("$buildDir/customJacocoReportDir")
}
task unzipExternalModel(type: Copy){
from zipTree(configurations.externalSources.files.first())
into "$buildDir/tmp/externalSources"
from zipTree(configurations.externalClasses.files.first())
into "$buildDir/tmp/externalClasses"
}
jacocoTestReport {
dependsOn unzipExternalModel
reports {
xml.enabled true
csv.enabled true
html.enabled true
html.destination "${buildDir}/jacocoHtml"
}
additionalSourceDirs = files("$buildDir/tmp/externalSources")
additionalClassDirs = files("$buildDir/tmp/externalClasses")
}
Just for the record, using 7.1.1, this code (inpired on the Vincent's) works:
test {
useJUnitPlatform()
finalizedBy jacocoTestReport
jacoco {
includes = ["com.mycompany.**"]
includeNoLocationClasses true
}
}
task unzipLibrariesJar(type: Copy){
into "$buildDir/tmp/libClasses"
from {
configurations.runtimeClasspath
.filter { it.path.contains("mycompany") }
.collect {zipTree(it) }
}
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
task unzipLibrariesSrc(type: Copy){
into "$buildDir/tmp/libSources"
from {
configurations.runtimeClasspath
.filter { it.path.contains("mycompany") && new File(it.getPath().replace(".jar", "-sources.jar")).exists() }
.collect { new File(it.getPath().replace(".jar", "-sources.jar")) }
.collect { zipTree(it) }
}
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
jacocoTestReport {
dependsOn test
dependsOn unzipLibrariesJar
dependsOn unzipLibrariesSrc
reports {
html.enabled true
}
additionalClassDirs.from = files("$buildDir/tmp/libClasses")
additionalSourceDirs.from = files("$buildDir/tmp/libSources")
}
Note that, I am only interested on the logs from all classes of "com.mycompany" package. You should change this according to your requirements.
Related
I have the following classes in separated files:
#Path("/products")
#Produces(MediaType.APPLICATION_JSON)
class ProductController{
#Inject
private ProductService productService;
}
#ApplicationScoped
public class ProductService {
#Inject
private ProductRepository productRepository;
}
#ApplicationScoped
public class ProductRepository implements PanacheRepositoryBase<Product, UUID> {
}
And when I run the tests coverage, all controllers and services and other packages are correctly recognized by JaCoCo, but all repositories aren't, resulting in 0% coverage in this package. And I'm sure that they are being used by services.
My guess is because JaCoCo only recognizes up to 1 level of "proxy", due to #AppicationScoped annotation, but I'm not sure.
I also tried offline intrumentation, but the result is the same.
This is my gradle config now:
plugins {
id 'java'
id 'io.quarkus'
id 'jacoco'
}
repositories {
mavenLocal()
mavenCentral()
}
dependencies {
// a lot of deps not related to tests
}
java {
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11
}
compileJava {
options.encoding = 'UTF-8'
options.compilerArgs << '-parameters'
}
compileTestJava {
options.encoding = 'UTF-8'
}
test {
useJUnitPlatform()
jacoco {
destinationFile = file("$buildDir/jacoco/test.exec")
}
finalizedBy jacocoTestReport
}
jacoco {
toolVersion = '0.8.6'
reportsDir = file("$buildDir/reports/jacoco")
}
jacocoTestCoverageVerification {
executionData fileTree("$buildDir/jacoco/").include("*.exec")
afterEvaluate {
classDirectories.setFrom(files(classDirectories.files.collect {
fileTree(dir: it)
}))
}
}
jacocoTestReport {
executionData fileTree("$buildDir/jacoco/").include("*.exec")
reports {
xml.enabled false
csv.enabled false
html.destination file("${buildDir}/reports/jacoco")
}
afterEvaluate {
classDirectories.setFrom(files(classDirectories.files.collect {
fileTree(dir: it)
}))
}
}
And this is the other "version" of gradle config, with offline instrumentation:
configurations {
jacocoAnt
jacocoRuntime
}
dependencies {
jacocoAnt group: 'org.jacoco', name: 'org.jacoco.ant', version: '0.8.5', classifier: 'nodeps'
jacocoRuntime group: 'org.jacoco', name: 'org.jacoco.agent', version: '0.8.5', classifier: 'runtime'
}
test {
useJUnitPlatform()
jacoco {
destinationFile = file("$buildDir/reports/jacoco/jacoco-sonar/jacoco-coverage.exec")
}
}
jacoco {
toolVersion = "0.8.5"
reportsDir = file("$buildDir/customJacocoReportDir")
}
jacocoTestReport {
reports {
xml.enabled false
csv.enabled false
html.destination file("${buildDir}/jacocoHtml")
}
}
jacocoTestCoverageVerification {
executionData fileTree("$buildDir/reports/jacoco/jacoco-sonar/").include("*.exec")
afterEvaluate {
classDirectories.setFrom(files(classDirectories.files.collect {
fileTree(dir: it)
}))
}
violationRules {
rule {
limit {
minimum = 0.74
}
}
}
}
task instrument(dependsOn: ['classes']) {
ext.outputDir = buildDir.path + '/reports/classes-instrumented'
doLast {
ant.taskdef(name: 'instrument',
classname: 'org.jacoco.ant.InstrumentTask',
classpath: configurations.jacocoAnt.asPath)
ant.instrument(destdir: outputDir) {
sourceSets.main.output.classesDirs.each { fileset(dir: it) }
}
}
}
gradle.taskGraph.whenReady { graph ->
if (graph.hasTask(instrument)) {
tasks.withType(Test) {
doFirst {
classpath = files(instrument.outputDir) + classpath + configurations.jacocoRuntime
}
}
}
}
task report(dependsOn: ['instrument', 'test']) {
doLast {
ant.taskdef(name: 'report',
classname: 'org.jacoco.ant.ReportTask',
classpath: configurations.jacocoAnt.asPath)
ant.report() {
executiondata {
ant.file(file: buildDir.path + '/reports/jacoco/jacoco-sonar/jacoco-coverage.exec')
}
structure(name: 'Example') {
classfiles {
sourceSets.main.output.classesDirs.each { fileset(dir: it) }
}
sourcefiles {
fileset(dir: 'src/main/java')
}
}
html(destdir: buildDir.path + '/reports/jacoco')
}
}
}
Why it happens?
I was facing the same problem as you, after include the following extension the coverage problem for #ApplicationScoped annotated classes was solved.
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-jacoco</artifactId>
<scope>test</scope>
</dependency>
Obviously, since you are using Gradle, you should change it. I also needed to remove jacoco-maven-plugin plugin from maven plugins section. Probably you will need to remove everything that you included for custom instrumentation.
Quarkus bytecode instrumentation and JaCoCo can step on each other toes, JaCoCo default mode uses an agent the inject some bytecode and this can be incompatible with Quarkus' own bytecode injection.
You can switch JaCoCo to offline instrumentation instead, please follow this section of the Quarkus test coverage guide:
https://quarkus.io/guides/tests-with-coverage#instrumenting-the-classes-instead
I have into the same problem quite a lot of people have here, which is getting proper code coverage information when using Jacoco/Gradle and Powermock.
I have read all the various threads here and in other places and I have successfully managed to create a task (for Gradle 6.4) that does offline instrumentation of my project's classes. For reference the code that does this is the following:
task instrumentClasses(dependsOn: [ classes, project.configurations.jacocoAnt ]) {
inputs.files classes.outputs.files
File outputDir = new File(project.buildDir, 'instrumented')
outputs.dir outputDir
doFirst {
project.delete(outputDir)
ant.taskdef(
resource: 'org/jacoco/ant/antlib.xml',
classpath: project.configurations.jacocoAnt.asPath,
uri: 'jacoco'
)
def instrumented = false
jacocoOfflineSourceSets.each { sourceSetName ->
if (file(sourceSets[sourceSetName as String].output.classesDirs.singleFile.absolutePath).exists()) {
def instrumentedClassedDir = "${outputDir}/${sourceSetName}"
ant.'jacoco:instrument'(destdir: instrumentedClassedDir) {
fileset(dir: sourceSets[sourceSetName as String].output.classesDirs.singleFile, includes: '**/*.class')
}
//Replace the classes dir in the test classpath with the instrumented one
sourceSets.test.runtimeClasspath -= sourceSets[sourceSetName as String].output.classesDirs
sourceSets.test.runtimeClasspath += files(instrumentedClassedDir)
instrumented = true
}
}
if (instrumented) {
//Disable class verification based on https://github.com/jayway/powermock/issues/375
test.jvmArgs += '-noverify'
}
}
}
Now, for the most part this seems to work alright. I have successfully verified that my classes are now properly instrumented and I'm seeing a Jacoco produced report which has the correct information. Problem is though that my SonarQube server still lists the classes in question as non covered. Regarding this I have no idea as to what I need to do to resolve it.
For reference I am using the following version of the sonarqube plugin:
"org.sonarqube" version "2.7"
And my CI runs the Gradle task in the following manner:
- ./gradlew jacocoTestReport sonarqube ${SONAR_GRADLE_EXTRA_PARAMS} -Dsonar.projectKey=${CI_PROJECT_ID} -Dsonar.host.url=${SONAR_URL} -Dsonar.login=${SONAR_LOGIN} -Dsonar.branch.name=${CI_COMMIT_REF_NAME};
I do get that it must be some configuration issue with either SonarQube or the way I run the Gradle task but I am not really sure as to what is the culprit.
If you are able to generate the aggregated jacoco report (aggregated from all source sets), then you can simply specify that in your sonarqube task while running (and sonar will just pick the exact coverage info that jacoco calculated)
./gradlew sonarqube -Dsonar.host.url=https://sonarcloud.io -Dsonar.login=XXXX -Dsonar.organization=XXXXX -Dsonar.coverage.jacoco.xmlReportPaths=build/jacoco-report.xml
FYI I am creating the aggregated report at build/jacoco-report.xml
Below is my gradle configuration (might be useful for you)
plugins {
id 'org.springframework.boot' version '2.3.1.RELEASE'
id 'io.spring.dependency-management' version '1.0.9.RELEASE'
id 'java'
id 'jacoco'
id "org.sonarqube" version "2.8"
}
group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = JavaVersion.VERSION_11
repositories {
mavenCentral()
}
sourceSets {
intTest {
compileClasspath += sourceSets.main.output + sourceSets.test.output
runtimeClasspath += sourceSets.main.output + sourceSets.test.output
}
}
configurations {
intTestImplementation.extendsFrom testImplementation
intTestRuntimeOnly.extendsFrom testRuntimeOnly
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
test {
useJUnitPlatform()
testLogging.showStandardStreams = true //To print logs
}
task integrationTest(type: Test) {
testClassesDirs = sourceSets.intTest.output.classesDirs
classpath = sourceSets.intTest.runtimeClasspath
shouldRunAfter test
testLogging.showStandardStreams = true //To print logs
}
jacocoTestReport {
executionData(file("${project.buildDir}/jacoco/test.exec"), file("${project.buildDir}/jacoco/integrationTest.exec"))
reports {
xml.enabled true
csv.enabled false
xml.destination file("${buildDir}/jacoco-report.xml")
html.destination file("${buildDir}/jacocoHtml")
}
mustRunAfter(test, integrationTest) // integration tests are required to run before generating the report
}
jacocoTestCoverageVerification {
executionData(file("${project.buildDir}/jacoco/test.exec"), file("${project.buildDir}/jacoco/integrationTest.exec"))
violationRules {
rule {
limit {
counter = 'INSTRUCTION'
minimum = 0.94
}
limit {
counter = 'BRANCH'
minimum = 1.0
}
}
}
}
check.dependsOn(integrationTest, jacocoTestCoverageVerification)
tasks.withType(Test) {
finalizedBy jacocoTestReport
}
I'm testing an app with JUnit5 and using Jacoco for coverage report.
Tests are executed Ok and test reports are present.
However, Jacoco report has the following logs if service contains methods, annotated with #Transactional
[ant:jacocoReport] Classes in bundle 'my-service' do no match with execution data. For report generation the same class files must be used as at runtime.
[ant:jacocoReport] Execution data for class mypackage/SampleService does not match.
This error occurres for all #Service classes methods, annotated with #Transactional, plain classes coverage is calculated ok.
Here's a sample test:
#SpringBootTest
#ExtendWith(SpringExtension.class)
public class MyServiceTest {
#Autowired
private SampleService sampleService;
#Test
public void doWork(){
sampleService.doWork();
}
}
Works fine. Coverage is non-zero:
public class SampleService {
public void doWork(){
System.out.println("HEY");
}
}
0% coverage:
public class SampleService {
#Transactional
public void doWork(){
System.out.println("HEY");
}
}
Transactional creates a proxy around actuall class. But, isn't there an out-of-box way for Jacoco to handle such a common situation?
I've tried #EnableAspectJAutoProxy annotaion with different flag variations, checked that up-to-date Jupiter engine and Jacoco plugin are used
Here's gradle config:
subprojects {
test {
useJUnitPlatform()
}
jacocoTestReport {
afterEvaluate {
classDirectories.from = files(classDirectories.files.collect {
fileTree(dir: it, exclude: '*Test.java')
})
}
reports {
html.enabled = true
xml.enabled = true
csv.enabled = false
}
}
}
Any help appreciated
I tried it with a test project, similar to what you've described, however I couldn't reproduce the issue. The only difference that I see between your project and mine, is that I've used maven instead of gradle.
Here is the test project:
https://github.com/gybandi/jacocotest
And here is the jacoco result for it (using org.springframework.transaction.annotation.Transactional annotation):
If this doesn't help you, could you upload your test project to github or some other place?
Edit:
#MikaelF posted a link to another answer, which shows how to add offline instrumentation for jacoco.
The solution that was described there worked for me, after I added the following block to build.gradle:
task instrument(dependsOn: [classes, project.configurations.jacocoAnt]) {
inputs.files classes.outputs.files
File outputDir = new File(project.buildDir, 'instrumentedClasses')
outputs.dir outputDir
doFirst {
project.delete(outputDir)
ant.taskdef(
resource: 'org/jacoco/ant/antlib.xml',
classpath: project.configurations.jacocoAnt.asPath,
uri: 'jacoco'
)
def instrumented = false
if (file(sourceSets.main.java.outputDir).exists()) {
def instrumentedClassedDir = "${outputDir}/${sourceSets.main.java}"
ant.'jacoco:instrument'(destdir: instrumentedClassedDir) {
fileset(dir: sourceSets.main.java.outputDir, includes: '**/*.class')
}
//Replace the classes dir in the test classpath with the instrumented one
sourceSets.test.runtimeClasspath -= files(sourceSets.main.java.outputDir)
sourceSets.test.runtimeClasspath += files(instrumentedClassedDir)
instrumented = true
}
if (instrumented) {
//Disable class verification based on https://github.com/jayway/powermock/issues/375
test.jvmArgs += '-noverify'
}
}
}
test.dependsOn instrument
There seems to be an open ticket on the jacoco plugin's github about this as well:
https://github.com/gradle/gradle/issues/2429
Based on single module instrumentation example https://stackoverflow.com/a/31916686/7096763 (and updated version for gradle 5+ by #MikaelF) here's example for multimodule instrumentation:
subprojects { subproject ->
subproject.ext.jacocoOfflineSourceSets = [ 'main' ]
task doJacocoOfflineInstrumentation(dependsOn: [ classes, subproject.configurations.jacocoAnt ]) {
inputs.files classes.outputs.files
File outputDir = new File(subproject.buildDir, 'instrumentedClasses')
outputs.dir outputDir
doFirst {
project.delete(outputDir)
ant.taskdef(
resource: 'org/jacoco/ant/antlib.xml',
classpath: subproject.configurations.jacocoAnt.asPath,
uri: 'jacoco'
)
def instrumented = false
jacocoOfflineSourceSets.each { sourceSetName ->
if (file(sourceSets[sourceSetName].output.classesDirs[0]).exists()) {
def instrumentedClassedDir = "${outputDir}/${sourceSetName}"
ant.'jacoco:instrument'(destdir: instrumentedClassedDir) {
fileset(dir: sourceSets[sourceSetName].output.classesDirs[0], includes: '**/*.class')
}
//Replace the classes dir in the test classpath with the instrumented one
sourceSets.test.runtimeClasspath -= files(sourceSets[sourceSetName].output.classesDirs[0])
sourceSets.test.runtimeClasspath += files(instrumentedClassedDir)
instrumented = true
}
}
if (instrumented) {
//Disable class verification based on https://github.com/jayway/powermock/issues/375
test.jvmArgs += '-noverify'
}
}
}
test.dependsOn doJacocoOfflineInstrumentation
}
full example here:
https://github.com/lizardeye/jacocomultimodulesample
Still, I think this is a durty hack, which can be easily broken with gradle or jacoco updates
I have a multimodule project and some TestNG tests, covering them.
My gradle config looks like:
subprojects { subproject ->
jacocoTestReport {
additionalSourceDirs = files(sourceSets.main.allSource.srcDirs)
classDirectories = files(sourceSets.main.output)
sourceDirectories = files(sourceSets.main.allSource.srcDirs)
reports {
html.enabled = true
xml.enabled = true
csv.enabled = false
}
}
test {
useTestNG()
}
test.finalizedBy(project.tasks.jacocoTestReport)
}
In the report I expected to see code coverage based on sources from main package (provided by sourceSets.main),however all java classes from test package are also included. So the coverage result is invalid.
How can the config be fixed?
All my tests had a naming pattern, so I could exclude them by regex:
jacocoTestReport {
additionalSourceDirs = files(sourceSets.main.allSource.srcDirs)
classDirectories = files(sourceSets.main.output)
sourceDirectories = files(sourceSets.main.allSource.srcDirs)
afterEvaluate {
classDirectories = files(classDirectories.files.collect {
fileTree(dir: it, exclude: 'path/to/test/*Test.java')
})
}
reports {
html.enabled = true
xml.enabled = true
csv.enabled = false
}
}
My gradle pitest is not able to give me the right results. It looks like it is not able to locate my test files.
I have the following build.gradle file:
apply plugin: "java" apply plugin: "maven" apply plugin: "info.solidsoft.pitest"
group = "myorg" version = 1.0
repositories {
mavenCentral() }
sourceSets.all { set ->
def jarTask = task("${set.name}Jar", type: Jar) {
baseName = baseName + "-$set.name"
from set.output
}
artifacts {
archives jarTask
} }
sourceSets {
api
impl main{ java { srcDir 'src/api/java' srcDir 'src/impl/java' } } test { java { srcDir 'src/test/java' } } }
buildscript {
repositories {
mavenCentral()
//Needed only for SNAPSHOT versions
//maven { url "http://oss.sonatype.org/content/repositories/snapshots/" }
}
dependencies {
classpath 'info.solidsoft.gradle.pitest:gradle-pitest-plugin:1.1.6'
} }
dependencies {
apiCompile 'commons-codec:commons-codec:1.5'
implCompile sourceSets.api.output
implCompile 'commons-lang:commons-lang:2.6'
testCompile 'junit:junit:4.9'
testCompile sourceSets.api.output
testCompile sourceSets.impl.output
runtime configurations.apiRuntime
runtime configurations.implRuntime }
jar {
from sourceSets.api.output
from sourceSets.impl.output }
pitest { println sourceSets.main
targetClasses = ['doubler.*'] targetTests = ['doubler.*'] verbose="on" }
THe output is stored in the correct folder. And when I run gradle test, it also runs fine.
Some additional information about this issue was supplied in the pitest user group.
https://groups.google.com/forum/#!topic/pitusers/8C7BHh-Vb6Y
The tests being run look like this.
#Test
public void testIt2() {
assert new DoublerImpl().testIt(1) == 2;
}
Pitest is correctly reporting that these tests provide 0% coverage of the class. There is no coverage because the assert keyword has been used.
Unless the -ea flag is set in the JVM running the tests assertions are disabled. There is basically hidden if block around this code generated by the compiler
#Test
public void testIt2() {
if (assertionsEnabled) {
assert new DoublerImpl().testIt(1) == 2;
}
}
As assertions are not enabled no code is executed.
To fix the issue use the built in JUnit assertions instead.
http://junit.sourceforge.net/javadoc/org/junit/Assert.html