I am using jacoco for Test coverage verification in a Java Library created with Gradle. I have set the verification rule to limit minimum coverage to 1.0 (100%). I have a few classes in my code which are not meeting the criteria. based on the unit-tests that I have written, I don't think I am missing any line of code.
on those lines, I was wondering if there is a was to make jacoco print out which lines are not getting covered by unit tests during the build process?
currently jacoco causes a build failure reporting the coverage ration for each files that didn't meet the bar but it does not provide any information on specific lines that were not covered.
I was able to configure My Gradle-Groovy project to start generating Jacoco coverage reports by doing the following:
added jacoco plugin to buildscript in build.gradle
apply plugin: 'jacoco'
adding a custom configuration:
jacocoTestReport {
dependsOn test
}
chaining jacocoTestReport to your test task
test {
finalizedBy jacocoTestReport
}
This ensures, report is always generated after tests run.
Note: if you have added both jacoco and java plugin, reports are automatically generated and stored at $buildDir/reports/jacoco/test in such a scenario, you can skip above steps.
Related
I have a multi-module Android project and I'm seeing a discrepancy between the coverage that jacoco reports and what Sonarqube reports.
The project is also a multi-flavor project that generates a lot of different variants. I am using this plugin to help me generate all of the tasks. The tasks generate an individual report for each variant.
When I run my jacoco reports I see this:
When I run the sonar scanner i see this:
I have some exclusions on my project,but even without them the coverage %s don't match.
I feel like I may not be providing the same bytecode as hinted in this question:
Here is my relevant info:
Sonar Version 5.6.
Gradle runner
plugins { id "org.sonarqube" version "2.0.1" }
Sonar config: (on root build.gradle)
sonarqube {
properties {
property "sonar.projectKey", "com.xxx.myApp"
property "sonar.projectName", "Android My App"
property "sonar.projectVersion", "3.0"
property "sonar.java.binaries", "build/classes"
property "sonar.coveragePlugin", "jacoco"
property "sonar.jacoco.reportMissing.force.zero", "false"
}
}
Sonar config (on app/build.gradle)
sonarqube {
properties {
property "sonar.sources", "src/main/java"
property "sonar.tests", "src/test/java"
property "sonar.java.tests", "src/test/java"
property "sonar.junit.reportsPath", "build/test-results/myAppGoogleMobileDebug"
property "sonar.java.binaries", "build/intermediates/classes/myAppGoogleMobile/debug"
property "sonar.jacoco.reportPath", "build/jacoco/testMyAppGoogleMobileDebugUnitTest.exec"
property "sonar.coverage.exclusions", coverageExclusions
}
}
Jacoco config on (app/build.gradle)
def coverageExclusions = ['**/AEWatchApp.*', '**/**_Factory.*',
'**/QaSettingsActivity.*',
'com/aetn/android/tveapps/activities/**',
'com/aetn/android/tveapps/test/**',
'com/aetn/android/tveapps/app/injection/modules/**',
'com/aetn/android/tveapps/app/injection/components/**',
'com.aetn.android.tveapps.mock/**',
'com/aetn/android/tveapps/databinding/**']
jacocoAndroidUnitTestReport {
csv.enabled false
html.enabled true
xml.enabled true
excludes += coverageExclusions
}
As far as I can see branch coverage is the same: 40% in both cases, 15 uncovered.
And comparison of "instructions" (shown in screenshot of JaCoCo report) with anything else is like comparison of apples and oranges - they don't represent the same thing. See http://www.eclemma.org/jacoco/trunk/doc/counters.html about counters that JaCoCo provides. And https://docs.sonarqube.org/latest/user-guide/metric-definitions about what SonarQube shows. Instructions coverage is presented only in JaCoCo.
"lines" ("27.1%" shown in screenshot of SonarQube) is not the same as "instructions": Single line of code usually contains many bytecode instructions. So for example if in total you have 100 instructions in 10 lines and cover 1 line with 20 instructions, then missed instructions 80%, but missed lines 90%.
So all in all, there is no discrepancy or at least it is not shown on your screenshots.
Gradle 'Jacoco' plugin's 'jacocoTestReport' generates code coverage report for all the unit tests.
How should I proceed to create a gradle task to generate 'Jacoco' code coverage reports of each unit test individually?
HelloWorld.java
HelloWorldTest.java contains Test1, Test2, Test3 methods
I want to generate individual Jacoco code coverage report for Test1, Test2, Test3 methods.
During running of the tests, Jacoco instruments the classes and records what was called. The task of type JacocoReport then just takes these results and makes an XML, HTML or CSV report out of them.
So to get what you want, you need to add several Test tasks that execute the single tests and then add several JacocoReport tasks that point to the different result files, then you can generate these reports in one run.
If you only want to do this manually, I think you can just call Gradle like gradlew test --tests HelloWorld.Test1 jacocoTestReport.
In the latter case it might be necessary to also set test { jacoco { append false } } to not have the results of former runs in the report.
I am using Gradle jacoco plugin and have JUnit tests in a gradle multi project.
I have installed jacoco in order to report to sonarqube my code coverage, which I did.
Gitlab-CI also offer a badge for the code coverage, I only need to output the code coverage in a terminal and use a regex to find it.
Is there a way to output the code coverage of all the project in the terminal ?
Yes, you can. Possible ways:
1) Use Gradle plugin gradle-console-reporter to report various kinds of summaries to console. JUnit, JaCoCo and Cobertura reports are supported.
In your case, following output will be printed to console:
...
BUILD SUCCESSFUL
Total time: 4.912 secs
Coverage summary:
project1: 72.2%
project2-with-long-name: 44.4%
Then you can use coverage with regular expression in Gitlab's .gitlab-ci.yml to parse code coverage.
2) Second option is a little bit tricky. You can print full JaCoCo HTML report (e.g. using cat target/site/jacoco/index.html) and then use regular expression (see this post) or grep (see this post) to parse coverage.
AFAIK Gradle does not support this, each project is treated separately.
To support your use case some aggregation task can be created to parse a report and to update some value at root project and finally print that value to stdout.
Update with approximate code for solution:
subprojects {
task aggregateCoverage {
// parse report from each module into ext.currentCoverage
rootProject.ext.coverage += currentCoverage
}
}
I'm using Maven Surefire plugin to run my project's unit tests and Jacoco for code coverage. I have a strict coverage criteria in my project but my problem is that logging level specific lines are not captured by the Jacoco and messing up with my covering stats.
For example:
if (log.isDebugEnabled()) {
log.debug("Log entry");
}
Will fail to identify the log entry code line. Any way to work around it?
I'm working on setting up the Jacoco Gradle plugin for my project. The plugin is executing fine and generating the coverage report but it's not excluding packages that I would excluded.
Classes I would like to include are in com/xxx/ws/hn/** and classes I would like excluded are in com/xxx/ws/enterprise/**.
Here's my Gradle script:
apply plugin: "jacoco"
jacoco {
toolVersion = "0.6.2.201302030002"
reportsDir = file("$buildDir/reports/jacoco")
}
test {
jacoco{
excludes = ["com/xx/ws/enterprise/**/*"]
includes = ["com/xxx/ws/hn/**/*"]
append = false
}
}
jacocoTestReport {
dependsOn test
description = "Generate Jacoco coverage reports after running tests."
executionData = files('build/jacoco/test.exec')
reports {
xml.enabled true
html.enabled true
}
}
Am I missing anything here? I've tried various patterns of exclusions including a '.' delimiter for packages instead of a '/' but nothing seems to work. Any help would be greatly appreciated.
I have not verified this but when I read the JaCoCo documentation, it states that wildcards can be used but all examples only have ONE *, not two as your example shows.
http://www.eclemma.org/jacoco/trunk/doc/agent.html
To solidify (and verify) Joachim's answer, I came across this SO post and found that specifying canonical class names, classpath style (with slashes, not periods), with single wildcard characters actually worked well. For example, here is my exclude list that works in my build, as defined as a list in a separate gradle file:
def testExcl = [
'android/*',
'com/android/*',
'com/nativelibs4java/*',
'com/ochafik/*',
'com/fasterxml/*',
'com/esotericsoftware/*',
'com/google/*',
'com/lmax/*',
'com/sun/*',
'jdk/*',
'mockit/*',
'org/apache/*',
'org/bridj/*',
'org/gradle/*',
'org/hamcrest/*',
'org/junit/*',
'org/slf4j/*',
'sun/*',
'worker/*',
'*Test',
'*TestIntegration',
'*AllTests',
'*Suite'
]
I tried probably 10 or 12 other ways of formatting the class names, and this is the only way that actually worked.
I don't know why something like this that is so simple/fundamental is so ungoogleable and difficult to find a definitive answer on in the context of Gradle (i.e. outside JaCoCo's official documentation, where no documentation exists about Gradle integration, but Ant and Maven integration docs are there).
Hopefully this helps if others are flailing trying to get JaCoCo to exclude classes.
UPDATE:
This changed in Gradle 7 (I went from 5.6.1 to 7.1.1, so not sure about Gradle 6) and no longer works.
For Gradle 7 you need to replace / with .; here are my updated exclusions:
def testExcl = [
'android.*',
'com.android.*',
'com.nativelibs4java.*',
'com.ochafik.*',
'com.fasterxml.*',
'com.esotericsoftware.*',
'com.google.*',
'com.lmax.*',
'com.sun.*',
'jdk.*',
'mockit.*',
'org.apache.*',
'org.bridj.*',
'org.gcpud.common.*',
'org.gcpud.testutil.*',
'org.gradle.*',
'org.hamcrest.*',
'org.junit.*',
'org.slf4j.*',
'sun.*',
'worker.*',
'*Test',
'*Test$*',
'*TestIntegration',
'*AllTests',
'*Suite'
]
Also see the following SO post for other means of excluding classes from reporting (exclude from reporting, not coverage itself):
Filter JaCoCo coverage reports with Gradle