Background:
I have 2 separate java projects, call them A and B.
Project A is the actual product (war application), with unit-tests. Gradle builds the project and then runs sonar analysis, and I can see the unit-tests coverage in Sonar.
Project B is an integration test for the first project. It is run by Jenkins in a pipeline after building project A and deploying it on an integration-environment. The deployment also involves instrumenting the code so that the jacoco-it report will correlate to Project A's classes.
My question:
How can I add to project A's sonar page, which currently has only unit-tests coverage, the integration tests coverage - as a second step?
The flow I need is:
build project A
run sonar analysis on project A (now it'll show unit-tests coverage on its sonar page)
build project B
run sonar analysis on project B - which will simply add integration-coverage to project A on sonar
It's currently not working. when I run sonar analysis on project B it messes up project A's sonar page, removing the unit-test coverage of A.
The flow that I have now, which is working but I want to change is:
build project A
run sonar analysis on project A
build project B
run jacoco report on project B. this outputs the jacoco-it.exec file into a specific location on my disk
run sonar analysis on project A (it has a setting to take the jacoco-it.exec file from the specified location
now project A's sonar page will show both unit-tests and integration-tests coverage, but step 5 is completely redundant and I want to avoid it.
Any suggestions?
what i will suggest is a bit different:
employ some build automation tool
create a sonar 'checker' job, which doesnt do the full analysis; just the incremental, this should break when add quality gate breaker issues
create a nightly job for your A project with full analysis: -Dsonar.analysis.mode=analysis (note: my experience shows that in this case you must run an incremental prior to this analysis, because in analysis mode sonar accepts new quality gate breaker issues
i don't think integration tests should count into project test coverage, so that might be just left out.
You cannot amend a previous analysis, which is why your 'B' job appears to replace the unit test numbers. What you need to do is generate your integration test coverage report before analysis, and make that report available so that both unit test and integration test numbers can be read in the same analysis.
From a build pipeline standpoint, that could get tricky depending on how you need to structure the jobs, but off-hand it sounds like you need 3 jobs:
build project A
build project B & generate integration test report
job 3 pulls integration test report from job 2, and either rebuilds project A or pulls code, classes, and unit test report from job 1 and performs analysis
Related
hi all i have build unit testing for controller, repository and Service but why in my sonarqube code coverage always give me code coverage only 0 percent. my question is how to make my percentage up ?
here what i was code in my testing
Sonarqube uses existing code coverage reports from JaCoCo (in the case of Java). Usually, you would set up the JaCoCo Maven plugin (or Gradle) to gather coverage info on test run and Sonarqube then loads this report.
See also the Sonarqube docs for info on setup. But if you are using Maven/Gradle, I believe Sonarqube is able to automatically pick up the correct file unless you have some special configuration.
I have a setup for multi Module module project something like this
Module1
|
submodule1
|
submodule2
I have written a Junit test in submodule 1 and it's covering the code of submodule 2 also but when I try to see coverage in sonar it's showing 0% for submodule 2 is there any way to show coverage of submodule2 ? Also I can generate aggregated xml report by running mvn clean install jacoco:report-aggregate but how can I feed this aggregated report into sonar ? How to setup pom of module as well as submodules ?
I'm not sure if this is what you are doing, but I'll say it anyway. If you are creating submodule1 as 'code' and submodule2 as 'tests', that's a horrible way to organize it. Tests (specifically, unit tests) should be stored with the code that it is testing. Thus, each submodule has src/main/java and src/test/java.
Now, that said, it is plausible that you have a 'common-lib' module, and then additional modules that depend on that lib (e.g. application tier). It is highly likely that you run a test at the application-tier and it invokes code in the lib module. But you get no code-coverage credit for that. Jacoco coverage of the lib will be solely based on the tests in the lib. The 'app' tests only give you coverage of the app. This has to do with the instrumentation that happens when jacoco runs - it's only going to instrument the things local to the module it is running in.
Yes, there is a jacoco-aggregate report, but this will merge the module reports together - single report of each module. It does NOT give you unified coverage of an app-tier test calling lib methods, etc.
Lastly, sonar. I believe as long as you have jacoco files hanging around, the sonar-scanner will make use of them. I think it operates on the raw jacoco.exec file, but may be able to interpret a jacoco-result.xml or jacoco-aggregate.xml. If it is giving you grief, include your pom.xml
I created a job in Jenkins that executes sonnar-runner against all the projects in my repository. This job is independent from code coverage and runs a few times a day.
On the other hand, I want code coverage to happen upon a build (using JaCoCo) and import the reports into Sonar using Ant.
I set up my environment and added the JaCoCo and Sonar plug-ins to the Ant lib directory. When I run the Ant target I can see the results of JaCoCo locally in XML and HTML.
All I want is to upload these results to a remote SonarQube instance. However, after setting up the Sonar properties and adding <sonar:sonar> to my build.xml I realized that Sonar seems to be running static analysis against my project.
How do I set up Ant to only run code coverage for a project?
You can't run only a small portion of the full analysis.
Imagine for a moment that this were possible: Let's say that once an hour a "coverage only" job runs that updates the coverage on my project.
At 8 a.m. I start with a project with 6 LoC and 50% coverage.
At 9 a.m. my project has "6 LoC" and 20% coverage
10a.m.: "6 Loc" and 10% coverage.
Has someone been removing tests? Or adding new, uncovered LoC?
I have no way of knowing because my partial analysis has put the SonarQube project in a bad state.
And this is why there's no way to run only part of the analysis.
I have two separate projects, Product and CompoTestProduct. Product is written in Scala and uses sbt while CompoTestProduct is written in Java and uses maven. Product will be deployed in a server and CompoTestProduct's tests classes will be ran locally. I tried to use scoverage as it says that it supports Multi project reports but I still have no idea how to do this and I'm still struggling to make the samples work. I am getting this error on the sample projects:
[error] Not a valid command: coverage
[error] Not a valid project ID: coverage
[error] Expected ':' (if selecting a configuration)
[error] Not a valid key: coverage (similar: homepage, package, compilerCache)
[error] coverage
[error] ^
I am new to this code coverage testing thing. Do you have other tutorials or maybe know other framework/toolkit that can generate test code coverage report for do this setup?
The multi project support refers to being able to aggregate the results of multiple reports.
Scoverage instruments your code when you activate coverage on the sbt command line. So sbt clean coverage compile would be enough in your Product project to get the classfiles instrumented.
Then you could run your unit tests as normal with Maven. At this point Maven needs to be configured to also use Scoverage, as it will need to write out the coverage data once it is completed.
Then you would need to run the report step after.
So, summary, it is possible, with a LOT of hassle, but why are you going through these hoops to have an awkward project setup? Just move your tests into the main project, do a combined java/scala compile and it should be much easier as you could run the entire build via sbt or maven, and not both mixed.
Is it possible to check in Sonar the quality of the *Test.java source code, e.g. Methods maximum size 100 lines?
The problem is, that the Java Junit tests are growing with the productive code, also the complexity.
We have unit test classes with more than 1000 lines and 2 methods.
We want to check in Sonar some rules for these *Test.java classes.
Since Sonar 3.1, it includes a plugin that has specific PMD rules to be executed against the unit tests (a JIRA was created for that). You can see them in the Configuration > Quality Profiles > Coding Rules.
However, it seems that you want to run a full analysis on the test source code, like you do on the production source code, and get additional metrics (for ex. a % rules compliance and also a % rules compliance for unit tests). I don't think that Sonar provides such feature natively. What you can do is to run 2 Sonar analysis:
Your first analysis is the current one;
The second analysis will consider the src/test/java as the "production" source code. Thus, this second analysis will give you the quality of your code. For this analysis, you can specify a specific Maven profile (or an alternative pom.xml) that will change the project information (for ex. it will indicate that src/test/java is the default sourceDirectory).
I also noticed that SonarQube will by default ignore the test resources for quality analysis. Using schnatterers answer, i found a simple way to create a separate project only including the test classes as sources in SonarQube, therefore triggering the quality anlysis on them. In the POM of the project i want to analyze i add a profile, which changes the sonar properties accordingly:
<profiles>
<profile>
<id>analyze-test-classes</id>
<properties>
<sonar.sources>src/test/java</sonar.sources>
<sonar.tests></sonar.tests>
<sonar.projectName>${project.name}-tests</sonar.projectName>
<sonar.projectKey>${project.groupId}:${project.artifactId}-tests</sonar.projectKey>
</properties>
</profile>
</profiles>
Running Maven with
mvn sonar:sonar -Panalyze-test-classes
will then activate this profile and create an additional project in SonarQube with the suffix -tests, which only contains the analysis of the test classes.
With SonarQube 4.5.2 (don't know when they changed the behavior) it seems to me that unit tests are no longer excluded from the analysis. When running sonar-runner with sonar.sources=src sonar also creates issues for src/test/java.
One approach to use a specific quality ruleset for test code would be to run two analyses: one for the main code and another one for the testing code.
This can be realized as follows:
sonar-project.properties:
sonar.projectName=testSonar
sonar.projectKey=testsonar
sonar.sources=src/main/java
sonar.projectVersion=1.0
Analyse main code: sonar-runner
Analyse test code: sonar-runner -Dsonar.projectKey=testsonar.test -Dsonar.sources=src/test/java -Dsonar.projectName="testSonar TEST"
The different quality profiles must be changed via the server (Dashboard | Project Configuration | Quality Profiles), because -Dsonar.profile is deprecated.
This should also work with analyses through maven or jenkins.