Is it possible to check in Sonar the quality of the *Test.java source code, e.g. Methods maximum size 100 lines?
The problem is, that the Java Junit tests are growing with the productive code, also the complexity.
We have unit test classes with more than 1000 lines and 2 methods.
We want to check in Sonar some rules for these *Test.java classes.
Since Sonar 3.1, it includes a plugin that has specific PMD rules to be executed against the unit tests (a JIRA was created for that). You can see them in the Configuration > Quality Profiles > Coding Rules.
However, it seems that you want to run a full analysis on the test source code, like you do on the production source code, and get additional metrics (for ex. a % rules compliance and also a % rules compliance for unit tests). I don't think that Sonar provides such feature natively. What you can do is to run 2 Sonar analysis:
Your first analysis is the current one;
The second analysis will consider the src/test/java as the "production" source code. Thus, this second analysis will give you the quality of your code. For this analysis, you can specify a specific Maven profile (or an alternative pom.xml) that will change the project information (for ex. it will indicate that src/test/java is the default sourceDirectory).
I also noticed that SonarQube will by default ignore the test resources for quality analysis. Using schnatterers answer, i found a simple way to create a separate project only including the test classes as sources in SonarQube, therefore triggering the quality anlysis on them. In the POM of the project i want to analyze i add a profile, which changes the sonar properties accordingly:
<profiles>
<profile>
<id>analyze-test-classes</id>
<properties>
<sonar.sources>src/test/java</sonar.sources>
<sonar.tests></sonar.tests>
<sonar.projectName>${project.name}-tests</sonar.projectName>
<sonar.projectKey>${project.groupId}:${project.artifactId}-tests</sonar.projectKey>
</properties>
</profile>
</profiles>
Running Maven with
mvn sonar:sonar -Panalyze-test-classes
will then activate this profile and create an additional project in SonarQube with the suffix -tests, which only contains the analysis of the test classes.
With SonarQube 4.5.2 (don't know when they changed the behavior) it seems to me that unit tests are no longer excluded from the analysis. When running sonar-runner with sonar.sources=src sonar also creates issues for src/test/java.
One approach to use a specific quality ruleset for test code would be to run two analyses: one for the main code and another one for the testing code.
This can be realized as follows:
sonar-project.properties:
sonar.projectName=testSonar
sonar.projectKey=testsonar
sonar.sources=src/main/java
sonar.projectVersion=1.0
Analyse main code: sonar-runner
Analyse test code: sonar-runner -Dsonar.projectKey=testsonar.test -Dsonar.sources=src/test/java -Dsonar.projectName="testSonar TEST"
The different quality profiles must be changed via the server (Dashboard | Project Configuration | Quality Profiles), because -Dsonar.profile is deprecated.
This should also work with analyses through maven or jenkins.
Related
I have a setup for multi Module module project something like this
Module1
|
submodule1
|
submodule2
I have written a Junit test in submodule 1 and it's covering the code of submodule 2 also but when I try to see coverage in sonar it's showing 0% for submodule 2 is there any way to show coverage of submodule2 ? Also I can generate aggregated xml report by running mvn clean install jacoco:report-aggregate but how can I feed this aggregated report into sonar ? How to setup pom of module as well as submodules ?
I'm not sure if this is what you are doing, but I'll say it anyway. If you are creating submodule1 as 'code' and submodule2 as 'tests', that's a horrible way to organize it. Tests (specifically, unit tests) should be stored with the code that it is testing. Thus, each submodule has src/main/java and src/test/java.
Now, that said, it is plausible that you have a 'common-lib' module, and then additional modules that depend on that lib (e.g. application tier). It is highly likely that you run a test at the application-tier and it invokes code in the lib module. But you get no code-coverage credit for that. Jacoco coverage of the lib will be solely based on the tests in the lib. The 'app' tests only give you coverage of the app. This has to do with the instrumentation that happens when jacoco runs - it's only going to instrument the things local to the module it is running in.
Yes, there is a jacoco-aggregate report, but this will merge the module reports together - single report of each module. It does NOT give you unified coverage of an app-tier test calling lib methods, etc.
Lastly, sonar. I believe as long as you have jacoco files hanging around, the sonar-scanner will make use of them. I think it operates on the raw jacoco.exec file, but may be able to interpret a jacoco-result.xml or jacoco-aggregate.xml. If it is giving you grief, include your pom.xml
I am working on understanding Maven and I'm learning about building your Java app with it.
So when I do a :
maven package
It does build my jar as expected but I see in the output console that Maven does build tests (it always say that the test a run and there are no failure).
I researched on the web about that and learned that Maven use a plugin called Maven Surefire. But I can't understand what does that plugin do to my code, what does the tests "means" ? What does the tests do with my code and how it works behind the console ?
The Maven surefire plugin runs the tests you have written. These are usually in the src/test/java folder. If you have none, the plugin does nothing.
Is this only one question? :D
So. Different things are going on.
You create an application with Java. To test the single components / packages / classes that you create most people use JUnit or TestNg. You usually have dedicated test classes that verify your production code behaves as intended without you clicking through all the things on every change.
When you now use maven to run your build the pom.xml file defines a packaging - in your case "jar" since you create a jar file. The packaging defines what set of default plugins run in the defined maven phases. You probably recognize package here. Maven executes all phases up to package and the registered / configured plugins.
To execute those tests maven provides the surefire plugin which supports running JUnit or TestNg tests. If you follow the directory conventions your tests reside in src/test/java and the surefire includes naming convention maven will execute those tests in every build (as this is the best practice). If you also want to write integration tests then there is the failsafe plugin. That plugin is not enabled by default and runs in different maven phases.
So the tests just run your production code - in fact they just do what you implement in the tests. They don't alter it in any way.
The maven introduction documentation has step by step explanations: Maven in 5 Minutes and the Getting Started Guide.
Starting from scratch this is probably a lot. So don't rush this. The build setup and test setup are very important things to have.
I've maven project1, which has src/main/java -- application source code
I do have another maven project2, which has all tests src/main/test -- all api tests which are all part of above project1
Now, Is there any way to configure Java Jacoco code coverage agent in project2 in such away that It should calculate code coverage of project1 when execute tests from project2 ?
Thanks in advance, I know it's kind of theoretical question.
Speaking in general - yes it is possible:
By default agent collects information about execution of all the
classes, no matter from where they come.
Report generation requires
collected information about execution, plus original class files and
sources for which you want to generate report.
If both projects are part of the same Maven reactor, then have a look at jacoco simple integration test solution . If not, then JaCoCo Ant Tasks that provide flexibility in specification of paths - http://www.jacoco.org/jacoco/trunk/doc/ant.html
Background:
I have 2 separate java projects, call them A and B.
Project A is the actual product (war application), with unit-tests. Gradle builds the project and then runs sonar analysis, and I can see the unit-tests coverage in Sonar.
Project B is an integration test for the first project. It is run by Jenkins in a pipeline after building project A and deploying it on an integration-environment. The deployment also involves instrumenting the code so that the jacoco-it report will correlate to Project A's classes.
My question:
How can I add to project A's sonar page, which currently has only unit-tests coverage, the integration tests coverage - as a second step?
The flow I need is:
build project A
run sonar analysis on project A (now it'll show unit-tests coverage on its sonar page)
build project B
run sonar analysis on project B - which will simply add integration-coverage to project A on sonar
It's currently not working. when I run sonar analysis on project B it messes up project A's sonar page, removing the unit-test coverage of A.
The flow that I have now, which is working but I want to change is:
build project A
run sonar analysis on project A
build project B
run jacoco report on project B. this outputs the jacoco-it.exec file into a specific location on my disk
run sonar analysis on project A (it has a setting to take the jacoco-it.exec file from the specified location
now project A's sonar page will show both unit-tests and integration-tests coverage, but step 5 is completely redundant and I want to avoid it.
Any suggestions?
what i will suggest is a bit different:
employ some build automation tool
create a sonar 'checker' job, which doesnt do the full analysis; just the incremental, this should break when add quality gate breaker issues
create a nightly job for your A project with full analysis: -Dsonar.analysis.mode=analysis (note: my experience shows that in this case you must run an incremental prior to this analysis, because in analysis mode sonar accepts new quality gate breaker issues
i don't think integration tests should count into project test coverage, so that might be just left out.
You cannot amend a previous analysis, which is why your 'B' job appears to replace the unit test numbers. What you need to do is generate your integration test coverage report before analysis, and make that report available so that both unit test and integration test numbers can be read in the same analysis.
From a build pipeline standpoint, that could get tricky depending on how you need to structure the jobs, but off-hand it sounds like you need 3 jobs:
build project A
build project B & generate integration test report
job 3 pulls integration test report from job 2, and either rebuilds project A or pulls code, classes, and unit test report from job 1 and performs analysis
We're using AspectJ in our project and also Jacoco for test coverage report, currently we're facing an issue that due to AspectJ changed the byte code during compiling phase, which makes the code coverage report not correct. One example is due to AspectJ adds extra if-else statement, then the branch coverage shows something like 1/4 but actually there's no condition branch in the source code. Is there some good way to tell Jacoco to ignore all code generated by AspectJ?
Thanks a lot.
I am copying here the answer I just wrote on the JaCoCo mailing list:
You have two options with AspectJ if you want to avoid it compiling from source:
Use LTW with the weaving agent.
Move your aspects into a separate Maven module. Compile your Java modules with the normal Maven Compiler Plugin and the aspect module with AspectJ Maven. Then create another module which just uses AspectJ Maven in order to do binary weaving on a Java module, using both previously created artifacts as dependencies. In this scenario you need to make sure that JaCoCo offline instrumentation is bound to a phase before binary weaving is done.
The easiest way out, though, would be to test your aspects in isolation and also the Java code without aspects and measure coverage there without any issues.
#RajeshTV:
Instructions how to use clover-aspectj-compiler are here:
https://confluence.atlassian.com/display/CLOVER/Clover+AspectJ+Compiler
These instructions are valid for OpenClover as well. Just download the:
org.openclover:clover-aspectj-compiler:1.0.0
org.openclover:clover:4.2.0
and aspectj-rt + aspectj-tools JARs
Next call them like this:
java -cp "clover-4.2.0.jar:clover-aspectj-compiler-1.0.0.jar:aspectjrt.jar:aspectjtools.jar" com.atlassian.clover.instr.aspectj.CloverAjc -d <output directory> <list of files>
It will produce *.class files in the specified directory as well as create clover.db database.
You have to call the command above from your Maven build, for instance by using the exec:exec goal.
Please note that the clover-aspectj-compiler does not have a dedicated Maven plugin to do this automatically, so it's your job to write the whole plumbing.