We're using AspectJ in our project and also Jacoco for test coverage report, currently we're facing an issue that due to AspectJ changed the byte code during compiling phase, which makes the code coverage report not correct. One example is due to AspectJ adds extra if-else statement, then the branch coverage shows something like 1/4 but actually there's no condition branch in the source code. Is there some good way to tell Jacoco to ignore all code generated by AspectJ?
Thanks a lot.
I am copying here the answer I just wrote on the JaCoCo mailing list:
You have two options with AspectJ if you want to avoid it compiling from source:
Use LTW with the weaving agent.
Move your aspects into a separate Maven module. Compile your Java modules with the normal Maven Compiler Plugin and the aspect module with AspectJ Maven. Then create another module which just uses AspectJ Maven in order to do binary weaving on a Java module, using both previously created artifacts as dependencies. In this scenario you need to make sure that JaCoCo offline instrumentation is bound to a phase before binary weaving is done.
The easiest way out, though, would be to test your aspects in isolation and also the Java code without aspects and measure coverage there without any issues.
#RajeshTV:
Instructions how to use clover-aspectj-compiler are here:
https://confluence.atlassian.com/display/CLOVER/Clover+AspectJ+Compiler
These instructions are valid for OpenClover as well. Just download the:
org.openclover:clover-aspectj-compiler:1.0.0
org.openclover:clover:4.2.0
and aspectj-rt + aspectj-tools JARs
Next call them like this:
java -cp "clover-4.2.0.jar:clover-aspectj-compiler-1.0.0.jar:aspectjrt.jar:aspectjtools.jar" com.atlassian.clover.instr.aspectj.CloverAjc -d <output directory> <list of files>
It will produce *.class files in the specified directory as well as create clover.db database.
You have to call the command above from your Maven build, for instance by using the exec:exec goal.
Please note that the clover-aspectj-compiler does not have a dedicated Maven plugin to do this automatically, so it's your job to write the whole plumbing.
Related
For example if 10 specs are run. then code coverage report should be generated for the same. I have my classes in src/test/Java folder.
Is there any way to achieve this?
I have tried using antrun plugin. It's getting executed but code coverage for all the classes is displayed as zeros. however execution of specs are successful and it's pass
I have a setup for multi Module module project something like this
Module1
|
submodule1
|
submodule2
I have written a Junit test in submodule 1 and it's covering the code of submodule 2 also but when I try to see coverage in sonar it's showing 0% for submodule 2 is there any way to show coverage of submodule2 ? Also I can generate aggregated xml report by running mvn clean install jacoco:report-aggregate but how can I feed this aggregated report into sonar ? How to setup pom of module as well as submodules ?
I'm not sure if this is what you are doing, but I'll say it anyway. If you are creating submodule1 as 'code' and submodule2 as 'tests', that's a horrible way to organize it. Tests (specifically, unit tests) should be stored with the code that it is testing. Thus, each submodule has src/main/java and src/test/java.
Now, that said, it is plausible that you have a 'common-lib' module, and then additional modules that depend on that lib (e.g. application tier). It is highly likely that you run a test at the application-tier and it invokes code in the lib module. But you get no code-coverage credit for that. Jacoco coverage of the lib will be solely based on the tests in the lib. The 'app' tests only give you coverage of the app. This has to do with the instrumentation that happens when jacoco runs - it's only going to instrument the things local to the module it is running in.
Yes, there is a jacoco-aggregate report, but this will merge the module reports together - single report of each module. It does NOT give you unified coverage of an app-tier test calling lib methods, etc.
Lastly, sonar. I believe as long as you have jacoco files hanging around, the sonar-scanner will make use of them. I think it operates on the raw jacoco.exec file, but may be able to interpret a jacoco-result.xml or jacoco-aggregate.xml. If it is giving you grief, include your pom.xml
I've maven project1, which has src/main/java -- application source code
I do have another maven project2, which has all tests src/main/test -- all api tests which are all part of above project1
Now, Is there any way to configure Java Jacoco code coverage agent in project2 in such away that It should calculate code coverage of project1 when execute tests from project2 ?
Thanks in advance, I know it's kind of theoretical question.
Speaking in general - yes it is possible:
By default agent collects information about execution of all the
classes, no matter from where they come.
Report generation requires
collected information about execution, plus original class files and
sources for which you want to generate report.
If both projects are part of the same Maven reactor, then have a look at jacoco simple integration test solution . If not, then JaCoCo Ant Tasks that provide flexibility in specification of paths - http://www.jacoco.org/jacoco/trunk/doc/ant.html
Is it possible to check in Sonar the quality of the *Test.java source code, e.g. Methods maximum size 100 lines?
The problem is, that the Java Junit tests are growing with the productive code, also the complexity.
We have unit test classes with more than 1000 lines and 2 methods.
We want to check in Sonar some rules for these *Test.java classes.
Since Sonar 3.1, it includes a plugin that has specific PMD rules to be executed against the unit tests (a JIRA was created for that). You can see them in the Configuration > Quality Profiles > Coding Rules.
However, it seems that you want to run a full analysis on the test source code, like you do on the production source code, and get additional metrics (for ex. a % rules compliance and also a % rules compliance for unit tests). I don't think that Sonar provides such feature natively. What you can do is to run 2 Sonar analysis:
Your first analysis is the current one;
The second analysis will consider the src/test/java as the "production" source code. Thus, this second analysis will give you the quality of your code. For this analysis, you can specify a specific Maven profile (or an alternative pom.xml) that will change the project information (for ex. it will indicate that src/test/java is the default sourceDirectory).
I also noticed that SonarQube will by default ignore the test resources for quality analysis. Using schnatterers answer, i found a simple way to create a separate project only including the test classes as sources in SonarQube, therefore triggering the quality anlysis on them. In the POM of the project i want to analyze i add a profile, which changes the sonar properties accordingly:
<profiles>
<profile>
<id>analyze-test-classes</id>
<properties>
<sonar.sources>src/test/java</sonar.sources>
<sonar.tests></sonar.tests>
<sonar.projectName>${project.name}-tests</sonar.projectName>
<sonar.projectKey>${project.groupId}:${project.artifactId}-tests</sonar.projectKey>
</properties>
</profile>
</profiles>
Running Maven with
mvn sonar:sonar -Panalyze-test-classes
will then activate this profile and create an additional project in SonarQube with the suffix -tests, which only contains the analysis of the test classes.
With SonarQube 4.5.2 (don't know when they changed the behavior) it seems to me that unit tests are no longer excluded from the analysis. When running sonar-runner with sonar.sources=src sonar also creates issues for src/test/java.
One approach to use a specific quality ruleset for test code would be to run two analyses: one for the main code and another one for the testing code.
This can be realized as follows:
sonar-project.properties:
sonar.projectName=testSonar
sonar.projectKey=testsonar
sonar.sources=src/main/java
sonar.projectVersion=1.0
Analyse main code: sonar-runner
Analyse test code: sonar-runner -Dsonar.projectKey=testsonar.test -Dsonar.sources=src/test/java -Dsonar.projectName="testSonar TEST"
The different quality profiles must be changed via the server (Dashboard | Project Configuration | Quality Profiles), because -Dsonar.profile is deprecated.
This should also work with analyses through maven or jenkins.
I would like to run some validation checks against projects that jenkins builds. The validation checks would run against files from the project being built and report violations. I already have a core java application which can test the file types I require but, being a complete beginner with jenkins I'm unsure where to start with the jenkins integration! Any help is welcome!
You can use ant to call your validation code and fail the build if your validation checks fail. Otherwise you are writing your own Jenkins plugin for this tool you have to run your validation. Anything that fails the ant build also fails the Jenkins build.
I'd try to modify your existing validation application, or make a transformation step that makes it produce files that Jenkins Violations Plugin can scan
https://wiki.jenkins-ci.org/display/JENKINS/Violations
Hopefully you can pretend that your application is e.g. findbugs or one of the already supported checkers by just producing output in the same format.
Violations is frecuently used, as mentioned previously, to collect and present the findings of many other tools: checkstyle, pmd, cpd, findbugs and many others, most of which allow to create custom rules, although not always easily.
Maven plugins Jenkins can launch for specific verifications the output of which you can, of course, grep:
maven dependency plugin. The goals analyze, analyze-dep-mgt, analyze-duplicate are useful to check multiple situations with the project dependencies
maven verifier plugin. Verifies the existence or non-existence of files/directories and optionally checks file content against a regular expression
maven enforcer plugin. To check whether the pom.xml elements satisfy certain rules. How to create your own rules is well documented. And repositories of existing rules exist out there such as: enforcer rules, extra-enforcer rules...