Develop Reference

I am using Gauge framework with Java and maven and wanted to implement Test coverage using Jacoco. At spec level,

For example if 10 specs are run. then code coverage report should be generated for the same. I have my classes in src/test/Java folder.
Is there any way to achieve this?
I have tried using antrun plugin. It's getting executed but code coverage for all the classes is displayed as zeros. however execution of specs are successful and it's pass

Aggregated Sonar report for mutiModule maven Project

I have a setup for multi Module module project something like this
Module1
|
submodule1
|
submodule2
I have written a Junit test in submodule 1 and it's covering the code of submodule 2 also but when I try to see coverage in sonar it's showing 0% for submodule 2 is there any way to show coverage of submodule2 ? Also I can generate aggregated xml report by running mvn clean install jacoco:report-aggregate but how can I feed this aggregated report into sonar ? How to setup pom of module as well as submodules ?

I'm not sure if this is what you are doing, but I'll say it anyway. If you are creating submodule1 as 'code' and submodule2 as 'tests', that's a horrible way to organize it. Tests (specifically, unit tests) should be stored with the code that it is testing. Thus, each submodule has src/main/java and src/test/java.
Now, that said, it is plausible that you have a 'common-lib' module, and then additional modules that depend on that lib (e.g. application tier). It is highly likely that you run a test at the application-tier and it invokes code in the lib module. But you get no code-coverage credit for that. Jacoco coverage of the lib will be solely based on the tests in the lib. The 'app' tests only give you coverage of the app. This has to do with the instrumentation that happens when jacoco runs - it's only going to instrument the things local to the module it is running in.
Yes, there is a jacoco-aggregate report, but this will merge the module reports together - single report of each module. It does NOT give you unified coverage of an app-tier test calling lib methods, etc.
Lastly, sonar. I believe as long as you have jacoco files hanging around, the sonar-scanner will make use of them. I think it operates on the raw jacoco.exec file, but may be able to interpret a jacoco-result.xml or jacoco-aggregate.xml. If it is giving you grief, include your pom.xml

What does the "Maven Surefire plugin" means when it test your build?

I am working on understanding Maven and I'm learning about building your Java app with it.
So when I do a :
maven package
It does build my jar as expected but I see in the output console that Maven does build tests (it always say that the test a run and there are no failure).
I researched on the web about that and learned that Maven use a plugin called Maven Surefire. But I can't understand what does that plugin do to my code, what does the tests "means" ? What does the tests do with my code and how it works behind the console ?

The Maven surefire plugin runs the tests you have written. These are usually in the src/test/java folder. If you have none, the plugin does nothing.

Is this only one question? :D
So. Different things are going on.
You create an application with Java. To test the single components / packages / classes that you create most people use JUnit or TestNg. You usually have dedicated test classes that verify your production code behaves as intended without you clicking through all the things on every change.
When you now use maven to run your build the pom.xml file defines a packaging - in your case "jar" since you create a jar file. The packaging defines what set of default plugins run in the defined maven phases. You probably recognize package here. Maven executes all phases up to package and the registered / configured plugins.
To execute those tests maven provides the surefire plugin which supports running JUnit or TestNg tests. If you follow the directory conventions your tests reside in src/test/java and the surefire includes naming convention maven will execute those tests in every build (as this is the best practice). If you also want to write integration tests then there is the failsafe plugin. That plugin is not enabled by default and runs in different maven phases.
So the tests just run your production code - in fact they just do what you implement in the tests. They don't alter it in any way.
The maven introduction documentation has step by step explanations: Maven in 5 Minutes and the Getting Started Guide.
Starting from scratch this is probably a lot. So don't rush this. The build setup and test setup are very important things to have.

Jacoco code coverage is affected by AspectJ

We're using AspectJ in our project and also Jacoco for test coverage report, currently we're facing an issue that due to AspectJ changed the byte code during compiling phase, which makes the code coverage report not correct. One example is due to AspectJ adds extra if-else statement, then the branch coverage shows something like 1/4 but actually there's no condition branch in the source code. Is there some good way to tell Jacoco to ignore all code generated by AspectJ?
Thanks a lot.

I am copying here the answer I just wrote on the JaCoCo mailing list:
You have two options with AspectJ if you want to avoid it compiling from source:
Use LTW with the weaving agent.
Move your aspects into a separate Maven module. Compile your Java modules with the normal Maven Compiler Plugin and the aspect module with AspectJ Maven. Then create another module which just uses AspectJ Maven in order to do binary weaving on a Java module, using both previously created artifacts as dependencies. In this scenario you need to make sure that JaCoCo offline instrumentation is bound to a phase before binary weaving is done.
The easiest way out, though, would be to test your aspects in isolation and also the Java code without aspects and measure coverage there without any issues.

#RajeshTV:
Instructions how to use clover-aspectj-compiler are here:
https://confluence.atlassian.com/display/CLOVER/Clover+AspectJ+Compiler
These instructions are valid for OpenClover as well. Just download the:
org.openclover:clover-aspectj-compiler:1.0.0
org.openclover:clover:4.2.0
and aspectj-rt + aspectj-tools JARs
Next call them like this:
java -cp "clover-4.2.0.jar:clover-aspectj-compiler-1.0.0.jar:aspectjrt.jar:aspectjtools.jar" com.atlassian.clover.instr.aspectj.CloverAjc -d <output directory> <list of files>
It will produce *.class files in the specified directory as well as create clover.db database.
You have to call the command above from your Maven build, for instance by using the exec:exec goal.
Please note that the clover-aspectj-compiler does not have a dedicated Maven plugin to do this automatically, so it's your job to write the whole plumbing.

Sonar Java: check the quality of the test classes source code?

Is it possible to check in Sonar the quality of the *Test.java source code, e.g. Methods maximum size 100 lines?
The problem is, that the Java Junit tests are growing with the productive code, also the complexity.
We have unit test classes with more than 1000 lines and 2 methods.
We want to check in Sonar some rules for these *Test.java classes.

Since Sonar 3.1, it includes a plugin that has specific PMD rules to be executed against the unit tests (a JIRA was created for that). You can see them in the Configuration > Quality Profiles > Coding Rules.
However, it seems that you want to run a full analysis on the test source code, like you do on the production source code, and get additional metrics (for ex. a % rules compliance and also a % rules compliance for unit tests). I don't think that Sonar provides such feature natively. What you can do is to run 2 Sonar analysis:
Your first analysis is the current one;
The second analysis will consider the src/test/java as the "production" source code. Thus, this second analysis will give you the quality of your code. For this analysis, you can specify a specific Maven profile (or an alternative pom.xml) that will change the project information (for ex. it will indicate that src/test/java is the default sourceDirectory).

I also noticed that SonarQube will by default ignore the test resources for quality analysis. Using schnatterers answer, i found a simple way to create a separate project only including the test classes as sources in SonarQube, therefore triggering the quality anlysis on them. In the POM of the project i want to analyze i add a profile, which changes the sonar properties accordingly:
<profiles>
<profile>
<id>analyze-test-classes</id>
<properties>
<sonar.sources>src/test/java</sonar.sources>
<sonar.tests></sonar.tests>
<sonar.projectName>${project.name}-tests</sonar.projectName>
<sonar.projectKey>${project.groupId}:${project.artifactId}-tests</sonar.projectKey>
</properties>
</profile>
</profiles>
Running Maven with
mvn sonar:sonar -Panalyze-test-classes
will then activate this profile and create an additional project in SonarQube with the suffix -tests, which only contains the analysis of the test classes.

With SonarQube 4.5.2 (don't know when they changed the behavior) it seems to me that unit tests are no longer excluded from the analysis. When running sonar-runner with sonar.sources=src sonar also creates issues for src/test/java.
One approach to use a specific quality ruleset for test code would be to run two analyses: one for the main code and another one for the testing code.
This can be realized as follows:
sonar-project.properties:
sonar.projectName=testSonar
sonar.projectKey=testsonar
sonar.sources=src/main/java
sonar.projectVersion=1.0
Analyse main code: sonar-runner
Analyse test code: sonar-runner -Dsonar.projectKey=testsonar.test -Dsonar.sources=src/test/java -Dsonar.projectName="testSonar TEST"
The different quality profiles must be changed via the server (Dashboard | Project Configuration | Quality Profiles), because -Dsonar.profile is deprecated.
This should also work with analyses through maven or jenkins.