My co-worker found this morning that compiling a project with Cobertura enabled changes the sonar results on the same project.
On this particular project we ran a build with sonar:sonar and then ran it again with cobertura:cobertura sonar:sonar.
The sonar results in the comparison are now showing that without Cobertura we have 7/78/153/24/0 violations of the 5 severities, but with Cobertura it changes to 7/81/94/24/0, and in particular finds 3 new critical violations and 15 new major violations that aren't found without Cobertura.
One of the biggest changes is that without Cobertura there are 60 violations of the rule against empty methods (many of them constructors) and with Cobertura only 3 of those are reported.
If Cobertura only prevented violations from being found we could run the two independently, but since some violations are only found with Cobertura enabled it seems like we would have to do two separate Sonar analyses.
Is this a known interaction? Is there any workaround other than doing Cobertura and Sonar in separate builds? And using both sets of results to get the best data?
Based on the comment you made let me explain what it seems to be happening:
You are using FindBugs via SonarQube (rules you are mentioning are findbugs rules)
First let's think about the two tools involved here and how they work (roughly) :
FindBugs : it is a static analysis tool based on bytecode : it will read bytecode and raise issue when it detects bad pattern.
Cobertura : Coverage tool : how does this work ? it instruments the bytecode to place probes and when tests are run keep track of which probes where hit or not.
Then you can understand where the issue might be : FindBugs ends up analyzing the bytecode instrumented by Cobertura. That would explain why you have some new issues and why some of the empty methods issues are removed when analyzing with cobertura.
To avoid this issue you have to be sure your bytecode files are not instrumented when you analyze them with FindBugs but (disclaimer, I develop the sonar java plugin so I might be a little biased here ;) ) I would recommend you to stop using FindBugs in favor of the SonarQube Java Analyzer which won't have this issue as its analyzer approach things slightly differently (see this blog post about that)
User error. :-(
It turns out that the user had run a mvn clean prior to running the sonar:sonar with cobertura, so as implied by benzonico, the findbugs rules that have to analyze compiled code didn't run. Only the rules that are run on source code, like the java plugin, generated results. That's why we were missing a bunch of rules and results.
We still have inconsistencies between Bamboo and manual builds, but that would be a topic for a separate post.
Related
I have pretty standard Maven multi-modules project (with JUnit, Arquillian and Selenium tests). I have Sonar 6.2 installed on a server. And on my project on Sonar the Code Coverage metric indicates 0.0%. But I know it's wrong as I do have some test coverage.
I found this Generic Test Data documentation page that explains that since 6.2 Sonar is supporting code coverage out of the box and that I have to pass a comma-delimited list of report paths to a parameter sonar.coverageReportPaths (I guess provided either in my pom or in command line).
I'm fine with that. But I cannot find out an example on how to setup this for a pretty classical Java project. What kind of file do I need to give in the list ? The relative paths to each of my Surefire/Failsafe reports ? Do I need to generate Jacoco reports in addition ? Can I give a "generic" path like report.xml if all of my reports have the same name ?
For standard Java project is it probably the easiest to use JaCoCo to generate coverage data and then feed it to SonarJava (SonarQube's plugin to analyze Java code). You can find documentation here
https://docs.sonarqube.org/display/PLUG/Code+Coverage+by+Unit+Tests+for+Java+Project
You might find mentions of separation between unit tests and integration tests, this has been deprecated and now there is only single kind of coverage.
Don't hesitate to reach out to mailing list or ask question if something is not clear, we are in the process of improving this documentation.
On my local machine for my project trying to see the code quality related issues that sonarqube shows.
Using eclipse IDE.
Installed sonarlint plugin and I am able to see most of the issues that I see in the sonarqube for my project.
But, don’t see issues related to duplicate code etc
From what I see on internet sonarqube uses other third party tools like PMD, checksyle, findbugs to show other issues apart from what sonarlint shows.
Who usually provide the xml rulesets for PMD, Checkstyle etc in the company? Is it the sonar team or the architecture team? or the project team leads create one and provide it to the team.
At the time of this writing, SonarLint runs analyses file by file,
so it cannot display errors involving multiple files, such as:
Duplications
Test coverage
Package-level issues (package-info.java is missing, etc)
Furthermore, it only shows issues from SonarSource analyzers,
third party analyzers such as PMD and checkstyle are excluded.
Finally, it will show the same issues as you see in SonarQube in connected mode.
Otherwise, in standalone mode,
it uses the default quality profile (= set of rules),
as defined by its embedded analyzers,
which may slightly vary depending on their version.
The differences that you see between SonarLint and SonarQube will come down to one or more of the reasons as explained above.
Who usually provide the xml rulesets for PMD, Checkstyle etc in the company? Is it the sonar team or the architecture team? or the project team leads create one and provide it to the team.
That depends on the company, and any answer to this would be subjective.
As discussed in Open JaCoCo report in Intellij IDEA, when I gather code coverage statistics using Jacoco (rather than native IntelliJ tracing) 0.0% (i.e. "no" coverage) is always shown in the coverage window.
This is after I have done the whole "Analyze -> Show coverage data..." and selected my generated "jacoco.exec" file.
The same "jacoco.exec" file works fine with other tools such as the Jacoco native report generation task, and with Sonar, and these produce the expected coverage report content.
Also native IntelliJ instrumentation works fine.
Can anybody advise me if there is some essential step I must perform to get IntelliJ to accept my Jacoco coverage?
When you add the class to PowerMockito annotation #PrepareForTest for the test classes, the corresponding class will not be showing the code coverage.
Buid source code through IntelliJ before importing jacoco.exec might be solved this showing 0% coverage problem.
For me the solution was to simply add the root package (in my case "de" in your case probably "com") to the "Packages and classes to include in coverage data".
Now i see a reasonable Code Coverage.
I may guess you are trying to see test coverage in same way as it is show by the TeamCity(jaCoCo).
If Yes there are simple workaround how to check all missed branches(just general statistic which branches was visited and how many times) :
That's all my advices , folks
"intellij idea" seems to have no way of showing correct coverage value of jacoco report created. It is indeed misleading and unfair when it shows coverage as 0.0 instead of giving an unsupport format error.
However, as an alternative, we can push jacoco report (created as part of maven build) to the sonar(qube) server using maven-sonar-plugin's target, sonar:sonar
mvn clean install sonar:sonar -Dsonar.host.url=http://<sonar-host>:9000 -Dsonar.projectKey=<sonar project key> -Dsonar.branch=<sonar project branch> -Dsonar.login=<sonar user> -Dsonar.password=<sonar pwd>
sonar.projectKey and sonar.branch properties value can be retrieved from corresponding project created in sonarqube.
Actually in our project, we are planning to start developing code (with SONAR to analyze from beginning only) , So we are making use of sonar plugin in eclipse. We know how to analyze code by configuring project to SONARQUEBE and making use of maven build tool as well as using sonarrunner. Instead of building maven for every code changes and analysing , is there any way that prompts (as well as show error lines ) sonar to immediately analyze just after saving the java file ?
Help will be appreciated..... Thanks in advance
There is currently no way to tie an sonar analysis to a save action.
However, you do not need to run the maven based analysis. Assuming you use a recent version of SonarQube and SonarQube ide (4+ and 3.3), you can simply run the analysis using the shortcut (ALT-Ctrl-Q, by default).
Using incremental mode with SonarQube 4+, this is actually really fast, because it analyzes only the changed files as compared to the last successful SonarQube run.
Of course, you need make sure that your buildserver regularily runs your full SonarQube builds as well.
Eclipse support incremental compiling. If I save a source file then it will compile the modified files.
Is it possible after such incremental compile also to run the JUnit tests of the same package and show the fail in the error view. Then I can see the JUnit test failing and compiling errors in the same view without extra action. Are there any plugins that can do it?
You have to look at these plugins:
JUnit Max: Not free, developed by Kent Benk (one of the men behind the TDD practice);
MoreUnit: Free, but essentially dedicated to help you write the tests;
Infinitest: Now free, this plugin is dedicated to run the tests related to the files you have just modified.
So regarding your needs, I suggest that you install MoreUnit and Infinitest plugins.
Use ExternalToolBuilder.
It can be triggered by source modify.
There’s Eclipse customized feature(integrate external tool builder) which may meet your need. But it needs extra effort to write the scripts I never used. Automatic test cases is not a convenient way, at least single click to see green bar in Eclipse is enough for me:)
You can run all tests in a project using Alt+Shift+X,T. I think that making it any more automated than this could take a serious performance toll. Incremental compilation is compiling at most 1 file at a time, but you're talking about running potentially hundreds of tests.