We use cobertura to measure code coverage in unit testing and integrated/system testing.
Some classes are more note suitable for getting a coverage in plain unit test. (Some require database, GUI testing etc). While mock objects, stubs, gui-unit-testing framework will help here, we are considering a way to mark some classes not to be included in instrumentation.
Cobertura provides a pattern based processing to include/exclude classes. Is there a way to annotate class and skip from instrumenting?
Without above, I will have to write a custom annotation processor to get list of classes, and then user the result to skip from instrumenting phase.
if you are using Ant to run cobertura this uses fileset so you could use the excludesFile parameter to specify a file containing a list of files to ignore.
This list of files could be produced by a custom annotation parser or you could stick a unique comment in the files and search for that maybe something like
for file in `find ./ -name '*.java'`; do if grep --silent "some unique tag" $file ;then echo $file > excludeFile.lst; fi ; done
Related
I am trying to integrate Clover with a Java project at work. Clover has been added to the classpath, the plugin has been added to the POM and the license has been set up.
When I run mvn clean clover:setup test clover:aggregate clover:clover Clover starts up correctly. However, tests that generate csv's fail because the CSV output is not what is expected. This is what gets generated for one of the tests:
<ID,NAME,[__CLR4_1_2_TEST_NAME_SNIFFER,"DESCRIPTION,Long","DESCRIPTION,Short",country
001,Fred,com_atlassian_clover.TestNameSniffer$1#e628ccb,Mr. Fred Flinstone,Fred Flinstone,US
002,Bob,com_atlassian_clover.TestNameSniffer$1#e628ccb,Mr. Bob the builder,Bob the builder,UK
003,Tintin,com_atlassian_clover.TestNameSniffer$1#e628ccb,Mr. Tin Tin,Tin Tin,FR
004,,com_atlassian_clover.TestNameSniffer$1#e628ccb,,Rob the Bob,
005,,com_atlassian_clover.TestNameSniffer$1#e628ccb],,Tom the Bob,
>
This is what is expected:
<ID,NAME,["DESCRIPTION,Long","DESCRIPTION,Short",country
001,Fred,Mr. Fred Flinstone,Fred Flinstone,US
002,Bob,Mr. Bob the builder,Bob the builder,UK
003,Tintin,Mr. Tin Tin,Tin Tin,FR
004,,,Rob the Bob,
005,],,Tom the Bob,
Why would Clover add a new column in to the CSV output, is it possible to stop it? I'm using maven 3.3, JDK 8 and Clover version 4.0.2.
I have tried looking for an answer but cannot seem to find anything similar to this issue. When I remove the failing tests Clover works fine.
I'm pretty sure it's connected to the code (or a library) you're using to create that CSV file. Clover on its own won't modify a CSV file. I suppose there's a code which access Java Object fields by reflection. Since class is instrumented by Clover, it adds some instrumentation code (like CLV_TEST_SNIFFER.
There are basically two solutions to your problem:
Modify the code which dumps classes to CSV to exclude static class members (this will effectively exclude Clover objects). Anyway, I believe it's not desired to dump static members to CSV which, as I assume, is a data projection of your Java object (static members don't belong to object but to class). Though, I don't know your full use case, so I may be simply wrong.
You can simply exclude files which cause problems (the ones which are serialised to CSV?) from Clover instrumentation, effectively you'll loose coverage data for those classes.
I am using EclEmma to test the coverage of my scenario tests and use case tests on my project.
I have a Base package which contains the most general classes and the use case tests. The coverage looks like this:
What I want is to exclude the use case tests (e.g. BugReportTest) from the coverage calculation. But I do want the tests inside it to be considered. I know how to exclude the entire class from the coverage but if I do that, my coverage % drops because the actual tests that check which lines of my code are tested are forgotten. These use case tests do need to stay in the Base package because of privacy reasons.
For technical reasons it might be necessary to exclude certain classes from code coverage analysis. The following options configure the coverage agent to exclude certain classes from analysis. Except for performance optimization or technical corner cases these options are normally not required.
Excludes: A list of class names that should be excluded from
execution analysis. The list entries are separated by a colon (:)
and may use wildcard characters (* and ?). (Default: empty)
Exclude classloaders: A list of class loader names that should be
excluded from execution analysis. The list entries are separated by
a colon (:) and may use wildcard characters (* and ?). This option
might be required in case of special frameworks that conflict with
JaCoCo code instrumentation, in particular class loaders that do not
have access to the Java runtime classes. (Default:
sun.reflect.DelegatingClassLoader)
Warning: Use these options with caution! Invalid entries might break
the code coverage launcher. Also do not use these options to define
the scope of your analysis. Excluded classes will still show as not
covered.
Resource Link:
EclEmma Code Coverage Preferences
The following examples all specify the same set of inclusion/exclusion patterns:
<filter includes="com.foo.*" excludes="com.foo.test.*,
com.foo.*Test*" />
<filter includes="com.foo.*" /> <filter excludes="com.foo.test.*,
com.foo.*Test*" />
<filter value="+com.foo.*, -com.foo.test.*, -com.foo.*Test*" />
<filter excludes="com.foo.*Test*" file="myfilters.txt" />
where myfilters.txt file contains these lines:
-com.foo.test.*
+com.foo.*
Resource Link:
Coverage filters
I am certain that all of my classes are built with -g(debug='true')
and yet EMMA still complains about missing debug info!
Ignore code coverage for unit tests in EclEmma
Preferences->Java->Code Coverage and set the "Only path entries matching" option to src/main/java - seems to work nicely
I use Spring JavaConfig to define my Spring configuration. Since unit tests use a different Spring configuration than production code, the production configuration shows up as 100% uncovered when I use Clover code coverage with unit tests.
I can use the #Configuration annotation to identify all these classes. Alternatively, I can use the #Bean annotation to identify all the methods within these classes.
Is there a Clover exclude or a code context filter that I can set up to globally exclude code by using these annotations? I use the maven-clover2-plugin to run Clover.
At the moment Clover allows to exclude code from coverage measurement using five methods:
excluding entire file
in Ant it is <clover-setup> / <fileset> / <excludes> tag
in Maven it is <excludes>, <excludesFile> or <excludesList> option
excluding certain methods
in Ant it is <clover-setup> / <methodContext> tag combined with the <clover-report> / <current> / <
in Maven it is not available directly - you'd have to use the <reportDescriptor> option to configure report format in an external XML file; the problem is with clover-setup which you'd also have to customise - maybe calling the Clover via maven-antrun-plugin could help here
excluding certain statements
in Ant it is <clover-setup> / <statementContext> tag combined with the <clover-report> / <current> / <
in Maven you'd have to use the <reportDescriptor>
excluding certain code blocks
in Ant it is <clover-report> / <current> / < with a predefined names of blocks, e.g. 'constructor' or 'switch' - see Using+Coverage+Contexts for more details
in Maven you'd have to use the <reportDescriptor>
excluding arbitrary source lines
You can put //CLOVER:OFF and //CLOVER:ON inline comments in the source code
Unfortunately, at the moment it's not possible to exclude the given class. However, I can see few workarounds available:
Add the //CLOVER:OFF and //CLOVER:ON comments around classes you want to exclude.
Exclude entire files
Treat these #Configuration classes as test code.
This is a hack. Clover allows to declare which classes are test classes. The match is based on file name, class signature and test signature. See the <clover-setup> / <testsources> / <testclass> tag. In this testclass tag you can define a regular expression matching entire class signature. In your case it could be:
<testclass annotation=".*#Configuration.*"/>
While this will not disable code instrumentation for these classes, they would be treated as test code and they should not contribute to application code metrics.
Note that Clover calculates code metrics for application and test code separately.
References:
https://docs.atlassian.com/clover-maven-plugin/latest/clover-setup.html
https://docs.atlassian.com/clover-maven-plugin/latest/clover-mojo.html
https://confluence.atlassian.com/display/CLOVER/Creating+custom+reports
https://confluence.atlassian.com/display/CLOVER/Using+Coverage+Contexts
I am using Cobertura for code coverage of my Java code using maven. When the reports are generated, I found out that there is no coverage generated for Interfaces.java.
While going through the logs, I found below:
[cobertura] INFO [main]
net.sourceforge.cobertura.reporting.html.HTMLReport - Data file does
not contain instrumentation information for the file
com/example/Interface.java. Ensure this class was instrumented, and
this data file contains the instrumentation information.
[file name mocked up]
Please let me know the reason for this. How can I have code coverage for interfaces in java.
Thanks
As commented by Tassos Bassoukos, Interfaces don't have any code, thus there is no possibility of instrumentation. Instead, instrument all classes that implement that interface.
So, cobertura will not be able to show code coverage for interfaces, and skip the interfaces.java files.
I am using EclEmma (inside of Eclipse) to scan my JUnit code coverage. This works correctly - however, I do not want EclEmma to scan my src/view folder since it contains Swing code that I consider not worthy of testing.
Is there any way to ignore this folder when EclEmma runs so that it: a) runs faster, and b) does not skew the coverage percentage?
EDIT:
My project's structure is:
src/view
src/model
src/controller
I have tried these (possibly others) with the Path Entries section in the Preferences page:
"src/view"
"src/view/*"
"view"
"view/*"
src/view
These are using the Excludes section in the Preferences page:
*
*View*
*View*.class
src/view/*View*
src/view/*View*.class
They all leave me with the same result of it analysing my entire src folder.
[Edit] The maintainers says you cannot, except one the source directory level: https://github.com/jacoco/eclemma/issues/70
I thought eclemma wasn't excluding files: it is. Just not as I thought.
When you go into excludes in preferences and specify your.classes.here.*, for example, that means those classes won't count towards your getting all your code covered, not that those classes won't be included in what needs to be covered by tests.
Try it and see. Try to exclude a class you know have coverage in it. Once you put that to the excludes preference, on a new coverage run they'll still be there in the coverage window, but they'll come up as 0% and will all be in red.
Rather useless if you ask me. I'm still searching for an adequate solution to exclude classes by name from the classes that need to be covered by tests.
You can specify an exclude field:
Excludes: A list of class names that should be excluded from execution
analysis. The list entries are separated by a colon (:) and may use
wildcard characters (* and ?). (Default: empty)
However, it might be easier to use their options for classpath matching:
Only path entries matching: Comma separated list of strings that must
match with the class path entry. A class path entry matches the
filter, if it contains one of the given strings. (e.g.
"src/main/java", Default: no filter)
See eclemma - how to ignore source about how to ignore src folders.
Also please note their caution,
Warning: If your settings do not match any of the class path entries
in your project(s), every new launch in coverage mode will have an
empty analysis scope.
I have given up on EclEmma because I can't get it to do the things I want it to do, so I use a different method - I'll document it here in case it helps anyone else.
To exclude classes from test, I name all my test classes as *Case.java and then include or exclude them via SuiteClasses. You can read more about that at https://github.com/junit-team/junit4/wiki/Aggregating-tests-in-suites
To measure coverage, I use Maven and Cobertura. This will test just the files specified in my test suites and produce coverage reports accordingly.