I use Spring JavaConfig to define my Spring configuration. Since unit tests use a different Spring configuration than production code, the production configuration shows up as 100% uncovered when I use Clover code coverage with unit tests.
I can use the #Configuration annotation to identify all these classes. Alternatively, I can use the #Bean annotation to identify all the methods within these classes.
Is there a Clover exclude or a code context filter that I can set up to globally exclude code by using these annotations? I use the maven-clover2-plugin to run Clover.
At the moment Clover allows to exclude code from coverage measurement using five methods:
excluding entire file
in Ant it is <clover-setup> / <fileset> / <excludes> tag
in Maven it is <excludes>, <excludesFile> or <excludesList> option
excluding certain methods
in Ant it is <clover-setup> / <methodContext> tag combined with the <clover-report> / <current> / <
in Maven it is not available directly - you'd have to use the <reportDescriptor> option to configure report format in an external XML file; the problem is with clover-setup which you'd also have to customise - maybe calling the Clover via maven-antrun-plugin could help here
excluding certain statements
in Ant it is <clover-setup> / <statementContext> tag combined with the <clover-report> / <current> / <
in Maven you'd have to use the <reportDescriptor>
excluding certain code blocks
in Ant it is <clover-report> / <current> / < with a predefined names of blocks, e.g. 'constructor' or 'switch' - see Using+Coverage+Contexts for more details
in Maven you'd have to use the <reportDescriptor>
excluding arbitrary source lines
You can put //CLOVER:OFF and //CLOVER:ON inline comments in the source code
Unfortunately, at the moment it's not possible to exclude the given class. However, I can see few workarounds available:
Add the //CLOVER:OFF and //CLOVER:ON comments around classes you want to exclude.
Exclude entire files
Treat these #Configuration classes as test code.
This is a hack. Clover allows to declare which classes are test classes. The match is based on file name, class signature and test signature. See the <clover-setup> / <testsources> / <testclass> tag. In this testclass tag you can define a regular expression matching entire class signature. In your case it could be:
<testclass annotation=".*#Configuration.*"/>
While this will not disable code instrumentation for these classes, they would be treated as test code and they should not contribute to application code metrics.
Note that Clover calculates code metrics for application and test code separately.
References:
https://docs.atlassian.com/clover-maven-plugin/latest/clover-setup.html
https://docs.atlassian.com/clover-maven-plugin/latest/clover-mojo.html
https://confluence.atlassian.com/display/CLOVER/Creating+custom+reports
https://confluence.atlassian.com/display/CLOVER/Using+Coverage+Contexts
Related
I arrived to a legacy project where multiple files are developed in Java and many others in Kotlin. I have be able to configure Pitest to execute the mutation test and i have a correct report.
Now I would like to execute the mutation test only over the Kotlin files.
I tried to use the <targetClasses> but the param expresion is able to include certain packages, but I didn't discover a way to include certain types of files only.
I also tried to use the <excludedClasses> to add there a Java identificator that exclude this type of files, but again it doesn't work.
Do you know a way to use the targetClasses or the excludedClasses to let the kotlin files only in the scope of the Pitest execution?
Thank you in advance.
There is no built in way to limit mutation to only kotlin files. You would need to implement an mutation interceptor.
https://pitest.org/quickstart/advanced/
Or use the exclusions functionality provided by the arcmutate extentions to ignore files with a .java extension.
https://docs.arcmutate.com/docs/exclusions.html
I'm running automation tests using cucumber-junit project and i've roughly around 200 scenario's in my project.. now the problem is, it's hard to find unused step definitions in my project as we constantly need to update features.. Is there any solution to detect step definitions that is no longer useful.. Any help much appreciated!!
Since cucumber-jvm 4.4.0 it is possible to use cucumber-jvm built in plugin - unused.
for cucumber junit runner it could look like this:
#RunWith(Cucumber.class)
#CucumberOptions(
plugin = {
"json:build/report/cucumber.json",
"unused:build/report/usage.txt"} //mind this plugin
glue = "stepdefs",
features = "features"
)
public class CucumberRunner {
}
After run unused stepdefs should be found in build/report/usage.txt
Original pull request: https://github.com/cucumber/cucumber-jvm/pull/1648
There can be a case where a single step can be a part of different scenarios and will have single method only in step definition. So, It is easy to map the feature file's step with the corresponding method while executing the feature file using 'cucumber feature' pulg-in.
However, It is literally not possible to cross validate the same from step definition file to identify the single step in number of feature files.
Perhaps, The only possible way out is to design your application in modular way:
1) with feature files and corresponding step definition files specific to a specific feature/module.
2) Keep generic methods in a generic parent step definition file.
Thus, Designing the application in a modular way can easily lead you to identify the unused methods that can be removed from the step definitions.
IntelliJ's cucumber plugin can search for usages of a step definition. It will not give you all unused ones in one go, but at least you can check individual usages one by one. The plugin is also available in the Community Edition of IDEA.
I am using EclEmma to test the coverage of my scenario tests and use case tests on my project.
I have a Base package which contains the most general classes and the use case tests. The coverage looks like this:
What I want is to exclude the use case tests (e.g. BugReportTest) from the coverage calculation. But I do want the tests inside it to be considered. I know how to exclude the entire class from the coverage but if I do that, my coverage % drops because the actual tests that check which lines of my code are tested are forgotten. These use case tests do need to stay in the Base package because of privacy reasons.
For technical reasons it might be necessary to exclude certain classes from code coverage analysis. The following options configure the coverage agent to exclude certain classes from analysis. Except for performance optimization or technical corner cases these options are normally not required.
Excludes: A list of class names that should be excluded from
execution analysis. The list entries are separated by a colon (:)
and may use wildcard characters (* and ?). (Default: empty)
Exclude classloaders: A list of class loader names that should be
excluded from execution analysis. The list entries are separated by
a colon (:) and may use wildcard characters (* and ?). This option
might be required in case of special frameworks that conflict with
JaCoCo code instrumentation, in particular class loaders that do not
have access to the Java runtime classes. (Default:
sun.reflect.DelegatingClassLoader)
Warning: Use these options with caution! Invalid entries might break
the code coverage launcher. Also do not use these options to define
the scope of your analysis. Excluded classes will still show as not
covered.
Resource Link:
EclEmma Code Coverage Preferences
The following examples all specify the same set of inclusion/exclusion patterns:
<filter includes="com.foo.*" excludes="com.foo.test.*,
com.foo.*Test*" />
<filter includes="com.foo.*" /> <filter excludes="com.foo.test.*,
com.foo.*Test*" />
<filter value="+com.foo.*, -com.foo.test.*, -com.foo.*Test*" />
<filter excludes="com.foo.*Test*" file="myfilters.txt" />
where myfilters.txt file contains these lines:
-com.foo.test.*
+com.foo.*
Resource Link:
Coverage filters
I am certain that all of my classes are built with -g(debug='true')
and yet EMMA still complains about missing debug info!
Ignore code coverage for unit tests in EclEmma
Preferences->Java->Code Coverage and set the "Only path entries matching" option to src/main/java - seems to work nicely
I am using EclEmma (inside of Eclipse) to scan my JUnit code coverage. This works correctly - however, I do not want EclEmma to scan my src/view folder since it contains Swing code that I consider not worthy of testing.
Is there any way to ignore this folder when EclEmma runs so that it: a) runs faster, and b) does not skew the coverage percentage?
EDIT:
My project's structure is:
src/view
src/model
src/controller
I have tried these (possibly others) with the Path Entries section in the Preferences page:
"src/view"
"src/view/*"
"view"
"view/*"
src/view
These are using the Excludes section in the Preferences page:
*
*View*
*View*.class
src/view/*View*
src/view/*View*.class
They all leave me with the same result of it analysing my entire src folder.
[Edit] The maintainers says you cannot, except one the source directory level: https://github.com/jacoco/eclemma/issues/70
I thought eclemma wasn't excluding files: it is. Just not as I thought.
When you go into excludes in preferences and specify your.classes.here.*, for example, that means those classes won't count towards your getting all your code covered, not that those classes won't be included in what needs to be covered by tests.
Try it and see. Try to exclude a class you know have coverage in it. Once you put that to the excludes preference, on a new coverage run they'll still be there in the coverage window, but they'll come up as 0% and will all be in red.
Rather useless if you ask me. I'm still searching for an adequate solution to exclude classes by name from the classes that need to be covered by tests.
You can specify an exclude field:
Excludes: A list of class names that should be excluded from execution
analysis. The list entries are separated by a colon (:) and may use
wildcard characters (* and ?). (Default: empty)
However, it might be easier to use their options for classpath matching:
Only path entries matching: Comma separated list of strings that must
match with the class path entry. A class path entry matches the
filter, if it contains one of the given strings. (e.g.
"src/main/java", Default: no filter)
See eclemma - how to ignore source about how to ignore src folders.
Also please note their caution,
Warning: If your settings do not match any of the class path entries
in your project(s), every new launch in coverage mode will have an
empty analysis scope.
I have given up on EclEmma because I can't get it to do the things I want it to do, so I use a different method - I'll document it here in case it helps anyone else.
To exclude classes from test, I name all my test classes as *Case.java and then include or exclude them via SuiteClasses. You can read more about that at https://github.com/junit-team/junit4/wiki/Aggregating-tests-in-suites
To measure coverage, I use Maven and Cobertura. This will test just the files specified in my test suites and produce coverage reports accordingly.
We use cobertura to measure code coverage in unit testing and integrated/system testing.
Some classes are more note suitable for getting a coverage in plain unit test. (Some require database, GUI testing etc). While mock objects, stubs, gui-unit-testing framework will help here, we are considering a way to mark some classes not to be included in instrumentation.
Cobertura provides a pattern based processing to include/exclude classes. Is there a way to annotate class and skip from instrumenting?
Without above, I will have to write a custom annotation processor to get list of classes, and then user the result to skip from instrumenting phase.
if you are using Ant to run cobertura this uses fileset so you could use the excludesFile parameter to specify a file containing a list of files to ignore.
This list of files could be produced by a custom annotation parser or you could stick a unique comment in the files and search for that maybe something like
for file in `find ./ -name '*.java'`; do if grep --silent "some unique tag" $file ;then echo $file > excludeFile.lst; fi ; done