I am using EclEmma (inside of Eclipse) to scan my JUnit code coverage. This works correctly - however, I do not want EclEmma to scan my src/view folder since it contains Swing code that I consider not worthy of testing.
Is there any way to ignore this folder when EclEmma runs so that it: a) runs faster, and b) does not skew the coverage percentage?
EDIT:
My project's structure is:
src/view
src/model
src/controller
I have tried these (possibly others) with the Path Entries section in the Preferences page:
"src/view"
"src/view/*"
"view"
"view/*"
src/view
These are using the Excludes section in the Preferences page:
*
*View*
*View*.class
src/view/*View*
src/view/*View*.class
They all leave me with the same result of it analysing my entire src folder.
[Edit] The maintainers says you cannot, except one the source directory level: https://github.com/jacoco/eclemma/issues/70
I thought eclemma wasn't excluding files: it is. Just not as I thought.
When you go into excludes in preferences and specify your.classes.here.*, for example, that means those classes won't count towards your getting all your code covered, not that those classes won't be included in what needs to be covered by tests.
Try it and see. Try to exclude a class you know have coverage in it. Once you put that to the excludes preference, on a new coverage run they'll still be there in the coverage window, but they'll come up as 0% and will all be in red.
Rather useless if you ask me. I'm still searching for an adequate solution to exclude classes by name from the classes that need to be covered by tests.
You can specify an exclude field:
Excludes: A list of class names that should be excluded from execution
analysis. The list entries are separated by a colon (:) and may use
wildcard characters (* and ?). (Default: empty)
However, it might be easier to use their options for classpath matching:
Only path entries matching: Comma separated list of strings that must
match with the class path entry. A class path entry matches the
filter, if it contains one of the given strings. (e.g.
"src/main/java", Default: no filter)
See eclemma - how to ignore source about how to ignore src folders.
Also please note their caution,
Warning: If your settings do not match any of the class path entries
in your project(s), every new launch in coverage mode will have an
empty analysis scope.
I have given up on EclEmma because I can't get it to do the things I want it to do, so I use a different method - I'll document it here in case it helps anyone else.
To exclude classes from test, I name all my test classes as *Case.java and then include or exclude them via SuiteClasses. You can read more about that at https://github.com/junit-team/junit4/wiki/Aggregating-tests-in-suites
To measure coverage, I use Maven and Cobertura. This will test just the files specified in my test suites and produce coverage reports accordingly.
Related
I am trying to integrate Clover with a Java project at work. Clover has been added to the classpath, the plugin has been added to the POM and the license has been set up.
When I run mvn clean clover:setup test clover:aggregate clover:clover Clover starts up correctly. However, tests that generate csv's fail because the CSV output is not what is expected. This is what gets generated for one of the tests:
<ID,NAME,[__CLR4_1_2_TEST_NAME_SNIFFER,"DESCRIPTION,Long","DESCRIPTION,Short",country
001,Fred,com_atlassian_clover.TestNameSniffer$1#e628ccb,Mr. Fred Flinstone,Fred Flinstone,US
002,Bob,com_atlassian_clover.TestNameSniffer$1#e628ccb,Mr. Bob the builder,Bob the builder,UK
003,Tintin,com_atlassian_clover.TestNameSniffer$1#e628ccb,Mr. Tin Tin,Tin Tin,FR
004,,com_atlassian_clover.TestNameSniffer$1#e628ccb,,Rob the Bob,
005,,com_atlassian_clover.TestNameSniffer$1#e628ccb],,Tom the Bob,
>
This is what is expected:
<ID,NAME,["DESCRIPTION,Long","DESCRIPTION,Short",country
001,Fred,Mr. Fred Flinstone,Fred Flinstone,US
002,Bob,Mr. Bob the builder,Bob the builder,UK
003,Tintin,Mr. Tin Tin,Tin Tin,FR
004,,,Rob the Bob,
005,],,Tom the Bob,
Why would Clover add a new column in to the CSV output, is it possible to stop it? I'm using maven 3.3, JDK 8 and Clover version 4.0.2.
I have tried looking for an answer but cannot seem to find anything similar to this issue. When I remove the failing tests Clover works fine.
I'm pretty sure it's connected to the code (or a library) you're using to create that CSV file. Clover on its own won't modify a CSV file. I suppose there's a code which access Java Object fields by reflection. Since class is instrumented by Clover, it adds some instrumentation code (like CLV_TEST_SNIFFER.
There are basically two solutions to your problem:
Modify the code which dumps classes to CSV to exclude static class members (this will effectively exclude Clover objects). Anyway, I believe it's not desired to dump static members to CSV which, as I assume, is a data projection of your Java object (static members don't belong to object but to class). Though, I don't know your full use case, so I may be simply wrong.
You can simply exclude files which cause problems (the ones which are serialised to CSV?) from Clover instrumentation, effectively you'll loose coverage data for those classes.
I'm running automation tests using cucumber-junit project and i've roughly around 200 scenario's in my project.. now the problem is, it's hard to find unused step definitions in my project as we constantly need to update features.. Is there any solution to detect step definitions that is no longer useful.. Any help much appreciated!!
Since cucumber-jvm 4.4.0 it is possible to use cucumber-jvm built in plugin - unused.
for cucumber junit runner it could look like this:
#RunWith(Cucumber.class)
#CucumberOptions(
plugin = {
"json:build/report/cucumber.json",
"unused:build/report/usage.txt"} //mind this plugin
glue = "stepdefs",
features = "features"
)
public class CucumberRunner {
}
After run unused stepdefs should be found in build/report/usage.txt
Original pull request: https://github.com/cucumber/cucumber-jvm/pull/1648
There can be a case where a single step can be a part of different scenarios and will have single method only in step definition. So, It is easy to map the feature file's step with the corresponding method while executing the feature file using 'cucumber feature' pulg-in.
However, It is literally not possible to cross validate the same from step definition file to identify the single step in number of feature files.
Perhaps, The only possible way out is to design your application in modular way:
1) with feature files and corresponding step definition files specific to a specific feature/module.
2) Keep generic methods in a generic parent step definition file.
Thus, Designing the application in a modular way can easily lead you to identify the unused methods that can be removed from the step definitions.
IntelliJ's cucumber plugin can search for usages of a step definition. It will not give you all unused ones in one go, but at least you can check individual usages one by one. The plugin is also available in the Community Edition of IDEA.
I am using EclEmma to test the coverage of my scenario tests and use case tests on my project.
I have a Base package which contains the most general classes and the use case tests. The coverage looks like this:
What I want is to exclude the use case tests (e.g. BugReportTest) from the coverage calculation. But I do want the tests inside it to be considered. I know how to exclude the entire class from the coverage but if I do that, my coverage % drops because the actual tests that check which lines of my code are tested are forgotten. These use case tests do need to stay in the Base package because of privacy reasons.
For technical reasons it might be necessary to exclude certain classes from code coverage analysis. The following options configure the coverage agent to exclude certain classes from analysis. Except for performance optimization or technical corner cases these options are normally not required.
Excludes: A list of class names that should be excluded from
execution analysis. The list entries are separated by a colon (:)
and may use wildcard characters (* and ?). (Default: empty)
Exclude classloaders: A list of class loader names that should be
excluded from execution analysis. The list entries are separated by
a colon (:) and may use wildcard characters (* and ?). This option
might be required in case of special frameworks that conflict with
JaCoCo code instrumentation, in particular class loaders that do not
have access to the Java runtime classes. (Default:
sun.reflect.DelegatingClassLoader)
Warning: Use these options with caution! Invalid entries might break
the code coverage launcher. Also do not use these options to define
the scope of your analysis. Excluded classes will still show as not
covered.
Resource Link:
EclEmma Code Coverage Preferences
The following examples all specify the same set of inclusion/exclusion patterns:
<filter includes="com.foo.*" excludes="com.foo.test.*,
com.foo.*Test*" />
<filter includes="com.foo.*" /> <filter excludes="com.foo.test.*,
com.foo.*Test*" />
<filter value="+com.foo.*, -com.foo.test.*, -com.foo.*Test*" />
<filter excludes="com.foo.*Test*" file="myfilters.txt" />
where myfilters.txt file contains these lines:
-com.foo.test.*
+com.foo.*
Resource Link:
Coverage filters
I am certain that all of my classes are built with -g(debug='true')
and yet EMMA still complains about missing debug info!
Ignore code coverage for unit tests in EclEmma
Preferences->Java->Code Coverage and set the "Only path entries matching" option to src/main/java - seems to work nicely
I use Spring JavaConfig to define my Spring configuration. Since unit tests use a different Spring configuration than production code, the production configuration shows up as 100% uncovered when I use Clover code coverage with unit tests.
I can use the #Configuration annotation to identify all these classes. Alternatively, I can use the #Bean annotation to identify all the methods within these classes.
Is there a Clover exclude or a code context filter that I can set up to globally exclude code by using these annotations? I use the maven-clover2-plugin to run Clover.
At the moment Clover allows to exclude code from coverage measurement using five methods:
excluding entire file
in Ant it is <clover-setup> / <fileset> / <excludes> tag
in Maven it is <excludes>, <excludesFile> or <excludesList> option
excluding certain methods
in Ant it is <clover-setup> / <methodContext> tag combined with the <clover-report> / <current> / <
in Maven it is not available directly - you'd have to use the <reportDescriptor> option to configure report format in an external XML file; the problem is with clover-setup which you'd also have to customise - maybe calling the Clover via maven-antrun-plugin could help here
excluding certain statements
in Ant it is <clover-setup> / <statementContext> tag combined with the <clover-report> / <current> / <
in Maven you'd have to use the <reportDescriptor>
excluding certain code blocks
in Ant it is <clover-report> / <current> / < with a predefined names of blocks, e.g. 'constructor' or 'switch' - see Using+Coverage+Contexts for more details
in Maven you'd have to use the <reportDescriptor>
excluding arbitrary source lines
You can put //CLOVER:OFF and //CLOVER:ON inline comments in the source code
Unfortunately, at the moment it's not possible to exclude the given class. However, I can see few workarounds available:
Add the //CLOVER:OFF and //CLOVER:ON comments around classes you want to exclude.
Exclude entire files
Treat these #Configuration classes as test code.
This is a hack. Clover allows to declare which classes are test classes. The match is based on file name, class signature and test signature. See the <clover-setup> / <testsources> / <testclass> tag. In this testclass tag you can define a regular expression matching entire class signature. In your case it could be:
<testclass annotation=".*#Configuration.*"/>
While this will not disable code instrumentation for these classes, they would be treated as test code and they should not contribute to application code metrics.
Note that Clover calculates code metrics for application and test code separately.
References:
https://docs.atlassian.com/clover-maven-plugin/latest/clover-setup.html
https://docs.atlassian.com/clover-maven-plugin/latest/clover-mojo.html
https://confluence.atlassian.com/display/CLOVER/Creating+custom+reports
https://confluence.atlassian.com/display/CLOVER/Using+Coverage+Contexts
If you are using Findbugs for compiled code inspection, is it possible to fail a build based on the result of a single detector or category of detectors?
For example, I would like to ensure that I don't have any null pointer-related detections (prefix of "NP" in this list) of any priority. Likewise, we really don't want to have any wait not in loop situations. That said, I don't necessarily want to fail a build based on internationalization detections as those aren't immediately critical to our application.
The desired end-state would be a process that we could tune for a variety of development phases ranging from the IDE level (Eclipse and Netbeans) to the release level (builds are generated using CruiseControl).
NOTE: I am aware that Eclipse and Netbeans both have similar detection methods built-in but this is a FindBugs specific question.
From the FindBugs Using the Ant Task section:
includeFilter
Optional attribute. It specifies the filename of a filter specifying which bugs are reported. See Chapter 8, Filter Files.
From Chapter 8:
However, a filter could also be used to select bug instances to specifically report:
$ findbugs -textui -include myIncludeFilter.xml myApp.jar
and
Match certain tests from all classes by specifying their abbreviations.
<Match>
<Bug code="DE,UrF,SIC" />
</Match></pre>
So I would assume something along the lines of:
<Match>
<Bug code="Wa,NP" />
</Match>
In your include filter and
<findbugs includeFilter="path/to/includefilter.xml"...
Would be what you're looking for.
The path/to /includeFilter (or excludeFilter) could be a property that gets set based on the value of another property which could default to something like dev for regular builds, test for CI builds, and deploy for deployment builds and specify which specific warnings you want to see or don't want to see at each stage.
Hope that helps.