I am using Jacoco with Ant on a Java project which has a combination of unit tests as well as integration tests. We also have tests for REST APIs. In case there is a failure in any of the test cases, the entire class is not getting covered as part of Jacoco code coverage. Is there any configuration that I need to set to ensure that at least a part of code that has been executed as part of tests is covered? Here is my jacoco report XML
<jacoco:report>
<structure name="Jacoco Report">
<classfiles>
<fileset dir="${build.classes.dir}">
<exclude name="ut/**"/>
<exclude name="com/**/test/**"/>
</fileset>
</classfiles>
<sourcefiles>
<fileset dir="${build.source.dir}">
<exclude name="ut/**"/>
<exclude name="com/**/test/**"/>
</fileset>
</sourcefiles>
</structure>
<!-- to produce reports in different formats. -->
<html destdir="${build.root.dir}/jacoco"/>
<csv destfile="${build.root.dir}/jacoco/report.csv"/>
<xml destfile="${build.root.dir}/jacoco/report.xml"/>
</jacoco:report>
Related
I am new to jacoco and trying to add jacoco coverage for a web application. i saw some links and came to know that we have to add javaagent below are the things that i did.
build.xml
<?xml version="1.0" ?>
<project xmlns:jacoco="antlib:org.jacoco.ant" name="Example Ant Build with JaCoCo" default="rebuild">
<description>
Example Ant build file that demonstrates how a JaCoCo coverage report can be itegrated into an existing build in three simple steps.
</description>
<property name="src.dir" location="./src"/>
<property name="result.dir" location="./target"/>
<property name="result.classes.dir" location="${result.dir}/classes"/>
<property name="result.report.dir" location="${result.dir}/site/jacoco"/>
<property name="result.exec.file" location="${result.dir}/jacoco.exec"/>
<!-- Step 1: Import JaCoCo Ant tasks -->
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
<classpath path="../../../lib/jacocoant.jar"/>
</taskdef>
<target name="clean">
<delete dir="${result.dir}"/>
</target>
<target name="compile">
<mkdir dir="${result.classes.dir}"/>
<javac srcdir="${src.dir}" destdir="${result.classes.dir}" debug="true" includeantruntime="false"/>
</target>
<target name="test" depends="compile">
<!--
Step 2: Wrap test execution with the JaCoCo coverage task
-->
<jacoco:coverage destfile="${result.exec.file}" includes="*">
<java classname="com.antspringmvc.exp.Math" fork="true" >
<classpath path="${result.classes.dir}"/>
</java>
</jacoco:coverage>
</target>
<target name="report" depends="test">
<!-- Step 3: Create coverage report -->
<jacoco:report>
<!--
This task needs the collected execution data and ...
-->
<executiondata>
<file file="${result.exec.file}"/>
</executiondata>
<!-- the class files and optional source files ... -->
<structure name="JaCoCo Ant Example">
<classfiles>
<fileset dir="${result.classes.dir}"/>
</classfiles>
<sourcefiles encoding="UTF-8">
<fileset dir="${src.dir}"/>
</sourcefiles>
</structure>
<!-- to produce reports in different formats. -->
<html destdir="${result.report.dir}"/>
<csv destfile="${result.report.dir}/report.csv"/>
<xml destfile="${result.report.dir}/report.xml"/>
</jacoco:report>
</target>
<target name="rebuild" depends="clean,compile,test,report"/>
</project>
I have added the below code to my tomcat:
if ""%1"" == ""stop"" goto skip_agent
set "JAVA_OPTS=%JAVA_OPTS% -javaagent: C:\Users\kk\Desktop\Personal\jars\jacocoagent.jar=destfile=C:\Users\kk\Desktop\Personal\WorkSpace\JacocoExample\target\jacoco.exec,append=true,output=tcpserver,address=localhost,port=8080,includes=*"
i thought after stopping the server the report will be generated but that has not happened.
am i missing something in the configuration
From your build.xml seems that you expect JaCoCo agent to generate jacoco.exec file, however according to
if ""%1"" == ""stop"" goto skip_agent
set "JAVA_OPTS=%JAVA_OPTS% -javaagent: C:\Users\kk\Desktop\Personal\jars\jacocoagent.jar=destfile=C:\Users\kk\Desktop\Personal\WorkSpace\JacocoExample\target\jacoco.exec,append=true,output=tcpserver,address=localhost,port=8080,includes=*"
you configure agent with output=tcpserver, which means quoting documentation at http://www.jacoco.org/jacoco/trunk/doc/agent.html :
The agent listens for incoming connections on the TCP port specified by the address and port attribute. Execution data is written to this TCP connection.
Im generating a jacoco coverage report using the following target :
<target name="report" depends="test">
<!-- This task needs the collected execution data and ... -->
<executiondata>
<file file="${result.exec.file}" />
</executiondata>
<!-- the class files and optional source files ... -->
<structure name="JaCoCo Ant Example">
<classfiles>
<fileset dir="${result.classes.dir}" />
</classfiles>
<sourcefiles encoding="UTF-8">
<fileset dir="${src.dir}" >
<exclude name="**/*Test*.class"/>
</fileset>
</sourcefiles>
</structure>
<!-- to produce reports in different formats. -->
<html destdir="${result.report.dir}" />
<csv destfile="${result.report.dir}/report.csv" />
<xml destfile="${result.report.dir}/report.xml" />
</jacoco:report>
</target>
The problem is that the report takes into account the code from unit tests and I consider this fact a mistake. This way, your line coverage percent and instruction coverage will be artificially increased(because test lines are considered 100% covered) and report correctness is pretty affected. I tried to add this tag
<exclude name="**/*Test*.class"/
under fileset tag, hoping that testClasses will be excluded, but it doesnt works. Do you have any ideas for my problem? I want to avoid programmatically
report modification.
Thanks!
You need to exclude the Test class files from the classfiles fileset:
<structure name="JaCoCo Ant Example">
<classfiles>
<fileset dir="${result.classes.dir}">
<exclude name="**/*Test*.class"/>
</fileset>
</classfiles>
<sourcefiles encoding="UTF-8">
<fileset dir="${src.dir}" />
</sourcefiles>
</structure>
See documentation:
"Note that the classfiles and sourcefiles elements accept any Ant resource collection. Therefore also filtering the class file set is possible and allows to narrow the scope of the report, for example:
<classfiles>
<fileset dir="classes">
<include name="org/jacoco/examples/important/**/*.class"/>
</fileset>
</classfiles>
This is because the actual report is done from the classfiles. The sourcefiles are there to include highlighted source code in the report - as of course the human eye can't read compiled code.
Again from the documentation:
classfiles: Container element for Ant resources and resource
collections that can specify Java class files, archive files (jar,
war, ear etc. or Pack200) or folders containing class files. Archives
and folders are searched recursively for class files.
sourcefiles:
Optional container element for Ant resources and resource collections
that specify corresponding source files. If source files are
specified, some report formats include highlighted source code.
Source files can be specified as individual files or as source
directories.
I have an Ant Build file. I added jacoco coverage task to get the coverage of my junit testcases. It says process fork failed.
<jacoco:coverage output="file" destfile="${result.exec.file}">
<!-- forkmode="perTest" printsummary="on" maxmemory="3072m" failureProperty="test.failure" haltonerror="false" includeAntRuntime="true"-->
<junit printsummary="on" fork="true">
<classpath refid="test.run.path" />
<syspropertyset>
<propertyref builtin="commandline" />
</syspropertyset>
<!-- usefile="true" -->
<formatter type="xml" usefile="true" />
<!-- failureproperty="test.failure" haltonerror="false" -->
<batchtest todir="reports/junit-xml" fork="yes">
<fileset dir="src/test/java">
<include name="**/AllTest.java" />
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
<junitreport todir="${reports.dir}/junit-xml">
<fileset dir="${reports.dir}/junit-xml">
<include name="TEST-*.xml" />
</fileset>
<report format="frames" todir="${reports.dir}/junit-html" />
</junitreport>
Have you tried this without jacoco? I ask because I'm having the same problem with fork failure and I'm not using jacoco. I also have another project's ant script doing the same thing and it works fine. Yet somehow the project and ant script I'm currently working on fails the fork. In both cases I have to have fork set to "yes" because I'm specifying an alternate jvm for the junit block.
Maybe the classpath length issue is worth exploring. I know that in my case the ant script that's failing the fork has a longer class path than the one that's working. I had another project with a longer classpath that was also failing to fork but I didn't really need to fork so I just set it to "no".
I have a java project that has just been imported into TFS and I've been trying to get TFS to output the results of the unit tests.
At the end of the TFSBuild.proj file I have the following:
<ItemGroup>
<!-- Ant Call Configuration.
The build file called should be included in the workspace of the build definition.
-->
<AntBuildFile Include="$/PROJECT_NAME/Main/tfsbuild.xml">
<Targets>build,test</Targets>
<Properties>BinariesRoot=$(BinariesRoot);BuildDefinitionName=$(BuildDefinitionName);BuildDefinitionUri=$(BuildDefinitionUri);BuildDirectory=$(BuildDirectory);BuildNumber=$(BuildNumber);DropLocation=$(DropLocation);LogLocation=$(LogLocation);SourceGetVersion=$(SourceGetVersion);TestResultsRoot=$(TestResultsRoot);TeamProject=$(TeamProject);WorkspaceName=$(WorkspaceName);WorkspaceOwner=$(WorkspaceOwner)</Properties>
<Lib></Lib>
</AntBuildFile>
<!-- JUnit XML Results files should be created using the XML formatter
and be located in the following path
-->
<JUnitLogFiles Include="$(BinariesRoot)\**\TEST-*.xml" />
</ItemGroup>
This kicks off the build and tells TFS where to find the junit test results.
The issue is, TFS is not finding the unit test results even though I can see through the logs that the tests were run.
I almost gave up on this, and changed my ant file to produce a junit report and store it with the build artifacts. I changed my ant task to be:
<target name="test" depends="compile-tests">
<echo>Running unit tests, output should be in ${junit.output}</echo>
<junit printsummary="yes">
<classpath>
<pathelement path="${compile.classpath}" />
<pathelement path="${lib.dir}/junit-4.0.jar" />
<pathelement path="${build}" />
<pathelement path="${dist-classes}" />
</classpath>
<formatter type="xml" />
<batchtest fork="yes" todir="${junit.output}">
<fileset dir="${src.test}">
<include name="**/*Test.java" />
</fileset>
</batchtest>
</junit>
<mkdir dir="${DropLocation}/${BuildNumber}/test_results" />
<junitreport todir="${junit.output}">
<fileset dir="${junit.output}">
<include name="TEST-*.xml" />
</fileset>
<report todir="${DropLocation}/${BuildNumber}/test_results" />
</junitreport>
</target>
But after checking the output from the next build, I realised that the JUnit output was saved as TESTS-TestSuites.xml, not TEST-*.xml. I changed my TFSBuild.proj file accordingly, and now have the build results appearing in TFS.
Somehow the junitreport task still picks up the output though.
I have a custom runner that ships Junit tests over a socket connection to a Junit server running on other hardware. The tests run as intended with the following target:
<target name="run">
<mkdir dir="reports" />
<junit fork="yes" haltonfailure="no">
<test name="${CurrentTest}" />
<formatter type="xml" />
<classpath refid="mastersuite.classpath" />
</junit>
<junitreport todir="${JunitReport.dir}">
<fileset dir=".">
<include name="TEST-*.xml" />
</fileset>
<report todir="${JunitReport.dir}" />
</junitreport>
</target>
However, when I add in the following <batchtest> element...
<target name="run">
<delete dir="reports" failonerror="false" />
<!-- Make the reports directory -->
<mkdir dir="reports" />
<!-- Execute the tests and saves the results to XML -->
<junit fork="yes" printsummary="no" haltonfailure="no">
<batchtest fork="yes" todir="${JunitReport.dir}">
<fileset dir="${APITesting.classes}">
<include name="test/api/**/*Test.class" />
</fileset>
</batchtest>
<formatter type="xml" />
<classpath refid="mastersuite.classpath" />
</junit>
<!-- Compile the resulting XML file into an HTML based report. -->
<junitreport todir="${JunitReport.dir}">
<fileset dir="${JunitReport.dir}">
<include name="TEST-*.xml" />
</fileset>
<report todir="${JunitReport.dir}" />
</junitreport>
</target>
nothing gets shipped over to the hardware, which leads me to believe that my #RunWith(com.company.name.RemoteTestCaseRunner.class) annotation is not being honored within the context of <batchtest>. Is there something I am forgetting to do, or perhaps that must be done additionally for my #RunWith annotation to be called?
The tests are still being run and the reports are created, and certain tests that are not platform dependent do run and pass, just not the ones that require communication with services on the target hardware.
UPDATE I have determined that this works fine when using #RunWith(Suite.class) paired with #SuiteClasses({}), just not if I explicitly give it a test case. So now I'm really not sure where the problem lies.
UPDATE While I haven't found anything solid on this, the behavior of my tests seem to imply the following: based on the way my tests are formatted (they extend TestCase) I think Ant is executing my test cases as Junit3 tests. As stated above, when I run a test suite formatted for Junit4 (using only annotations) my tests run and are executed as intended. It seems that when I pass the Junit3 formatted test case directly is when my annotations are not honored, which implies that the Junit3 runner is being used.
My new question is this: is there a way to explicitly tell ant to use the Junit 4 runner?
It seems that Ant automatically resolves which runner to use based on if your test case uses the Junit4TestAdapter. I ended up having to add the following method to all of my test cases:
public class MyTestCase extends TestCase
public static junit.framework.Test suite() {
return new JUnit4TestAdapter(MyTestCase.class);
}
...
}
Most of you will probably not need to extend TestCase, but in my case this is required because Junit4 is acting as a client to a Junit3 server.