I have a java project that has just been imported into TFS and I've been trying to get TFS to output the results of the unit tests.
At the end of the TFSBuild.proj file I have the following:
<ItemGroup>
<!-- Ant Call Configuration.
The build file called should be included in the workspace of the build definition.
-->
<AntBuildFile Include="$/PROJECT_NAME/Main/tfsbuild.xml">
<Targets>build,test</Targets>
<Properties>BinariesRoot=$(BinariesRoot);BuildDefinitionName=$(BuildDefinitionName);BuildDefinitionUri=$(BuildDefinitionUri);BuildDirectory=$(BuildDirectory);BuildNumber=$(BuildNumber);DropLocation=$(DropLocation);LogLocation=$(LogLocation);SourceGetVersion=$(SourceGetVersion);TestResultsRoot=$(TestResultsRoot);TeamProject=$(TeamProject);WorkspaceName=$(WorkspaceName);WorkspaceOwner=$(WorkspaceOwner)</Properties>
<Lib></Lib>
</AntBuildFile>
<!-- JUnit XML Results files should be created using the XML formatter
and be located in the following path
-->
<JUnitLogFiles Include="$(BinariesRoot)\**\TEST-*.xml" />
</ItemGroup>
This kicks off the build and tells TFS where to find the junit test results.
The issue is, TFS is not finding the unit test results even though I can see through the logs that the tests were run.
I almost gave up on this, and changed my ant file to produce a junit report and store it with the build artifacts. I changed my ant task to be:
<target name="test" depends="compile-tests">
<echo>Running unit tests, output should be in ${junit.output}</echo>
<junit printsummary="yes">
<classpath>
<pathelement path="${compile.classpath}" />
<pathelement path="${lib.dir}/junit-4.0.jar" />
<pathelement path="${build}" />
<pathelement path="${dist-classes}" />
</classpath>
<formatter type="xml" />
<batchtest fork="yes" todir="${junit.output}">
<fileset dir="${src.test}">
<include name="**/*Test.java" />
</fileset>
</batchtest>
</junit>
<mkdir dir="${DropLocation}/${BuildNumber}/test_results" />
<junitreport todir="${junit.output}">
<fileset dir="${junit.output}">
<include name="TEST-*.xml" />
</fileset>
<report todir="${DropLocation}/${BuildNumber}/test_results" />
</junitreport>
</target>
But after checking the output from the next build, I realised that the JUnit output was saved as TESTS-TestSuites.xml, not TEST-*.xml. I changed my TFSBuild.proj file accordingly, and now have the build results appearing in TFS.
Somehow the junitreport task still picks up the output though.
Related
I'm using Jacoco code coverage in my Ant build and build is success after using instrument for my junit classes as they have powermockito in it.
I'm getting errors in the junit classes when i ran through the ant and tests failed but build success.
I'm comipiling the src and test classes using WAS server jars from another build.xml and placed the class files in the dir.build folder. This folder is given as input for the instrument task to give me the instrumented classes. When the junit task is started it is saying that errors occured in the test classes and test failed. I used the printsummary ="on" and tried to generate logs using verbose.
I want to know what are the errors occuring while running the junit. Can somebody please tell me how can i see the errors in the Test classes.
<?xml version="1.0" encoding="UTF-8"?>
<project name = "JunitIntegration" default = "report" xmlns:jacoco="antlib:org.jacoco.ant">
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
<classpath path="./ant/lib/jacocoant.jar"/>
</taskdef>
<property name= "project.name" value = "Services"/>
<property name= "source.dir" value = "C:\\Thejesh\\Workspaces\\firstSetup\\Services"/>
<property name= "dir.build" value = "C:\\Thejesh\\Workspaces\\edpm_firstSetup\\build\\Services-classes"/>
<!-- Code coverage report -->
<property name= "result.dir" value = "C:\\Thejesh\\Workspaces\\edpm_firstSetup\\junitResults"/>
<property name= "result.classes.instr.dir" value = "${result.dir}/classes-instr"/>
<property name= "result.report.dir" value = "${result.dir}/site/jacoco"/>
<property name= "result.exec.file" value = "${result.dir}/jacoco.exec"/>
<import file="build-utils.xml"/>
<target name ="cleanL">
<delete dir ="${result.dir}"/>
</target>
<target name ="createL">
<mkdir dir ="${result.dir}"/>
<mkdir dir ="${result.classes.instr.dir}"/>
</target>
<target name ="instrument" depends = "cleanL, createL">
<jacoco:instrument destdir="${result.classes.instr.dir}">
<fileset dir="${dir.build}"/>
</jacoco:instrument>
</target>
<target name ="test" depends = "instrument">
<record name="${result.dir}/loggerinfo.log" loglevel="verbose" action="start"/>
<jacoco:coverage destfile="${result.exec.file}" xmlns:jacoco="antlib:org.jacoco.ant"
exclclassloader = "sun.reflect.DelegatingClassLoader">
<junit fork="true" forkmode="once" printsummary="on">
<formatter type= "xml"/>
<classpath>
<pathelement path="${dir.build}" />
<pathelement path="./ant/lib/jacocoagent.jar" />
<fileset dir ="C:\\Thejesh\\PIF\\Jars\\Common">
<include name ="*.jar*" />
</fileset>
</classpath>
<batchtest todir ="${result.dir}">
<fileset dir= "${result.classes.instr.dir}">
<include name ="**/*Test.class*"/>
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
<record name="${result.dir}/loggerinfo.log" loglevel="verbose" action="stop"/>
</target>
<!-- Generating code coverage reports -->
<target name ="report" depends ="test">
<jacoco:report>
<executiondata>
<file file="${result.exec.file}"/>
</executiondata>
<structure name="JUnit intergration report">
<classfiles>
<fileset dir="${dir.build}"/>
</classfiles>
<sourcefiles encoding="UTF-8">
<fileset dir="${source.dir}"/>
</sourcefiles>
</structure>
<html destdir="${result.report.dir}"/>
<xml destfile ="${result.report.dir}/report.xml"/>
</jacoco:report>
</target>
<target name ="compiler" >
<junit.compile
projectdir="${source.dir}"
destfile="${dir.build}"
/>
</target>
</project>
One way that may help you. Hopefully someone can suggest an easier way - this is snippet from build.xml I used a few years ago before upgrade to JUNIT5.
<junit printsummary="on" showoutput="off" fork="true"
haltonerror="false" haltonfailure="false"
errorproperty="junit.errors" failureproperty="junit.failures" >
<formatter type="plain"/>
<formatter type="xml"/>
You'd need to check what outout files you get from JUNIT4 in your ${result.dir} and add ant action to grab and print out part of the junit output files to the console so that the final error lines were visible
<loadfile srcfile="${junit.stdout}" property="junit.summary">
<filterchain><tailfilter lines="30"/></filterchain>
</loadfile>
<echo>${junit.summary}</echo>
and then bail out of the build if there are errors:
<fail if="junit.failures">JUNIT failed - see ${result.dir}</fail>
<fail if="junit.errors" >JUNIT error - see ${result.dir}</fail>
I have an Ant Build file. I added jacoco coverage task to get the coverage of my junit testcases. It says process fork failed.
<jacoco:coverage output="file" destfile="${result.exec.file}">
<!-- forkmode="perTest" printsummary="on" maxmemory="3072m" failureProperty="test.failure" haltonerror="false" includeAntRuntime="true"-->
<junit printsummary="on" fork="true">
<classpath refid="test.run.path" />
<syspropertyset>
<propertyref builtin="commandline" />
</syspropertyset>
<!-- usefile="true" -->
<formatter type="xml" usefile="true" />
<!-- failureproperty="test.failure" haltonerror="false" -->
<batchtest todir="reports/junit-xml" fork="yes">
<fileset dir="src/test/java">
<include name="**/AllTest.java" />
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
<junitreport todir="${reports.dir}/junit-xml">
<fileset dir="${reports.dir}/junit-xml">
<include name="TEST-*.xml" />
</fileset>
<report format="frames" todir="${reports.dir}/junit-html" />
</junitreport>
Have you tried this without jacoco? I ask because I'm having the same problem with fork failure and I'm not using jacoco. I also have another project's ant script doing the same thing and it works fine. Yet somehow the project and ant script I'm currently working on fails the fork. In both cases I have to have fork set to "yes" because I'm specifying an alternate jvm for the junit block.
Maybe the classpath length issue is worth exploring. I know that in my case the ant script that's failing the fork has a longer class path than the one that's working. I had another project with a longer classpath that was also failing to fork but I didn't really need to fork so I just set it to "no".
I'm in the process of setting up a project which uses TeamCity to build the project and run the unit tests.
The first has been done successfully and the project builds. However I can't figure out how to set up TeamCity or my build.xml such that I can view the success or failure of each individual test. Although I'm quite sure I'm attacking this from the wrong angle.
Below is my build.xml:
<project>
<property name="test" location="tests"/>
<target name="test">
<junit printsummary="yes" fork="yes" haltonfailure="yes">
<formatter type="plain"/>
<batchtest fork="true">
<fileset dir="${test}">
<include name="**/*Test*.java"/>
</fileset>
</batchtest>
</junit>
</target>
</project>
This post on stack overflow answered my main problem. So I'll close this.
Simply generating the ANT build file from within eclipse did the trick.
Now for the reporting...
I have a custom runner that ships Junit tests over a socket connection to a Junit server running on other hardware. The tests run as intended with the following target:
<target name="run">
<mkdir dir="reports" />
<junit fork="yes" haltonfailure="no">
<test name="${CurrentTest}" />
<formatter type="xml" />
<classpath refid="mastersuite.classpath" />
</junit>
<junitreport todir="${JunitReport.dir}">
<fileset dir=".">
<include name="TEST-*.xml" />
</fileset>
<report todir="${JunitReport.dir}" />
</junitreport>
</target>
However, when I add in the following <batchtest> element...
<target name="run">
<delete dir="reports" failonerror="false" />
<!-- Make the reports directory -->
<mkdir dir="reports" />
<!-- Execute the tests and saves the results to XML -->
<junit fork="yes" printsummary="no" haltonfailure="no">
<batchtest fork="yes" todir="${JunitReport.dir}">
<fileset dir="${APITesting.classes}">
<include name="test/api/**/*Test.class" />
</fileset>
</batchtest>
<formatter type="xml" />
<classpath refid="mastersuite.classpath" />
</junit>
<!-- Compile the resulting XML file into an HTML based report. -->
<junitreport todir="${JunitReport.dir}">
<fileset dir="${JunitReport.dir}">
<include name="TEST-*.xml" />
</fileset>
<report todir="${JunitReport.dir}" />
</junitreport>
</target>
nothing gets shipped over to the hardware, which leads me to believe that my #RunWith(com.company.name.RemoteTestCaseRunner.class) annotation is not being honored within the context of <batchtest>. Is there something I am forgetting to do, or perhaps that must be done additionally for my #RunWith annotation to be called?
The tests are still being run and the reports are created, and certain tests that are not platform dependent do run and pass, just not the ones that require communication with services on the target hardware.
UPDATE I have determined that this works fine when using #RunWith(Suite.class) paired with #SuiteClasses({}), just not if I explicitly give it a test case. So now I'm really not sure where the problem lies.
UPDATE While I haven't found anything solid on this, the behavior of my tests seem to imply the following: based on the way my tests are formatted (they extend TestCase) I think Ant is executing my test cases as Junit3 tests. As stated above, when I run a test suite formatted for Junit4 (using only annotations) my tests run and are executed as intended. It seems that when I pass the Junit3 formatted test case directly is when my annotations are not honored, which implies that the Junit3 runner is being used.
My new question is this: is there a way to explicitly tell ant to use the Junit 4 runner?
It seems that Ant automatically resolves which runner to use based on if your test case uses the Junit4TestAdapter. I ended up having to add the following method to all of my test cases:
public class MyTestCase extends TestCase
public static junit.framework.Test suite() {
return new JUnit4TestAdapter(MyTestCase.class);
}
...
}
Most of you will probably not need to extend TestCase, but in my case this is required because Junit4 is acting as a client to a Junit3 server.
I'm trying to make an ant target that only runs the JUnit tests on a project without any other prior actions (no depends). I'm using Emma to instrument these in another target, and then I have another target that does a bytecode mutation on these instrumented classes. All that is working and I can get JUnit to run in that target after I've performed the instrumentation/mutation steps but I need to have the ability to just run the JUnit tests separately from this compile-instrument-mutate chain.
I've build a target that looks like this:
<target name="mutant-test-run" description="Run tests with mutation applied">
<path id="run.classpath">
<pathelement location="${out.dir}" />
</path>
<mkdir dir="${reports}" />
<junit printsummary="yes" fork="true">
<classpath>
<pathelement location="${out.instr.dir}" />
<path refid="junit.classpath" />
<path refid="famaLib.classpath" />
<path refid="run.classpath" />
<path refid="emma.lib" />
</classpath>
<jvmarg value="-Demma.coverage.out.file=${coverage.dir}/coverage.emma" />
<jvmarg value="-Demma.coverage.out.merge=true" />
<formatter type="plain" />
<formatter type="xml" />
<batchtest todir="${reports}">
<fileset dir="${src.dir}">
<include name="test1.java" />
<include name="test2.java" />
<include name="test3.java" />
<include name="test4.java" />
<include name="test5.java" />
</fileset>
</batchtest>
</junit>
<emma enabled="${emma.enabled}">
<report sourcepath="${src.dir}" sort="+block,+name,+method,+class" metrics="method:70,block:80,line:80,class:100">
<fileset dir="${coverage.dir}">
<include name="*.emma" />
</fileset>
<!-- for every type of report desired, configure a nested
element; various report parameters can be inherited from the parent <report>
and individually overridden for each report type:-->
<txt outfile="${coverage.dir}/coverage.txt" depth="package" columns="class,method,block,line,name" />
<xml outfile="${coverage.dir}/coverage.xml" depth="package" />
<html outfile="${coverage.dir}/coverage.html" depth="method" columns="name,class,method,block,line" />
</report>
</emma>
</target>
The jUnit task within the target doesn't get executed however, all I get is the output from the mkdir task and the emma task.
m1> ${ANT_HOME}/bin/ant -f nnbuild.xml -verbose mutant-test-run
Apache Ant(TM) version 1.8.2 compiled on July 5 2011
Buildfile: (Elided path here)/nnbuild.xml
Detected Java version: 1.6 in: /usr/lib64/jvm/java-1.6.0-sun-1.6.0/jre
Detected OS: Linux
parsing buildfile (Elided path)/nnbuild.xml with URI = file:(Elided path)/nnbuild.xml
Project base dir set to: (Elided path)
parsing buildfile jar:file:(Elided ANT_HOME path)/ant/lib/ant.jar!(Elided ANT_HOME path)/ant/antlib.xml with URI = jar:file:(Elided ANT_HOME path)/ant/lib/ant.jar!/org/apache/tools/ant/antlib.xml from a zip file
Build sequence for target(s) `mutant-test-run' is [mutant-test-run]
Complete build sequence is [mutant-test-run, emma, clean, init, compile, run, emma-run, merge, all, sofya, ]
mutant-test-run:
[mkdir] Skipping (Elided path)/reports because it already exists.
[emma] [EMMA v2.0, build 5312 (2005/06/12 19:32:43)]
[report] processing input files ...
[report] 1 file(s) read and merged in 48 ms
[report] nothing to do: no runtime coverage data found in any of the data files
BUILD SUCCESSFUL
Total time: 0 seconds
m1>
How do you setup an ant target to only do JUnit tests?
It looks like your fileset is wrong for the batchtest element, I think maybe you need to include a **:
<batchtest todir="${reports}">
<fileset dir="${src.dir}">
<include name="test1.java" />
<include name="test2.java" />
<include name="test3.java" />
<include name="test4.java" />
<include name="test5.java" />
</fileset>
</batchtest>
From the examples on the page JUnit Task, you can use:
<batchtest fork="yes" todir="${reports.tests}">
<fileset dir="${src.tests}">
<include name="**/*Test*.java"/>
<exclude name="**/AllTests.java"/>
</fileset>
</batchtest>
Where the ** means in any sub-directory.
Another example:
<batchtest todir="${collector.dir}" unless="hasFailingTests">
<fileset dir="${collector.dir}" includes="**/*.java" excludes="**/${collector.class}.*"/>
</batchtest>