I have a custom runner that ships Junit tests over a socket connection to a Junit server running on other hardware. The tests run as intended with the following target:
<target name="run">
<mkdir dir="reports" />
<junit fork="yes" haltonfailure="no">
<test name="${CurrentTest}" />
<formatter type="xml" />
<classpath refid="mastersuite.classpath" />
</junit>
<junitreport todir="${JunitReport.dir}">
<fileset dir=".">
<include name="TEST-*.xml" />
</fileset>
<report todir="${JunitReport.dir}" />
</junitreport>
</target>
However, when I add in the following <batchtest> element...
<target name="run">
<delete dir="reports" failonerror="false" />
<!-- Make the reports directory -->
<mkdir dir="reports" />
<!-- Execute the tests and saves the results to XML -->
<junit fork="yes" printsummary="no" haltonfailure="no">
<batchtest fork="yes" todir="${JunitReport.dir}">
<fileset dir="${APITesting.classes}">
<include name="test/api/**/*Test.class" />
</fileset>
</batchtest>
<formatter type="xml" />
<classpath refid="mastersuite.classpath" />
</junit>
<!-- Compile the resulting XML file into an HTML based report. -->
<junitreport todir="${JunitReport.dir}">
<fileset dir="${JunitReport.dir}">
<include name="TEST-*.xml" />
</fileset>
<report todir="${JunitReport.dir}" />
</junitreport>
</target>
nothing gets shipped over to the hardware, which leads me to believe that my #RunWith(com.company.name.RemoteTestCaseRunner.class) annotation is not being honored within the context of <batchtest>. Is there something I am forgetting to do, or perhaps that must be done additionally for my #RunWith annotation to be called?
The tests are still being run and the reports are created, and certain tests that are not platform dependent do run and pass, just not the ones that require communication with services on the target hardware.
UPDATE I have determined that this works fine when using #RunWith(Suite.class) paired with #SuiteClasses({}), just not if I explicitly give it a test case. So now I'm really not sure where the problem lies.
UPDATE While I haven't found anything solid on this, the behavior of my tests seem to imply the following: based on the way my tests are formatted (they extend TestCase) I think Ant is executing my test cases as Junit3 tests. As stated above, when I run a test suite formatted for Junit4 (using only annotations) my tests run and are executed as intended. It seems that when I pass the Junit3 formatted test case directly is when my annotations are not honored, which implies that the Junit3 runner is being used.
My new question is this: is there a way to explicitly tell ant to use the Junit 4 runner?
It seems that Ant automatically resolves which runner to use based on if your test case uses the Junit4TestAdapter. I ended up having to add the following method to all of my test cases:
public class MyTestCase extends TestCase
public static junit.framework.Test suite() {
return new JUnit4TestAdapter(MyTestCase.class);
}
...
}
Most of you will probably not need to extend TestCase, but in my case this is required because Junit4 is acting as a client to a Junit3 server.
Related
I am trying to migrate my test suite from Junit4 to Junit5. Have a bunch of System Properties given as parameters in the older targets which run tests on Junit4 but now as i am migrating to Junit5, JunitLauncher doesn't support this parameter .
Older Target which runs tests on Junit4:
<target name="test">
<mkdir dir="${junit_reports.dir}" />
<junit printsummary="${junit.printsummary}" haltonfailure="${junit.haltonfailure}" haltonerror="${junit.haltonerror}" showoutput="${junit.showoutput}" fork="true" forkmode="once" failureProperty="failed">
<sysproperty key="clover.initstring" value="${clover.dbdir}/${clover.dbfile}" />
<sysproperty key="rules.location" value="${classes.dir}/rules/impl" />
<classpath>
<path refid="classes.classpath" />
<path refid="test.classpath" />
<pathelement path="${basedir}/../../.." />
<pathelement path="${test.classes.dir}" />
<path location="${basedir}/../common/target/test_classes" />
<pathelement location="${3rdparty.dir}/prime-server-framework/framework-core-mock.jar" />
</classpath>
<formatter type="${unittest.output.type}" />
<batchtest fork="true" todir="${junit_reports.dir}">
<fileset dir="${test.classes.dir}" includes="${tests.patternset}" />
</batchtest>
</junit>
</target>
New Target which runs tests on Junit5:
<target name = "sampletest">
<mkdir dir="${junit_reports.dir}" />
<junitlauncher>
<classpath>
<path refid="classes.classpath" />
<path refid="test.classpath" />
<pathelement path="${basedir}/../../.." />
<pathelement path="${test.classes.dir}" />
<path location="${basedir}/../common/target/test_classes" />
</classpath>
<!--<testclasses outputdir="${junit_reports.dir}">
<fileset dir="${test.classes.dir}">
<include name = "**/*Test.class"/>
</fileset>
</testclasses>-->
<test name = "impl.RulesEngineValidationTest"/>
</junitlauncher>
</target>
How do i give system properties in new target?
Ant 1.10.4 does support JUnit 5. However, it does not support all the features that Ant integration JUnit 4 does. In particular, it does not support forking the junit process and therefore passing system properties.
I found this question because I was trying to do the same thing. I found a workaround though. You can set the system properties in code before calling junitlauncher.
This code is what I used to set a single system property for file encoding. You could do something similar for your properties.
<script language="javascript">
<![CDATA[
var imports = new JavaImporter(java.lang.System);
imports.System.setProperty('file.encoding', 'ISO8859_1')
]]>
</script>
Yours is a little more complicated since your properties use others. You can read an Ant variable from inside the code. (I don't know how to read one with a dot in the name so I got rid of the dot in this example)
<property name="cloverdbdir" value="clover-dir-property-value" />
<property name="cloverdbfile" value="clover-db-file-property-value" />
<script language="javascript">
<![CDATA[
var imports = new JavaImporter(java.lang.System);
imports.System.setProperty('clover.initstring', cloverdbdir + '/' + cloverdbfile);
print(imports.System.getProperty('clover.initstring'));
]]>
</script>
There are a few things to be aware of if you use this technique:
Nashorn is deprecated for removal. It is definitely in Java 11. However, it isn't guaranteed to all future versions. It seems likely that Ant will add the system property functionality natively by then so I'm not worried about it.
The system property remains set for the remainder of the build. This doesn't look like a problem for you. If it is, you'd need another script block after calling JUnit to null it out.
I have an Ant Build file. I added jacoco coverage task to get the coverage of my junit testcases. It says process fork failed.
<jacoco:coverage output="file" destfile="${result.exec.file}">
<!-- forkmode="perTest" printsummary="on" maxmemory="3072m" failureProperty="test.failure" haltonerror="false" includeAntRuntime="true"-->
<junit printsummary="on" fork="true">
<classpath refid="test.run.path" />
<syspropertyset>
<propertyref builtin="commandline" />
</syspropertyset>
<!-- usefile="true" -->
<formatter type="xml" usefile="true" />
<!-- failureproperty="test.failure" haltonerror="false" -->
<batchtest todir="reports/junit-xml" fork="yes">
<fileset dir="src/test/java">
<include name="**/AllTest.java" />
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
<junitreport todir="${reports.dir}/junit-xml">
<fileset dir="${reports.dir}/junit-xml">
<include name="TEST-*.xml" />
</fileset>
<report format="frames" todir="${reports.dir}/junit-html" />
</junitreport>
Have you tried this without jacoco? I ask because I'm having the same problem with fork failure and I'm not using jacoco. I also have another project's ant script doing the same thing and it works fine. Yet somehow the project and ant script I'm currently working on fails the fork. In both cases I have to have fork set to "yes" because I'm specifying an alternate jvm for the junit block.
Maybe the classpath length issue is worth exploring. I know that in my case the ant script that's failing the fork has a longer class path than the one that's working. I had another project with a longer classpath that was also failing to fork but I didn't really need to fork so I just set it to "no".
I want to define in command line what unit tests are to be run. Tests are defined in a logical structure based on Java packages. From the command line I want to define what Java packages are to be included in the test run (i.e. what tests to include).
In this way I want to loop through a tokenized list of path values, and include Java tests from these paths. However the batchtest element does not support nested for loops. How can I modify this to do what I want? So the amount of includes are as many as there are defined path values.
Updated:
I have defined target like this:
<target name="test">
<property name="path" value="**"/>
<junit fork="yes" failureproperty="true" forkmode="once">
<formatter type="xml" />
<classpath>
<pathelement path="bin.path"/>
</classpath>
<batchtest haltonerror="false" todir="${test.dir}">
<fileset dir="/src/test/">
<for list="${path}" param="path">
<sequential>
<var name="path" value="#{path}"/>
<include name="**/${path}/**/*.java" />
</sequential>
</for>
</fileset>
</batchtest>
</junit>
You could use reguler expression.
For example;
<batchtest haltonfailure="yes" todir="/test">
<fileset dir="/build/classes/"
includes="**/TestBlaBla.class" excludes="**/BlaBla.class" />
</batchtest>
I hope it helps to you.
I have a java project that has just been imported into TFS and I've been trying to get TFS to output the results of the unit tests.
At the end of the TFSBuild.proj file I have the following:
<ItemGroup>
<!-- Ant Call Configuration.
The build file called should be included in the workspace of the build definition.
-->
<AntBuildFile Include="$/PROJECT_NAME/Main/tfsbuild.xml">
<Targets>build,test</Targets>
<Properties>BinariesRoot=$(BinariesRoot);BuildDefinitionName=$(BuildDefinitionName);BuildDefinitionUri=$(BuildDefinitionUri);BuildDirectory=$(BuildDirectory);BuildNumber=$(BuildNumber);DropLocation=$(DropLocation);LogLocation=$(LogLocation);SourceGetVersion=$(SourceGetVersion);TestResultsRoot=$(TestResultsRoot);TeamProject=$(TeamProject);WorkspaceName=$(WorkspaceName);WorkspaceOwner=$(WorkspaceOwner)</Properties>
<Lib></Lib>
</AntBuildFile>
<!-- JUnit XML Results files should be created using the XML formatter
and be located in the following path
-->
<JUnitLogFiles Include="$(BinariesRoot)\**\TEST-*.xml" />
</ItemGroup>
This kicks off the build and tells TFS where to find the junit test results.
The issue is, TFS is not finding the unit test results even though I can see through the logs that the tests were run.
I almost gave up on this, and changed my ant file to produce a junit report and store it with the build artifacts. I changed my ant task to be:
<target name="test" depends="compile-tests">
<echo>Running unit tests, output should be in ${junit.output}</echo>
<junit printsummary="yes">
<classpath>
<pathelement path="${compile.classpath}" />
<pathelement path="${lib.dir}/junit-4.0.jar" />
<pathelement path="${build}" />
<pathelement path="${dist-classes}" />
</classpath>
<formatter type="xml" />
<batchtest fork="yes" todir="${junit.output}">
<fileset dir="${src.test}">
<include name="**/*Test.java" />
</fileset>
</batchtest>
</junit>
<mkdir dir="${DropLocation}/${BuildNumber}/test_results" />
<junitreport todir="${junit.output}">
<fileset dir="${junit.output}">
<include name="TEST-*.xml" />
</fileset>
<report todir="${DropLocation}/${BuildNumber}/test_results" />
</junitreport>
</target>
But after checking the output from the next build, I realised that the JUnit output was saved as TESTS-TestSuites.xml, not TEST-*.xml. I changed my TFSBuild.proj file accordingly, and now have the build results appearing in TFS.
Somehow the junitreport task still picks up the output though.
I'm using JUnit Categories and ClassPathSuite in a setup similar to that described in this answer. To recap:
public interface FastTests {
}
#RunWith(Categories.class)
#Categories.IncludeCategory(FastTests.class)
#Suite.SuiteClasses(AllTests.class)
public class FastTestSuite {
}
#RunWith(ClasspathSuite.class)
public class AllTests {
}
...where AllTests makes use of the ClasspathSuite library.
A test class that's part of the FastTests category would look like this:
#Category(FastTests.class)
public class StringUtilsTest {
// ...
}
When I run "FastTestSuite" in my IDE, all tests with the FastTests annotation are executed, nice & smooth:
Now, I want to do the same thing with Ant. (To my surprise, I couldn't easily find instructions for this on SO.) In other words, I need an Ant target that runs all tests with the FastTests annotation.
I've tried some simplistic approaches using <test> or <batchtest>...
<junit showoutput="true" printsummary="yes">
<test name="fi.foobar.FastTestSuite"/>
<formatter type="xml"/>
<classpath refid="test.classpath"/>
</junit>
... but no luck, so far.
Edit: Besides the IDE, it works fine with JUnitCore on the command line:
$ java -classpath "classes:WebContent/WEB-INF/lib/*" org.junit.runner.JUnitCore fi.foobar.FastTestSuite
.............
Time: 0.189
OK (13 tests)
Right, I got it working with <batchtest> quite simply:
<junit showoutput="true" printsummary="yes" fork="yes">
<formatter type="xml"/>
<classpath refid="test.classpath"/>
<batchtest todir="${test.reports}">
<fileset dir="${classes}">
<include name="**/FastTestSuite.class"/>
</fileset>
</batchtest>
</junit>
I had tried <batchtest> earlier, but had made the silly mistake of using "**/FastTestSuite.java" instead of "**/FastTestSuite.class" in the <include> element... Sorry about that :-)
NB: it's necessary to set fork="yes" (i.e., run the tests in a separate VM); otherwise this will also produce "initializationError" at java.lang.reflect.Constructor.newInstance(Constructor.java:513) like <test> (see comments on the question). However, I couldn't get <test> working even with fork="yes".
The only shortcoming is that this produces just one JUnit XML report file (TEST-fi.foobar.FastTestSuite.xml) which makes it look like all the (hundreds) of tests are in one class (FastTestSuite). If anyone knows how to tweak this to show the tests in their original classes & packages, please let me know.
I found a workaround to use ant's junit task to run test of a specific Category:
<target name="test" description="Run Selenium tests by category">
<fail unless="category.name" message="Please specify 'category.name' property"/>
<junit showoutput="true" printsummary="true" fork="true">
<formatter type="xml"/>
<classpath refid="test.classpath"/>
<batchtest todir="${test.reports}" haltonfailure="false">
<fileset dir="${classes}">
<!-- regex '^\s*#Category' part added to ensure #Category annotation is not commented out -->
<containsregexp expression="^\s*#Category.*${category.name}"/>
</fileset>
</batchtest>
</junit>
</target>
Execute test by supplying category.name property to ant like this:
ant -Dcategory.name=FastTests test
Using batchtest will also produce separate JUnit XML report files per test (e.g. TEST-fi.foobar.FastTestClassN.xml).