We use non-java tests. Each one of them executes our tool which is written in Java.
I'm trying to use Eclemma for creating the coverage report of the tests.
Lets start with one test. We compile the code with build.xml. I would like somehow to create a coverage report for each test and then to merge them into one main report.
I found out that Jacoco has CMD interface I could use in order to merge those reports. But I don't understand how do I run the tool with coverage package?
Which coverage package should I use? Is it Eclemma or Jacoco?
How do I run the tool with the coverage package? Should I add it into the build.xml file? Should I add it to the command line?
I'm a bit confused about the whole idea of coverage in Java. In dynamic langues such as Python and Perl, I just execute the code with the coverage module, which creates the coverage report.
The command we use to execute out tool:
gandu -vm /usr/pkgs/java/1.6.0.25-64/bin/java -configuration /.ganduData -data /.ganduData -configuration /ganduInternalConfig --session_id 1582722179
Should I add some options?
The build in build.xml:
<target name="pde-build" depends="clean, init">
<java classname="org.eclipse.equinox.launcher.Main" fork="true" failonerror="true">
<arg value="-application" />
<arg value="org.eclipse.ant.core.antRunner" />
<arg value="-buildfile" />
<arg value="${eclipseLocation}/plugins/org.eclipse.pde.build_${pdeBuildPluginVersion}/scripts/productBuild/productBuild.xml" />
<arg value="-Dtimestamp=${timestamp}" />
<classpath>
<pathelement location="${eclipseLocation}/plugins/org.eclipse.equinox.launcher_${equinoxLauncherPluginVersion}.jar" />
</classpath>
</java>
</target>
Should I add the following command?
<arg value="-autVMArgs" />
<arg value="-Xmx800m;-XX:MaxPermSize=600M;-javaagent:${jacoco-agent-path}=destfile=${jacoco-report},append=true"/>
Update: Thanks to the two answer I have managed to create the report using the CLI. But it looks like it shows (almost) zero coverage. It shows red (uncovered) on printed lines that I see that have been executed (they were printed to the stdout).
The project contains a lot of packages but for first try I'm trying to create coverage report for a specific package. I ran:
gandu -vm /usr/pkgs/java/1.6.0.25-64/bin/java -configuration /.ganduData -data /.ganduData --session_id 1583967465 -vmargs -Xmx256m -Xms128m -javaagent:/jacoco_coverage/jacoco-0.8.5/lib/jacocoagent.jar
and then I ran:
/usr/pkgs/java/1.6.0.16-64/bin/java -jar /jacoco_coverage/jacoco-0.8.5/lib/jacococli.jar report jacoco.exec --classfiles /gandu_repo/com.core.gandu/bin/ --html temp_dir --sourcefiles /gandu_repo/com.core.gandu/src/
The output:
[WARN] Some classes do not match with execution data.
[WARN] For report generation the same class files must be used as at runtime.
[WARN] Execution data for class <PATH> does not match.
[INFO] Analyzing 8 classes.
I have trouble understanding how classfiles and sourcefiles. The file that I'm trying to get coverage on is Application. In the bin I have in some inner folder Application.class and in src I have in some inner folder Application.java. I even tried to add full path for the those files in classfiles and sourcefiles but still don't have coverage (all red). What could be the problem?
The structure of the tool:
The root contains alot of packages and each one has folder bin and src. Each src contains folders recursively with Java files. The bin has the same structure for class files.
As I know, the only place to add java agent is in the configuration of the unit tests.
Agent should be added to the JVM that executes application under test. Your confusion probably comes from the fact that usually unit tests are executed in the same JVM as code under test.
From your description unclear how JVM with the application is started.
However consider following src/Main.java as an example:
import java.io.BufferedReader;
import java.io.InputStreamReader;
public class Main {
public static void main(String[] args) throws Exception {
String input;
try (BufferedReader reader = new BufferedReader(new InputStreamReader(System.in))) {
input = reader.readLine();
}
if ("hello".equals(input)) {
System.out.println("Hello, World!");
}
}
}
after compilation
javac -d classes src/Main.java
This application can be executed in JVM with JaCoCo Java Agent taken from JaCoCo zip distribution as following
java -javaagent:jacoco/lib/jacocoagent.jar -cp classes Main
Then tests are executed - e.g. in this case manual input of "hello".
After graceful termination of JVM, you'll get jacoco.exec file.
By passing this jacoco.exec, source and class files to the JaCoCo Command Line Interface you can create HTML report:
java -jar jacoco/lib/jacococli.jar report jacoco.exec --classfiles classes --sourcefiles src --html report
Which coverage package should I use? Is it Eclemma or Jacoco?
EclEmma is a plugin for Eclipse IDE based on JaCoCo.
If for testing you execute your application from Eclipse IDE then you can start it with JaCoCo using EclEmma. Here is the same example executed in Eclipse IDE:
However this is not suitable for other scenarios such as Ant builds.
How do I run the tool with the coverage package? Should I add it into the build.xml file? Should I add it to the command line?
This depends on how you execute your application - if for tests your application is executed by Ant, then you can use JaCoCo Ant Tasks for configuration of agent and generation of report. As an example here is build.xml for Ant that produces the same report for src/Example.java as the above commands:
<project xmlns:jacoco="antlib:org.jacoco.ant" name="Example Ant Build with JaCoCo" default="rebuild">
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
<classpath path="jacoco/lib/jacocoant.jar"/>
</taskdef>
<target name="clean">
<delete dir="classes"/>
<delete dir="report"/>
<delete file="jacoco.exec"/>
</target>
<target name="compile">
<mkdir dir="classes"/>
<javac srcdir="src" destdir="classes" debug="true" includeantruntime="false"/>
</target>
<target name="run" depends="compile">
<jacoco:agent property="jacocoAgent" />
<java classname="Main" fork="true">
<classpath path="classes"/>
<jvmarg value="${jacocoAgent}"/>
</java>
</target>
<target name="report" depends="run">
<jacoco:report>
<executiondata>
<file file="jacoco.exec"/>
</executiondata>
<structure name="JaCoCo Ant Example">
<classfiles>
<fileset dir="classes"/>
</classfiles>
<sourcefiles encoding="UTF-8">
<fileset dir="src"/>
</sourcefiles>
</structure>
<html destdir="report"/>
</jacoco:report>
</target>
<target name="rebuild" depends="clean,report"/>
</project>
If application is executed by some custom homemade tool, then maybe be better to add the agent to the command line directly.
Related
I'm trying to use Daikon to test a large project so I needed ant to compile and run everything for me. Since I'm new to ant I'm trying to get familiar with it through a simple example given to Daikon. So, the best candidate example is the associated StackAr package with Daikon.
To run Daikon normally I will execute the following command after compiling all the java files I'm targeting.
java daikon.Chicory --daikon DataStructures.StackArTester
which is equivalent to this command
java -cp "/usr/lib/daikonparent/daikon-5.2.20/daikon.jar:." daikon.Chicory --daikon DataStructures.StackArTester
Both commands will run daikon.jar using Chicory interface with the flag --daikon and will take the StackArTester.class to instrument it and run it.
Now, my attempt to run the same command using ant looks as the following:
<project name="StackAr" default="compile" basedir=".">
<description>
simple example build file
</description>
<!-- set global properties for this build -->
<property name="src" location="DataStructures"/>
<property name="build" location="DataStructures"/>
<target name="init">
<!-- Create the time stamp -->
<tstamp/>
</target>
<target name="compile" depends="init"
description="compile the source">
<!-- Compile the java code from ${src} into ${build} -->
<javac srcdir="${src}"
destdir="."
debug="on"
/>
</target>
<target name="run" description="run StackArTester">
<java classname="DataStructures.StackArTester">
<classpath>
<pathelement location="."/>
</classpath>
</java>
</target>
<target name="inv" description="run daikon">
<java classname="daikon.Chicory" >
<arg value="--daikon" />
<arg value="DataStructures.StackArTester"/>
<classpath>
<pathelement path="/usr/lib/daikonparent/daikon-5.2.20/daikon.jar"/>
</classpath>
</java>
</target>
<target name="clean"
description="clean up">
<!-- Delete the ${build} and ${dist} directory trees -->
</target>
</project>
The target compile is used to compile all the source files and is working. Also, the target run is used as a test to run the targeted .class file and is working. The target inv is where I'm having trouble and it's suppose to be equivalent to the two command above.
My issue is that I'm not sure how I'm suppose to pass the .calss file to Daikon. From the results I have seen I'm positive that first argument/flag --daikon is recognized. However, for the second argument DataStructures.StackArTester in most of my tires I see that it's recognized, but Daikon has an issue with its classpath. Runing the script above ant inv will generate the following error message:
Buildfile: /usr/lib/daikonparent/daikon-5.2.20/examples/java-examples/StackAr/build.xml
inv:
[java]
[java] Executing target program: java -cp /usr/share/ant/lib/ant-launcher.jar:/usr/lib/daikonparent/daikon-5.2.20/daikon.jar:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/rt.jar:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/share/ant/lib/ant-testutil.jar:/usr/share/ant/lib/ant-apache-log4j.jar:/usr/share/ant/lib/ant-javamail.jar:/usr/share/ant/lib/ant-jsch.jar:/usr/share/ant/lib/ant-apache-regexp.jar:/usr/share/ant/lib/ant-antlr.jar:/usr/share/ant/lib/ant-swing.jar:/usr/share/ant/lib/ant-apache-oro.jar:/usr/share/ant/lib/ant-apache-bcel.jar:/usr/share/ant/lib/ant-apache-xalan2.jar:/usr/share/ant/lib/ant-junit.jar:/usr/share/ant/lib/ant-launcher.jar:/usr/share/ant/lib/ant-apache-bsf.jar:/usr/share/ant/lib/ant-jdepend.jar:/usr/share/ant/lib/ant-apache-resolver.jar:/usr/share/ant/lib/ant-jmf.jar:/usr/share/ant/lib/ant-commons-logging.jar:/usr/share/ant/lib/ant-commons-net.jar:/usr/share/ant/lib/ant-junit4.jar:/usr/share/ant/lib/ant.jar:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar -ea -Xmx1000m -javaagent:/usr/lib/daikonparent/daikon-5.2.20/java/ChicoryPremain.jar=--daikon --dtrace-file=StackArTester.dtrace.gz DataStructures.StackArTester
[java] entered daikon.chicory.Runtime.setDtrace(./StackArTester.dtrace.gz, false)...
[java] Error: Could not find or load main class DataStructures.StackArTester
[java] Chicory warning: No methods were instrumented.
[java] Warning: Did not run Daikon because target exited with 1 status
[java] Java Result: 1
BUILD SUCCESSFUL
Total time: 0 seconds
Note that the error is
Error: Could not find or load main class DataStructures.StackArTester
I'm out of ideas about what I might try. Hopefully the information above is enough. Please note that I'm in the same directory when executing the original commands and the ant command.
Thakns ...
The error
[java] Error: Could not find or load main class DataStructures.StackArTester
means that Java did not find the class DataStructures.StackArTester on your classpath.
On the line that starts with [java] Executing target program, the -cp command-line argument is your classpath. The classpath must contain a directory or jar file containing DataStructures/StackArTester.class. You just need to augment the classpath.
I have my Eclipse selenium project with few classes in it,I want to view the test reports in XSLT, my project has been complied and ran successfully through ANT, however when I try to run makexsltreports (either through Eclipse or through ANT command prompt am getting the below error:
C:\Automation\Automation\build.xml:89: javax.xml.transform.TransformerFactoryConfigurationError: Provider net.sf.saxon.TransformerFactoryImpl not found
Below is my build.xml:
<path id="test.c">
<fileset dir="${ws.jars}" includes="*.jar"/>
</path>
<target name="makexsltreports">
<mkdir dir="${ws.home}/XSLT_Reports/output"/>
<xslt in="${ng.result}/testng-results.xml" style="src/xslt/testng-results.xsl"
out="${ws.home}/XSLT_Reports/output/index.html" classpathref="test.c" processor="SaxonLiaison">
<param name="testNgXslt.outputDir" expression="${ws.home}/XSLT_Reports/output/"/>
<param name="testNgXslt.showRuntimeTotals" expression="true"/>
</xslt>
</target>
This issue is because of missing JARs, Please add all required JAR files into project and class path and try again
I added classpath to an ant script. Then, in a java file, I wrote "java.class.path" property to get the classpath. When I run a test suite in Eclipse, the classpath is in the property value. However, when I ran the test suite with ant script in terminal, the classpath is not in the property value. I couldn't find the reason.
Here is the ant script snippet. I added "target/class" and jar files to the classpath.
<target name="test">
<echo message="Running unit tests ..."/>
<junit printsummary="true"
showoutput="true"
haltonfailure="false">
<formatter type="plain" usefile="false"/>
<classpath>
<pathelement path="${result.classes.dir}"/>
<path refid="jmeter_cp"/>
</classpath>
<test name="test.JMeterTestSuite"/>
</junit>
</target>
And here is java code snippet. It retrieves "java.class.path" property value.
stPaths = new StringTokenizer(
System.getProperty("java.class.path"),
System.getProperty("path.separator"));
I looked at the "java.class.path" value, but it did not have "target/class" and the jar files...
FYI, I'm using an modified ant from a mutation tool(http://mutation-testing.org/). "java.class.path" value contains only libraries for the tool. The tool might delete the classpath that I added. But I'm not sure, so this is why I'm posting this question here.
I'm trying to run TestNG tests (in a contained Java project) from Jenkins but having no luck.
It appears as though the TestNG plugin for Jenkins (https://wiki.jenkins-ci.org/display/JENKINS/testng-plugin) only publishes the results of TestNG tests, but doesn't actually run test classes... or am I wrong?
In any case, how do I actually run TestNG tests in a TestNG project with Jenkins, or is that even possible? Do I have to use a command line statement or batch file (on Windows Server 2008), for example?
Any help much appreciated.
Note I tried entering a post-build command line in Jenkins for the project to run TestNG tests but had a hard time with class paths not being found for TestNG. I posted an earlier question about running TestNG from the command line which I couldn't get working, so I've given up on that route:
How to run TestNG from command line
There are two steps to accomplished this task:-
Step 1:-
Go to localhost:8080/configure (Jenkins configure section)
Now go to JDK section and uncheck Install automatically (If you don't uncheck that then it will download latest java every time whenever it is available, and can cause for build failed)
put JAVA_HOME in name section and jdk home path in JAVA_HOME section
Apply and save
Step 2:-
Go to Jenkins and add new Item, also select "Free Style Project" and click on Ok.
Click on "Advanced in "Advanced Project Options"
Now check option: - "Use custom workspace" and specify your project absolute path in Directory section
Apply
Now to go "Build" and select "Execute windows batch command"
Here in command column give the file name of your batch file
Apply and save
Now go to the Jenkins and select your Jenkins project and click on Build :)
I use gradle to run my testNG tests from Jenkins.
Take a look at the gradle docs.
I run the testNG tests using configuration xml files.
Take a look at the testNG docs.
There is quite a lot to cover so I suggest reading these sources but I'll provide some relevant pieces from one of my configurations.
The relevant parts from my build.gradle
tasks.withType(Test) {
useTestNG {
useDefaultListeners = true
}
options {
outputDirectory = file('test-report')
listeners << 'org.uncommons.reportng.JUnitXMLReporter'
}
testLogging.showStandardStreams = true
systemProperties System.getProperties()
systemProperty "org.uncommons.reportng.escape-output", "false"
systemProperty "org.uncommons.reportng.title", "Test Report"
ignoreFailures = true
}
task Smoke_Test(type: Test) {
description "SmokeTest"
options.suites("resources/testng-smoketest.xml")
ignoreFailures = false
}
My testNG xml as referenced above 'testng-smoketest.xml'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Smoke Tests" >
<test name="BootCheck" parallel="false" thread-count="1">
<classes>
<class name="com.x.automation.y.tests.smoke.BootCheck" />
</classes>
</test>
</suite>
And from Jenkins, as an 'execute shell' build step run the gradle task, I use gradle wrapper for convenience.
./gradlew clean Smoke_Test
Ensure you're in the correct directory, 'Smoke_Test' is the name specified in the build.gradle.
You can use the testNG Jenkins plugin for saving your results.
I also recommend using reportng for nice formatting of your test reports which can also be shown and saved in Jenkins using the HTML Publisher plugin.
Try getting this to run from a CLI on your local machine first, trying to debug when running from Jenkins will drive you crazy.
As commented above, please use the following ant script to run TestNG unit tests. Please tweak the below code to meet your requirements.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project basedir="." default="build" name="Ant Play">
<property name="classes.dir" value="bin" />
<property name="report.dir" value="test-output" />
<path id="classpath">
<fileset dir="lib">
<include name="**/*.jar"/>
</fileset>
<pathelement path="${basedir}\${classes.dir}"/>
</path>
<target name="init">
<mkdir dir="${classes.dir}"/>
<copy includeemptydirs="false" todir="${classes.dir}">
<fileset dir="src">
<exclude name="**/*.java"/>
</fileset>
</copy>
</target>
<target name="clean">
<delete dir="${classes.dir}"/>
</target>
<target depends="clean" name="cleanall"/>
<target depends="build-project" name="build"/>
<target depends="init" name="build-project">
<echo message="${ant.project.name}: ${ant.file}"/>
<javac debug="true" includeantruntime="false" destdir="${classes.dir}">
<src path="src"/>
<classpath refid="classpath"/>
</javac>
</target>
<target depends="build" name="runTests" description="Running tests" >
<echo>Running Tests...</echo>
<taskdef resource="testngtasks" classpathref="classpath"/>
<testng outputDir="${report.dir}"
haltonfailure="true"
useDefaultListeners="false"
listeners="org.uncommons.reportng.HTMLReporter"
classpathref="classpath">
<xmlfileset dir="${basedir}" includes="testng.xml"/>
<!--<classfileset dir="${classes.dir}" includes="**/*.class" />-->
</testng>
</target>
</project>
Let me know if you encounter any issues. BTW, please use the Jenkins ant plugin/task to run this script
My program getting command line arguments. How can I pass it when I use Ant?
Extending Richard Cook's answer.
Here's the ant task to run any program (including, but not limited to Java programs):
<target name="run">
<exec executable="name-of-executable">
<arg value="${arg0}"/>
<arg value="${arg1}"/>
</exec>
</target>
Here's the task to run a Java program from a .jar file:
<target name="run-java">
<java jar="path for jar">
<arg value="${arg0}"/>
<arg value="${arg1}"/>
</java>
</target>
You can invoke either from the command line like this:
ant -Darg0=Hello -Darg1=World run
Make sure to use the -Darg syntax; if you ran this:
ant run arg0 arg1
then ant would try to run targets arg0 and arg1.
If you do not want to handle separate properties for each possible argument, I suggest you'd use:
<arg line="${args}"/>
You can check if the property is not set using a specific target with an unless attribute and inside do:
<input message="Type the desired command line arguments:" addProperty="args"/>
Putting it all together gives:
<target name="run" depends="compile, input-runargs" description="run the project">
<!-- You can use exec here, depending on your needs -->
<java classname="Main">
<arg line="${args}"/>
</java>
</target>
<target name="input-runargs" unless="args" description="prompts for command line arguments if necessary">
<input addProperty="args" message="Type the desired command line arguments:"/>
</target>
You can use it as follows:
ant
ant run
ant run -Dargs='--help'
The first two commands will prompt for the command-line arguments, whereas the latter won't.
The only effective mechanism for passing parameters into a build is to use Java properties:
ant -Done=1 -Dtwo=2
The following example demonstrates how you can check and ensure the expected parameters have been passed into the script
<project name="check" default="build">
<condition property="params.set">
<and>
<isset property="one"/>
<isset property="two"/>
</and>
</condition>
<target name="check">
<fail unless="params.set">
Must specify the parameters: one, two
</fail>
</target>
<target name="build" depends="check">
<echo>
one = ${one}
two = ${two}
</echo>
</target>
</project>
Can you be a bit more specific about what you're trying to do and how you're trying to do it?
If you're attempting to invoke the program using the <exec> task you might do the following:
<exec executable="name-of-executable">
<arg value="arg0"/>
<arg value="arg1"/>
</exec>
What I did in the end is make a batch file to extract the CLASSPATH from the ant file, then run java directly using this:
In my build.xml:
<target name="printclasspath">
<pathconvert property="classpathProp" refid="project.class.path"/>
<echo>${classpathProp}</echo>
</target>
In another script called 'run.sh':
export CLASSPATH=$(ant -q printclasspath | grep echo | cut -d \ -f 7):build
java "$#"
It's no longer cross-platform, but at least it's relatively easy to use, and one could provide a .bat file that does the same as the run.sh. It's a very short batch script. It's not like migrating the entire build to platform-specific batch files.
I think it's a shame there's not some option in ant whereby you could do something like:
ant -- arg1 arg2 arg3
mpirun uses this type of syntax; ssh also can use this syntax I think.