Develop Reference

maven surfire testng execution not working

I have problems setting up my project for testng usage.
Background is that I want to execute my test.xml with maven command line.
I use this plugin within the build section of my pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>test.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
However... when I call mvn clean test -DxmlSuiteFile=test.xml
all my unit tests are executed but the test.xml is untouched.
when I change the version of surefire to 2.22.2 everything works fine.
BUT I get the message: testSuiteXmlFiles0 has null value
when running only mvn test
I am confused how to get that to run with the surefire 3.0.0-M5 ?

-DxmlSuiteFile=test.xml is not required if it already defined in pom.xml.
Try to define surefire testNG provider explicitly:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>test.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-testng</artifactId>
<version>3.0.0-M5</version>
</dependency>
</dependencies>
</plugin>
test.xml file should be in the project root.
See also:
https://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html

Cobertura on Jenkins Duplicate Test Runs

I have a build that includes cobertura as part of my Jenkins run. However, I realize now that it is running all my tests twice.
I have the project setup as a Maven project in Jenkins. and I had the goal to run set to: clean coberture:cobertura install
I had thought that the install portion of the command would simply run the remaining portions of the install goal, such as package. However, it reruns the compile and all the tests. This is despite the fact that there are already tests there.
I've tried configuring different combinations of pre build and post build steps, but I keep running into issues. In some combinations, the build artifacts, such as jars, never get published on Jenkins. In other cases the test results are missing.
I've thought that perhaps I need to remake the build as just a shell build. I think I could then run the command: mvn clean cobertura:cobertura && mvn install -Dmaven.test.skip=true
I think this would do what I want. It would at least stop running all the tests twice.
Is this the best approach or is there another way?
This is how I am including Cobertura in my POM:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.foo.bar</groupId>
<artifactId>my-parent</artifactId>
<version>1.2.1</version>
</parent>
<groupId>com.foo.bar.baz</groupId>
<artifactId>osom</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<description>TODO</description>
<modules>
<module>module1</module>
<module>module2</module>
<module>module3</module>
</modules>
<dependencies>
</dependencies>
<build>
<!-- To define the plugin version in your parent POM -->
<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>3.0.3</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18</version>
<configuration>
<useUnlimitedThreads>true</useUnlimitedThreads>
<parallel>suites</parallel>
<reuseForks>false</reuseForks>
<includes>
<include>**/*Test.java</include>
<include>**/Test*.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
<!-- use mvn cobertura:cobertura to generate cobertura reports -->
<reporting>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.7</version>
<configuration>
<aggregate>true</aggregate>
<format>xml</format>
</configuration>
</plugin>
</plugins>
</reporting>
</project>

The best approach I could come up with is to use a "Freestyle" project in Jenkins.
I made the project run two separate maven commands. mvn clean cobertura:cobertura and mvn install -Dmaven.test.skip=true
This prevents the tests from running twice.

If you dont want to run your tests twice, use qualinsight-mojo-cobertura maven plugin instead of cobertura-maven-plugin.

Unable to generate Allure reports using allure-maven plugin

I am unable to generate Allure test html reports using allure-maven plugin.
I am using the same version of testNG-adapter and allure maven plugin (1.4.0.RC8). But I am able to generate the allure html reports using Allure CLI.
My pom.xml excluding dependencies is
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.14</version>
<configuration>
<testFailureIgnore>false</testFailureIgnore>
<argLine>
-javaagent:${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar
</argLine>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>1.8.1</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
<reporting>
<excludeDefaults>true</excludeDefaults>
<plugins>
<plugin>
<groupId>ru.yandex.qatools.allure</groupId>
<artifactId>allure-maven-plugin</artifactId>
<version>1.4.0.RC8</version>
<configuration>
<outputDirectory>${basedir}/target/allure-reports/</outputDirectory>
<allureResultsDirectory>${basedir}/target/allure-results</allureResultsDirectory>
</configuration>
</plugin>
</plugins>
</reporting>

Add property allure.version to your pom.xml
See https://github.com/allure-framework/allure-core/wiki/Allure-Maven-Plugin

As you are not showing full POM, this example you can follow to make sure all plugin & property is correct.
https://github.com/sarkershantonu/Automation-Getting-Started/blob/master/AllureJunit/pom.xml
And, for command..
Testing => mvn clean test
Generating report => mvn site
Now, if you are running in CI servers, that need allure plugins to show report but if you are running locally, use jetter server (see my POM)
To see report via jettey : mvn jetty:run

maven surefire plugin does not inject system properties

This is how I set the values
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<systemProperties>
<property>spring.active.profiles</property>
<value>Test</value>
</systemProperties>
<systemPropertyVariables>
<spring.active.profiles>development</spring.active.profiles>
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</build>
The test fails
#Test
public void testGetProfile() throws Exception {
assertEquals("development", System.getProperty("spring.active.profiles"));
}
I see
java.lang.AssertionError:
Expected :development
Actual :null
What am I missing here?

Nothing wrong!
Run it in a console mvn test
OR
Eclipse --> [project] --> Run as --> Maven test
This test maybe fail if you run it via Eclipse --> Run as --> JUnit, sometimes this action ignores the pom.xml variables.

My bad, plugin was in a different module that test

Junit4 and TestNG in one project with Maven with inheritance

I have the same problem as Junit4 and TestNG in one project with Maven but I still have issues.
I created simple project where I need to use junit4 and testng
Here is root pom:
<groupId>com.dimas.testmodule</groupId>
<artifactId>TheParent</artifactId>
<version>0.0.001</version>
<name>TheParent</name>
<packaging>pom</packaging>
<properties>
<spring.version>3.0.5.RELEASE</spring.version>
<spring.scope>test</spring.scope>
<maven.surefire.plugin.version>2.9</maven.surefire.plugin.version>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.7</version>
<scope>test</scope>
</dependency>
...etc
</dependencies>
<build>
<finalName>${project.artifactId}</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>default</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<modules>
<module>JUnitOnly</module>
<module>TestNGOnly</module>
<module>testmodule</module>
</modules>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.plugin.version}</version>
<configuration>
<redirectTestOutputToFile>false</redirectTestOutputToFile>
<excludes>
<exclude>**/*NGTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>testNGonly</id>
<modules>
<module>JUnitOnly</module>
<module>TestNGOnly</module>
<module>testmodule</module>
</modules>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.plugin.version}</version>
<configuration>
<redirectTestOutputToFile>false</redirectTestOutputToFile>
<includes>
<include>**/*NGTest.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
modules JUnitOnly and TestModule just inherits and do not overrite any properties. And here are TestNgOnly
<groupId>com.dimas.testmodule.testngonly</groupId>
<artifactId>TestNGOnly</artifactId>
<version>0.0.001</version>
<name>testngonly</name>
<parent>
<groupId>com.dimas.testmodule</groupId>
<artifactId>TheParent</artifactId>
<version>0.0.001</version>
</parent>
<dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.3.1</version>
<scope>provided</scope>
</dependency>
</dependencies>
<profiles>
<profile>
<id>testNGtest</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.plugin.version}</version>
<configuration>
<redirectTestOutputToFile>false</redirectTestOutputToFile>
<suiteXmlFiles>
<suiteXmlFile>
<!--test/resources/my-testng.xml-->
src\test\resources\my-testng.xml
</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
Well I need to ignore testng tests for default profile and launch only testng tests for profile testNGOnly.

There are three modules - junit, testng and mixed.
The default profile should execute only junit tests - i.e. run junit module and run junit tests of mixed module. testNG profile should execute only testNG tests - i.e. run testNG module and run testNG tests of mixed module.
The way profiles work in maven, it is not possible to exclude a module in a profile. However, there is no need to build a module which is not required.
Thus, in default profile, have modules junit and mixed.
In testNG profile, add module testNG.
Following maven best practice, define <dependencies> in the parent pom within <dependencyManagement>. This allows us to only use junit dependency in junit module and testNG dependency in testNG module.
There is nothing common about surefire configuration across the three modules and as such not required to be specified in parent pom.
Since it not required to run junit tests in testNG profile, add a plugin configuration to skipTests in testNG profile for junit module.
Configure the mixed pom using the tip in this answer for the related SO question. This gets tricky since surefire needs to use junit or testng runner based on the profile.
I have incorporated the above suggestions in a sample project here.