Develop Reference

maven surfire testng execution not working

I have problems setting up my project for testng usage.
Background is that I want to execute my test.xml with maven command line.
I use this plugin within the build section of my pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>test.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
However... when I call mvn clean test -DxmlSuiteFile=test.xml
all my unit tests are executed but the test.xml is untouched.
when I change the version of surefire to 2.22.2 everything works fine.
BUT I get the message: testSuiteXmlFiles0 has null value
when running only mvn test
I am confused how to get that to run with the surefire 3.0.0-M5 ?

-DxmlSuiteFile=test.xml is not required if it already defined in pom.xml.
Try to define surefire testNG provider explicitly:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>test.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-testng</artifactId>
<version>3.0.0-M5</version>
</dependency>
</dependencies>
</plugin>
test.xml file should be in the project root.
See also:
https://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html

how to avoid runing testcases without using skipTests in maven while using the mvn clean install

I have written the test cases but it is activating when I am running the mvn clean install test cases also running but my case no need to run the test cases while running the mvn clean install when I am using the particular profile then only activate the test cases.

You can configure surefire plugin in your pom.xml to include/exclude tests based on their names, and use different configuration for the profiles you want to run these tests on.
Something like:
<profiles>
<profile>
<id>without-tests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<excludes>
<exclude>**/someTests.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>without-other-tests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<excludes>
<exclude>**/otherTests.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
See more about it here

Name the tests you dont want to run with a suffix that enables you to group them ...
e.g. sometimes we use ...FunctionalTest to label functional tests. These tests can then be grouped/included/excluded when running unit tests, as a part of the standard build.
See this post for more info How to permanently exclude one test class in a Maven build

Running Tests in a separte JVM

Is there a clean way to implement running #Testmethods in each own separate jvm.
I got surefire setup to fork each TestClass but I need to run methods.
Each should start in it's own separate JVM. Ideally through an annotation so I could parametrise which tests can fork testcases and which one test methods

You can achieve forked test execution using Maven's Surefire Plugin.
See the section about forked test execution in http://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html
This is the example from the page above illustrating how to create 3 forked executions:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20.1</version>
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
<systemPropertyVariables>
<databaseSchema>MY_TEST_SCHEMA_${surefire.forkNumber}</databaseSchema>
</systemPropertyVariables>
<workingDirectory>FORK_DIRECTORY_${surefire.forkNumber}</workingDirectory>
</configuration>
</plugin>

How to build maven artifact that run test suites? How 'sbt test' works?

I'm scala developer, so i'll try to explain my thoughts with sbt example but i belive java developers can understand me
Traditional scala/java project structure is like follow:
src ->
- main
- test
- it
test folder is conceptually for unit testing (for testing app logic). So you can run 'sbt test' and will test over your application. This code must be close to your application.
may be that was offtopic, here is my question:
is it possible to build jar artifact that would run all my tests? I don't want to start sbt just in order to run my tests. I want to build many jar artifacts with integration test suites
java my-super-service-it-case1.jar
this would give me exit code 0 if tests have passed
java my-super-service-it-case2.jar
so on...
I belive i need to run test within main block..

I would not recommend to mix any of your tests within the main functionality of your program, there is a very good plugin for exactly what you want to do, Maven Surefire Plugin and it should be working fine with Scala but have not ever messed around with Scala my self.
It would be to much to write out what you need here but it's well covered in the documentation and a lot of examples. You can specify different goals which has includes and excludes so you can for example just execute Unit tests or Acceptance Tests with for example:
mvn verify -P acceptance-tests -Dbuild.env=sit
Which has following profile configured in the POM.XML file
<profile>
<id>acceptance-tests</id>
<activation>
<property>
<name>env.BUILD_STAGE</name>
<value>ACCEPTANCE</value>
</property>
</activation>
<build>
<plugins>
<!-- skip unit tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
<!-- run acceptance tests (during integration-test phase) -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/*AT*.*</include>
</includes>
<excludes>
<exclude>**/*Test.*</exclude>
<exclude>**/*IT.*</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
All my Acceptance Tests are named with AT like, myTestAT.java

Execute TestNG.xml from Jenkins (Maven Project)

My Project dosnt have a bin folder
Have tried the following:
D:>java -cp "Pathtolibfolder\lib*;Pathtobinfolder\bin"
org.testng.TestNG testng.xml
Any Ideas?

If you are using Maven then i would simply use the surefire plugin and run tests as part of the build:
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>RELEASE</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
...
</plugins>
Just specify the path to your testng.xml and take advantage of it if you can use this kind of configuration.
It allows for tons of parametrization and i have used it extensively in my projects.
Check out this tutorial to get a hang of it: http://maven.apache.org/surefire/maven-surefire-plugin/examples/testng.html