I'm using JUnit 4.8.1 and Surefire 2.8.1. In my machine I can run one test method only by doing:
$$ mvn clean test -Dtest=TestClass#TestMethod
But when running the same command in the grid configuration, all tests from TestClass are run.
The only difference in the pom is the
<configuration>
<parallel>both</parallel>
</configuration>
configuration in the surefire-plugin.
Any idea, how do I run only one test method in the grid?
possible solution is to use #Categories annotation. But I use simple web driver(not grid).
try step by step set up here
The idea is in following:
you annotate a set of tests you want to exclude and then you show in POM file that you exclude'em in a way somthing like that:
<configuration>
<excludes>
<exclude>blablabla.class</exclude>
</excludes>
</configuration>
Hope this somehow helps you.
Related
I've been able to configure the project to run test cases in parallel using a TestNG runner; however, there are a handful of scenarios that are not very thread safe. If these test cases were to run in parallel, they'd interfere with each other. Now there are a couple of ways I could make these scenarios thread safe, but I was wondering if there was a way to specify these Cucumber scenarios not to run in parallel.
Is there a specific tag I could configure to tag scenarios not to run in parallel? Specify certain feature files not to run in parallel? I believe I might have come across something like that for JUnit 5, but does this exist with TestNG?
Unlike JUnit 5, TestNG does not provide such fine-grained controls. At best you can create multiple runner classes with a different selection of features and different configurations for parallel/serial execution.
Another thing I just realized was I could make the number of threads an argument; with a default of 1 thread.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.version}</version>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
<properties>
<property>
<name>dataproviderthreadcount</name>
<value>1</value>
</property>
</properties>
<includes>
<include>api.automation.tests.runners.TestNGRunnerTest</include>
</includes>
</configuration>
</plugin>
I have one Jenkins build that has all the tests tagged with the same name to run in parallel with however many threads needed.
mvn clean test -Dcucumber.filter.tags="${CucumberFilterTag} and not #opentofix" -Dspring.profiles.active=${Environment} -Ddataproviderthreadcount=4
And in the other build, I have a very similar command (with a different tag) just without the added arguement; to use the default of just one thread. This will run all those tagged tests one at a time.
mvn clean test -Dcucumber.filter.tags="${CucumberFilterTag} and not #opentofix" -Dspring.profiles.active=${Environment}
Either way, the first answer still works, but just thought to add this possibility.
In order to avoid race condition and isolate usage of a shared resource, I want to run some specific tests in a forked jvm.
How should I do it with Junit 5 & Gradle?
I'm not good in Gradle, but in maven we can do that by excluding class or packages from a profile like that :
<configuration>
<excludes>
<exclude>**/TestCircle.java</exclude>
<exclude>**/TestSquare.java</exclude>
</excludes>
</configuration>
With a simple search, I found that it is possible with gradle.
You can check that link : Gradle documentation
I have a Java JUnit Selenium test framework running some tests. There are two classes with two tests each.
I have maven surefire configured like this
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<parallel>methods</parallel>
<threadCount>100</threadCount>
<redirectTestOutputToFile>false</redirectTestOutputToFile>
</configuration>
<version>2.12.4</version>
</plugin>
I'd like it to run 4 tests simultaneously, but no matter what combination of threadCount, parallel and fork settings I use, I can only seem to get 1 class worth of test cases to run at a time. It seems like this should work, can anyone provide a solution?
Do you want to run suites in parallel or methods or tests in parallel?
The only working solution that I found for running Suites in parallel is setting
<property>
<name>suitethreadpoolsize</name>
<value>8</value>
</property>
in the pom.xml. Every other combination did not work as I needed the tests to run on same JVM, not start forked processes.
I use the following configuration for surefire v2.20.1 in maven v3.5.0
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<useUnlimitedThreads>true</useUnlimitedThreads>
<rerunFailingTestsCount>1</rerunFailingTestsCount>
<parallel>methods</parallel>
<forkedProcessExitTimeoutInSeconds>2</forkedProcessExitTimeoutInSeconds>
</configuration>
</plugin>
I believe this is working as our test suite is a lot quicker than it used to be and windows reports the running processes as largely increased specifically when surefire is running.
Probably 1 year too late for you, but just in case it can help someone.
Using parallel=methods like you did will launch all tests (methods) at a time, but 1 class at a time (sequential). So in your example, 2 classes having 2 tests, you will have all tests of of ClassA to execute, then all tests of ClassB.
If you were to use parallel=classes, then all the classes would launch at the same time, but running 1 test (method) at a time (sequential). So in your example, 2 classes having 2 tests, you will have Test1 of ClassA and Test1 of ClassB to start in parallel, and then Test2 of ClassA and Test2 of ClassB to execute afterward.
Since you want all 4 tests to execute in parallel, then use parallel=all.
Both methods and classes will execute in parallel.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<parallel>all</parallel>
<threadCount>10</threadCount>
</configuration>
<version>2.22.0</version>
</plugin>
Note : alternatively, you may want to remove the < configuration > block and set them as parameters in your mvn command line.
Ex: mvn clean test -Dparallel=all -DthreadCount=10
Regards,
I'm running some service testing using restassured and cucumber and they work fine locally just using Maven test.
The issue is if I run Maven clean, then I must run Maven update or it will not work (Says it can't find my Cucumber feature files). For reference it says:
No features found at [classpath:classpath/classpath]
This wouldn't be a huge issue except I need to have this running through Bamboo where I can't call Maven update.
So I either need to figure out what is wrong with my POM to begin with to cause this issue, or how I can run Maven update through the goals/environment variables.
The POM is fairly simple, only having the needed dependencies/reporting stuff.
The build part of the POM is as follows:
<build>
<finalName>Test</finalName>
<directory>target</directory>
<outputDirectory>target/classes</outputDirectory>
<testOutputDirectory>target/test-classes</testOutputDirectory>
<sourceDirectory>src/main/java</sourceDirectory>
<testSourceDirectory>src/test/java</testSourceDirectory>
<resources>
<resource>
<directory>src/test/resources</directory>
</resource>
</resources>
</build>
This is all in Java 8 using Eclipse as the IDE.
I would avoid specifying anything in the build section in my pom and instead use the default values.
That is, I would keep my feature files in the same package as the runner or a sub package.
The runner could for example live in the package se/thinkcode/tage
As in the directory:
./test/java/se/thinkcode/tage
This means that the feature files should live in the directory:
./test/resources/se/thinkcode/tage
This would allow me to minimize the configuration in the runner. I typically use runners that looks like this:
package se.thinkcode.tage;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
#RunWith(Cucumber.class)
public class RunCukesTest {
}
This is the smallest configuration possible if you want to run Cucumber using JUnit from Maven.
It is even smaller that the example supplied by the Cucumber team: https://github.com/cucumber/cucumber-java-skeleton
Looks like defining the features/glue in the cucumber options fixed this.
I do believe there is a better option though.
I added the following cucumber options:
features ="src/test/java",
glue = "packagename",
I have TestNG tests that are grouped with the (groups = "unit") annotation as unit or integration and run using the following Maven config:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7.2</version>
<configuration>
<groups>${test.group}</groups>
<includes>
<include>**/*Test.java</include>
<include>**/Test*.java</include>
<include>**/When*.java</include>
</includes>
<argLine>${test.args.unit} -Xmx1024m</argLine>
</configuration>
</plugin>
I would like to separate test groups by directory structure instead of annotations and place the unit tests under a directory called unit.
So for example a unit tests in: src/test/java/foo/FooTests/MyFoo.java
I want to move it to: src/test/java/foo/unit/FooTests/MyFoo.java.
Then change the Maven config to pick up Java files under a unit subdir for unit tests.
I tried doing this by changing the plugin config to use the following include:
<groups>${test.group}</groups>
<includes>
<include>**/unit/**/*Test.java</include>
<include>**/unit/**/Test*.java</include>
<include>**/unit/**/When*.java</include>
</includes>
But that doesn't seem to work. Any ideas?
The first thing is that you should use a more up-to-date maven-surefire-plugin (2.12.4) and furthermore dont use includes, cause maven-surefire-plugin has defaults for Unit tests. If you have integration tests better use maven-failsafe-plugin and name them appropriately. If you use TestNG in combination it sounds more like integration tests.
If you name your Unit testes like '*Test' and your Integration Tests as '*IT' then the plugins will pick the right tests.