I recently stumbled upon a simple way to parallelize the execute of tests via jUnit by specifying the following in a java project's pom.xml file:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<parallel>classes</parallel>
</configuration>
</plugin>
I've discovered that there are 2 test-classes (let's call them "badtestclass1" and "badtestclass2") that keep getting penalized by this parallel execution due to the way the tests in them have been written. Ideally, I would refactor those test-classes to behave better, but in the interim, I was wondering if there's a nifty way to "exclude" these specific classes from being executed in parallel. Basically, is there a way to execute everything else in parallel and then these 2 in sequence (or the other order, doesn't matter). Would something like the following work?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<parallel>classes</parallel>
<excludes>
<excludesFile>badtestclass1</excludesFile>
<excludesFile>badtestclass2</excludesFile>
</excludes>
</configuration>
</plugin>
#polaretto's answer suggests the jcip #NotThreadSafe annotation, which works with the surefire plugin in a build, but does not work with command line mvn test. #patson-luk's answer is on the right track, unfortunately by excluding the "bad test" in the default configuration, it remained excluded and was not run in the separate <execution>.
I managed to get this working using the following configuration:
For JUnit 5, these are sufficient
In my src/test/resources/junit-platform.properties file (or you can pass as command line parameters):
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
At the top of my not-thread-safe class (instead of the jcip annotation):
import static org.junit.jupiter.api.parallel.ExecutionMode.SAME_THREAD;
#Execution(SAME_THREAD)
class SingleThreadedTest {
// ...
}
For JUnit 4, modify the accepted answer as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<!-- Skip default, define executions separately below -->
<skip>true</skip>
</configuration>
<executions>
<execution>
<id>single-thread-test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<!-- Not thread safe, run separately -->
<includes>
<include>**/SingleThreadedTest.java</include>
</includes>
<forkCount>1</forkCount>
<reuseForks>false</reuseForks>
<threadCount>1</threadCount>
<skip>false</skip>
</configuration>
</execution>
<execution>
<id>multi-thread-test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>**/SingleThreadedTest.java</exclude>
</excludes>
<forkCount>4</forkCount>
<reuseForks>true</reuseForks>
<parallel>all</parallel>
<useUnlimitedThreads>true</useUnlimitedThreads>
<skip>false</skip>
</configuration>
</execution>
</executions>
</plugin>
You can annotate the classes you don't want parallelized with jcip #NotThreadSafe and leave the surefire configuration as it was in your starting example. This way, whenever surefire finds an annotated class it just executes it in a single thread. It's explained right here in the "Parallel Test Execution and Single Thread Execution" paragraph.
Exclude those 2 tests in the original test phrase and then create a new execution with those 2 classes running in single thread? :)
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>path/to/your/class/badtestclass1.java</exclude>
<exclude>path/to/your/class/badtestclass2.java</exclude>
</excludes>
<parallel>classes</parallel>
</configuration>
<executions>
<execution>
<id>single-thread-test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>
<include>path/to/your/class/badtestclass1.java</include>
<include>path/to/your/class/badtestclass2.java</include>
</includes>
<threadCount>1</threadCount>
</configuration>
</execution>
</executions>
</plugin>
Related
I am using TestNG and maven failsafe for my integration tests.
The tests are executed and they pass however there are absolutely no details printed out.
I even created a dummy test and have System.out.println in that and none of that is printed out.
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
<properties>
<property>
<name>surefire.testng.verbose</name>
<value>10</value>
</property>
</properties>
</configuration>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
<configuration>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<skip>false</skip>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
I want the testNG summary to be printed and logging statements to be printed ...
I am also using the groovy-eclipse-compiler plugin .. not sure if that makes a difference though
When you use a testing framework, you need to use the maven surefire plugin to have a summary of your results. I dont think that anything is executed the way you do it right now, so no output is expectable.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${basedir}/src/main/resources/testng.xml</suiteXmlFile>
</suiteXmlFiles>
<skipTests>${skipTests}</skipTests>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
This question already has answers here:
Maven does not find JUnit tests to run
(34 answers)
Closed 7 years ago.
I am struggeling on executing all junit tests by maven test.
There are 57 Tests in 10 classes but somehow maven executes only
12 Tests in 6 classes. The ignored classes are in the same folder
as another class, beeing executed.
However, when run my test source folter as JUnit Test it does execute
all tests.
Funny thing: I copied a test, which had been executed, but even the copy
is ignored by maven.
Any help will be appreciated.
<build>
<sourceDirectory>src</sourceDirectory>
<testSourceDirectory>test</testSourceDirectory>
<resources>
<resource>
<directory>resources</directory>
<excludes>
<exclude>**/*.java</exclude>
</excludes>
</resource>
</resources>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>2.3</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.5.201505241946</version>
<configuration>
<append>true</append>
</configuration>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
By default, it is the maven-surefire-plugin that is launching the unit test and it includes only classes whose name match a condition.
This is documented in the includes parameter of this plugin. Quoting:
When not specified and when the test parameter is not specified, the default includes will be
<includes>
<include>**/Test*.java</include>
<include>**/*Test.java</include>
<include>**/*TestCase.java</include>
</includes>
Therefore, you need to make sure all of your test classes respect this naming convention. Otherwise, you need to override this default configuration to suit your needs.
I'd like to remove a dependency for a unit test. I found how to do it in this answer.
But I'd like to remove a dependency for only one specific test, not for all my tests. Is there a way to do that?
Not by using one Surefire execution.
You will have to define two executions of the Surefire plugin: one containing the full Classpath for most of the tests, and one containing the specialized Classpath for the single test that requires it.
Follow the Surefire plugin's documentation: http://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html
You'll have to create two executions, and bind them both to the test phase. Use the following example as a skeleton (you'll have to adjust the include and exclude patterns, as well as the excluded Classpath artifact):
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>full-cp</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>
<include>**/Test*.java</include>
</includes>
<excludes>
<exclude>MyFancyTest.java</exclude>
</excludes>
</configuration>
</execution>
<execution>
<id>special-cp</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>
<include>MyFancyTest.java</include>
</includes>
<classpathDependencyExcludes>
<classpathDependencyExcludes>excluded-artifact</classpathDependencyExcludes>
</classpathDependencyExcludes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
I am trying to get a JBehave story to execute in Maven it is completely ignoring the JBehave plugin. I've spent several hours using different configurations but it looks like the plugin isn't being executed at all. Any recommendations/tips would be appreciated!
All my JBehave classes live in:
src/at/java
Relevant parts of my pom.xml:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.5</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/at/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jbehave</groupId>
<artifactId>jbehave-maven-plugin</artifactId>
<executions>
<execution>
<id>run-stories-as-embeddables</id>
<phase>integration-test</phase>
<configuration>
<includes>
<include>**/*.java</include>
</includes>
</configuration>
<goals>
<goal>run-stories-as-embeddables</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<includes>
<include>**/*.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
Best is to change the location of your test classes to src/test/java and change the name of the stories based on the documentation of JBehave.
JBehave running with maven follow the Maven rules for location of code and text artifacts.
For test scope you must put them in src/test/java and src/test/resources. For compile scopes is src/main/java and src/main/resources.
With JBehave with maven you could use two scopes (test or compile), you just need to set which one you want in the plugin configuration, so you choose where to put your artifacts. it defaults to compile.
In your case you are adding a new test source so you must set the scope to test:
see detail here.
Maybe the jbehave-maven-plugin could not find the compiled test classes (scenarios) because it looks in the wrong classpath.
Please look at your target directory and search the embeddable classes -> target/classes or target/test-classes?
To solve the problem i must set the scope of jbehave-maven-plugin to test in the configuration of my project pom.xml.
here is a example
<plugin>
<groupId>org.jbehave</groupId>
<artifactId>jbehave-maven-plugin</artifactId>
<executions>
<execution>
<id>run-stories-as-embeddables</id>
<phase>integration-test</phase>
<goals>
<goal>run-stories-as-embeddables</goal>
</goals>
<configuration>
<scope>test</scope>
<includes>
<include>**/*Scenarios.java</include>
</includes>
<ignoreFailureInStories>true</ignoreFailureInStories>
<ignoreFailureInView>false</ignoreFailureInView>
</configuration>
</execution>
I want to run the maven compiler plugin in a different phase and with different sourceDirectories and destinationDirectories such that code from directories other than src/main/java and src/test/java can be used.
I thought the solution would look something like the below, where the phase I was linking it to was pre-integration-test. However the properties for testSourceDirectory and testOutputDirectory don't seem to be specified in this way as they are in the section of the POM.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<executions>
<execution>
<id>compile mytests</id>
<goals>
<goal>testCompile</goal>
</goals>
<phase>pre-integration-test</phase>
<configuration>
<testSourceDirectory>${basedir}/src/inttest/java</testSourceDirectory>
<testOutputDirectory>${basedir}/target/inttest-classes</testOutputDirectory>
</configuration>
</execution>
</executions>
</plugin>
Is there a way to get this plug-in to compile different directories in different phases without affecting its default operation?
The source directories are set outside the compiler-plugin inside the <build> element, so this won't work.
You can use the build-helper-maven-plugin's add-source and add-test-source to specify additional source directories for your integration tests, but this will not remove the existing source dirs.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.3</version>
<executions>
<execution>
<id>add-it-source</id>
<phase>pre-integration-test</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>${basedir}/src/inttest/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
If you bind the add-test-source goal to run just before the testCompile goal, your integration tests will be included. Note you want them to be output to target/test-classes so the surefire plugin will find them.
To handle removal of the standard test sources, I wrote a small plugin to modify the model to remove existing testSource locations before adding the ones for integration tests.
After more research it is apparent this is not actually possible in Maven 2 in the way I want, a hack of some form is necessary to introduce integration tests. While you can add additional directories (as suggested by Rich Seller) there is no plugin to remove the other sources or to compile a directory separately from the main compilation.
The best solution I have found for adding integration tests is to first use the build helper plugin to add the directory inttest directory to be compiled as tests.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/inttest/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
Now in order to get the integration tests to execute on the integration-test phase you need to use excludes and includes to manipulate when they get run as below. This allow any custom parameters you might want (in my case an agent being added via argline).
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/itest/**</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>inttests</id>
<goals>
<goal>test</goal>
</goals>
<phase>integration-test</phase>
<configuration>
<excludes><exclude>none</exclude></excludes>
<includes>
<include>**/itest/**/*Test.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>