According to the maven failsafe plugin documentation
http://maven.apache.org/surefire/maven-failsafe-plugin/
mvn verify should execute all the integration test class files but it looks like it is not doing so. The unit tests does get executed but not the integration tests even after following proper file naming conventions.
mvn verify
HelloWorldControllerTest.java (Executes)
HelloWorldControllerIT.java (Does NOT Execute)
pom.xml
<build>
<finalName>${project.artifactId}</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<fork>true</fork>
</configuration>
</plugin>
</plugins>
</build>
Do I need to do any explicit configuration? Is it mandatory to to bind fail safe's integration-test goal to maven's verify phase.
Related
I have a Spring Boot multi module Maven project, I can run integration tests with:
mvn clean verify
and it works well. I now want to run the same integration tests from a container and I don't want to embed all the source code in the container.
My question is : how can I run the Maven Failsafe Plugin without using the source code?
I tried to run the failsafe:integration-test goal and setting the dependenciesToScan parameter, from the command line:
mvn failsafe:integration-test -DdependenciesToScan=com.myorg:proj-tests.jar
but no tests are found.
P.S.1: I've seen this similar question Running spring tests from executable jar. But I don't need to run the tests without Maven. I prefer to run tests from the command line with Maven than adding code or modifying the structure of my project.
P.S.2: I'm using maven-failsafe-plugin 2.22.2 which is the version provided with Spring Boot 2.1.8.
From the docs:
Since version 2.22.0 you can scan for test classes from a project
dependency of your multi-module project.
It means the tests (proj-tests.jar) must be a dependency of the project. As you cannot have a dependency to the tests jar in the same project where you build them, the solution is to have another module or pom file. Example:
<groupId>failsafe.use.jar</groupId>
<artifactId>failsafe-use-jar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<dependencies>
...
<dependency>
<groupId>com.myorg</groupId>
<artifactId>proj-tests</artifactId>
<version>0.0.1-SNAPSHOT</version>
<classifier>tests</classifier>
</dependency>
...
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
</plugin>
</plugins>
</build>
The proj-tests is a project dependency and can be created with:
<groupId>com.myorg</groupId>
<artifactId>proj-tests</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.1.2</version>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
See Guide to using attached tests
To run the integration tests from the container you obviously need all the dependencies installed in the local
(container) maven repository or deployed in remote. Then you can run with:
mvn failsafe:integration-test -DdependenciesToScan=com.myorg:proj-tests
Note that the format of the dependenciesToScan property is groupId:artifactId (you run with the name of jar instead of artifactid)
As another note, for integration tests failsafe searches by default for class files ending in IT (integration test).
I have a JUnit 4.12 SlowTests test suite that I want to exclude from execution unless specifically requested on the Maven command line.
I have added the following to my pom file:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<configuration>
<excludedGroups>com.Example.SlowTests</excludedGroups>
<includes>
<include>**/*TestSuite.class</include>
</includes>
<excludes>
<exclude></exclude>
</excludes>
</configuration>
</plugin>
I have defined the category as SlowTests and applied it to the MySlowTests class.
I have annotated the test suite as follows:
#RunWith(Categories.class)
#IncludeCategory(SlowTests.class)
#SuiteClasses({ MySlowTests.class })
public class MySlowTestSuite
When I run mvn package all the unit tests except MySlowTests are executed.
However, looking at various answers such as https://stackoverflow.com/a/25639372/820657 and https://stackoverflow.com/a/21830866/820657 I expected the following command:
mvn package -Dgroups=com.Example.MySlowTests
to run the excluded MySlowTests tests but they don't run. In fact no tests run.
What am I doing wrong?
The Maven Surefire plugin has some issues w.r.t categories in versions < 2.13 (IIRC) but as long as you are using Surefire >= 2.13 the following will run any classes annotated with #Category(com.yourcompany.SlowTests.class):
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<configuration>
<groups>com.yourcompany.SlowTests</groups>
</configuration>
</plugin>
This approach is often used with profiles, the following configuration ...
<profiles>
<profile>
<id>slow</id>
<properties>
<testCategories>com.yourcompany.SlowTests</testCategories>
</properties>
</profile>
<profile>
<id>fast</id>
<properties>
<testCategories>com.yourcompany.FastTests</testCategories>
</properties>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<configuration>
<groups>${testCategories}</groups>
</configuration>
</plugin>
</plugins>
</build>
... can be used to run:
mvn install -P slow: runs the slow tests only
mvn install -P fast: runs the fast tests only
mvn install -P fast,slow: runs the fast and slow tests
Update 1: for this question: "Is there a way to use this approach so I can run all fast tests by default?"
You can define two properties:
<properties>
<includedCategories></includedCategories>
<excludedCategories>com.yourcompany.SlowTests</excludedCategories>
</properties>
Then update your surefire plugin definition like so:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<configuration>
<groups>${includedCategories}</groups>
<excludedGroups>${excludedCategories}</excludedGroups>
</configuration>
</plugin>
And, finally, add this profile:
<profile>
<id>slow</id>
<properties>
<includedCategories>com.yourcompany.SlowTests</includedCategories>
<excludedCategories></excludedCategories>
</properties>
</profile>
This just toggles the includedCategories and excludedCategories properties. By default, you include everything except those tests annotated with com.yourcompany.SlowTests (i.e. everything except your 'slow' tests). When you run with -P slow you exclude everything except those tests annotated with com.yourcompany.SlowTests (i.e. everything except your 'slow' tests).
Note: what I said in my original answer about Surefire versions < 2.13 misbehaving with Categories still stands so to make this work you need to be using a version of the Maven Surefire plugin >= 2.13.
Something like the following.
I would like a way to skip my dao tests in surefire. Trying to avoid overhead of defining Suites.
With CI I'd like to have one nightly that runs all tests and another 5 minute poll of SCM that runs only 'fast' tests.
mvn -DskipPattern=**.dao.** test
Let me extend Sean's answer. This is what you set in pom.xml:
<properties>
<exclude.tests>nothing-to-exclude</exclude.tests>
</properties>
<profiles>
<profile>
<id>fast</id>
<properties>
<exclude.tests>**/*Dao*.java</exclude.tests>
</properties>
</profile>
</profiles>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>${exclude.tests}</exclude>
</excludes>
</configuration>
</plugin>
Then in CI you start them like this:
mvn -Pfast test
That's it.
Sure, no problem:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.6</version>
<configuration>
<excludes>
<!-- classes that include the name Dao -->
<exclude>**/*Dao*.java</exclude>
<!-- classes in a package whose last segment is named dao -->
<exclude>**/dao/*.java</exclude>
</excludes>
</configuration>
</plugin>
Reference:
Maven Surefire Plugin > Inclusions and Exclusions of Tests
(The excludes can not be configured via command line, so if you want to turn this behavior on conditionally, you will have to define a profile and activate that on the command line)
It is possible to exclude tests using the commandline; using ! to exclude.
Note: I'm not sure but possibly needs 2.19.1 or later version of surefire to work.
Examples:
This will not run TestHCatLoaderEncryption
mvn install '-Dtest=!TestHCatLoaderEncryption'
Exclude a package:
mvn install '-Dtest=!org.apache.hadoop.**'
This can be combined with positive filters as well. The following will run 0 tests:
mvn install '-Dtest=Test*CatLoaderEncryption,!TestHCatLoaderEncryption'
See the Maven Surefire docs.
I am completely new to maven and testng. I am using maven as build tool, and testng as my testing framework. I am not following the standard maven project structure. Now I want my pom.xml to execute the test cases in my project. The question is, how pom.xml knows what are the test cases to consider for execution?
If you keep this in a single place you need to set the testClassesDirectory argument of the maven-surefire-plugin configuration:
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<testClassesDirectory>path/to/compiled test classes</testClassesDirectory>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
All of this is well documented in the Maven Surefire Plugin Documentation
Using JUnit 4.8 and the new #Category annotations, is there a way to choose a subset of categories to run with Maven's Surefire plugin?
For example I have:
#Test
public void a() {
}
#Category(SlowTests.class)
#Test
public void b() {
}
And I'd like to run all non-slow tests as in: (note that the -Dtest.categories was made up by me...).
mvn test -Dtest.categories=!SlowTests // run non-slow tests
mvn test -Dtest.categories=SlowTests // run only slow tests
mvn test -Dtest.categories=SlowTests,FastTests // run only slow tests and fast tests
mvn test // run all tests, including non-categorized
So the point is that I don't want to have to create test suites (Maven just picks up all unit tests in the project which is very convenient) and I'd like Maven to be able to pick the tests by category.
I think I just made up the -Dtest.categories, so I was wondering if there's a similar facility I can use?
Maven has since been updated and can use categories.
An example from the Surefire documentation:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<configuration>
<groups>com.mycompany.SlowTests</groups>
</configuration>
</plugin>
This will run any class with the annotation #Category(com.mycompany.SlowTests.class)
Based on this blog post - and simplifying - add this to your pom.xml:
<profiles>
<profile>
<id>SlowTests</id>
<properties>
<testcase.groups>com.example.SlowTests</testcase.groups>
</properties>
</profile>
<profile>
<id>FastTests</id>
<properties>
<testcase.groups>com.example.FastTests</testcase.groups>
</properties>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.13</version>
</dependency>
</dependencies>
<configuration>
<groups>${testcase.groups}</groups>
</configuration>
</plugin>
</plugins>
</build>
then at the command line
mvn install -P SlowTests
mvn install -P FastTests
mvn install -P FastTests,SlowTests
I had a similar case where I want to run all test EXCEPT a given category (for instance, because I have hundreds of legacy uncategorized tests, and I can't / don't want to modify each of them)
The maven surefire plugin allows to exclude categories, for instance:
<profiles>
<profile>
<id>NonSlowTests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludedGroups>my.category.SlowTest</excludedGroups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
You can use
mvn test -Dgroups="com.myapp.FastTests, com.myapp.SlowTests"
But ensure that you configure properly the maven surefire plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.12.2</version>
</dependency>
</dependencies>
</plugin>
See docs in: https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html
I lost lot of time on this error "groups/excludedGroups require TestNG or JUnit48+ on project test classpath" because I thought I was using a bad version of junit, or a bad version of the surefire plugin, or a combination that does not fit.
It was none of that: in my project I had a "config" module that was built before the module I wanted to test. This module had no junit dependency -> it had no junit on the classpath...
This mistake may help others...
Not exactly the same thing but using surefire plugin, test classes can be chosen based on file name. You are not using Junit Categories though.
An example for running just DAO tests.
<executions>
<execution>
<id>test-dao</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/com/proy/core/dao/**/*Test.java</include>
</includes>
</configuration>
</execution>
http://maven.apache.org/plugins/maven-surefire-plugin/examples/inclusion-exclusion.html
I was having issue running test with categories ex: #Category(com.mycompany.SlowTests.class) when running the test via mvn test -Dgroups=com.mycompany.SlowTests.class
I discovered that tests in a classes without the word Test in their name would not run. After adding the word Test to the class the the tests in the class ran.
For the use-case where you want to ignore certain tests by default and then conditionally run all tests from the command line, this setup will work on JUnit 4.9.
First create the marker interface:
public interface IntegrationTest {}
Annotate tests to be excluded:
#Category(IntegrationTest.class)
public class MyIntegrationTest() {}
Add plugins and profiles to pom.xml:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<!-- Read JUnit categories to be excluded from Maven property -->
<configuration>
<excludedGroups>${test.excluded.groups}</excludedGroups>
</configuration>
</plugin>
</plugins>
<profiles>
<profile>
<id>Default</id>
<!-- Set profile to be active by default -->
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<test.excluded.groups>com.example.IntegrationTest</test.excluded.groups>
</properties>
</profile>
<profile>
<id>AllTest</id>
<!-- Set groups property to blank value which will match nothing -->
<properties>
<test.excluded.groups></test.excluded.groups>
</properties>
</profile>
</profiles>
When running tests as usual from command line, integration test will be excluded:
mvn test
All tests including integration tests can be activated with the corresponding profile:
mvn test -P AllTest
Junit 5 allows you to use the #Tag annotation. More info about that here:
https://www.baeldung.com/junit-filtering-tests
I find it looks a little cleaner:
#Tag("SlowTests")
#Test
public void b() {
}