Develop Reference

Maven JUnit 5: Run all unit tests by default or run only a specific test suite from the command line [duplicate]

I have just upgraded my solution to use JUnit5. Now trying to create tags for my tests that have two tags: #Fast and #Slow. To start off I have used the below maven entry to configure which test to run with my default build. This means that when I execute mvn test only my fast tests will execute. I assume I can override this using the command line. But I can not figure out what I would enter to run my slow tests....
I assumed something like.... mvn test -Dmaven.IncludeTags=fast,slow which does not work
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<properties>
<includeTags>fast</includeTags>
<excludeTags>slow</excludeTags>
</properties>
</configuration>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.0.0-M3</version>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>1.0.0-M3</version>
</dependency>
</dependencies>
</plugin>

You can use this way:
<properties>
<tests>fast</tests>
</properties>
<profiles>
<profile>
<id>allTests</id>
<properties>
<tests>fast,slow</tests>
</properties>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<groups>${tests}</groups>
</configuration>
</plugin>
</plugins>
</build>
This way you can start with mvn -PallTests test all tests (or even with mvn -Dtests=fast,slow test).

Using a profile is a possibility but it is not mandatory as groups and excludedGroups are user properties defined in the maven surefire plugin to respectively include and exclude any JUnit 5 tags (and it also works with JUnit 4 and TestNG test filtering mechanism).
So to execute tests tagged with slow or fast you can run :
mvn test -Dgroups=fast,slow
If you want to define the excluded and/or included tags in a Maven profile you don't need to declare a new property to convey them and to make the association of them in the maven surefire plugin. Just use groups and or excludedGroups defined and expected by the maven surefire plugin :
<profiles>
<profile>
<id>allTests</id>
<properties>
<groups>fast,slow</groups>
</properties>
</profile>
</profiles>

You can omit profile and just use properties, it is more elastic way.
<properties>
<tests>fast</tests>
</properties>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<groups>${tests}</groups>
</configuration>
</plugin>
</plugins>
</build>
Then you can runt fast tests by typing mvn test, all test by typing mvn test -Dtests=fast | slow or only slow by typing mvn test -Dtests=slow. When you have more test tags then you can also run all of them except the selected type by typing mvn test -Dtests="! contract".

Does mvn verify executes all integration test classes suffixed with *IT.java

According to the maven failsafe plugin documentation
http://maven.apache.org/surefire/maven-failsafe-plugin/
mvn verify should execute all the integration test class files but it looks like it is not doing so. The unit tests does get executed but not the integration tests even after following proper file naming conventions.
mvn verify
HelloWorldControllerTest.java (Executes)
HelloWorldControllerIT.java (Does NOT Execute)
pom.xml
<build>
<finalName>${project.artifactId}</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<fork>true</fork>
</configuration>
</plugin>
</plugins>
</build>
Do I need to do any explicit configuration? Is it mandatory to to bind fail safe's integration-test goal to maven's verify phase.

How to exclude some directory in maven command line for test? [duplicate]

Something like the following.
I would like a way to skip my dao tests in surefire. Trying to avoid overhead of defining Suites.
With CI I'd like to have one nightly that runs all tests and another 5 minute poll of SCM that runs only 'fast' tests.
mvn -DskipPattern=**.dao.** test

Let me extend Sean's answer. This is what you set in pom.xml:
<properties>
<exclude.tests>nothing-to-exclude</exclude.tests>
</properties>
<profiles>
<profile>
<id>fast</id>
<properties>
<exclude.tests>**/*Dao*.java</exclude.tests>
</properties>
</profile>
</profiles>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>${exclude.tests}</exclude>
</excludes>
</configuration>
</plugin>
Then in CI you start them like this:
mvn -Pfast test
That's it.

Sure, no problem:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.6</version>
<configuration>
<excludes>
<!-- classes that include the name Dao -->
<exclude>**/*Dao*.java</exclude>
<!-- classes in a package whose last segment is named dao -->
<exclude>**/dao/*.java</exclude>
</excludes>
</configuration>
</plugin>
Reference:
Maven Surefire Plugin > Inclusions and Exclusions of Tests
(The excludes can not be configured via command line, so if you want to turn this behavior on conditionally, you will have to define a profile and activate that on the command line)

It is possible to exclude tests using the commandline; using ! to exclude.
Note: I'm not sure but possibly needs 2.19.1 or later version of surefire to work.
Examples:
This will not run TestHCatLoaderEncryption
mvn install '-Dtest=!TestHCatLoaderEncryption'
Exclude a package:
mvn install '-Dtest=!org.apache.hadoop.**'
This can be combined with positive filters as well. The following will run 0 tests:
mvn install '-Dtest=Test*CatLoaderEncryption,!TestHCatLoaderEncryption'
See the Maven Surefire docs.

How to run JUnit tests by category in Maven?

Using JUnit 4.8 and the new #Category annotations, is there a way to choose a subset of categories to run with Maven's Surefire plugin?
For example I have:
#Test
public void a() {
}
#Category(SlowTests.class)
#Test
public void b() {
}
And I'd like to run all non-slow tests as in: (note that the -Dtest.categories was made up by me...).
mvn test -Dtest.categories=!SlowTests // run non-slow tests
mvn test -Dtest.categories=SlowTests // run only slow tests
mvn test -Dtest.categories=SlowTests,FastTests // run only slow tests and fast tests
mvn test // run all tests, including non-categorized
So the point is that I don't want to have to create test suites (Maven just picks up all unit tests in the project which is very convenient) and I'd like Maven to be able to pick the tests by category.
I think I just made up the -Dtest.categories, so I was wondering if there's a similar facility I can use?

Maven has since been updated and can use categories.
An example from the Surefire documentation:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<configuration>
<groups>com.mycompany.SlowTests</groups>
</configuration>
</plugin>
This will run any class with the annotation #Category(com.mycompany.SlowTests.class)

Based on this blog post - and simplifying - add this to your pom.xml:
<profiles>
<profile>
<id>SlowTests</id>
<properties>
<testcase.groups>com.example.SlowTests</testcase.groups>
</properties>
</profile>
<profile>
<id>FastTests</id>
<properties>
<testcase.groups>com.example.FastTests</testcase.groups>
</properties>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.13</version>
</dependency>
</dependencies>
<configuration>
<groups>${testcase.groups}</groups>
</configuration>
</plugin>
</plugins>
</build>
then at the command line
mvn install -P SlowTests
mvn install -P FastTests
mvn install -P FastTests,SlowTests

I had a similar case where I want to run all test EXCEPT a given category (for instance, because I have hundreds of legacy uncategorized tests, and I can't / don't want to modify each of them)
The maven surefire plugin allows to exclude categories, for instance:
<profiles>
<profile>
<id>NonSlowTests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludedGroups>my.category.SlowTest</excludedGroups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>

You can use
mvn test -Dgroups="com.myapp.FastTests, com.myapp.SlowTests"
But ensure that you configure properly the maven surefire plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.12.2</version>
</dependency>
</dependencies>
</plugin>
See docs in: https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html

I lost lot of time on this error "groups/excludedGroups require TestNG or JUnit48+ on project test classpath" because I thought I was using a bad version of junit, or a bad version of the surefire plugin, or a combination that does not fit.
It was none of that: in my project I had a "config" module that was built before the module I wanted to test. This module had no junit dependency -> it had no junit on the classpath...
This mistake may help others...

Not exactly the same thing but using surefire plugin, test classes can be chosen based on file name. You are not using Junit Categories though.
An example for running just DAO tests.
<executions>
<execution>
<id>test-dao</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/com/proy/core/dao/**/*Test.java</include>
</includes>
</configuration>
</execution>
http://maven.apache.org/plugins/maven-surefire-plugin/examples/inclusion-exclusion.html

I was having issue running test with categories ex: #Category(com.mycompany.SlowTests.class) when running the test via mvn test -Dgroups=com.mycompany.SlowTests.class
I discovered that tests in a classes without the word Test in their name would not run. After adding the word Test to the class the the tests in the class ran.

For the use-case where you want to ignore certain tests by default and then conditionally run all tests from the command line, this setup will work on JUnit 4.9.
First create the marker interface:
public interface IntegrationTest {}
Annotate tests to be excluded:
#Category(IntegrationTest.class)
public class MyIntegrationTest() {}
Add plugins and profiles to pom.xml:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<!-- Read JUnit categories to be excluded from Maven property -->
<configuration>
<excludedGroups>${test.excluded.groups}</excludedGroups>
</configuration>
</plugin>
</plugins>
<profiles>
<profile>
<id>Default</id>
<!-- Set profile to be active by default -->
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<test.excluded.groups>com.example.IntegrationTest</test.excluded.groups>
</properties>
</profile>
<profile>
<id>AllTest</id>
<!-- Set groups property to blank value which will match nothing -->
<properties>
<test.excluded.groups></test.excluded.groups>
</properties>
</profile>
</profiles>
When running tests as usual from command line, integration test will be excluded:
mvn test
All tests including integration tests can be activated with the corresponding profile:
mvn test -P AllTest

Junit 5 allows you to use the #Tag annotation. More info about that here:
https://www.baeldung.com/junit-filtering-tests
I find it looks a little cleaner:
#Tag("SlowTests")
#Test
public void b() {
}

Is there a way to separate long running (e.g. stress tests) out so they're not run by default in Maven 2?

We've had an ongoing need here that I can't figure out how to address using the stock Maven 2 tools and documentation.
Some of our developers have some very long running JUnit tests (usually stress tests) that under no circumstances should be run as a regular part of the build process / nightly build.
Of course we can use the surefire plugin's exclusion mechanism and just punt them from the build, but ideally we'd love something that would allow the developer to run them at will through Maven 2.

Normally you would add a profile to your maven configuration that runs a different set of tests:
run this with mvn -Pintegrationtest install
<profile>
<id>integrationtest</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-client -Xmx896m -XX:MaxPermSize=192m</argLine>
<forkMode>once</forkMode>
<includes>
<include>**/**/*Test.java</include>
<include>**/**/*IntTest.java</include>
</includes>
<excludes>
<exclude>**/**/*SeleniumTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<activation>
<property>
<name>integrationtest</name>
</property>
</activation>
</profile>

Adding to krosenvold's answer, to ensure no unexpected behavior, make sure you also have a default profile that is active by default that excludes the integration or stresstests you want to run in your special profile.
<profile>
<id>normal</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/**/*IntTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
You will need to create a profile like this, simply listing the surefire-plugin outside of a profile will override the profile should it be selected with:
mvn -P integrationtest clean install

Use an integration test plugin such as the Super Helpful Integration Test Thingy to separate Integration Tests (long running, systemic) from Unit Test (purists say 30 seconds max for all true unit tests to run). Make two Java packages for your unit tests versus integration tests.
Then do not bind this plugin to a phase (the normal maven lifecycle) and only run it when it is explicitly called as a target, like so:
mvn shitty:clean shitty:install shitty:test
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>shitty-maven-plugin</artifactId>
</plugin>
</plugins>
This way, your normal developers will not be impacted, and you'll be able to run integration tests on demand.

Another option is to have the stress test detect it is running in maven and run only once or twice. i.e. turn into a regular functional test. This way you can check the code is still good, but not run for a long time.