surefire-junit47 from parent POM prevents JUnit 5 test detection

surefire-junit47 from parent POM prevents JUnit 5 test detection - java

Due to company policy, my project must inherit from a parent POM, which I cannot change. The parent POM defines the following surefire plugin in the pluginManagement section:
<build>
<pluginManagment>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M4</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>3.0.0-M4</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</dependencyManagement>
</build>
This forces the surefire Provider to JUnit Core (4.7+). However, my project uses JUnit 5.7.0 exclusively. I can run unit tests from the IDE, no problem. But when I run mvn clean test, none of the unit tests are detected:
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
To debug, I removed the reference to the parent POM and copied the contents of the parent POM to my project's POM and got the same results. When I deleted the surefire plugin definition above from pluginManagement, everything worked as expected. All tests detected and run.
Back to the original setup with parent POM, in my pluginManagement section, I tried overriding maven-surefire-plugin without the surefire-junit47 dependency. I added a junit-platform-surefire-provider dependency, but get a ForkedProcessEvent error, since both JUnit 4 and 5 providers conflict. It seems like my dependency is just added to the one from parent POM.
How can I exclude or override the parent POM's maven-surefire-plugin dependency from the child POM?
maven version: 3.6.3
maven compiler version: 3.8.1

Related

How to run Maven Failsafe integration tests from a jar file?

I have a Spring Boot multi module Maven project, I can run integration tests with:
mvn clean verify
and it works well. I now want to run the same integration tests from a container and I don't want to embed all the source code in the container.
My question is : how can I run the Maven Failsafe Plugin without using the source code?
I tried to run the failsafe:integration-test goal and setting the dependenciesToScan parameter, from the command line:
mvn failsafe:integration-test -DdependenciesToScan=com.myorg:proj-tests.jar
but no tests are found.
P.S.1: I've seen this similar question Running spring tests from executable jar. But I don't need to run the tests without Maven. I prefer to run tests from the command line with Maven than adding code or modifying the structure of my project.
P.S.2: I'm using maven-failsafe-plugin 2.22.2 which is the version provided with Spring Boot 2.1.8.

From the docs:
Since version 2.22.0 you can scan for test classes from a project
dependency of your multi-module project.
It means the tests (proj-tests.jar) must be a dependency of the project. As you cannot have a dependency to the tests jar in the same project where you build them, the solution is to have another module or pom file. Example:
<groupId>failsafe.use.jar</groupId>
<artifactId>failsafe-use-jar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<dependencies>
...
<dependency>
<groupId>com.myorg</groupId>
<artifactId>proj-tests</artifactId>
<version>0.0.1-SNAPSHOT</version>
<classifier>tests</classifier>
</dependency>
...
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
</plugin>
</plugins>
</build>
The proj-tests is a project dependency and can be created with:
<groupId>com.myorg</groupId>
<artifactId>proj-tests</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.1.2</version>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
See Guide to using attached tests
To run the integration tests from the container you obviously need all the dependencies installed in the local
(container) maven repository or deployed in remote. Then you can run with:
mvn failsafe:integration-test -DdependenciesToScan=com.myorg:proj-tests
Note that the format of the dependenciesToScan property is groupId:artifactId (you run with the name of jar instead of artifactid)
As another note, for integration tests failsafe searches by default for class files ending in IT (integration test).

Generate test-jar of unit testing project and use in other maven projects

I created a unit-test maven project as a base project which other project can extend and use Described here. Here is the pom.xml -
<dependencies>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-mockito-release-full</artifactId>
<version>1.6.4</version>
<type>pom</type>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
This generates two jars in target -
unit-test-0.0.1-SNAPSHOT.jar
unit-test-0.0.1-SNAPSHOT-tests.jar
Now, I have some spring boot microservice projects say service-a and service-b which are using another maven project say super-service as dependency. service-a and service-b are using super-service as following declaration in respective services pom.xml -
<dependency>
<groupId>com.super.service</groupId>
<artifactId>super-service</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
I have written unit test for classes in super-service by using above unit-test maven project which is working fine. The pom.xml of super-service is -
<dependency>
<groupId>com.unit-test</groupId>
<artifactId>unit-test</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
But service-a and service-b is not working in same manner. I thought above dependency should get resolved through the base i.e. super-service but it's not and test gets fail. Then I have tried to repeat the same dependency declaration in each services pom.xml but still maven test gets fail.
I tried the other way described in that URL which tells to move source files from src/test/java to src/main/java but that also worked only for super-service project and not for service-a and service-b.

You're doing the wrong thing. You should not run the unit-tests of your dependencies -- if you don't think it can pass its tests you shouldn't be using it.
Only use a dependency when you have evidence that it passed its tests. This is controlled by your CI process which should only put successfully tested binaries in your binary repository. For most people this means that Jenkins should run your tests and if they all pass only then should it put the binary in nexus for you to depend on it. As it does the release to nexus the CI process, through maven, should also update the version number from a snapshot to a release version so that you're never depending on snapshots.

Surefire rerun failing tests not working

I want to rerun a test that I know will fail cause I am trying to test the Surefire parameter for re-running the failing tests.
I tried running Maven with these two commands neither of them works as expected
-Dsurefire.rerunFailingTestsCount=2 -Dtest=TestThatFails test
and
-Dsurefire.rerunFailingTestsCount=2 -Dtest=TestThatFails surefire:test
Here is part of pom.xml
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-api</artifactId>
<version>2.19.1</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.53.1</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
I was expecting that Surefire would restart the test after failure but Maven just throws this error, which I know how to solve but I want the test to be rerun.
Results :
Tests in error:
testA(selenium.services.TestThatWillFail): Element is not currently visible and so may not be interacted with(..)
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 55.060 s
[INFO] Finished at: 2016-11-24T12:58:02+01:00
[INFO] Final Memory: 18M/173M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project eskn_selenium: There are test failures.

Just to add to Wim Rutgeerts's answer - rerunFailingTestsCount must be in the configuration section, not in properties, like this:
<configuration>
<rerunFailingTestsCount>2</rerunFailingTestsCount>
</configuration>
In my case with maven-surefire-plugin 2.19.1 it worked this way. When it was in properties it did not work.

Instead of using the command line property -Dsurefire.rerunFailingTestsCount=2, you can also define it in the pom in the properties section
<properties>
<surefire.rerunFailingTestsCount>2</surefire.rerunFailingTestsCount>
</properties>

Update for JUnit 5: Maven Surefire version 3.0.0-M4 or later now lets you use rerunFailingTestsCount system property when executing JUnit 5 tests.
Make sure to pass the below property when running your mvn clean stage:
-Dsurefire.rerunFailingTestsCount=3

Although that is missing from the documentation, the parameter rerunFailingTestsCount was introduced in version 2.18 of the Maven Surefire Plugin, as mentioned in SUREFIRE-1087. Since you're using the default version of 2.12.4 (that comes from the Super POM), that option is not available.
Therefore, the fix is simply to update the Surefire version to a version that is at least 2.18; for example, the latest, which is currently 2.19.1:
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
</plugin>
</plugins>
</pluginManagement>
Note that this parameter only works with JUnit 4+ (which is your case, since you have JUnit 4.12).

Maven does not find unit tests to run, no Junit tests are executed Java [duplicate]

I have a maven program, it compiles fine. When I run mvn test it does not run any tests (under TESTs header says There are no tests to run.).
I've recreated this problem with a super simple setup which I will include below as well as the output when run with -X.
The unit tests run fine from eclipse (both with its default junit package and when I instead include the junit.jar downloaded by maven). Also mvn test-compile correctly creates the class under test-classes. I am running this on OSX 10.6.7 with Maven 3.0.2 and java 1.6.0_24.
Here is the directory structure:
/my_program/pom.xml
/my_program/src/main/java/ClassUnderTest.java
/my_program/src/test/java/ClassUnderTestTests.java
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>my_group</groupId>
<artifactId>my_program</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>My Program</name>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
ClassUnderTest.java:
public class ClassUnderTest {
public int functionUnderTest(int n) {
return n;
}
}
ClassUnderTestTests.java:
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
public class ClassUnderTestTests {
private ClassUnderTest o;
#Before
public void setUp() {
o = new ClassUnderTest();
}
#Test
public void testFunctionUnderTest_testCase1() {
Assert.assertEquals(1, o.functionUnderTest(1));
}
#Test
public void testFunctionUnderTest_testCase2() {
Assert.assertEquals(2, o.functionUnderTest(2));
}
}
End of mvn -X test:
[DEBUG] Configuring mojo org.apache.maven.plugins:maven-surefire-plugin:2.7.1:test from plugin realm ClassRealm[plugin>org.apache.maven.plugins:maven-surefire-plugin:2.7.1, parent: sun.misc.Launcher$AppClassLoader#5224ee]
[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-surefire-plugin:2.7.1:test' with basic configurator -->
[DEBUG] (s) basedir = /Users/aaron/Programs/my_program
[DEBUG] (s) childDelegation = false
[DEBUG] (s) classesDirectory = /Users/aaron/Programs/my_program/target/classes
[DEBUG] (s) disableXmlReport = false
[DEBUG] (s) enableAssertions = true
[DEBUG] (s) forkMode = once
[DEBUG] (s) junitArtifactName = junit:junit
[DEBUG] (s) localRepository = id: local
url: file:///Users/aaron/.m2/repository/
layout: none
[DEBUG] (f) parallelMavenExecution = false
[DEBUG] (s) pluginArtifactMap = {org.apache.maven.plugins:maven-surefire-plugin=org.apache.maven.plugins:maven-surefire-plugin:maven-plugin:2.7.1:, org.apache.maven.surefire:surefire-booter=org.apache.maven.surefire:surefire-booter:jar:2.7.1:compile, org.apache.maven.surefire:surefire-api=org.apache.maven.surefire:surefire-api:jar:2.7.1:compile, org.apache.maven.surefire:maven-surefire-common=org.apache.maven.surefire:maven-surefire-common:jar:2.7.1:compile, org.apache.maven.shared:maven-common-artifact-filters=org.apache.maven.shared:maven-common-artifact-filters:jar:1.3:compile, org.codehaus.plexus:plexus-utils=org.codehaus.plexus:plexus-utils:jar:2.0.5:compile, junit:junit=junit:junit:jar:3.8.1:compile, org.apache.maven.reporting:maven-reporting-api=org.apache.maven.reporting:maven-reporting-api:jar:2.0.9:compile}
[DEBUG] (s) printSummary = true
[DEBUG] (s) project = MavenProject: my_group:my_program:1.0-SNAPSHOT # /Users/aaron/Programs/my_program/pom.xml
[DEBUG] (s) projectArtifactMap = {junit:junit=junit:junit:jar:4.8.1:test}
[DEBUG] (s) redirectTestOutputToFile = false
[DEBUG] (s) remoteRepositories = [ id: central
url: http://repo1.maven.org/maven2
layout: default
snapshots: [enabled => false, update => daily]
releases: [enabled => true, update => never]
]
[DEBUG] (s) reportFormat = brief
[DEBUG] (s) reportsDirectory = /Users/aaron/Programs/my_program/target/surefire-reports
[DEBUG] (s) session = org.apache.maven.execution.MavenSession#dfbb43
[DEBUG] (s) skip = false
[DEBUG] (s) skipTests = false
[DEBUG] (s) testClassesDirectory = /Users/aaron/Programs/my_program/target/test-classes
[DEBUG] (s) testFailureIgnore = false
[DEBUG] (s) testNGArtifactName = org.testng:testng
[DEBUG] (s) testSourceDirectory = /Users/aaron/Programs/my_program/src/test/java
[DEBUG] (s) trimStackTrace = true
[DEBUG] (s) useFile = true
[DEBUG] (s) useManifestOnlyJar = true
[DEBUG] (s) workingDirectory = /Users/aaron/Programs/my_program
[DEBUG] -- end configuration --
[INFO] Surefire report directory: /Users/aaron/Programs/my_program/target/surefire-reports
[DEBUG] Setting system property [user.dir]=[/Users/aaron/Programs/my_program]
[DEBUG] Setting system property [localRepository]=[/Users/aaron/.m2/repository]
[DEBUG] Setting system property [basedir]=[/Users/aaron/Programs/my_program]
[DEBUG] Using JVM: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java
[DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10 for /Users/aaron/.m2/repository
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG] org.apache.maven.surefire:surefire-booter:jar:2.7.1:compile (selected for compile)
[DEBUG] org.apache.maven.surefire:surefire-api:jar:2.7.1:compile (selected for compile)
[DEBUG] Adding to surefire booter test classpath: /Users/aaron/.m2/repository/org/apache/maven/surefire/surefire-booter/2.7.1/surefire-booter-2.7.1.jar Scope: compile
[DEBUG] Adding to surefire booter test classpath: /Users/aaron/.m2/repository/org/apache/maven/surefire/surefire-api/2.7.1/surefire-api-2.7.1.jar Scope: compile
[DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10 for /Users/aaron/.m2/repository
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG] org.apache.maven.surefire:surefire-junit4:jar:2.7.1:test (selected for test)
[DEBUG] org.apache.maven.surefire:surefire-api:jar:2.7.1:test (selected for test)
[DEBUG] Adding to surefire test classpath: /Users/aaron/.m2/repository/org/apache/maven/surefire/surefire-junit4/2.7.1/surefire-junit4-2.7.1.jar Scope: test
[DEBUG] Adding to surefire test classpath: /Users/aaron/.m2/repository/org/apache/maven/surefire/surefire-api/2.7.1/surefire-api-2.7.1.jar Scope: test
[DEBUG] Test Classpath :
[DEBUG] /Users/aaron/Programs/my_program/target/test-classes
[DEBUG] /Users/aaron/Programs/my_program/target/classes
[DEBUG] /Users/aaron/.m2/repository/junit/junit/4.8.1/junit-4.8.1.jar
[DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10 for /Users/aaron/.m2/repository
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG] org.apache.maven.surefire:surefire-booter:jar:2.7.1:compile (selected for compile)
[DEBUG] org.apache.maven.surefire:surefire-api:jar:2.7.1:compile (selected for compile)
[DEBUG] Adding to surefire booter test classpath: /Users/aaron/.m2/repository/org/apache/maven/surefire/surefire-booter/2.7.1/surefire-booter-2.7.1.jar Scope: compile
[DEBUG] Adding to surefire booter test classpath: /Users/aaron/.m2/repository/org/apache/maven/surefire/surefire-api/2.7.1/surefire-api-2.7.1.jar Scope: compile
Forking command line: /bin/sh -c cd /Users/aaron/Programs/my_program && /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java -jar /Users/aaron/Programs/my_program/target/surefire/surefirebooter6118081963679415631.jar /Users/aaron/Programs/my_program/target/surefire/surefire4887918564882595612tmp /Users/aaron/Programs/my_program/target/surefire/surefire9012255138269731406tmp
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.089s
[INFO] Finished at: Mon May 30 12:03:09 EDT 2011
[INFO] Final Memory: 7M/62M
[INFO] ------------------------------------------------------------------------

By default Maven uses the following naming conventions when looking for tests to run:
Test*
*Test
*Tests (has been added in Maven Surefire Plugin 2.20)
*TestCase
If your test class doesn't follow these conventions you should rename it or configure Maven Surefire Plugin to use another pattern for test classes.

I also found that the unit test code should put under the src/test/java folder, it can not be recognized as test class if you put it under the main folder.
eg.
Wrong
/my_program/src/main/java/NotTest.java
Right
/my_program/src/test/java/MyTest.java

UPDATE:
Like #scottyseus said in the comments, starting from Maven Surefire 2.22.0 the following is sufficient:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
</plugin>
When using JUnit 5, i ran into the same problem. Maven Surefire needs a plugin to run JUnit 5 tests. Add this to our pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>1.2.0-M1</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.2.0-M1</version>
</dependency>
</dependencies>
</plugin>
Source:
https://junit.org/junit5/docs/current/user-guide/#running-tests-build-maven
UPDATE 2021
The junit-platform-surefire-provider, which was originally developed by the JUnit team, was deprecated in JUnit Platform 1.3 and discontinued in 1.4. Please use Maven Surefire’s native support instead.

Another thing that can cause Maven to not find the tests if if the module's packaging is not declared correctly.
In a recent case, someone had <packaging>pom</packaging> and my tests never ran. I changed it to <packaging>jar</packaging> and now it works fine.

In my case it was adding the junit-vintage-engine which makes it compatible with older version of JUnit tests and can run them. As I'm using JUnit 5.
<dependency>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
<scope>test</scope>
</dependency>

If you created a Spring Boot application using Spring Initializr, tests are running all right from Intellij Idea.
But, if try to run tests from a command-line:
mvn clean test
You might have been surprised, that no tests were run at all. I tried to add surefire plugin with no luck.
The answer was simple: pom.xml contained the following dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
The exclusion, junit-vintage-engine, is dedicated for keeping backward compatibility with JUnit 4.x. So, new versions of Spring Boot Initializr do not support it by default.
After I removed the exclusion, Maven started to see project's tests.

Also, check if your test classes directory (e.g. src/test/java) corresponds to directory listed in property <testSourceDirectory> in your pom.xml under <build> property. Took me a while to find that.

Many of these answers were quite useful to me in the past, but I would like to add an additional scenario that has cost me some time, as it may help others in the future:
Make sure that the test classes and methods are public.
My problem was that I was using an automatic test class/methods generation feature of my IDE (IntelliJ) and for some reason it created them as package-private. I find this to be easier to miss than one would expect.

I struggle with this problem. In my case I wasn't importing the right #Test annotation.
1) Check if the #Test is from org.junit.jupiter.api.Test (if you are using Junit 5).
2) With Junit5 instead of #RunWith(SpringRunner.class), use #ExtendWith(SpringExtension.class)
import org.junit.jupiter.api.Test;
#ExtendWith(SpringExtension.class)
#SpringBootTest
#AutoConfigureMockMvc
#TestPropertySource(locations = "classpath:application.properties")
public class CotacaoTest {
#Test
public void testXXX() {
}
}

Maven will not run your tests if the project has <packaging>pom</packaging>
You need to set the packaging to jar (or some other java artefact type) for the tests to run: <packaging>jar</packaging>

Check that (for jUnit - 4.12 and Eclipse surefire plugin)
Add required jUnit version in POM.xml in dependencies. Do Maven -> Update project to see required jars exported in project.
Test class is under the folder src/test/java and subdirectories of this folder (or base folder can be specified in POM in config testSourceDirectory). Name of the class should have tailng word 'Test'.
Test Method in the test class should have annotation #Test

Discovered if you prefix a test with 'Abstract' it will be ignored by default aswell.

Following worked just fine for me in Junit 5
https://junit.org/junit5/docs/current/user-guide/#running-tests-build-maven
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.0</version>
</plugin>
</plugins>
</build>
<!-- ... -->
<dependencies>
<!-- ... -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.4.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.4.0</version>
<scope>test</scope>
</dependency>
<!-- ... -->
</dependencies>
<!-- ... -->

I also had similar issue, after exploring found that testng dependency is causing this issue. After removing the testng dependency from pom (as I dont need it anymore), it started to work fine for me.
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.8</version>
<scope>test</scope>
</dependency>

In my case we are migration multimodule application to Spring Boot. Unfortunately maven didnt execute all tests anymore in the modules. The naming of the Test Classes didnt change, we are following the naming conventions.
At the end it helped, when I added the dependency surefire-junit47 to the plugin maven-surefire-plugin. But I could not explain, why, it was trial and error:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>${maven-surefire-plugin.version}</version>
</dependency>
</dependencies>

The Maven Surefire plugin supports several test frameworks. It tries to autodetect which framework you are using, then looks for tests written using that framework. If that autodetection is confused, and chooses the wrong framework, the second stage will not find your tests.
The autodetection works by scanning the classpath for the presence of significant "driver" classes for the test frameworks it supports. Therefore the autodetection can go wrong if your POM, or a depended on module, has an incorrect dependency on one of those "driver" classes.
At present (2020), a particular problem is the difference between JUnit 4 and JUnit 5. The Surefire plugin treats them as different frameworks. But because of the similarity in the package names, a project can have a dependency on the wrong framework but seem OK to a casual inspection.
In particular, beware that junit-platform-console is for JUnit 5, but junit-platform-runner is for JUnit 4. If your project has a dependency on the latter, Surefire will not run your JUnit 5 tests.

Here's the exact code I had to add to my pom.xml:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>1.2.0-M1</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.2.0</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
And here's my dependencies:
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.2.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.easytesting</groupId>
<artifactId>fest-assert-core</artifactId>
<version>2.0M10</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>1.2.0-M1</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.2.0-M1</version>
</dependency>
</dependencies>

If your test class name does not follow the standard naming convention (as highlighted by #axtavt above), you need to add the pattern/class name in the pom.xml in order to Maven pick the test -
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/*_UT.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
...

If you have a shared Java / Groovy application and all you have are Groovy unit tests, then Maven won't find any tests. This can be fixed by adding one unit test under src/test/java.

/my_program/src/test/java/ClassUnderTestTests.java
should be
/my_program/src/test/java/ClassUnderTestTest.java
The Maven finds those ends Test or starts with Test to run automatically.
However, you can using
mvn surefire:test -Dtest=ClassUnderTestTests.java
to run your tests.

I faced the same issue , it resolved by below change in pom.xml :
<build>
<testSourceDirectory>test</testSourceDirectory>
...
changed to:
<build>
<testSourceDirectory>${project.basedir}/src/test/java</testSourceDirectory>

One more tip (in addition to the previous answers):
In Eclipse, go to your project's Properties > click Run/Debug Settings:
"This page allows you to manage launch configurations with the
currently selected resource"
In there you can add (New...) or remove (Delete) any JU (JUnit) tests you have in your project (under the src/test/java folder, or course).

In my case, my parent pom had a parent:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>some version</version>
<relativePath/>
</parent>
After changing to importing a spring pom:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>some version</version>
<type>pom</type>
<scope>import</scope>
</dependency>
My unit tests started to run

Probably not a common mistake but in addition to #João Matos's answer. Besides the requirement of your methods being public:
You're methods also should be void and of course have the #Test annotation.
This won't work:
#Test
public Integer fooBarTest() {
// omitted
}
It must return a void:
#Test
public void fooBarTest() {
// omitted
}

Another reason for not running the test cases happened to me - I had a property named "test" for completely different purposes, but it interfered with the surefire plugin. Thus, please check your POMs for:
<properties>
<test>.... </test>
...
</properties>
and remove it.

If you have written your tests in JUnit 4 and added JUnit 5 dependencies to the surefire plugin, your tests will not run.
In that case, just comment JUnit 5 dependencies from surefire plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<!--<dependencies>-->
<!--<dependency>-->
<!--<groupId>org.junit.platform</groupId>-->
<!--<artifactId>junit-platform-surefire-provider</artifactId>-->
<!--<version>1.0.0</version>-->
<!--</dependency>-->
<!--<dependency>-->
<!--<groupId>org.junit.jupiter</groupId>-->
<!--<artifactId>junit-jupiter-engine</artifactId>-->
<!--<version>${junit.version}</version>-->
<!--</dependency>-->
<!--</dependencies>-->
</plugin>

Such problem might occur when you use surfire plugin 3.x.x+ with JUnit5 and by mistake annotate the test class with #Test annotation from JUnit4.
Use: org.junit.jupiter.api.Test (JUnit5) instead of org.junit.Test (Junit4)
NOTE: this might be hard to notice as the IDE might run this wihout problems just as JUnit4 test.

Another easily overlooked problem I recently experienced - Ensure your Test class' file has the .java extension. If there's no tests to compile, there are no tests to run

junitArtifactName might also be the case if the JUnit in use isn't the standard (junit:junit) but for instance...
<dependency>
<groupId>org.eclipse.orbit</groupId>
<artifactId>org.junit</artifactId>
<version>4.11.0</version>
<type>bundle</type>
<scope>test</scope>
</dependency>

In case someone has searched and I do not solve it, I had a library for different tests:
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>${org.junit.jupiter.version}</version>
<scope>test</scope>
</dependency>
When I installed junit everything worked, I hope and help this:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>

Maven install passes but there are tests failing

I am running a maven install for a spring boot project with multiple modules in STS.
I can see that there are few test cases failing but the maven install passes and shows build successful.
The tests are not being skipped, they are running during the install stage and the surefire reports are also being generated with failed tests but somehow the build passes.
I want the build to fail as the tests are failing, any pointers are appreciated?
<properties>
<maven-surefire-plugin-version>2.16</maven-surefire-plugin-version>
<junit.version>4.12</junit.version>
...
...
</properties>
..
..
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
...
...
</dependency>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
</plugins>
...
...
...
</pluginManagement>
</build>
Maven logs:
Results :
Failed tests:
MyClassTest.testMyClass:35 null
Tests in error:
MyClass1Test.testMyClass:64 » NullPointer
Tests run: 3, Failures: 1, Errors: 1, Skipped: 0
[ERROR] There are test failures.
Please refer to /Users/XXX/yyyy/zzzz/target/surefire-reports for the individual test results.
[INFO]
[INFO] --- maven-war-plugin:2.6:war (default-war) # zzzzz ---
[INFO] Packaging webapp
[INFO] Assembling webapp [zzzzz] in [/Users/XXXX/yyyyy/zzzz/target/zzzz-1.0.0]
[INFO] Processing war project
[INFO] Copying webapp resources [/Users/XXXX/zzzz/zzzz/src/main/webapp]
[INFO] Webapp assembled in [232 msecs]
[INFO] Building war: /Users/XXX/yyyyy/zzzzz/target/zzzzz-1.0.0.war
At last, it just shows all the modules build successfully. Shouldn't it stop building the war itself as soon as it detects a failure?

The test phase in the build lifecycle is executed by the Surefire plugin. One of its options, testFailureIgnore, is used to execute tests and sucessfully finish the build regardless of the result. This is useful eg when generating metrics to Sonar.
One way to activate such option is using -Dmaven.test.failure.ignore, perhaps you have enabled such option on your build.

According to documentation of surefire-maven-plugin, it does not fail on test failures by design. The reason is: surefire-maven-plugin should execute all tests and present the general statistics.
If you would like to save time, you can use this parameter:
<skipAfterFailureCount>N</skipAfterFailureCount>
where you stop executing tests after N of them failed.
maven-surefire-report-plugin can be used for reports analysis:
https://maven.apache.org/surefire/maven-surefire-report-plugin/

Any chance the failing tests are run using the failsafe plugin? This plugin does not fail the build on failing tests, as documented on their site:
If you use the Surefire Plugin for running tests, then when you have a test failure, the build will stop at the integration-test phase and your integration test environment will not have been torn down correctly.
The Failsafe Plugin is used during the integration-test and verify phases of the build lifecycle to execute the integration tests of an application. The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute.

Develop Reference

Java is a programming language and computing platform first released by Sun Microsystems in 1995.

surefire-junit47 from parent POM prevents JUnit 5 test detection - java

Related

How to run Maven Failsafe integration tests from a jar file?

Generate test-jar of unit testing project and use in other maven projects

Surefire rerun failing tests not working

Maven does not find unit tests to run, no Junit tests are executed Java [duplicate]

Maven install passes but there are tests failing

Categories

Resources