I am attempting to configure my Maven project to have unit tests and integration tests. The unit tests are already working fine using the Maven Surefire plugin and are named according to the pattern *Test.java.
After adding the Failsafe plugin, like so:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.18.1</version>
</dependency>
</dependencies>
<configuration>
<includes>
<include>**/*IT.java</include>
</includes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
</plugin>
I added an integration test named SomeTestIT.java. However, when I run:
mvn failsafe:integration-test
I get the following:
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building MyApp 1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-failsafe-plugin:2.18.1:integration-test (default-cli) # MyApp ---
[INFO] Tests are skipped.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.368 s
[INFO] Finished at: 2015-03-04T14:43:50-06:00
[INFO] Final Memory: 11M/219M
[INFO] ------------------------------------------------------------------------
My test class (buried a few package levels deep beneath the test hierarchy) looks something like:
package com.example.my.package;
import org.junit.Test;
import org.junit.Assert;
import com.example.my.package.SomeService;
import org.junit.runner.RunWith;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.beans.factory.annotation.Autowired;
#RunWith(SpringJUnit4ClassRunner.class)
public class SomeTestIT
{
#Autowired
private SomeService target;
#Test
public void testTest()
{
Assert.assertTrue(false);
}
#Test
public void basicListTest()
{
Assert.assertNotNull(target);
}
}
Where the tests are stubs to make sure that I have the Maven integration working.
I have ensured that:
My POM includes a direct, test scoped dependency on JUnit.
I have tried the plugin configuration both with and without an explicit test runner dependency.
I have tried the configuration above both with and without an explicit include pattern (on the assumption that my class ought to have been picked up by the defaults).
Nonetheless, the tests never actually run. Is there anything else necessary to get the integration tests running?
I finally figured out what was going on and wanted to keep anyone else from spinning their wheels like I did. In the <properties> tag at the head of the POM, someone had added a property that read <skipITs>true</skipITs>. There were no integration tests previously, so the property was useless. It was probably cargo-culted out of some other POM without any real regard for what it is or what it does.
My problem was that maven failsafe plugin was already defined in some parent pom and was configured oddly.
In general, the best way I have found to handle this kind of issues is to first look at the effective pom:
mvn help:effective-pom
and check (1) all properties (2) the configuration of the problematic plugin
IntelliJ users should be sure to check their settings!
As shown in the screenshot below, there is an option in
Settings -> Build, Execution, Deployment -> Build Tools -> Maven -> Runner
If checked, Maven will skip integration tests in the failsafe plugin! It seems to have a similar effect as skipITs in one of the other answers. However, because IntelliJ passes it as an argument to its Maven command, it of course does not show up in the effective POM (also mentioned in a different answer).
Related
Small question regarding Maven and running unit test please.
I have a very basic Java SpringBoot project, with some unit tests.
I run the basic mvn clean install very straightforward.
For some reason I do not understand, I am seeing all of my unit tests run twice.
one time here:
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) # project ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running MyTest
and one time here:
[INFO] --- maven-surefire-plugin:2.22.2:test (integration-test) # project ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running MyTest
And here is my maven:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
<reportsDirectory>target/reports/junit</reportsDirectory>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>test</goal>
</goals>
<phase>integration-test</phase>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
</configuration>
</execution>
</executions>
</plugin>
May I ask what did I do wrong, as I do not want unit tests to run twice.
Thank you
The Surefire and Failsafe plugins are very similar, and the primary distinction is the phase that they run in. Generally, Surefire runs during the test phase and Failsafe runs during the integration-test phase. (The default configuration has them looking for slightly different test names, such as *Test vs. *IT, to know when to run each.)
You have explicitly told Surefire (the unit-test plugin) to run during integration testing. If you don't want this, simply remove the entire executions stanza. I also strongly recommend against telling Maven not to fail when the tests fail, and changing the report directory is simply likely to confuse tools like CI that are looking in the default location. Best? Take out the plugin definition entirely and use the defaults.
I wrote a custom maven plugin that scaffolds java-code from a custom schema.
The project-structure is like this:
Project
+ plugin
+ web-application
The reactor compiles first the plugin, then the application.
The usual mvn-command is:
mvn
... who is triggering the <defaultGoal>plugin:scaffold package</defaultGoal>
On fresh machines the build fails because the plugin is not yet known at the time the reactor plan the build-phases. So I have to call mvn install first. Then mvn plugin:scaffold package works like a charm.
The problem is: Whenever I modify the scaffolder-plugin and call mvn plugin:scaffold package the modifications of the scaffolder-plugin is not yet used because it is not yet installed into the repository. So I have to call mvn install first again.
Is there a way to:
Install the modification to the plugin
Build the webapplication using the modifications of the plugin
in one step?
First your plugin must be a module of the root project for the resolution to work correctly:
<modules>
<module>plugin</module>
<module>app</module>
</modules>
Then declare the plugin in the build/plugins section your application pom
<build>
<plugins>
<plugin>
<groupId>org.example.plugin</groupId>
<artifactId>plugin</artifactId>
<version>${project.parent.version}</version>
<executions>
<execution>
<id>sayhi</id>
<phase>generate-sources</phase>
<goals>
<goal>sayhi</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
The first time you run the plugin or when the plugin changes you need to run at least the package phase so the plugin jar is created. It must be run from the root project:
mvn package
The plugin will be executed during the generate-sources phase:
[INFO] --- plugin:1.0-SNAPSHOT:sayhi (sayhi) # app ---
[INFO] Hello, world.
[INFO]
When you change the plugin just run (again from root project):
mvn package
and you will see the changes:
[INFO] --- plugin:1.0-SNAPSHOT:sayhi (sayhi) # app ---
[INFO] Hello, worldxxxx.
[INFO]
See a full example on Github
I want to rerun a test that I know will fail cause I am trying to test the Surefire parameter for re-running the failing tests.
I tried running Maven with these two commands neither of them works as expected
-Dsurefire.rerunFailingTestsCount=2 -Dtest=TestThatFails test
and
-Dsurefire.rerunFailingTestsCount=2 -Dtest=TestThatFails surefire:test
Here is part of pom.xml
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-api</artifactId>
<version>2.19.1</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.53.1</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
I was expecting that Surefire would restart the test after failure but Maven just throws this error, which I know how to solve but I want the test to be rerun.
Results :
Tests in error:
testA(selenium.services.TestThatWillFail): Element is not currently visible and so may not be interacted with(..)
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 55.060 s
[INFO] Finished at: 2016-11-24T12:58:02+01:00
[INFO] Final Memory: 18M/173M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project eskn_selenium: There are test failures.
Just to add to Wim Rutgeerts's answer - rerunFailingTestsCount must be in the configuration section, not in properties, like this:
<configuration>
<rerunFailingTestsCount>2</rerunFailingTestsCount>
</configuration>
In my case with maven-surefire-plugin 2.19.1 it worked this way. When it was in properties it did not work.
Instead of using the command line property -Dsurefire.rerunFailingTestsCount=2, you can also define it in the pom in the properties section
<properties>
<surefire.rerunFailingTestsCount>2</surefire.rerunFailingTestsCount>
</properties>
Update for JUnit 5: Maven Surefire version 3.0.0-M4 or later now lets you use rerunFailingTestsCount system property when executing JUnit 5 tests.
Make sure to pass the below property when running your mvn clean stage:
-Dsurefire.rerunFailingTestsCount=3
Although that is missing from the documentation, the parameter rerunFailingTestsCount was introduced in version 2.18 of the Maven Surefire Plugin, as mentioned in SUREFIRE-1087. Since you're using the default version of 2.12.4 (that comes from the Super POM), that option is not available.
Therefore, the fix is simply to update the Surefire version to a version that is at least 2.18; for example, the latest, which is currently 2.19.1:
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
</plugin>
</plugins>
</pluginManagement>
Note that this parameter only works with JUnit 4+ (which is your case, since you have JUnit 4.12).
I am running a maven install for a spring boot project with multiple modules in STS.
I can see that there are few test cases failing but the maven install passes and shows build successful.
The tests are not being skipped, they are running during the install stage and the surefire reports are also being generated with failed tests but somehow the build passes.
I want the build to fail as the tests are failing, any pointers are appreciated?
<properties>
<maven-surefire-plugin-version>2.16</maven-surefire-plugin-version>
<junit.version>4.12</junit.version>
...
...
</properties>
..
..
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
...
...
</dependency>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
</plugins>
...
...
...
</pluginManagement>
</build>
Maven logs:
Results :
Failed tests:
MyClassTest.testMyClass:35 null
Tests in error:
MyClass1Test.testMyClass:64 ยป NullPointer
Tests run: 3, Failures: 1, Errors: 1, Skipped: 0
[ERROR] There are test failures.
Please refer to /Users/XXX/yyyy/zzzz/target/surefire-reports for the individual test results.
[INFO]
[INFO] --- maven-war-plugin:2.6:war (default-war) # zzzzz ---
[INFO] Packaging webapp
[INFO] Assembling webapp [zzzzz] in [/Users/XXXX/yyyyy/zzzz/target/zzzz-1.0.0]
[INFO] Processing war project
[INFO] Copying webapp resources [/Users/XXXX/zzzz/zzzz/src/main/webapp]
[INFO] Webapp assembled in [232 msecs]
[INFO] Building war: /Users/XXX/yyyyy/zzzzz/target/zzzzz-1.0.0.war
At last, it just shows all the modules build successfully. Shouldn't it stop building the war itself as soon as it detects a failure?
The test phase in the build lifecycle is executed by the Surefire plugin. One of its options, testFailureIgnore, is used to execute tests and sucessfully finish the build regardless of the result. This is useful eg when generating metrics to Sonar.
One way to activate such option is using -Dmaven.test.failure.ignore, perhaps you have enabled such option on your build.
According to documentation of surefire-maven-plugin, it does not fail on test failures by design. The reason is: surefire-maven-plugin should execute all tests and present the general statistics.
If you would like to save time, you can use this parameter:
<skipAfterFailureCount>N</skipAfterFailureCount>
where you stop executing tests after N of them failed.
maven-surefire-report-plugin can be used for reports analysis:
https://maven.apache.org/surefire/maven-surefire-report-plugin/
Any chance the failing tests are run using the failsafe plugin? This plugin does not fail the build on failing tests, as documented on their site:
If you use the Surefire Plugin for running tests, then when you have a test failure, the build will stop at the integration-test phase and your integration test environment will not have been torn down correctly.
The Failsafe Plugin is used during the integration-test and verify phases of the build lifecycle to execute the integration tests of an application. The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute.
I am literally trying to do exactly this:
http://maven.apache.org/plugins/maven-dependency-plugin/usage.html#The_dependency:build-classpath_mojo
What's amazing is that after finding an explicit example of exactly what I want Maven to do.. I still can't get it to work.
From the command line, I can run ...
mvn -Dmdep.outputFile=classpath.txt dependency:build-classpath
... which does indeed produce a file called classpath.txt with the information I'd like.
I would like to be able to issue a command like "mvn compile" and have the production of this classpath.txt file be a part of that process. The example provided at the link above associates it with generate-sources, which to my understanding should suffice.
When executing a command like "mvn compile" with this pom snippet below, nothing regarding the build-classpath goal seems to execute.
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.8</version>
<executions>
<execution>
<id>build-classpath</id>
<phase>generate-sources</phase>
<goals>
<goal>build-classpath</goal>
</goals>
<configuration>
<outputFile>myfile.txt</outputFile>
<mdep.outputFile>myFile1.txt</mdep.outputFile>
<ihavenoidea>whatgoeshere</ihavenoidea>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
And here is what I end up with:
$ mvn compile
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building someproj 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # someproj ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) # someproj ---
[INFO] Nothing to compile - all classes are up to date
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.600s
[INFO] Finished at: Fri Jan 31 14:05:29 CST 2014
[INFO] Final Memory: 9M/156M
[INFO] ------------------------------------------------------------------------
$ ls
bin html log pom.xml resources sql src target test-output wwwroot
Your plugin definition is inside <pluginManagement>, which means that when you will declare a "real" execution of that plugin inside a pom that has this pom as parent (or this pom itself), it will use that configuration.
This is generaly a good idea to use <pluginManagement> when a common configuration has to be applied on multiple execution, through multiple modules in the same global project.
Here, I would personally keep the compiler plugin inside <pluginManagement>, as you probably always want that plugin to be configured like this, but I woul move the dependency-plugin inside the <plugins> section (outside the <pluginManagement> section, well yes, this can be confusive...)
You may think of <pluginManagement> as a kind of template. It's often used in parent POMs to define a common configuration. Only plugins in <build><plugins> are included in the build.
That said, Maven does do some "magic" depending on the packaging type. I answered a similar question here.