My integration tests take long to run, and I don't want my developers to waste time every time they need to compile. I only want my integration tests to run:
In my CI/CD server.
When I tell my IDE (NetBeans) to specifically run a test file.
Is this possible? How can I achieve it?
Add a configuration to your POM that excludes the integration tests during your normal build (assuming that your integration tests are named something like UserService_IT.java.
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>${skip.surefire.tests}</skipTests>
<excludes>
<exclude>**/*_IT*.java</exclude>
</excludes>
</configuration>
To run the integration tests, say in your CI build, add a profile and activate it in your pipeline, like mvn verify -Pintegration-tests.
<profile>
<id>integration-tests</id>
<activation>
<activeByDefault>false</activeByDefault>
<property>
<name>integration-tests</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<includes>
<include>**/*_IT.java</include>
</includes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
Related
I have profile configuration in my POM with surefire-maven-plugin & junit connection to run only specific tests by profile. For example:
mvn clean test -Pgroup1 --also-make -DfailIfNoTests=false
It works as expected with following versions:
<maven-surefire-plugin.version>2.22.1</maven-surefire-plugin.version>
<junit.version>4.12</junit.version>
But stops working normally when I try to upgrade them:
<maven-surefire-plugin.version>3.0.0-M5</maven-surefire-plugin.version>
<junit.version>4.13</junit.version>
In this case mvn test always run all tests as I wouldn't set profile in command line.
My config of profiles is:
<profiles>
<profile>
<id>default</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/unit/*Test.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>group1</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/unit/**</exclude>
</excludes>
<groups>com.Group1</groups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>group2</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/unit/**</exclude>
</excludes>
<groups>com.Group2</groups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
......................
</profiles>
Every test class has connected interface linked to profile:
#Category(Group1.class)
#RunWith(JUnitParamsRunner.class)
public class Group1Test {
Playing with 'default' profile and 'activeByDefault' property also gave me no result. Any ideas how to fix it?
I got this to work by using "executions" in both the default plugin and in the profile plugin (which is not, by the way, an override of the default one)
<project>
...
<build>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M8</version>
<executions>
<execution>
<id>default-test</id>
<phase>test</phase>
<goals><goal>test</goal></goals> <!-- REQUIRED -->
<configuration>
<enableAssertions>true</enableAssertions>
<systemPropertyVariables>
<log4j.debug>true</log4j.debug>
<client.test.url>http://localhost:8080/axis/services/MyService</client.test.url>
</systemPropertyVariables>
<excludes>
<exclude>**/TestAccountOp</exclude> <!-- Tomcat required - use the profile -->
<exclude>**/TestWsdl</exclude>
</excludes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>myprofileid-testwithtomcat</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M8</version>
<executions>
<execution> <id>default-test</id> <phase>none</phase> </execution> <!-- Disable default Maven execution -->
<execution>
<id>myexecutionid-testwithtomcat</id>
<phase>test</phase>
<goals><goal>test</goal></goals> <!-- REQUIRED -->
<configuration>
<enableAssertions>true</enableAssertions>
<systemPropertyVariables>
<log4j.debug>true</log4j.debug>
<client.test.url>http://localhost:8080/axis/services/MyService</client.test.url>
</systemPropertyVariables>
<excludes>
<exclude>**/TestWsdl</exclude>
</excludes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
...
</profiles>
</project>
In the profile, I removed the "default-test", otherwise the "excludes" were set from the default one, but that might have been before I moved the "configurations" into the "executions". You have to remember that "default-test" is active unless you disable it, but not using an "execution" in the main body did not work for me.
"default-test" is Maven's "id" for the one in the main body, so that is why I used that name in the "execution" in the main body.
I think you can get away with not bothering with the "phase" elements, because that's the default for the "test" goal, but I'm pretty sure that you need the goal.
Good luck!
I want to run test classes whose name end with ResourceTest.java, so I defined following execution.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/*.java</exclude>
</excludes>
</configuration>
<version>2.12.2</version>
<executions>
<execution>
<id>resource-tests</id>
<phase>resource-tests</phase>
<goals>
<goal>resource-tests</goal>
</goals>
<configuration>
<includes>**/*ResourceTest.java</includes>
<!-- <exludes>**/*.java</exludes> -->
</configuration>
</execution>
</executions>
</plugin>
But I'm not sure how to run this, I've searched a lot and I'm missing something.
I tried surefire:test, it skipped all the test cases as defined in above configuration. So, I tried surefire:resource-tests, maven is saying no goal is not defined.
I'm using eclipse to run my maven build, by passing these parameters. How can I run by the execution id?
How to select a specific execution when running with surefire:test when I've mulltiple executions defined in my pom?
What am I missing? Any help would be appreciated.
There are several problems with your current configuration :
you are forcing the maven-surefire-plugin to be executed in the resource-tests phase but this phase does not exist. You should delete that declaration to keep the default plugin binding phase, which is test.
you are invoking the goal resource-tests but maven-surefire-plugin does not define such a goal.
the <includes> element is ill-defined. There should be a <include> tag under it.
you are excluding all Java files from the plugin configuration so no test will be run
the configuration of the plugin should be done under the <configuration> element not for each <executions>.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.2</version>
<configuration>
<includes>
<include>**/*ResourceTest.java</include>
</includes>
</configuration>
</plugin>
When you have multiple executions and you want "select" one of them, you can use a profile:
<profiles>
<profile>
<id>resource-tests</id>
<properties>
<test-classes>**/*ResourceTest.java</test-classes>
</properties>
</profile>
<profile>
<id>task-tests</id>
<properties>
<test-classes>**/*TaskTest.java</test-classes>
</properties>
</profile>
</profiles>
with the following plugin configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.2</version>
<configuration>
<includes>
<include>${test-classes}</include>
</includes>
</configuration>
</plugin>
With such a configuration:
when you run mvn clean test -Presource-tests, only the classes matching **/*ResourceTest.java will be tested
when you run mvn clean test -Ptask-tests, only the classes matching **/*TaskTest.java will be tested
I'm trying to separate different kinds of tests (unit, integration, acceptance) by using maven profiles. This is the part of the main pom file:
<properties>
<build.profile.id>dev</build.profile.id>
<skip.unit.tests>false</skip.unit.tests>
<skip.integration.tests>true</skip.integration.tests>
<skip.acceptance.tests>true</skip.acceptance.tests>
</properties>
<profiles>
<profile>
<id>dev</id>
</profile>
<profile>
<id>integration-test</id>
<properties>
<build.profile.id>integration-test</build.profile.id>
<skip.unit.tests>true</skip.unit.tests>
<skip.integration.tests>false</skip.integration.tests>
<skip.acceptance.tests>true</skip.acceptance.tests>
</properties>
</profile>
<profile>
<id>acceptance-test</id>
<properties>
<build.profile.id>acceptance-test</build.profile.id>
<skip.unit.tests>true</skip.unit.tests>
<skip.integration.tests>true</skip.integration.tests>
<skip.acceptance.tests>false</skip.acceptance.tests>
</properties>
</profile>
</profiles>
<build>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<skipTests>${skip.unit.tests}</skipTests>
<includes>
<include>**/*UnitTests.java</include>
</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.17</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<skipTests>${skip.integration.tests}</skipTests>
<includes>
<include>**/*IntegrationTests.java</include>
</includes>
</configuration>
</execution>
<execution>
<id>acceptance-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<skipTests>${skip.acceptance.tests}</skipTests>
<includes>
<include>**/*AcceptanceTests.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
As you can see I'm using the profile information to run certain types of tests based on the profile that is used. Default profile is dev and it will only run unit tests. They can be executed like this:
mvn clean test
For integration and acceptance tests I use failsafe plugin: Example of running integartion tests would be:
mvn clean verify -P integration-test
This works fine when I run it from the main pom module, but it doesn't work when running it from the child module. Tests are just ignored. Looking at the effective pom for child module I don't see the profiles. Am I doing something wrong or is this is expected behavior from maven? If profile inheritance (needs to cascade to deepest modules in the hierarchy) can't be achieved this way how can it be done?
Update: This is the project hierarchy
project directory
--main module
--commons module
--administration
----domain
----data
----business
----web
With multimodule project you usually don't execute modules directly. Instead you should always execute the main module, but specify only your desired submodule via -pl parameter. There are a lot more issues connected with running modules directly.
Just double checked some multi-module projects I have participated on and I we are using <pluginManagement> to propagate plugin configuration form parent POM to child projects.
I've setup Cargo to start an instance of glassfish during the pre-integration-test phase in a maven profile. My tests are then run in the integration-test phase and, finally, cargo shuts down the tomcat instance in the post-integration-test phases.
This works great when all tests pass, but if any test fails, the maven build fails, and it appears that the post-integration-test phase is never reached, which leaves the glassfish instance running (and me unable to stop it without killing the process).
Am I doing something wrong? Is there a way to make sure cargo shuts down my glassfish instance, even if the integration-test phase fails?
My maven profile:
<profile>
<!-- run integration tests against the app deployed to a container -->
<id>integration</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<!-- override the exclusion and include integration tests -->
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>***IntegrationTest.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<version>${cargo.plugin.version}</version>
<executions>
<execution>
<id>start-server</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop-server</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
<configuration>
<container>
<containerId>glassfish3x</containerId>
<artifactInstaller>
<groupId>org.glassfish.main.distributions</groupId>
<artifactId>glassfish</artifactId>
<version>${glassfish.version}</version>
</artifactInstaller>
</container>
<configuration>
<properties>
<cargo.datasource.datasource.mysql>
cargo.datasource.jndi=jdbc/TrackerPool|
cargo.datasource.driver=com.mysql.jdbc.Driver|
cargo.datasource.url=jdbc:mysql://localhost/[database]|
cargo.datasource.transactionsupport=LOCAL_TRANSACTION|
cargo.datasource.username=[username]|
cargo.datasource.password=[password]
</cargo.datasource.datasource.mysql>
</properties>
</configuration>
</configuration>
</plugin>
</plugins>
</build>
</profile>
The problem is simply based in the wrong usage of the maven-surefire-plugin which is intended for using in relation with unit test but not for integration tests. For such purposes the maven-failsave-plugin exist which will solve your problem.
The usage of the maven-failsave-plugin release you from defining of include rule for integration tests. The usual naming convention in Maven for integration tests is like this:
IT*.java
*IT.java
*ITCase.java
So i would suggest to name your integration tests accordingly so you don't need any kind of exlude/include rule neither for maven-surefire-plugin (unit tests) nor for maven-failsafe-plugin (integration tests).
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.13</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
The verify goal is only needed if you like to fail your build in case of failing integration tests. You have to call maven like this:
mvn -Pprofile clean verify
I am using TestNg and Maven with the surefire plugin to run my tests. I have several different tests that I want to run in the same pom. Currently to do this I have two different XML files defined and i've defined two profiles to run different xml files but this keeps throwing errors!
I don't know how I should solve this issue.
Any help will be greatly appreciated!
<profile>
<id>run</id>
<activation>
<property>
<name>start</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
<suiteXmlFiles>
<suiteXmlFile>${project.build.directory}/resources/testng.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
<executions>
<execution>
<id>test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
Imagine profile 2 to be pretty much the same BUT the xml suite file is different
I get a parsing error: It can't find the test class in the suiteXmlFile. However, if I just use one profile have just one xml file, it's acceptable. The one that was failing also works.
Try to move suiteXmlFiles to execution configuration and use maven-failsafe-plugin that is more suitable for integration tests