Using a Spring maven context, I would like to run specific tests based on a maven profile. I would like to have an easy way of tagging the test groups. If possible I would like to use the annotations.
Which options are there, like maven command line parameters, maven profiles specification, etc.
Say I have the following tests:
Example:
// annotation("integration")
public class GeopointFormatterTest {
#Test
public void testIntegration1() { ... }
#Test
public void testIntegration2() { ... }
Annotations like #Profile (which is for creating beans) and #ActiveProfile (which is for selecting specific profiles for creating beans) cannot be used for selecting tests, of course. All tests just run for statements like:
mvn clean install -Pdevelopment
mvn clean install -Pdevelopment -Dspring.profiles.active=acceptance
mvn clean install -Pdevelopment -Dspring.profiles.active=integration
As suggested, I used also #IfProfileValue. This is a good way for selecting tests based on system property values. System property values can be overruled by a CustomProfileValueSource class, like in: #ProfileValueSourceConfiguration(CustomProfileValueSource.class)
EDIT and ALTERNATIVE
The GREAT answers below focus on JUnit's #Category mechanism. Thanks to all!
A different approach is via these steps: [1] set a property within a maven profile and [2] use the property to skip tests via the of the standard surefire test plugin.
[1] Setting the properties via a profile:
<profiles>
<profile>
<id>integrationtests</id>
<properties>
<integration.skip>false</integration.skip>
<acceptance.skip>true</acceptance.skip>
</properties>
</profile>
... other profiles
[2] Using the properties in the surefire test plugin to skip tests.
<build>
<plugins>
<plugin>
<!-- Run the integration test-->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.plugin.version}</version>
<configuration>
<skipTests>${acceptance.skip}</skipTests>
Start in maven: mvn clean install –Pintegrationtests
Take a look at junit categories.
You'd tag your tests with specific category annotations
public interface FastTests { /* category marker */ }
public interface SlowTests { /* category marker */ }
#Category(SlowTests.class)
public class A {
#Test public void a() {}
}
then form a suite like
#RunWith(Categories.class)
#IncludeCategory({FastTests.class})
#SuiteClasses({A.class, B.class})
public static class FastTestSuite {
//
}
And then run it with
mvn -Dtest=FastTestSuite test
Note also that if you don't want to manually specify your unit test case classes in the suite class, you can also use the help of ClasspathSuite and then just limit based on categories.
You will probably need to categorize your tests using the #Category annotation. A complete example has been provided in the Surefire documentation provided here - search for the string Using JUnit Categories.
Assuming that you have categorized your tests accordingly, you will now be able to setup one or more profiles in your maven build which will trigger these tests as per the category
<profiles>
<profile>
<id>slow-tests</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<configuration>
<groups>com.mycompany.SlowTests</groups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>fast-tests</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<configuration>
<groups>com.mycompany.FastTests</groups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
You can specify one or more profiles on the command line when running the tests.
mvn test -Pslow-tests,fast-tests
You can specify the profile with this flag:
mvn test -Dspring.profiles.active=acceptance
In my latest project I have an "integration" profile that I use to run the integration tests against an embedded H2 database.
We did solved categorization of junit in the following steps.
I did create a project for you in github.
https://github.com/djaganathan/unit-test-samples
Caveat:- Junit Categorization packages still says as experimental.
1) Created a category of interfaces
/**
* This interface used to categories Junit Test
* those Tests will be executed during bamboo build run
*/
public interface ReleaseTest {
}
2) Tagged the unit testcases with the category you want
import org.junit.Test;
import org.junit.experimental.categories.Category;
import org.junit.internal.runners.JUnit4ClassRunner;
import org.junit.runner.RunWith;
import com.github.djaganathan.unit.test.category.ReleaseTest;
#RunWith(JUnit4ClassRunner.class)
public class GeneralTest {
#Test
#Category(value={ReleaseTest.class})
public void doTestForRelease(){}
#Test
public void doTestForDev(){}
}
3) Create profile in maven and attach to it
4) Run the command as mvn test -PreleaseTest
Based on the answers of a number of posts, I created a simple test project that demonstrates a number of code quality & testing features:
Performing either unit tests OR integration tests with plugins Surefire and Failsafe.
Improving code quality via the plugin findbugs.
Determine test coverage stats with plugin Jacoco.
Enjoy!
Related
I have been working with cucumber/Java and JUnit4 (CucumberOptions) for years without trouble running the tests in both IntelliJ and maven command line.
Recently, i have been trying to make the move to JUnit5 and i was able to have all tests running in IntelliJ (only, unfortunately)
My POC project has the following structure:
junit5
-Features (folder with feature files)
-resources (folder with files used in tests)
-src
--test
---java
----stepdefs
-----SetupEnvHook
-----StepDefs
----AllTest (testrunner wip)
----JU4Test (testrunner JUnit4)
----JU5Test (testrunner Junit5)
---resources (test resources)
-junit5.iml
-pom.xml
The JU5Test.java file :
import org.junit.platform.suite.api.ConfigurationParameter;
import org.junit.platform.suite.api.SelectDirectories;
import org.junit.platform.suite.api.Suite;
import stepdefs.SetupEnvHook;
import io.cucumber.java.Before;
import static io.cucumber.core.options.Constants.*;
#Suite
#SelectDirectories("Features")
//#ConfigurationParameter(key = PARALLEL_EXECUTION_ENABLED_PROPERTY_NAME, value = "true")
#ConfigurationParameter(key = PLUGIN_PUBLISH_ENABLED_PROPERTY_NAME, value = "false")
#ConfigurationParameter(key = PLUGIN_PUBLISH_QUIET_PROPERTY_NAME, value = "true")
#ConfigurationParameter(key = PLUGIN_PROPERTY_NAME, value = "json:target/cucumber-reports/cucumber.json")
#ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "stepdefs, my.external.steps.stepdefinition")
public class JU5Test {
#Before
public static void beforeSuite() {
SetupEnvHook.setEnvironment("QA");
}
}
The beforeSuite() method is also used in the JU4Test.
When i set a breakpoint in SetupEnvHook.setEnvironment("QA"); it is completely ignored due to the fact that the Before Annotation is not working, while another breakpoint inside the same
#io.cucumber.java.BeforeAll(order = 9999)
Annotation in SetupEnvHook class is triggered correctly.
My pom file is as follows :
<dependencies>
<dependency>
<groupId>my.external</groupId>
<artifactId>steps</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.9.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
</plugins>
</build>
Please ignore the my external dependancy. This dependancy is related to the stepdefinitions in the test runner file glue property.
I know that group and version values are also missing but these are all fed from the same dependancy in red so as to have more control on the versions everyone uses.
This is all done in Java 8 using
org.apache.maven.plugins:maven-compiler-plugin:3.10.1
org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7
io.cucumber:cucumber-java:7.8.1
io.cucumber:cucumber-junit:7.8.1
io.cucumber:cucumber-junit-platform-engine:7.8.1
org.junit.jupiter:junit-jupiter-api:5.9.1
org.junit.jupiter:junit-jupiter-engine:5.9.1
org.junit.platform:junit-platform-suite-api:1.9.1
org.junit.platform:junit-platform-suite-engine:1.9.1
I already tried using different Annotations not only from io.cucumber.java but also from org.junit (which is basically JUnit4) and org.junit.jupiter.api with no success obviously.
Running through maven command line ends up with :
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project junit5: No tests were executed!
It does not however state that 'no tests were found', had that issue initially and got it solved.
From looking at the error i suspect i may have something missing from the pom.xml surefire plugin but i cannot figure out what. (this pom is the same used to run the JU4Test without issues)
Anyone else have any thoughts on what i can try next? or better yet, the solution for this xD
Edit: remove images
It does not however state that 'no tests were found', had that issue initially and got it solved.
From looking at the error i suspect i may have something missing from the pom.xml surefire plugin but i cannot figure out what. (this pom is the same used to run the JU4Test without issues)
From your description it is impossible to say what is wrong with your project. Your list of depencies includes dependencies not included in your POM.
You may want to consider starting your project from scratch. You can use the https://github.com/cucumber/cucumber-java-skeleton for that.
When i set a breakpoint in SetupEnvHook.setEnvironment("QA"); it is completely ignored due to the fact that the Before Annotation is not working
The reason the #Before annotation is ignored is because you are using a Cucumber annotation on a class that is not part of the glue path.
Though I suspect you are trying to find a mapping for JUnit 4s #BeforeClass. Currently there is not such thing in JUnit 5s Suite Engine. If you need it, you should consider making a pull requests.
Alternatively you could create a package with a single class for each environment and use Cucumbers #BeforeAll hooks to set the environment. Then for each #Suite you configure a different glue path to include those hooks.
Though I think it would be even better to read the target environment from an environment variable and have it default to something sane. You can then use different CI jobs for each environment.
I have a testsuite say JunitTest1 and JunitTest2 as below,
#RunWith(Suite.class)
#SuiteClasses({
com.sample.test1.class,
com.sample.test2.class,
com.sample.test3.class,
})
public class JunitTest1 {
}
#RunWith(Suite.class)
#SuiteClasses({
com.sample.xxx1.class,
com.sample.xxx2.class,
com.sample.xxx3.class,
})
public class JunitTest2 {
}
I want to run both the testsuite in parallel but the test class inside the testsuite should run in the specified order. I have added the below plugin in maven pom.xml,
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20.1</version>
<configuration>
<parallel>suites</parallel>
<threadCount>2</threadCount>
<!-- <forkCount>1</forkCount>
<reuseForks>true</reuseForks> -->
</configuration>
</plugin>
I have followed this link - http://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html
I have tried all the possible combination for parallel tag like methods, classes, both, suites, suitesAndClasses, suitesAndMethods, classesAndMethods, all but nothing seems to solve my problem. The test suite are still running in sequential if i use suites and for other options the test classes in test suite are running in random order
Is there anything i am doing it wrong? can anyone please guide me to solve this issue ?
junit tests should be independent of one-another. junit does not provide a means to order tests to prevent you from writing tests which are dependent on other tests.
There are other SO threads on this topic:
How to run test methods in order with Junit
and
http://stackoverflow.com/questions/3693626/how-to-run-test-methods-in-specific-order-in-junit4
I'm running some service testing using restassured and cucumber and they work fine locally just using Maven test.
The issue is if I run Maven clean, then I must run Maven update or it will not work (Says it can't find my Cucumber feature files). For reference it says:
No features found at [classpath:classpath/classpath]
This wouldn't be a huge issue except I need to have this running through Bamboo where I can't call Maven update.
So I either need to figure out what is wrong with my POM to begin with to cause this issue, or how I can run Maven update through the goals/environment variables.
The POM is fairly simple, only having the needed dependencies/reporting stuff.
The build part of the POM is as follows:
<build>
<finalName>Test</finalName>
<directory>target</directory>
<outputDirectory>target/classes</outputDirectory>
<testOutputDirectory>target/test-classes</testOutputDirectory>
<sourceDirectory>src/main/java</sourceDirectory>
<testSourceDirectory>src/test/java</testSourceDirectory>
<resources>
<resource>
<directory>src/test/resources</directory>
</resource>
</resources>
</build>
This is all in Java 8 using Eclipse as the IDE.
I would avoid specifying anything in the build section in my pom and instead use the default values.
That is, I would keep my feature files in the same package as the runner or a sub package.
The runner could for example live in the package se/thinkcode/tage
As in the directory:
./test/java/se/thinkcode/tage
This means that the feature files should live in the directory:
./test/resources/se/thinkcode/tage
This would allow me to minimize the configuration in the runner. I typically use runners that looks like this:
package se.thinkcode.tage;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
#RunWith(Cucumber.class)
public class RunCukesTest {
}
This is the smallest configuration possible if you want to run Cucumber using JUnit from Maven.
It is even smaller that the example supplied by the Cucumber team: https://github.com/cucumber/cucumber-java-skeleton
Looks like defining the features/glue in the cucumber options fixed this.
I do believe there is a better option though.
I added the following cucumber options:
features ="src/test/java",
glue = "packagename",
How can I write equivalent maven plugin for the following gradle plugin defined?
/*
* Plugin to copy system properties from gradle JVM to testing JVM
* Code was copied from gradle discussion froum:
* http://forums.gradle.org/gradle/topics/passing_system_properties_to_test_task
*/
class SystemPropertiesMappingPlugin implements Plugin{
public void apply(Project project){
project.tasks.withType(Test){ testTask ->
testTask.ext.mappedSystemProperties = []
doFirst{
mappedSystemProperties.each{mappedPropertyKey ->
def systemPropertyValue = System.getProperty(mappedPropertyKey)
if(systemPropertyValue){
testTask.systemProperty(mappedPropertyKey, systemPropertyValue)
}
}
}
}
}
}
It really depends on what exactly you want to achieve.
In case you want to help with writing a maven plugin in general, you'll have to read the documentation.
In case you want to filter system properties that Maven JVM passes to your test JVM, I don't see any other option than extending the maven-surefire-plugin plugin and add there an option to do such mapping. (Note that by default Maven passes all its System Properties to the test JVM.) That is definitely doable but maybe you can achieve your goal with something maven already offers.
You can definitely pass additional system properties to your test JVM from Maven by using:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<configuration>
<systemPropertyVariables>
<propertyName>propertyValue</propertyName>
<anotherProperty>${myMavenProperty}</buildDirectory>
</systemPropertyVariables>
</configuration>
</plugin>
as documented http://maven.apache.org/surefire/maven-surefire-plugin/examples/system-properties.html.
In this case you can set the value of anotherProperty from command line by invoking maven
mvn test -DmyMavenProperty=theValueThatWillBePassedToTheTestJVMAsProperty_anotherProperty
You can also use Surefire argline to pass multiple properties to the JVM. For instance
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19</version>
<configuration>
<argLine>${propertiesIWantToSetFromMvnCommandLine}</argLine>
</configuration>
</plugin>
and execute maven as follows
mvn test -DpropertiesIWantToSetFromMvnCommandLine="-Dfoo=bar -Dhello=ahoy"
in this case, you'll see properties foo and hello with values bar and ahoy, respectively, in your test JVM.
I have TestNG tests that are grouped with the (groups = "unit") annotation as unit or integration and run using the following Maven config:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7.2</version>
<configuration>
<groups>${test.group}</groups>
<includes>
<include>**/*Test.java</include>
<include>**/Test*.java</include>
<include>**/When*.java</include>
</includes>
<argLine>${test.args.unit} -Xmx1024m</argLine>
</configuration>
</plugin>
I would like to separate test groups by directory structure instead of annotations and place the unit tests under a directory called unit.
So for example a unit tests in: src/test/java/foo/FooTests/MyFoo.java
I want to move it to: src/test/java/foo/unit/FooTests/MyFoo.java.
Then change the Maven config to pick up Java files under a unit subdir for unit tests.
I tried doing this by changing the plugin config to use the following include:
<groups>${test.group}</groups>
<includes>
<include>**/unit/**/*Test.java</include>
<include>**/unit/**/Test*.java</include>
<include>**/unit/**/When*.java</include>
</includes>
But that doesn't seem to work. Any ideas?
The first thing is that you should use a more up-to-date maven-surefire-plugin (2.12.4) and furthermore dont use includes, cause maven-surefire-plugin has defaults for Unit tests. If you have integration tests better use maven-failsafe-plugin and name them appropriately. If you use TestNG in combination it sounds more like integration tests.
If you name your Unit testes like '*Test' and your Integration Tests as '*IT' then the plugins will pick the right tests.