I'm trying to execute some test cases in an very big applications. There are 100s of error test cases and some of them
requires specific property files which won't be available in development environment.
are outdated, but can't be simply removed
Is there anyway in which I can provide an XML or any other input which can list all the test classes to ignore?
I'm using maven with surefire plugin for executing test cases.
PS
I'm aware of #ignore annotation which can be used to ignore test cases or test classes. I don't want to use this because it requires changing each class which I want to ignore.
What I want is a single configuration file where I can mention all the classes to ignore.
There are 2 options:
1.To skip running the tests for a particular project, you can set skipTests to true.
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
2) Also you can skip the tests via command line by executing the following command:
mvn install -DskipTests
You can potentially create a TestSuite that includes all the tests you do want to run, and then specify to Maven that it should use that particular Test Suite with:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<includes>
<include>TestSuite.java</include>
</includes>
</configuration>
</plugin>
with TestSuite.java being the name of the suite you wish to run.
You could potentially have two and just switch that setting in your pom.xmlwhen you wish to target only certain classes.
This setting affects what tests are executed when you run mvn test.
Related
Something like the following.
I would like a way to skip my dao tests in surefire. Trying to avoid overhead of defining Suites.
With CI I'd like to have one nightly that runs all tests and another 5 minute poll of SCM that runs only 'fast' tests.
mvn -DskipPattern=**.dao.** test
Let me extend Sean's answer. This is what you set in pom.xml:
<properties>
<exclude.tests>nothing-to-exclude</exclude.tests>
</properties>
<profiles>
<profile>
<id>fast</id>
<properties>
<exclude.tests>**/*Dao*.java</exclude.tests>
</properties>
</profile>
</profiles>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>${exclude.tests}</exclude>
</excludes>
</configuration>
</plugin>
Then in CI you start them like this:
mvn -Pfast test
That's it.
Sure, no problem:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.6</version>
<configuration>
<excludes>
<!-- classes that include the name Dao -->
<exclude>**/*Dao*.java</exclude>
<!-- classes in a package whose last segment is named dao -->
<exclude>**/dao/*.java</exclude>
</excludes>
</configuration>
</plugin>
Reference:
Maven Surefire Plugin > Inclusions and Exclusions of Tests
(The excludes can not be configured via command line, so if you want to turn this behavior on conditionally, you will have to define a profile and activate that on the command line)
It is possible to exclude tests using the commandline; using ! to exclude.
Note: I'm not sure but possibly needs 2.19.1 or later version of surefire to work.
Examples:
This will not run TestHCatLoaderEncryption
mvn install '-Dtest=!TestHCatLoaderEncryption'
Exclude a package:
mvn install '-Dtest=!org.apache.hadoop.**'
This can be combined with positive filters as well. The following will run 0 tests:
mvn install '-Dtest=Test*CatLoaderEncryption,!TestHCatLoaderEncryption'
See the Maven Surefire docs.
I have multiple java packages in my java project. Is it possible to run a single package on eclipse using maven. I want to do it with the project level pom. I dont want to create POMs for every package.
Based on your comment, not sure if this is what you are looking for but I use a configuration variable to run test cases for a particular package. You have to add a plugin to the POM to achieve this.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7.2</version>
<configuration>
<includes>
<include>**/${testGroup}/*Test.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
Now if you want to run the test cases for lets say package org/myPackage you would use a command like this.
mvn test -DtestGroup=org/myPackage
I've got many test suites in TestNG. These are XML files. I want to be able to choose multiple XML suites when running integration-test from maven.
Currently I can add the suite files to pom.xml like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${pathToMySuiteFile_1}</suiteXmlFile>
<suiteXmlFile>${pathToMySuiteFile_1}</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
This solution has some limitations. I can only change a path to the test suite I've got defined in pom.xml. So in my example it always has to be two files. I'm not able to run, lets say, 5 suites or just one.
Is there a way to somehow parametrize the whole section "suiteXmlFiles" in pom.xml ?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<suiteXmlFiles>
${multiple_paths_ToMySuiteFiles}
</suiteXmlFiles>
</configuration>
</plugin>
Running everything that matches given test group is not an option for me:
I don't want to load all the suites I've got and then run just the selected tests using groups in TestNG suite. The reason being that a report that gets generated after running all the test suites with group filters is different from a report when just the selected test suites were run.
According to User Property of suiteXmlFiles You can use:
mvn test -Dsurefire.suiteXmlFiles=test1.xml,test2.xml
We also hit this issue with our tests.
The current workaround that we use now is to define a property variable in the property section and inject it in sure-fire suiteXmlFiles block.
<properties>
<!-- Default suites -->
<batsSuiteFile>${project.build.testOutputDirectory}/BatsTests.xml</batsSuiteFile>
<smokeSuiteFile>${project.build.testOutputDirectory}/SmokeTests.xml</smokeSuiteFile>
<!-- Default suite files if not being specified from mvn command line -->
<defaultSuiteFiles>${batsSuiteFile},${smokeSuiteFile}</defaultSuiteFiles>
<suiteFile>${defaultSuiteFiles}</suiteFile>
</properties>
Then in the plugin section...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<suiteXmlFiles>
<!-- Suite file injection parameter from command line -->
<suiteXmlFile>${suiteFile}</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
If nothing is specified from the command line it will fall back and default to the 2 suites specified above in the properties section.
Then if you want to kick off a specified set of suite files you can do:
mvn test -DsuiteFile=test1.xml,test2.xml
Surprisingly from the maven docs you expect that the arg suiteXmlFiles should just override this from the command line and should just accept a comma delimited list of testng xmls
http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#suiteXmlFiles
If anyone has any other better way please share.
You can use groups parameter or specify -Dgroups=... in the command line:
(TestNG/JUnit47 provider with JUnit4.8+ only) Groups for this test.
Only classes/methods/etc decorated with one of the groups specified
here will be included in test run, if specified. For JUnit, this
parameter forces the use of the 4.7 provider This parameter is ignored
if the suiteXmlFiles parameter is specified. .
I'm currently working on a Maven-backed project. I've chosen TestNg to implement my unitary tests. In order to run my unitary tests at each Maven build, I have added the maven-surefire-plugin to my pom.xml :
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<!-- Configuring the test suites to execute -->
<suiteXmlFiles>
<suiteXmlFile>testsuite-persistence-layer.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
Moreover, I want to specify the tests to be executed using TestNg's TestSuiteXmlFile. For instance, in my pom.xml, I've configured the surefire plugin so that it will execute the tests defined in the xml file named "testsuite-persistence-layer.xml".
The problem is that by default, the surefire plugin seems to be looking for this xml file at the root of my project. How can I specify the directory in which the surefire plugin should look for the TestSuite xml files?
According to the TestNg documenation, this could be specified through the "maven.testng.suitexml.dir" property but the Surefire plugin does not seem to take it into account.
I am not sure if I understand your problem. You can easily specify the exact location of xml file, both as relative and fully qualified path.
<suiteXmlFile>c:/some/dir/testsuite-persistence-layer.xml</suiteXmlFile>
or
<suiteXmlFile>src/test/java/com/something/project/testsuite-persistence-layer.xml</suiteXmlFile>
But that's too easy, so I am guessing you are looking for a way to parametrize the directory where xmls are located. The quick solution that comes to my mind would be
<suiteXmlFile>${xmlPath}/testSuite.xml</suiteXmlFile>
Now you can run
mvn test -DxmlPath=c:/some/path
Of course xmlPath is just made-up name, you can use any other variable name you wish.
If you don't want to pass the path as an argument from command line you can specify the value of xmlPath variable in properties section of your POM. Properties is one of the main sections located just under < project > branch.
<project ... >
...
<properties>
<xmlPath>c:/some/path</xmlPath>
</properties>
...
<build>
...
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.9</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${xmlPath}/testSuite.xml</suiteXmlFile>
</suiteXmlFiles>
...
</configuration>
</plugin>
...
</plugins>
...
</build>
...
</project>
We've had an ongoing need here that I can't figure out how to address using the stock Maven 2 tools and documentation.
Some of our developers have some very long running JUnit tests (usually stress tests) that under no circumstances should be run as a regular part of the build process / nightly build.
Of course we can use the surefire plugin's exclusion mechanism and just punt them from the build, but ideally we'd love something that would allow the developer to run them at will through Maven 2.
Normally you would add a profile to your maven configuration that runs a different set of tests:
run this with mvn -Pintegrationtest install
<profile>
<id>integrationtest</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-client -Xmx896m -XX:MaxPermSize=192m</argLine>
<forkMode>once</forkMode>
<includes>
<include>**/**/*Test.java</include>
<include>**/**/*IntTest.java</include>
</includes>
<excludes>
<exclude>**/**/*SeleniumTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<activation>
<property>
<name>integrationtest</name>
</property>
</activation>
</profile>
Adding to krosenvold's answer, to ensure no unexpected behavior, make sure you also have a default profile that is active by default that excludes the integration or stresstests you want to run in your special profile.
<profile>
<id>normal</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/**/*IntTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
You will need to create a profile like this, simply listing the surefire-plugin outside of a profile will override the profile should it be selected with:
mvn -P integrationtest clean install
Use an integration test plugin such as the Super Helpful Integration Test Thingy to separate Integration Tests (long running, systemic) from Unit Test (purists say 30 seconds max for all true unit tests to run). Make two Java packages for your unit tests versus integration tests.
Then do not bind this plugin to a phase (the normal maven lifecycle) and only run it when it is explicitly called as a target, like so:
mvn shitty:clean shitty:install shitty:test
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>shitty-maven-plugin</artifactId>
</plugin>
</plugins>
This way, your normal developers will not be impacted, and you'll be able to run integration tests on demand.
Another option is to have the stress test detect it is running in maven and run only once or twice. i.e. turn into a regular functional test. This way you can check the code is still good, but not run for a long time.