I want to run the build for Apache James which has a huge test suite that is running very long due to tests that irrelevant to me, e.g. tests concerning RabbitMQ. Thus I'd like to exclude those and I want to do so from the command line (not by editing POMs). I'm using Maven 3.6.3 on Java 11 OpenJDK. The project uses JUnit5 and maven-surefire-plugin 2.22.2.
Now, I would expect the following to work:
For example, to run only test methods in the org.example.MyTest test
class you can execute mvn -Dtest=org.example.MyTest test from the
command line.
But it doesn't work. In fact, as soon as I set the test parameter to anything else than an empty string, all tests will be skipped. I tried some of the syntax that is supposedly supported...
mvn package -Dtest=*
mvn package -Dtest=".*"
mvn package -Dtest=\!SomethingFishy
mvn package -Dtest='!MavenHeadache'
mvn package -Dtest='!%regex[.*HelpMe.*]'
...but the result is always the same:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test
(default-test) on project testing-base: No tests were executed!
(Set -DfailIfNoTests=false to ignore this error.) -> [Help 1]
I'm running the package goal, as it does the full build, but test behaves the same. Also tried specifying -Dincludes=... / -Dexcludes=..., which had no effect at all: All tests are executed regardless and the parameters don't even show up in the output of mvn -X .... This behavior doesn't change when update maven-surefire-plugin to the latest version which is 3.0.0-M5.
Do I understand something wrong here? How can i specify inclusions/exclusions in this setup?
Update: It looks like this is caused by nested projects and/or James' project structure in particular. If I enter a "leaf project", e.g. core, then inclusions/exclusions begin to work:
cd core
mvn test -Dtest=HostTest # will only run HostTest, as expected
mvn test -Dtest=\!HostTest # will run all tests but HostTest, as expected
As suggested by RobertScholte, I have looked at the maven-surefire-plugin configuration, but couldn't find anything that seems to be related to this behavior.
The error message tells what you need to know: in project testing-base which is the first to run, there's no test matching your pattern, so it fails to ensure you won't have a false impression of success.
It then suggests to use -DfailIfNoTests=false option to let maven ignore modules that don't have any test matching the pattern (it's probably what you need).
Related
In my project there are many tests marked with #SpringBootTest which I don't regard as unit tests and rather as integration tests. So I would like to run only the unit tests when I execute:
mvn clean install
actually I want to run this command as part of pre-commit git hook but #SpringBootTest makes it longer to finish execution.
Is there a way to exclude the tests marked with #SpringBootTest? May be there is a pattern we can pass to maven that excludes/certain tests. Or may be write a test suite that includes the spring boot tests.
I did google search to achieve the above but don't have much luck.
Is there even a better way?
#Update: Constraint is maven pom file can't be modified.
#Update2: I have a solution that looks promising:
1. Use #Category("IntegrationTests") for #SpringBootTests tests.
2. Create TestSuite with excludeCategory:
#RunWith(CategoryRunner.class)
#ExcludeCategory("IntegrationTests")
public class TestSuite {
}
3. From mvn command line, run only TestSuite.
I am not sure this is the best. Appreciate anyone's better approach.
If you have different kinds of tests, and want to be able to specify which tests to run, you can do that with #Conditionals or with #Profile.
Examples:
#ConditionalOnProperty("test.run.integration") The class will only be loaded by Spring when property test.run.integration is defined and not false.
#Profile("integrationtest") The class will only be loaded by Spring when profile integrationtest is active.
If you're on JUnit 4, use #IfProfileValue annotation on the test class or method.
Example:
#IfProfileValue(name ="spring.profiles.active", value ="IntegrationTests")
If you're on JUnit 5, as you should be by this time, use #EnabledIf or #DisabledIf.
Example:
#DisabledIf(
expression = "#{systemProperties['os.name'].toLowerCase().contains('mac')}",
reason = "Disabled on Mac OS"
)
See the docs for more details.
try either
mvn clean install -DskipTests
or
mvn clean install -Dmaven.test.skip=true
For more options refer to below links
https://mkyong.com/maven/how-to-skip-maven-unit-test/
https://www.baeldung.com/maven-skipping-tests
I have TestNg unit tests which is supposed to run with my maven clean install.
I don't have any test-suite.xmls in my pom to run testes. Expectation is to run all my test files without any configuration with the maven build.
But this is not happening.
My test class goes like this
public class CreateUtilty{
#Test
public void testScope(){
Creationutiltiy.create("myApp");
// remaing code
}
}
What could have I done wrong ?
Running testNG as you are with no configuration, the surefire plugin expects your test classes to end with Test. Try changing your test class name to CreateUtilityTest and it should be picked up.
The documentation for the maven surefile plugin contains useful information to help you get started.
To find out more about how to include/exlude tests based on naming convention read this.
Try execute your class with following syntax
mvn -Dtest=CreateUtilty test
My RunCukeTest class looks like:
package runsupport;
import ....
#RunWith(Cucumber.class)
#CucumberOptions(
monochrome = true,
features = "classpath:features",
plugin = {"pretty", "html:target/cucumber-html-report", "json:target/cucumber.json"},
glue = { "classpath:steps", "classpath:runsupport" },
tags = {"#search"}
)
public class RunCukesTest{
}
How do I convert this to mvn test -Dcucumber.options="" format so that I can run it from the Goals line of a Maven Build run configuration? I want to be able to change the tags on the run configuration rather than having to edit the RunCukeTest class each time.
The answer turned out to be:
clean test -Dcucumber.options="--tags #search --monochrome --plugin pretty:STDOUT --plugin html:target/cucumber-html-report --plugin json:target/cucumber.json --glue steps --glue runsupport classpath:features"
Note that since I have two glue paths that I needed two --glue statements. Also note that only the package name of the two -glue paths were specified.
Further note that STDOUT needed to be specified on --plugin pretty:STDOUT.
Finally note that the feature keyword was dropped completely. The path specified at the end (without a keyword) tells cucumber-jvm where to find the feature files.
If you get any of this wrong cucumber-jvm gives you cryptic error messages. I thought that I would post this for the next person to save him or her an hour or two.
BTW, to get a Maven Build configuration in Eclipse, click on Run > Run Configurations... On the left pane double-click Maven Build. A default Maven Build configuration is created. Give it a good name and on the Goals line paste your clean test -Dcucumber.options line. It wouldn't hurt to click on the JRE tab and make sure that the correct JDK is being used. Back on the Main tab, press Apply, press Run. It completely overrides the #CucumberOptions line in your RunCukesTest (or whatever you named it) class. It is a lot easier to run tests this way and you aren't confusing git with unnecessary local file edits.
I know that it's possible to run a specific test class with -Dtest=MyTest. But is it possible to run a specific test within that class?
I.e. if MyTest defines testFoo() and testBar(), is there a way to specify that only testfoo() should be run?
I'm aware that it's trivially easy to do this in an IDE, but I occasionally need to run tests on the command line on another server.
From Running a Single Test Using Maven Surefire Plugin
With version 2.7.3, you can run only n tests in a single Test Class.
NOTE : it's supported for junit 4.x and TestNG.
You must use the following syntax
mvn -Dtest=TestCircle#mytest test
You can use patterns too
mvn -Dtest=TestCircle#test* test
It will be available as of Surefire 2.8, see SUREFIRE-577
Don't think its available. You can work around it by passing some system properties & ignore execution of tests based on the property value. However it does not seem to add a great value add. There is also TestNG which offers additional features.
http://maven.apache.org/plugins/maven-surefire-plugin/examples/testng.html
To execute one Test at a time, run mvn test
mvn -Dtest=MyUnitlTest test
To execute one Test at a time and a specific method from it:
mvn -Dtest=MyUnitTest#method test
where MyUnitTest is the name of your test and #method is the name of your method.
Execute tests with surefire:
mvn surefire:test
I run test cases from maven, that is from command line. Is there a way to see a reason why test failed in terminal output? I've tried raising verbosity in testng.xml to 4,hoping that it will activate TextReporter, which seems like class that prints, well, text reports, but it didn't help.
Try mvn -Dsurefire.useFile=false ... test or corresponding setting in surefire plugin configuration.