Develop Reference

Include / exclude tests with Maven Surefire + JUnit5 from command line

I want to run the build for Apache James which has a huge test suite that is running very long due to tests that irrelevant to me, e.g. tests concerning RabbitMQ. Thus I'd like to exclude those and I want to do so from the command line (not by editing POMs). I'm using Maven 3.6.3 on Java 11 OpenJDK. The project uses JUnit5 and maven-surefire-plugin 2.22.2.
Now, I would expect the following to work:
For example, to run only test methods in the org.example.MyTest test
class you can execute mvn -Dtest=org.example.MyTest test from the
command line.
But it doesn't work. In fact, as soon as I set the test parameter to anything else than an empty string, all tests will be skipped. I tried some of the syntax that is supposedly supported...
mvn package -Dtest=*
mvn package -Dtest=".*"
mvn package -Dtest=\!SomethingFishy
mvn package -Dtest='!MavenHeadache'
mvn package -Dtest='!%regex[.*HelpMe.*]'
...but the result is always the same:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test
(default-test) on project testing-base: No tests were executed!
(Set -DfailIfNoTests=false to ignore this error.) -> [Help 1]
I'm running the package goal, as it does the full build, but test behaves the same. Also tried specifying -Dincludes=... / -Dexcludes=..., which had no effect at all: All tests are executed regardless and the parameters don't even show up in the output of mvn -X .... This behavior doesn't change when update maven-surefire-plugin to the latest version which is 3.0.0-M5.
Do I understand something wrong here? How can i specify inclusions/exclusions in this setup?
Update: It looks like this is caused by nested projects and/or James' project structure in particular. If I enter a "leaf project", e.g. core, then inclusions/exclusions begin to work:
cd core
mvn test -Dtest=HostTest # will only run HostTest, as expected
mvn test -Dtest=\!HostTest # will run all tests but HostTest, as expected
As suggested by RobertScholte, I have looked at the maven-surefire-plugin configuration, but couldn't find anything that seems to be related to this behavior.

The error message tells what you need to know: in project testing-base which is the first to run, there's no test matching your pattern, so it fails to ensure you won't have a false impression of success.
It then suggests to use -DfailIfNoTests=false option to let maven ignore modules that don't have any test matching the pattern (it's probably what you need).

TestNg test not running with maven build

I have TestNg unit tests which is supposed to run with my maven clean install.
I don't have any test-suite.xmls in my pom to run testes. Expectation is to run all my test files without any configuration with the maven build.
But this is not happening.
My test class goes like this
public class CreateUtilty{
#Test
public void testScope(){
Creationutiltiy.create("myApp");
// remaing code
}
}
What could have I done wrong ?

Running testNG as you are with no configuration, the surefire plugin expects your test classes to end with Test. Try changing your test class name to CreateUtilityTest and it should be picked up.
The documentation for the maven surefile plugin contains useful information to help you get started.
To find out more about how to include/exlude tests based on naming convention read this.

Try execute your class with following syntax
mvn -Dtest=CreateUtilty test

Gradle jacoco multi project. How to output the code coverage of all the project in the terminal?

I am using Gradle jacoco plugin and have JUnit tests in a gradle multi project.
I have installed jacoco in order to report to sonarqube my code coverage, which I did.
Gitlab-CI also offer a badge for the code coverage, I only need to output the code coverage in a terminal and use a regex to find it.
Is there a way to output the code coverage of all the project in the terminal ?

Yes, you can. Possible ways:
1) Use Gradle plugin gradle-console-reporter to report various kinds of summaries to console. JUnit, JaCoCo and Cobertura reports are supported.
In your case, following output will be printed to console:
...
BUILD SUCCESSFUL
Total time: 4.912 secs
Coverage summary:
project1: 72.2%
project2-with-long-name: 44.4%
Then you can use coverage with regular expression in Gitlab's .gitlab-ci.yml to parse code coverage.
2) Second option is a little bit tricky. You can print full JaCoCo HTML report (e.g. using cat target/site/jacoco/index.html) and then use regular expression (see this post) or grep (see this post) to parse coverage.

AFAIK Gradle does not support this, each project is treated separately.
To support your use case some aggregation task can be created to parse a report and to update some value at root project and finally print that value to stdout.
Update with approximate code for solution:
subprojects {
task aggregateCoverage {
// parse report from each module into ext.currentCoverage
rootProject.ext.coverage += currentCoverage
}
}

How to run single Android local unit test using gradle 2.4. Test filtering is not supported

I'm using gradle to build my android project and am not able to run single local unit test. I have several test classes and one of them is MockServerTest and I only want to run test methods in this class.
I tried using gradle -Dtest.single=MockServerTest test but it turned out running all my tests, including these in other test classes.
I also tried gradle test --tests MockServerTest but an error occurred said
Test filtering is not supported for given version of junit. Please upgrade junit version to at least 4.6.
But I'm using junit 4.12 in my gradle file
testCompile 'junit:junit:4.12'
I'm using gradle 2.4 with com.android.tools.build:gradle:1.2.3.
Also, how can I run a single test method inside a single test class?
BTW, I'm able to run single test method inside Android Studio, by right clicking on the test method and select run targetTestMethod() from the menu. But how can I achieve this in the terminal? I guess Android Studio also trigger a certain command to do this. How can I see what that command is?

Figured it out myself. I have to run
gradle testDebug --tests com.my.package.TestClassName
There are two things to note here.
1. You have to use gradle testDebug or gradle testRelease instead of just gradle test. If you have build variant, you have to use gradle testVariantNameDebug or gradle testVariantNameRelease
2. You have to specify the whole qualified class name, means including the package name.

You can use Android Gradle plugin DSL to set up test tasks filters like this:
android {
testOptions {
unitTests.all {
it.testNameIncludePattern = "*.SomeTest"
}
}
}
You can find more information on testOptions here and filters here.

Have you tried running gradle test -Dtest.single=MockServerTest? More information can be found here.

Testng terminal report

I run test cases from maven, that is from command line. Is there a way to see a reason why test failed in terminal output? I've tried raising verbosity in testng.xml to 4,hoping that it will activate TextReporter, which seems like class that prints, well, text reports, but it didn't help.

Try mvn -Dsurefire.useFile=false ... test or corresponding setting in surefire plugin configuration.