I have been trying to run all JUnit tests in a directory with Bazel. As far as I know, the java_test rule is only capable of running a specific class. However, I'm looking for behavior more like mvn test which runs all JUnit tests in the project. How can I accomplish this?
The typical way to organize this is to have a java_test rule per Java test class, or per group of related Java test classes. Then java_tests can be grouped together using test_suite, if that's something you want to do.
You can run all tests in a package with:
bazel test //some/package:all
or in a package and its subpackages:
bazel test //some/package/...
or in the entire workspace:
bazel test //...
More about target patterns: https://docs.bazel.build/versions/master/guide.html#target-patterns
If you just want a java_test that runs all the tests in a directory, you can do something like
java_test(
name = "tests",
srcs = glob(["*Test.java"]),
deps = [ ..... ],
)
but that may or may not be the right thing to do. In particular, if you want to just run one test or one test method (e.g. using --test_filter), bazel will still build all the java_test's dependencies. Also, note that globs only glob within a build package, and won't cross over into other packages.
Related
I am trying to find an approach that will allow me to run a single test from a JUnit class using only command-line and java.
I can run the whole set of tests from the class using the following:
java -cp .... org.junit.runner.JUnitCore org.package.classname
What I really want to do is something like this:
java -cp .... org.junit.runner.JUnitCore org.package.classname.method
or:
java -cp .... org.junit.runner.JUnitCore org.package.classname#method
I noticed that there might be ways to do this using JUnit annotations, but I would prefer to not modify the source of my test classes by hand (attempting to automate this). I did also see that Maven might have a way to do this, but if possible I would like to avoid depending on Maven.
So I am wondering if there is any way to do this?
Key points I'm looking for:
Ability to run a single test from a JUnit test class
Command Line (using JUnit)
Avoid modifying the test source
Avoid using additional tools
You can make a custom, barebones JUnit runner fairly easily. Here's one that will run a single test method in the form com.package.TestClass#methodName:
import org.junit.runner.JUnitCore;
import org.junit.runner.Request;
import org.junit.runner.Result;
public class SingleJUnitTestRunner {
public static void main(String... args) throws ClassNotFoundException {
String[] classAndMethod = args[0].split("#");
Request request = Request.method(Class.forName(classAndMethod[0]),
classAndMethod[1]);
Result result = new JUnitCore().run(request);
System.exit(result.wasSuccessful() ? 0 : 1);
}
}
You can invoke it like this:
> java -cp path/to/testclasses:path/to/junit-4.8.2.jar SingleJUnitTestRunner
com.mycompany.product.MyTest#testB
After a quick look in the JUnit source I came to the same conclusion as you that JUnit does not support this natively. This has never been a problem for me since IDEs all have custom JUnit integrations that allow you to run the test method under the cursor, among other actions. I have never run JUnit tests from the command line directly; I have always let either the IDE or build tool (Ant, Maven) take care of it. Especially since the default CLI entry point (JUnitCore) doesn't produce any result output other than a non-zero exit code on test failure(s).
NOTE:
for JUnit version >= 4.9 you need hamcrest library in classpath
I use Maven to build my project, and use SureFire maven plugin to run junit tests.
Provided you have this setup, then you could do:
mvn -Dtest=GreatTestClass#testMethod test
In this example, we just run a test method named "testMethod" within Class "GreatTestClass".
For more details, check out http://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html
The following command works fine.
mvn -Dtest=SqsConsumerTest -DfailIfNoTests=false test
We used IntelliJ, and spent quite a bit of time trying to figure it out too.
Basically, it involves 2 steps:
Step 1: Compile the Test Class
% javac -cp .:"/Applications/IntelliJ IDEA 13 CE.app/Contents/lib/*" SetTest.java
Step 2: Run the Test
% java -cp .:"/Applications/IntelliJ IDEA 13 CE.app/Contents/lib/*" org.junit.runner.JUnitCore SetTest
Is there a way to run only the tests with each of the given tags? For example, can we run only the tests with tag1 AND tag2?
We are using Maven 3.6.2 with version 2.22.2 of Surefire and JUnit 5.5.2 to run tests against many application API endpoints. Each test has at least 3 tags specifying test type, application and method type. Some have more.
We are currently running our tests from the command line to give the tester control over each run:
mvn test -Dgroups=app1,fast
The problem we run into is that the above will run all the tests tagged with app1 OR fast. The result is many dozens of tests being run. The tester's goal is to run only the "fast" tests for "app1".
While it is possible to use excludedGroups, this does not help the tester until after the run. Additionally, we are adding tests and tags every day so what worked today may not work tomorrow.
One of our goals is enabling the specification of tags on the command line and not require the tester to edit the POM to run a different combination of tests. We would like to specify our test sets from the command line and not have to touch the POM between runs. Due to the growing list of test combinations, our POM would become too large to manage efficiently.
Is ANDing JUnit5 tags together even possible with Surefire?
In Maven Surefire/Maven Failsafe you can define groups as you already mentioned but you can define it for JUnit Jupiter (aka JUnit 5) as like this:
mvn test -Dgroups="app1&fast"
or
mvn test -Dgroups="app1|fast"
for more details take a look into the documentation.
BTW: I recommend to upgrade to the most recent versions of maven-surefire-plugin or maven-failsafe-plugin.
They way I've done it was setup interfaces. If there are sub groups I just make the interface extend the parent.
For your case it seems like you have app1 tag and some of those tests tagged fast. Create interface for app1 and another for fast that extends app1. In your tests you can add #Category(app1Fast.class)
When you call mvn test -Dgroups=tests.groups.app1Fast it will run just the fast tags. If you run mvn clean test -Dgroups=tests.groups.app1 it will run everything in app1.
Also, you can tell I created a package called groups to add my interfaces in.
Another also, the interface will be empty -
public interface app1 {}
public interface app1Fast extends app1{}
Sounds like you'll have multiple apps -- app1, app2, etc.
public interface app2{}
public interface app2Fast extends app2{}
Probably other ways, but this is simple enough.
call app1 to test everything in app1, or app1Fast for only the fast tests for app1. Same for app2.
Hope that helps.
I installed JUnit-Tools 1.1.0 in eclipse market. As per the documentation, they want the test-projects and the mock-projects to be created manually.
My class structure is like below :
I tried creating a test package like com.unifiedportal.core.unifiedportal.service.test and tried generating the test classes from the JUnit-Tools..
Got an warning like below :
Let me know how i should proceed with creating the test-project for this.
If you want to generate the tests under the same project you are in then remove Test-project-postfix in the junit tools preferences, otherwise let the postfix as it is and create a project (not a package) with the same name than your project plus that postfix.
Look an example of my config for generating the tests in the same project under src/test/java, because I don't want my tests to be in a separated project:
you can add the jars as maven or gradle dependencies.Change the test-source-folder-name to src/test/java and generate the test cases using the plugin.This will generate the test cases in your test folder
This question is more of a best practices approach. The application we have is Spring Boot 1.5.4 and builds using Gradle. I'm in the process of creating nightly builds with Jenkins, and want to make sure all the unit tests pass in the project.
The project has a number of "tests" like this however:
#SpringBootTest(classes = {Application.class})
#RunWith(SpringRunner.class)
public class DatabaseCreationProcessImplTest {
This particular class creates a sample database image for developers to work off of. Granted we could make straight SQL scripts, but a Java process is useful since there's code that also queries for data from outside sources (e.g. Liferay.)
The reason we're using a unit test for this is because developers can easily run it in IntelliJ to load a new database image. However this isn't really a "test", it's using the test runner as a quick way to run a Java process.
I'm working on setting up nightly builds and I don't want this test to be included in the builds. I can do something like the following in the build script:
test {
exclude 'com/mydomain/service/util/impl/DatabaseCreationProcessImplTest.class'
}
However by doing this, if running the unit test individually in the IDE with the Spring test runner, it is unable to find any tests. I thought about passing in a Boolean value in the Jenkins task for doing this, e.g.
test {
systemProperties 'property': 'value'
testLogging {
events "passed", "skipped", "failed", "standardOut", "standardError"
}
// Exclusions go here
if (Boolean.getBoolean('exclude.tests')) {
exclude 'com/mydomain/service/util/impl/DatabaseCreationProcessImplTest.class'
}
}
However this seems like a hack/kludge... any ways looking for some "best practices" approach for handling this. Is JUnit the right way for quickly running Java processes? Are there other alternatives? Is it possible to create a Gradle script which developers can use to invoke common Java (Spring Boot) process as well?
I think you could group your not-really-tests in a test suite with JUnit's #SuiteClasses annotation:
#Suite.SuiteClasses(DatabaseCreationProcessImplTest.class)
public class NotReallyTests {}
And then use a condition that you pass from your Jenkins command line to exclude the not-really-tests suite:
test {
if (project.hasProperty('excludeNotReallyTests')) {
useJunit {
excludeCategories 'fully.qualified.name.of.your.NotReallyTests'
}
}
}
Your Jenkins command line would then be
$ gradle -PexcludeNotReallyTests=true
It's a little less hacky than your solution in that it keeps track of the grouping of tests that are not really tests in the codebase instead of the build.gradle file.
The Testing API for Android provides several annotations that are used to group tests together. Then you can specify which tests to run by giving one of the annotations on the command line. I do not know the details of how to implement this. It is just a suggestion for you to explore on your own, if you are interested.
I am trying to run individual spock unit tests using intellij idea.
Consider:
// rest of code
def "Test Something"() {
// test code below
}
In above test, when I goto the test body and right context menu, I get two kinds of tests for Test Something. One is the grails test and other is the junit test.
Referring to this question, the accepted answer recommends using the jUnit runner. But using it, the code simply does not compile(probably because certain plugins and other classes are not available).
(I am not sure though as this is the desired behavior because I am just running a single test and not all tests. So wonder why is it compiling all classes ,including plugin classes not required by the test target class.)
Using the grails runner, I check the configuration and here is the screenshot:
So nothing looks wrong with the command there.
But the test on running gives Test framework quit unexpectedly error.
I try running same command from grails console(CMD windows) and it runs without any error message.
But on checking the output html files(in target/test-reports) I see that none of the tests actually ran!
So what is going on here and why are not individual tests running?
PS:
When I run All tests using test-app command, tests run as expected. Only individual (unit)tests are not running.
Part of the price paid for Spock's nice test naming, is that you can't specify an individual test to run anymore.
Here are some articles about it. The first seems pretty on-point:
Run a specific test in a single test class with Spock and Maven
This one isn't about running a single test, but has some relevance and talks about Spock's test-name conversions, plus Peter Niederwieser chimes in with comments:
Can TestNG see my Spock (JUnit) test results?
A workaround for this could be the #IgnoreRest annotation. Simply annotate the test you want to run with #IgnoreRest, and then specify that test class to run, and only the annotated test will run. http://spockframework.github.io/spock/javadoc/1.0/spock/lang/IgnoreRest.html
Try using the grails unit test and add the following in the command line part:
-Dgrails.env=development
This will run the test as we change the running environment to development . Hope this will help to everyone facing such problems.