I have a project that we have many Junit tests. We just did a large migration from JUnit4 to JUnit5. We would like to run most of the tests in parallel but have a couple that need to be ran sequentially. Is there any way to use JUnit5 and run tests both ways?
The reason I ask is that I have 4 tests that load a database into memory and I am loading data into this database. Then I run tests on that database. These are the four tests I need to run sequentially and cannot run in parallel.
You most likely want to use #ResourceLock annotation on tests
https://junit.org/junit5/docs/snapshot/user-guide/#writing-tests-parallel-execution-synchronization
You might take a look at #Isolated annotation
https://junit.org/junit5/docs/5.7.1/api/org.junit.jupiter.api/org/junit/jupiter/api/parallel/Isolated.html.
It guarantees that test-class won't run in parallel with other classes.
According to Parallel Test Execution and Single Thread Execution:
Since of maven-surefire-plugin:2.18, you can apply the JCIP annotation
#net.jcip.annotations.NotThreadSafe on the Java class of JUnit test
(pure test class, Suite, Parameterized, etc.) in order to execute it
in single Thread instance. The Thread has name
maven-surefire-plugin#NotThreadSafe and it is executed at the end of
the test run.
So you could annotate your test classes with #NotThreadSafe in order to get them executed on 1 same thread (named maven-surefire-plugin#NotThreadSafe).
<dependency>
<groupId>com.github.stephenc.jcip</groupId>
<artifactId>jcip-annotations</artifactId>
<version>1.0-1</version>
<scope>test</scope>
</dependency>
Related
I've written multiple test classes to test my methods using junit 5.
all the test classes pass successfully when I run them indivually
but when I try to run them all at a time using a test suite as shown below, some of my tests get pending and the testing won't finish. it doesn't even jump to test other classes
as all the methods pass successfully, I don't think if there's any problem with the class ParametrizedMethodTest
I'm using junit-platform-runner version 1-6-2
From the current JavaDoc:
Please note that test classes and suites annotated with
#RunWith(JUnitPlatform.class) cannot be executed directly on the JUnit
Platform (or as a "JUnit 5" test as documented in some IDEs). Such
classes and suites can only be executed using JUnit 4 infrastructure.
In other words, JUnit 5 does not support test suites in the way you want to do it in your example. If you want to run all your tests classes just select the package and choose Run Tests from the context menu.
I have a framework that uses TestNG with Cucumber and runs in parallel.
Now i need some of my tests to be run sequentially.
There are some cases to do such trick with exclude option in TestNG xml. But, since im using Cucumber, i have only one #Test method, that parses cucumber features and runs its step definitions so i can't modify any particular #Test.
I think i can do what i need by creating only one step definition for sequential step, then synchronized it with some static lock. But in my opinion this is way too ugly and unconvenient to use and read. But i hope there is some more clear and easy to achieve way.
Any suggestions would be appreciated
Is there some way to have complete control over execution of test methods (including before/after methods) in JUnit5, similar to JUnit4 #RunWith annotation)?
I'm trying to build a JUnit5 Arquillian extension, but since Aquillian basically needs to execute each test in a container, I'm coming to a problem when running Arquillian from a Junit5 extension.
My code is here: BasicJunit5ArquillianTest.java
The test should run all methods (including before/after) in a separate container, which can be a separate JVM, remote or embedded server or anything isolated. My extension runs the test method from beforeEach hook, using Arquillian to transfer the test class and run it in a container using LauncherFactory.create(), collect test results and transfer them back.
The problem is that the test methods are executed twice - via normal JUnit5 execution and via my Arquillian extension from a beforeEach hook. I'd like to run tests only via Arquillian and skip normal execution of methods.
Is this possible in a JUnit5 extension?
Or I need to create a custom test engine, possibly extending the Jupiter test engine?
There is no extension point (yet?) that allows you to define where or how tests are run. This is already true for threads, which means there is no way to run them on JavaFX application thread or Swing EDT.
You might have to go deeper and implement an engine but that means that users have to choose between writing Arquillian tests or writing Jupiter tests.
UPDATE: In a newer version of JUnit 5 released since this answer was accepted, JUnit 5 now provides the InvocationInterceptor extension point, which is exactly what is needed to implement a custom runner as an extension, which has a full control over how the tests are executed and can even replace the body of the test method with something completely different (e.g. run the test in a different JVM and return the result).
I am running a suite of integration tests using maven and about 10% of the tests would fail or throw an error. However, when I start the server and run the individual failed tests manually from my IDE(intellij idea), they all pass with no problem. What could be the cause of this issue?
This is almost always caused by the unit tests running in an inconsistent order, or a race condition between two tests running in parallel via forked tests. If Test #1 finishes first, it passes. But if Test #2 finishes first, it leaves a test resource, such as a test database, in an alternate state causing Test #1 to fail. It is very common with database tests, esepecially when one or more alter the database. Even in IDEA, you may find all the tests in the com.example.FooTest class always pass when you run that class. But if you run all the tests in the com.example package or all tests in the project, sometimes (or even always) a test in FooTest fails.
The fix is to ensure your tests are always guaranteed a consistent state when run. (That is a guiding principle for good unit tests.) You need to pay attention to test setup and tear-down via the #Before, #BeforeClass, #After, and #AfterClass annotations (or TestNG equivalents). I recommend Googling database unit testing best practices. For database tests, running tests in a Transaction can prevent these type of issues. That way the database is rolled back to its starting state whether the test passes or fails. Spring has some great support for JDBC dtaabase tests. (Even if your project is not a Spring project, the classes can be very useful.) Read section 11.2.2 Unit Testing support Classes and take a look at the AbstractTransactionalJUnit4SpringContextTests / AbstractTransactionalTestNGSpringContextTests classes and the #TransactionConfiguration annotation (this latter one being from running Spring Contexts). There are also other Database testing tools out there such as DbUnit.
I need to setup a database in my tests (schema and some test data), this takes quite a bit of time, and as such I prefer to have it done once for all tests that are being run, and reset so that any chanages to the DB are rolled back between tests.
I'm not sure though which JUnit facilities should be used for this.
It seems like I can set a #BeforeClass/#AfterClass on a test suite, but than I can't run individual tests anymore.
Is there some way to add a setup/teardown for all tests that will run even when only executing a subset of the tests and not a specific suite? (For example NUnit has SetUpFixture)
I guess the transactions/truncation of the DB can be done using JUnit Rules...
You can use in-memory databases like HSQL or H2 to speed up test.
To roll back, you can use transactional feature.
Is there some way to add a setup/teardown for all tests that will run even when only executing a subset of the tests and not a specific suite?
For this, you can create a super class which is extended by other test classes. In super class, you can set up to setup/teardown.