In my project I have to do some repository setup before all tests and after successful tests I want to clean up, I know #afterClass is used for this but it requires static variables but i dont want to make my objects statics, so is there any other way by which I can achieve this??
If you're using JUnit4, then use #After. Documentation here. Note that your methods annotated this way will be executed after each test case, in a similar fashion to #Before being executed before each test case.
If you're writing an integration test with multiple test cases and your setup is heavy, you could use a combination of #BeforeClass to set up static objects and #After to mutate/clean up/reset certain parts of these objects' states. This, of course, would violate your requirement of not using static variables, but I can't really see the rationale behind that requirement. Recall that JUnit instantiates the test class once for every single test case.
If you want set up and tear down before and after each test method, use #Before and #After. If you want to set up once, run all your tests and then tear down, use #BeforeClass and #AfterClass.
Yes, #BeforeClass and #AfterClass are static methods, but note that JUnit recreates your test class instance for each test method, so you cannot maintain any information in non-static fields of a test class across different tests.
I, as the other commenters here, have a feeling that you want to avoid static fields because there's a common opinion that they are bad practice. Please note, however, that good practice in writing code often does not apply to good practice in writing tests. This is one such example.
Related
I have some unit test built with Junit in a Java project.
The only issue is that I want to define the order/priority of the tests.
I mean... I'm using #BeforeAll to execute first test, which is the login process and to get the access for the postlogin functionalities.
But right after that, I want to run another specific test.
And then the rest...
I checked and there is an option to use #Order() annotation, but my idea is not to be ordering every test like this... I just wanna run first the login, and then a test that I need to run all the others. So after the first two test, the others doesn't matter in which order they are.
First, having unit tests depend on other unit tests is typically bad design. The JUnit 5 User Guide mentions this:
Although true unit tests typically should not rely on the order in which they are executed, there are times when it is necessary to enforce a specific test method execution order — for example, when writing integration tests or functional tests where the sequence of the tests is important, especially in conjunction with #TestInstance(Lifecycle.PER_CLASS).
Consider if what you're doing is testing or set-up. If the action of logging in is just set-up for your tests then you should do this in a method annotated with #BeforeAll or #BeforeEach, whichever makes more sense in your context. Afterwards, you may need to clean-up using #AfterAll or #AfterEach, depending on which "before annotation" you used.
If you're actually trying to test the login code then try to separate these tests from the others. You could do this by moving them into a separate class file or even leveraging #Nested classes (if appropriate). Instead of having your later tests require a real login you should use a fake login. In other words, mock the dependencies needed for the later tests. This will remove the inter-test dependency situation. And don't be afraid to re-"login" for each test (e.g. by using #BeforeEach); if you're using mocks this shouldn't be too expensive.
Note: As of JUnit 5.4 you might even be able to abort some tests if previous tests fail using a TestWatcher extension, as mentioned in this Q&A. However, using such an approach seems better suited for integration tests rather than unit tests.
That said, what you want should be possible. You mention #Order but then say you're hesitant to use it because you don't want to order every method, only make sure that two tests run before all the others. You don't have to add the annotation to every method. If you look at the documentation of MethodOrderer.OrderAnnotation, you'll see:
MethodOrderer that sorts methods based on the #Order annotation.
Any methods that are assigned the same order value will be sorted arbitrarily adjacent to each other.
Any methods not annotated with #Order will be assigned a default order value of Integer.MAX_VALUE which will effectively cause them to appear at the end of the sorted list.
And from Order.value():
Elements are ordered based on priority where a lower value has greater priority than a higher value. For example, Integer.MAX_VALUE has the lowest priority.
This means you only need to annotate your two tests with #Order and leave the rest alone. All the test methods without the #Order annotation will run in any order, but after the first two tests.
#TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class MyTests {
#Test
#Order(1)
void firstTest() {}
#Test
#Order(2)
void secondTest() {}
#Test
void testFoo() {}
#Test
void testBar() {}
// other tests...
}
I have a super class that defines #BeforeEach and #AfterEach methods. The class also has a method that should run only when a system property is set - basically #EnabledIfSystemPropertyCondition. Normal tests with #Test annotation are in subclasses.
This works well, but the conditional test is shown as skipped in test report - which I want to avoid. Is there any way I can run a test method on a condition but not consider it as a test in normal situations?
This question is not related to inheritance of #Test annotations. The basic question is Is there any way I can run a test method on a condition but not consider it as a test in normal situations?
The easiest way to achieve that you want is to check clause manually inside the test case (just as #Paizo advised in comments). JUnit has several features which allow you skip tests execution such as Junit-ext project with #RunIf annotation or special Assume clause which will force test to be skipped if this assumption is not met. But these features also mark test as skipped which is not desired in your case. Another possibility you might think about is modifying your code with the magic of reflection to add/remove annotations at runtime, but this is not possible. Theoretically I can imagine a difficult way using cglib to subclass your test class at runtime and manage annotations just like Spring do, but first of all ask yourself if it worth?
My personal feeling is that you're trying to fix the thing which is working perfectly and not broken. Skipped test in the report is not a failed test. It's cool that you can understand from report was test executed or no.
Hope it helps!
Why does a JUnit Suite class, in my cases its called TestSuite.class, not execute its own Test, Before, and After annotations? It only exectutes its own BeforeClass, AfterClass, and then ALL annotations of the suite test classes. I proved this is the case by creating a test project around this theory: https://gist.github.com/djangofan/5033350
Can anyone refer me to where this is explained? I need to really understand this.
Because a TestSuite is not a Test itself. Those annotation are for unit tests only. See here for an example.
public class FeatureTestSuite {
// the class remains empty <----- important for your question
}
A TestSuite is way of identifying a group of tests you wish apply some common behaviour to.
Perhaps better explained with an example.
So say you were doing some basic CRUD tests on an Orders Table in a Database MyDB.
Everyone needs mydb to be there and the orders table to exist, so you put them in a suite. It sets up the db and the table, the tests run, then before the suite goes out of scope the db is dropped, everything is nice and clean for the next test run. Otherwise you'd have to do that in every test which is expensive, or worse have test data from previous tests cause the others to fail often apparently randomly as you would have created an implicit dependancy between them.
There are other ways of achieving the same thing, but they clutter up your tests and you have to remember to call them.
You don't have to test it. If it doesn't get done none of your tests will execute.
As others have said, it's because a TestSuite is not a Test. It's just a class with an annotation to group other tests, so that it is more convenient to run.
It does have one special property, however, and that is the execution of #BeforeClass and #AfterClass. These are enabled to allow a global setup/teardown for the suite. It does not execute any tests (including #After, #Before or any rules).
Is there a way to know in a JUnit 4 test Class, if the Class was initiated by a Test-Suite ?
I have global things that I want to run before all tests (regarding in-memory DB), so I thought doing it in the test-suit. However, I still want to be able to initiate one test at a time without a Test-Suit, So I need to know if I need to initialize the global things in the #Before part of the test... Does any-one know if it's possible ?
There are several ways to achieve this. The easiest and simplest is to have a 'test' which is run at the beginning and end of your suite which set up your database and then set a global flag. In your #Before and #After tests you check this flag and if necessary do the setup/teardown.
#RunWith(Suite.class)
#SuiteClasses({SetupTest.class, RealTest.class, TeardownTest.class});
This is the simplest solution, but it isn't very nice, so a neater solution would be to use a TestRule. Look at extending ExternalResource. This implements before & after logic which surrounds your test methods. This would allow you to factor out your #Before and #After methods, to reuse the same code everywhere.
Then, for your suite, you need to implement before/after logic as well. Unfortunately, the class annotated with #RunWith(Suite.class) isn't actually instantiated, so you can't use the constructor of that class, but you can extend Suite. Depending upon how you will run this, you will will need to implement one of the constructors, using #RunWith as an example:
public class MySuite extends Suite {
/**
* Called reflectively on classes annotated with <code>#RunWith(Suite.class)</code>
*
* #param klass the root class
* #param builder builds runners for classes in the suite
* #throws InitializationError
*/
public MySuite(Class<?> klass, RunnerBuilder builder) throws InitializationError {
this(builder, klass, getAnnotatedClasses(klass));
// put your global setup here
// set global variable
}
}
Then run your test suite with
#RunWith(MySuite.class)
There are several constructors which are used in different situations, look at the comments next to each for specifics. You still need to use a global variable so that your Rules don't re-execute the setup code. The above will work if you're wanting to execute only setup code, executing teardown code is harder, but can be done. Let me know if you need it :-)
If you're wanting more flexibility (say executing setup code only for specific methods), then see my answer to How to define JUnit method rule in a suite?.
I would use JUnit's #BeforeClass and #AfterClass annotations to indicate methods to do this type of work.
From the #BeforeClass Javadoc:
Sometimes several tests need to share computationally expensive setup
(like logging into a database). While this can compromise the
independence of tests, sometimes it is a necessary optimization.
Annotating a public static void no-arg method with #BeforeClass causes
it to be run once before any of the test methods in the class. The
#BeforeClass methods of superclasses will be run before those the
current class.
You could place some #BeforeClass code in each test suite, which delegates to an auxiliary class that does the common set up. The auxiliary class could have a static boolean that records whether the set up has already been done. If it has already been done, the auxiliary class would do nothing.
I have a problematic situation with some quite advanced unit tests (using PowerMock for mocking and JUnit 4.5). Without going into too much detail, the first test case of a test class will always succeed, but any following test cases in the same test class fails. However, if I select to only run test case 5 out of 10, for example, it will pass. So all tests pass when being run individually. Is there any way to force JUnit to run one test case at a time? I call JUnit from an ant-script.
I am aware of the problem of dependant test cases, but I can't pinpoint why this is. There are no saved variables across the test cases, so nothing to do at #Before annotation. That's why I'm looking for an emergency solution like forcing JUnit to run tests individually.
I am aware of all the recommendations, but to finally answer your question here is a simple way to achieve what you want. Just put this code inside your test case:
Lock sequential = new ReentrantLock();
#Override
protected void setUp() throws Exception {
super.setUp();
sequential.lock();
}
#Override
protected void tearDown() throws Exception {
sequential.unlock();
super.tearDown();
}
With this, no test can start until the lock is acquired, and only one lock can be acquired at a time.
It seems that your test cases are dependent, that is: the execution of case-X affects the execution of case-Y. Such a testing system should be avoided (for instance: there's no guarantee on the order at which JUnit will run your cases).
You should refactor your cases to make them independent of each other. Many times the use of #Before and #After methods can help you untangle such dependencies.
Your problem is not that JUnit runs all the tests at once, you problem is that you don't see why a test fails. Solutions:
Add more asserts to the tests to make sure that every variable actually contains what you think
Download an IDE from the Internet and use the built-in debugger to look at the various variables
Dump the state of your objects just before the point where the test fails.
Use the "message" part of the asserts to output more information why it fails (see below)
Disable all but a handful of tests (in JUnit 3: replace all strings "void test" with "void dtest" in your source; in JUnit 4: Replace "#Test" with "//D#TEST").
Example:
assertEquals(list.toString(), 5, list.size());
Congratulations. You have found a bug. ;-)
If the tests "shouldn't" effect each other, then you may have uncovered a situation where your code can enter a broken state. Try adding asserts and logging to figure out where the code goes wrong. You may even need to run the tests in a debugger and check your code's internal values after the first test.
Excuse me if I dont answer your question directly, but isn't your problem exactly what TestCase.setUp() and TestCase.tearDown() are supposed to solve? These are methods that the JUnit framework will always call before and after each test case, and are typically used to ensure you begin each test case in the same state.
See also the JavaDoc for TestCase.
You should check your whole codebase that there are no static variables which refer to mutable state. Ideally the program should have no static mutable state (or at least they should be documented like I did here). Also you should be very careful about cleaning up what you write, if the tests write to the file system or database. Otherwise running the tests may leak some side-effects, which makes it hard to make the tests independent and repeatable.
Maven and Ant contain a "forkmode" parameter for running JUnit tests, which specifies whether each test class gets its own JVM or all tests are run in the same JVM. But they do not have an option for running each test method in its own JVM.
I am aware of the problem of dependant
test cases, but I can't pinpoint why
this is. There are no saved variables
across the test cases, so nothing to
do at #Before annotation. That's why
I'm looking for an emergency solution
like forcing JUnit to run tests
individually.
The #Before statement is harmless, because it is called for every test case. The #BeforeClass is dangerous, because it has to be static.
It sounds to me that perhaps it isn't that you are not setting up or tearing down your tests properly (although additional setup/teardown may be part of the solution), but that perhaps you have shared state in your code that you are not aware of. If an early test is setting a static / singleton / shared variable that you are unaware of, the later tests will fail if they are not expecting this. Even with Mocks this is very possible. You need to find this cause. I agree with the other answers in that your tests have exposed a problem that should not be solved by trying to run the tests differently.
Your description shows me, that your unit tests depend each other. That is strongly not recommended in unit tests.
Unit test must be independent and isolated. You have to be able to execute them alone, all of them (in which order, it does not matter).
I know, that does not help you. The problem will be in your #BeforeClass or #Before statements. There will be dependencies. So refactor them and try to isolate the problem.
Probably your mocks are created in your #BeforeClass. Consider to put it into the #Before statement. So there's no instance that last longer than a test case.