We have DAO tests that should run against both the real DAO/database, and against a mock dao to verify that the mock dao behaves the same as the real dao. To this end, we have a structure like this:
public abstract class DAOTestBase
{
public void testSimple()
{
// dummy assertion
assertTrue(true, "Hello");
}
}
#Test(groups = "fast")
public class TestMockDAO extends DAOTestBase
{
// setUp/tearDown and helper methods for mock
}
#Test(groups = "slow")
public class TestDAO extends DAOTestBase
{
// setUp/tearDown and helper methods for real DB
}
Unfortunately this doesn't work - TestNG doesn't think that the testSimple method is a test and hence won't run it. So instead I tried to annotate the testSimple method (or the DAOTestBase class):
A #Test annotation without any groups will lead to the same effect - the test won't run for either fast nor slow groups.
A #Test annotation with groups fast and slow will lead to the opposite effect - both TestMockDAO and TestDAO will be run regardless of whether only fast or only slow tests should be run.
A #Test annotation with a different group, say common, plus added dependsOnGroups="common" annotations in both TestMockDAO and TestDAO will also not work unless common is included in the groups to run which leads again to case 2 above (both TestMockDAO and TestDAO are run).
In the end, what I'm looking for is a way to be able to define the group for the inherited tests in the sub class, but it seems as if the #Test annotation is only applied to test methods in that very same class, not also to inherited methods that don't have a #Test annotation. Is there any other way to achieve this (without overriding all methods in the sub classes) ?
I am currently working through a similar situation.
A way to make test cases run is to use something like:
#Test
public void someTest() {
TestNG testng = new TestNG();
testng.setTestClasses(new Class[] { SomeTests.class });
testng.run();
}
Reference: http://testng.org/doc/documentation-main.html#running-testng-programmatically
Unforunately I am currently unable to get it to report the test cases within SomeTests.
Have you tried simply adding a #Test annotation on top of DAOTestBase? Each subclass will override it with its own group and this should make the method in the base a test method.
I am using TestNG 6.14.3 version and I found a solution using priority annotation.
Example:
I have a base test class:
public class TestBase {
#Test(priority = 0)
public void testA() {
assertTrue(true, "testA");
}
}
And another extended test class:
public class Test2 extends TestBase {
#Test(priority = 1)
public void testB() {
assertTrue(true, "testB");
}
}
When I run Test2 test class, I obtain the following esult:
testA: true
testB: true
I solved it this way:
The methods in the base class are in "base" group, but need to check if the test have been initialized.
public abstract DaoTestBase {
private boolean initialized = false;
#Test(groups = "base")
public void testSimple() {
if (!initialized) { return; }
// dummy assertion
assertTrue(true, "Hello");
}
}
The test is initialized in the child, in the BeforeClass annotated method.
#BeforeClass
protected void initialize() {
super.initialized = true;
}
If you annotate the parent class instead of the methods, you must pass inheritGroups=false and the group, since it inherits also the group of the base class and it will not work.
Now, you must run TestNG to check groups base,fast or base,slow. Both tests will be executed, but the one not initialized will do nothing.
It is ugly, and I would not recommend it (it looks better to redefine the methods in child and call the according super method), but in my case I need priority in my test methods, and I want to avoid that repetition in each child class.
Related
I have a test class containing test cases for say blue and non-blue devices. If the parameter isBlue is set then it should run only those test case that have say the #Blue annotation. I am not sure how to implement this specific annotation that will achieve this functionality.
public class TestClass {
boolean isBlue = false;
#Before
public void setUp () {
isBlue = MyApplication.instance().isBlue();
}
#Test
public void testA () { ... }
#Test #Blue
public void testB() { ... }
}
In this example if isBlue is true then it should only run testB() otherwise it should run all test cases
Take a look at JUnit runners. Nice explanation of runners
You can create your own runner which will extend on BlockJUnit4ClassRunner.
You can override
#Override
protected List<FrameworkMethod> getChildren() {
// scan test class for methonds annotated with #Test
}
To additionally filter by methods that have your custom annotation set, and your property in the application is set as well.
I think you are on the wrong path.
You want your test cases to help identifying problems; and assist you in fixing them. Anything that "distracts" you from that purpose reduces the values of your unit tests.
Thus: don't put tests that have really "different" scope into the same test class. Especially from the point of view: how would you control what "MyApplication.instance()" would return? That sounds like a problem in itself.
Meaning: you want to have two independent tests; and those don't rely on some "static" object telling them something. They test what is in their scope; nothing else.
By using jUnit Assume
#Test
public void testA () {
assumeTrue(MyApplication.instance().isBlue());
...
}
A failing assumption in a #Before or #BeforeClass method will have the same effect as a failing assumption in each #Test method of the class.
I have a class, which I use as a basis for my unit tests. In this class I initialize the whole environment for my tests, setting up database mappings, enter a number of database records across multiple tables, etc. That class has a method with a #BeforeClass annotation which does the initialization. Next thing, I extend that class with specific classes in which I have #Test methods.
My question is, since the before class is exactly the same for all these test classes, how can I ensure that they are run only once for all the tests.
One simple solution is that I could keep all the tests in one class. However, the number of tests is huge, also they are categorised based on functional heads. So they are located in different classes. However since they need the exact same setup, they inherit the #BeforeClass. As a result the whole setup is done at least once per test class, taking much more time in total than I would prefer.
I could, though, put them all in various subpackages under one package, hence if there is a way, how I can run set up once for all the tests within that package, it would be great.
With JUnit4 test suite you can do something like this :
#RunWith(Suite.class)
#Suite.SuiteClasses({ Test1IT.class, Test2IT.class })
public class IntegrationTestSuite
{
#BeforeClass
public static void setUp()
{
System.out.println("Runs before all tests in the annotation above.");
}
#AfterClass
public static void tearDown()
{
System.out.println("Runs after all tests in the annotation above.");
}
}
Then you run this class as you would run a normal test class and it will run all of your tests.
JUnit doesn't support this, you will have to use the standard Java work-arounds for singletons: Move the common setup code into a static code block and then call an empty method in this class:
static {
...init code here...
}
public static void init() {} // Empty method to trigger the execution of the block above
Make sure that all tests call init(), for example my putting it into a #BeforeClass method. Or put the static code block into a shared base class.
Alternatively, use a global variable:
private static boolean initialize = true;
public static void init() {
if(!initialize) return;
initialize = false;
...init code here...
}
Create one base class for all tests:
public class BaseTest {
static{
/*** init code here ***/
}
}
and every test should inherit from it:
public class SomeTest extends BaseTest {
}
You can make one BaseTest class with a #BeforeClass method, then have all the other tests inherit from it. This way, when each test object is constructed, #BeforeClass gets executed.
Also avoid executing it just once for all the test suite, since all the test cases should be independent. #BeforeClass should execute only once each test case, not test suite.
If you can tolerate adding spring-test to your project, or you are using it already, then a good approach is to use the technique described here: How to load DBUnit test data once per case with Spring Test
Not sure if anyone still is using JUnit and trying to fix it without using Spring Runner (aka no spring integration). TestNG has this feature. But here is a JUnit based solution.
Create a RunOnce per thread operation like so. This maintains a list of classes for which the operation has run.
public class RunOnceOperation {
private static final ThreadLocal t = new ThreadLocal();
public void run(Function f) {
if (t.get() == null) {
t.set(Arrays.asList(getClass()));
f.apply(0);
} else {
if (!((List) t.get()).contains(getClass())) {
((List) t.get()).add(getClass());
f.apply(0);
}
}
}
}
Back in your unit test
#Before
public beforeTest() {
operation.run(new Function<Integer, Void>() {
#Override
public Void apply(Integer t) {
checkBeanProperties();
return null;
}
});
}
private void checkBeanProperties() {
//I only want to check this once per class.
//Also my bean check needs instance of the class and can't be static.
}
My function interface is like this:
interface Function<I,O> {
O apply(I i);
}
When you use this way, you can perform operations once per class using ThreadLocal.
Let's say I have a test class called testFixtureA with several methods testA, testB, testC, etc, each with #Test annotation.
Let's now say I subclass testFixtureA into class called testFixtureAB and I don't overwrite anything. testFixtureAB is empty as for now.
When I run tests from testFixtureAB, methods testA, testB and testC are executed by test runner because test runner doesn't distinguish between test methods from class and baseclass.
How can I force test runner to leave out tests from baseclass?
and I don't overwrite anything.
testFixtureAB is empty as for now
There's your answer. If you want to not run testB from the main class, overrride it:
public class testFixtureAB extends testFixtureA {
#Override
public void testB() {}
}
Restructure your test classes.
If you don't want to use the tests from the baseclass, then don't extend it
If you need other functionality from the base class, split that class in two - the tests, and the other functionality
ignoring the whole base class:
#Ignore
class BaseClass {
// ...
}
check out this example
It's quite easy to achieve implementing some few classes:
Create your own TestRunner
Create an annotation like #IgnoreInheritedTests
Create a class that extends org.junit.runner.manipulation.Filter
On the filter class:
public class InheritedTestsFilter extends Filter {
#Override
public boolean shouldRun(Description description) {
Class<?> clazz = description.getTestClass();
String methodName = description.getMethodName();
if (clazz.isAnnotationPresent(IgnoreInheritedTests.class)) {
try {
return clazz.getDeclaredMethod(methodName) != null;
} catch (Exception e) {
return false;
}
}
return true;
}
#Override
public String describe() {
// TODO Auto-generated method stub
return null;
}
}
on your custom runner:
/**
* #param klass
* #throws InitializationError
* #since
*/
public CustomBaseRunner(Class<?> klass) throws InitializationError {
super(klass);
try {
this.filter(new InheritedTestsFilter());
} catch (NoTestsRemainException e) {
throw new IllegalStateException("class should contain at least one runnable test", e);
}
}
In Junit 5, you can make base class as abstract and extends it with a concrete class.
When you run the abstract in your IDE, your subclass will get executed instead.
In the latest JUnit you can use the #Rule annotation on the subclass to inspect the test name and intercept the test run to ignore the test dynamically. But I would suggest that #Bozho's idea is the better one - the fact that you need to do this indicates a bigger problem that probably shows inheritance is not the right solution here.
I know, it's not the answer...
Consider the reason why you extend concrete test classes. You do duplicate test methods that way.
If you share code between tests then consider writing base test classes with helper and fixture setup methods or test helper class.
If for running tests then try organizing tests with suites and categories.
What if you want to execute the same test for different configurations of the same test suite?
For example, let's say you have Class A with test1, test2 and test3 methods that hit an embedded database then you want to create separated "setUp" and "tearDown" for every embedded vendor (H2, HyperSQL, etc) but runs the same testing for each one.
I would like to extend a class that contain those test methods and configure it in a subclass. My problem is that the super class SHOULD NOT be considered as eligible for the test runner. The problem arises when the test runner executes the super class and given that don't found the corresponding setup and teardown methods, it crashs :(
In the base test class' #Test methods:
assumeTrue(getClass().equals(BaseClassTest.class));
It will ignore those in the subclass tests but not completely leave them out.
If for any reason you need two JUnit classes for same functionality, the best approach for me is:
put the common code in a parent class TestFixture with only constants and mocked services.
create two subclasses: TestFixtureA and TestFixtureB
This way you will not have duplicated code, nor double runs.
I ran into a similar issue where I had a generic abstract base class TestBase and 2 subclasses TestA, TestB that extended the base class. I wanted all tests from TestBase to run for TestA but to skip some for TestB. You can use the JUnit annotations Nested and Disabled to group tests in the base class into categories and then enable a full disable of that entire category of tests.
public class TestB extends TestBase {
// disable entire suite of inherited tests
// from the nested class declared on the parent
#Nested
#Disabled
public class ComponentCTests {}
}
public abstract class TestBase {
#Nested
public class ComponentCTests {
#Test
public void testSomething() {
// test something
}
}
}
I have some common set up code that I've factored out to a method marked with #Before. However, it is not necessary for all this code to run for every single test. Is there a way to mark it so the #Before method only runs before certain tests?
Just move out the tests that don't need the setup code into a separate test class. If you have some other code common to the tests that would be helpful to keep, move that out into a helper class.
#Nested + #BeforeEach
Totally agree with the point of moving the related code to an inner class. So here what I have done.
Create an inner class inside your test class
Annotate the inner class with #Nested
Move all the test methods you want to use in the inner class
Write the init code inside the inner class and annotate it with #BeforeEach
Here is the code:
class Testing {
#Test
public void testextmethod1() {
System.out.println("test ext method 1");
}
#Nested
class TestNest{
#BeforeEach
public void init() {
System.out.println("Init");
}
#Test
public void testmethod1() {
System.out.println("This is method 1");
}
#Test
public void testmethod2() {
System.out.println("This is method 2");
}
#Test
public void testmethod3() {
System.out.println("This is method 3");
}
}
#Test
public void testextmethod2() {
System.out.println("test ext method 2");
}
}
Here is the output
test ext method 1
test ext method 2
Init
This is method 1
Init
This is method 2
Init
This is method 3
Note: I am not sure if this is supported in Junit4. I am doing this in JUnit5
It is possible to achieve also via Assume from JUnit. And then you can check the method name for which you want to process #Before.
public class MyTest {
#Rule
public TestName testName = new TestName();
#Before
public void setUp() {
assumeTrue(testName.getMethodName().equals("myMethodName"));
// setup follows
}
}
Check the topic for more insights about #Rule.
Now that it's 2023, I'd recommend sticking with JUnit 5.x
I'd also say that this is probably a micro-optimization. I would not go to the effort until I measured my test time and saw that running the code when it wasn't necessary added a significant amount of time.
Not sure about #Before, but I recently came up with a strategy for #After block to run selectively. The implementation was straight forward. I have some flags set to default values as part of the test class. They are reset to default values in #Before class. In the class I need to do things specific to a flag, I set those flags & in #After I check for flag values to do the respective jobs.
JUnit 4.12 provide Enclosed Runner like
#RunWith(Enclosed.class)
public class GlobalTest{
#RunWith(MockitoJUnitRunner.class)
public class InnerTest{
}
}
I have an abstract base class, which I use as a base for my unit tests (TestNG 5.10). In this class, I initialize the whole environment for my tests, setting up database mappings, etc. This abstract class has a method with a #BeforeClass annotation which does the initialization.
Next, I extend that class with specific classes in which I have #Test methods and also #BeforeClass methods. These methods do class-specific initialization of the environment (e.g. put some records into the database).
How I can enforce a specific order of the #BeforeClass annotated methods? I need the ones from the abstract base class to be executed before the ones of the extending class.
Example:
abstract class A {
#BeforeClass
doInitialization() {...}
}
class B extends A {
#BeforeClass
doSpecificInitialization() {...}
#Test
doTests() {...}
}
Expected order:
A.doInitialization
B.doSpecificInitialization
B.doTests
Actual order:
B.doSpecificInitialization // <- crashes, as the base init is missing
(A.doInitialization // <---not executed
B.doTests) // <-/
edit: Answer below is for JUnit, but I will leave it here anyway, because it could be helpful.
According to the JUnit api: "The #BeforeClass methods of superclasses will be run before those the current class."
I tested this, and it seems to work for me.
However, as #Odys mentions below, for JUnit you need to have the two methods named differently though as doing otherwise will result in only the subclass method being run because the parent will be shadowed.
Don't put the #BeforeClass on the abstract class. Call it from each subclass.
abstract class A {
void doInitialization() {}
}
class B extends A {
#BeforeClass
void doSpecificInitialization() {
super.doInitialization();
}
#Test
void doTests() {}
}
Seems like TestNG has #BeforeClass(dependsOnMethods={"doInitialization"}) - give it a try.
I added public to the abstract class and TestNG (6.0.1) executed the doInitialization() before doTests. TestNG does not execute doInitialization() if I remove public from class A.
public abstract class A {
#BeforeClass
doInitialization() {...}
}
class B extends A {
#Test
doTests() {...}
}
I just tried your example with 5.11 and I get the #BeforeClass of the base class invoked first.
Can you post your testng.xml file? Maybe you are specifying both A and B there, while only B is necessary.
Feel free to follow up on the testng-users mailing-list and we can take a closer look at your problem.
--
Cedric
I've just gone through this and found one more way to achieve this. Just use alwaysRun on #BeforeClass or #BeforeMethod in the abstract class, works as you would expect.
public class AbstractTestClass {
#BeforeClass(alwaysRun = true)
public void generalBeforeClass() {
// do stuff
specificBeforeClass();
}
}
For JUnit:
As #fortega has mentioned:
According to the JUnit api: "The #BeforeClass methods of superclasses will be run before those the current class."
But be careful not to name both methods with the same name. Since in this case the parent method will be hidden by child parent. Source.
When I run from: JUnitCore.runClasses(TestClass.class);
It will execute the parent properly, before the child (You do not need super.SetUpBeforeClass();)
If you run it from Eclipse:
For some reason it fails to run the base class.
The work around:
Call the base class explicitely: (BaseTest.setUpBeforeClass();)
You may want to have a flag in the base class in case you run it from an application, to determine if it is already setup or not. So it only runs once if you run it via both possible methods (such as from eclipse for personal testing, and through ANT for a build release).
This appears to be a bug with Eclipse, or at least unexpected results..
How about having your #BeforeClass method call an empty specificBeforeClass() method that may or may not be overwritten by sub classes like so:
public class AbstractTestClass {
#BeforeClass
public void generalBeforeClass() {
// do stuff
specificBeforeClass();
}
protected void specificBeforeClass() {}
}
public class SpecificTest {
#Override
protected void specificBeforeClass() {
// Do specific stuff
}
// Tests
}
dependsOnMethod can be used.
e.g. in case of Spring (AbstractTestNGSpringContextTests)
#BeforeClass(alwaysRun = true, dependsOnMethods = "springTestContextPrepareTestInstance")
Check your import statement.
It should be
import org.testng.annotations.BeforeClass;
not
import org.junit.BeforeClass;
This works for me --
abstract class A {
#BeforeClass
doInitialization() {...}
}
class B extends A {
#Override
#BeforeClass
doInitialization() {
//do class specific init
}
#Test
doTests() {...}
}
Why don't you try to create an abstract method doSpecialInit() in your super class, called from your BeforeClass annotated method in superclass.
So developpers inheriting your class is forced to implement this method.
There is another easy solution here.
My particular situation is that I need to inject mock services from "BeforeClass" in the subclass before "BeforeClass" in the superclass is executed.
To do this - simply use a #ClassRule in the subclass.
For example:
#ClassRule
public static ExternalResource mocksInjector = new ExternalResource() {
#Override
protected void before() {
// inject my mock services here
// Note: this is executed before the parent class #BeforeClass
}
};
I hope this helps. This can effectively execute static setup in "reverse" order.
I've faced a similar issue today, the only difference was a Base class was not abstract
Here's my case
public class A {
#BeforeClass
private void doInitialization() {...}
}
public class B extends A {
#BeforeClass
private void doSpecificInitialization() {...}
#Test
public void doTests() {...}
}
It occurred that a #BeforeClass method from class A was never executed.
A.doInitialization() -> THIS WAS NEVER EXECUTED silently
B.doSpecificInitialization()
B.doTests()
Playing with privacy modifiers I found that TestNG will not execute a #BeforeClass annotated method from inherited class if a method is not visible from a class-inheritor
So this will work:
public class A {
#BeforeClass
private void doInitialization() {...}
}
public class B extends A {
#BeforeClass
//Here a privacy modifier matters -> please make sure your method is public or protected so it will be visible for ancestors
protected void doSpecificInitialization() {...}
#Test
public void doTests() {...}
}
As a result following happens:
A.doInitialization()
B.doSpecificInitialization()
B.doTests()
In my case (JUnit) I have the same methods called setup() in the base class and the derived class. In this case only the derived class's method is called, and I have it call the base class method.
A better and cleaner way to achieve this using inheritance may be as following -
abstract class A {
#BeforeClass
void doInitialization() {}
}
class B extends A {
#Override
#BeforeClass
void doInitialization() {
super.doInitialization();
}
#Test
void doTests() {}
}