I'm trying to figure out a good solution to having specific unit tests run with certain runtime configurations. For example:
public class TestClassAlpha() {
#setup
public void setup() {
}
#After
public void tearDown() {
}
#Test
#<only run in particular env>
public void testA() {
//whatever A
}
//always run below test no mater what env
#Test
public void testB() {
//whatever B
}
}
I am contemplating a custom annotation or custom rule perhaps, but i thought this has to be a question that comes up frequently as running tests in certain conditions (envs) is a very valid scenario. I did some limited searching within stack, and I didn't find anything that really that helped solidify either way.
This post shows you exactly what you require.
You should Write a Custom TestRule and an annotation to mark the condition.
Related
I have a test class containing test cases for say blue and non-blue devices. If the parameter isBlue is set then it should run only those test case that have say the #Blue annotation. I am not sure how to implement this specific annotation that will achieve this functionality.
public class TestClass {
boolean isBlue = false;
#Before
public void setUp () {
isBlue = MyApplication.instance().isBlue();
}
#Test
public void testA () { ... }
#Test #Blue
public void testB() { ... }
}
In this example if isBlue is true then it should only run testB() otherwise it should run all test cases
Take a look at JUnit runners. Nice explanation of runners
You can create your own runner which will extend on BlockJUnit4ClassRunner.
You can override
#Override
protected List<FrameworkMethod> getChildren() {
// scan test class for methonds annotated with #Test
}
To additionally filter by methods that have your custom annotation set, and your property in the application is set as well.
I think you are on the wrong path.
You want your test cases to help identifying problems; and assist you in fixing them. Anything that "distracts" you from that purpose reduces the values of your unit tests.
Thus: don't put tests that have really "different" scope into the same test class. Especially from the point of view: how would you control what "MyApplication.instance()" would return? That sounds like a problem in itself.
Meaning: you want to have two independent tests; and those don't rely on some "static" object telling them something. They test what is in their scope; nothing else.
By using jUnit Assume
#Test
public void testA () {
assumeTrue(MyApplication.instance().isBlue());
...
}
A failing assumption in a #Before or #BeforeClass method will have the same effect as a failing assumption in each #Test method of the class.
i don't know if my question was clear, but i am using testNG and i have this:
#Test
public void passengerServiceTest() {
...
}
#AfterTest
public void deleteCreatedPassenger() {
...
}
I want to execute my deleteCreatedPassenger() method after passengerServiceTest, also, i want that in case of deleteCreatedPassenger fails, passengerServiceTest fails too, in other words, i want that both of them be the same test, so if one them fails, test fails.
So i tried with the annotations #AfterTest, #AfterMethod, #AfterClass and all make two tests as "separated" tests.
Do you know how to do this? Regards
You don't need annotations to achieve this, since it's exactly what the finally block is intended for:
#Test
public void passengerServiceTest() {
try {
//test code
} finally {
deleteCreatedPassenger();
}
}
public void deleteCreatedPassenger() {
...
}
If the delete throws an exception then your service test fails.
Annotations are useful in certain scenarios, you shouldn't aim to use them over core language constructs.
alwaysRun
https://testng.org/doc/documentation-main.html
Ex:
#AfterTest(alwaysRun = true)
public void deleteOldValuesFromDatabase() {
...
}
Docs say that this will cause the method to run "even if one or more methods invoked previously failed or was skipped"
At the moment I have the following code:
public class AlterServiceSelT
{
#Before
public void setupAndActivate()
{
System.out.println("setupAndActivate");
}
#Test
public void suspendService()
{
System.out.println("suspendService");
}
#Test
public void reActivateService()
{
System.out.println("reActivateService");
}
#After
public void terminateService()
{
System.out.println("terminateService");
}
}
and when ran I get the following in my console:
setupAndActivate
reActivateService
terminateService
setupAndActivate
suspendService
terminateService
The problem is that the full code for setupAndActivate() takes 15 minutes and its output is needed to run for the tests. Ideally I would like the console to output:
setupAndActivate
reActivateService
suspendService
terminateService
How could this be done?
Try looking at #BeforeClass in stead of using the #Before.
One of the downsides of BeforeClass is that it has to be defined on a static method, so all your fields that you set up have to be static.
Upside is that your setup is only done once for all the tests in your class.
Are you sure that a setup of 15 minutes is optimum for your application?
Is it possible to run a JUnit #Test method in a class that has a method annotated with #Before, but to ignore the #Before method only for this test?
Edit: I am interested if JUnit supports this functionality, not workarounds. I am aware of workarounds like moving the test(s) in another class or removing the annotation and manually calling setUp() in each test method.
Suppose in a class there are 30 tests, and for 29 of them #Before really simplifies the testing initialization, but for one (or more than one) of them is useless/it complicates things.
public class MyTestClass {
#Before
public void setUp() {
//setup logic
}
#Test
public void test1() {
//[...]
}
#Test
public void test2() {
//[...]
}
//more tests here
#Test(ignoreBefore = true, ignoreAfter = true //false by default)
//something equivalent to this
public void test20() {
//[...]
}
}
You can do this with a TestRule. See my answer to Exclude individual test from 'before' method in JUnit. Basically, implement ExternalResource, and in the apply method, check if there is a specific annotation on the method, and if there is, don't run the before/after method. You'll have to specifically call the before/after from your rule though.
If it useless it should not be a problem - does it harm to run the setUp once more?
However I don't think it's possible and looks for me as a cripple feature.
Another approach - move that test to a separate test-class.
With JUnit 5 You can have nested tests using #Nested annotation :
public class MainClass {
#Nested
class InnerClass1 {
#BeforeEach
void setup(){}
#Test
void test1(){}
}
#Nested
class InnerClass2 {
// No setup
#Test
void test2(){}
}
}
Just to add to this, something that I found out on a recent visit to this area.
Junit/Mockito seems to give preference to mocking behavior defined in a test, over a #Before annotated method.
OK, so the #Ignore annotation is good for marking that a test case shouldn't be run.
However, sometimes I want to ignore a test based on runtime information. An example might be if I have a concurrency test that needs to be run on a machine with a certain number of cores. If this test were run on a uniprocessor machine, I don't think it would be correct to just pass the test (since it hasn't been run), and it certainly wouldn't be right to fail the test and break the build.
So I want to be able to ignore tests at runtime, as this seems like the right outcome (since the test framework will allow the build to pass but record that the tests weren't run). I'm fairly sure that the annotation won't give me this flexibility, and suspect that I'll need to manually create the test suite for the class in question. However, the documentation doesn't mention anything about this and looking through the API it's also not clear how this would be done programmatically (i.e. how do I programatically create an instance of Test or similar that is equivalent to that created by the #Ignore annotation?).
If anyone has done something similar in the past, or has a bright idea of how else I could go about this, I'd be happy to hear about it.
The JUnit way is to do this at run-time is org.junit.Assume.
#Before
public void beforeMethod() {
org.junit.Assume.assumeTrue(someCondition());
// rest of setup.
}
You can do it in a #Before method or in the test itself, but not in an #After method. If you do it in the test itself, your #Before method will get run. You can also do it within #BeforeClass to prevent class initialization.
An assumption failure causes the test to be ignored.
Edit: To compare with the #RunIf annotation from junit-ext, their sample code would look like this:
#Test
public void calculateTotalSalary() {
assumeThat(Database.connect(), is(notNull()));
//test code below.
}
Not to mention that it is much easier to capture and use the connection from the Database.connect() method this way.
You should checkout Junit-ext project. They have RunIf annotation that performs conditional tests, like:
#Test
#RunIf(DatabaseIsConnected.class)
public void calculateTotalSalary() {
//your code there
}
class DatabaseIsConnected implements Checker {
public boolean satisify() {
return Database.connect() != null;
}
}
[Code sample taken from their tutorial]
In JUnit 4, another option for you may be to create an annotation to denote that the test needs to meet your custom criteria, then extend the default runner with your own and using reflection, base your decision on the custom criteria. It may look something like this:
public class CustomRunner extends BlockJUnit4ClassRunner {
public CTRunner(Class<?> klass) throws initializationError {
super(klass);
}
#Override
protected boolean isIgnored(FrameworkMethod child) {
if(shouldIgnore()) {
return true;
}
return super.isIgnored(child);
}
private boolean shouldIgnore(class) {
/* some custom criteria */
}
}
Additionally to the answer of #tkruse and #Yishai:
I do this way to conditionally skip test methods especially for Parameterized tests, if a test method should only run for some test data records.
public class MyTest {
// get current test method
#Rule public TestName testName = new TestName();
#Before
public void setUp() {
org.junit.Assume.assumeTrue(new Function<String, Boolean>() {
#Override
public Boolean apply(String testMethod) {
if (testMethod.startsWith("testMyMethod")) {
return <some condition>;
}
return true;
}
}.apply(testName.getMethodName()));
... continue setup ...
}
}
A quick note: Assume.assumeTrue(condition) ignores rest of the steps but passes the test.
To fail the test, use org.junit.Assert.fail() inside the conditional statement. Works same like Assume.assumeTrue() but fails the test.