java testng retry logic - java

I have written a code for retrying a failed test case in selenium web driver and java.
How should we enhance the script to hold only one record in the test case output even the same test case as been executed multiple times.
I don't want the test output report to contain any redundant result entries.
Code:
Retry Logic :
package tests;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class Retry implements IRetryAnalyzer {
private int retryCount = 0;
private int maxRetryCount = 2; // retry a failed test 2 additional times
#Override
public boolean retry(ITestResult result) {
if (retryCount <maxRetryCount) {
retryCount++;
return true;
}
return false;
}
}
Implementation in the class
#Test(retryAnalyzer=Retry.class)
public void example() throws CustomException
{
throw new CustomException("Example");
}
Please let me know what changes are needed.
Thanks and Regards
Sushanth.G

I had the same problem before.
We used jenkins to run CI, and need to make sure all test case results were SUCCESS (even after some times of retry) then we can deploy build.
The solution is adding TestListenerAdapter to re-write test result status to SKIP if this test will be run again.
public class MyTestListenerAdapter extends TestListenerAdapter {
#Override
public void onTestFailure(ITestResult result) {
if (result.getMethod().getRetryAnalyzer() != null) {
MyRetryAnalyzer retryAnalyzer = (MyRetryAnalyzer)result.getMethod().getRetryAnalyzer();
if(retryAnalyzer.isRetryAvailable()) {
result.setStatus(ITestResult.SKIP);
} else {
result.setStatus(ITestResult.FAILURE);
}
Reporter.setCurrentTestResult(result);
}
}
}
RetryAnalyzer needs to provide another method (isRetryAvailable() in this example) for TestListenerAdapter.
public class MyRetryAnalyzer implements IRetryAnalyzer {
private static int MAX_RETRY_COUNT = 3;
AtomicInteger count = new AtomicInteger(MAX_RETRY_COUNT);
public boolean isRetryAvailable() {
return (count.intValue() > 0);
}
#Override
public boolean retry(ITestResult result) {
boolean retry = false;
if (isRetryAvailable()) {
System.out.println("Going to retry test case: " + result.getMethod() + ", " + (MAX_RETRY_COUNT - count.intValue() + 1) + " out of " + MAX_RETRY_COUNT);
retry = true;
count.decrementAndGet();
}
return retry;
}
}
Now we can add this TestListenerAdapter to test class.
#Listeners({MyTestListenerAdapter.class})
public class AppTest {
#Test(retryAnalyzer=MyRetryAnalyzer.class)
public void testSum(){
MyClass c = new MyClass();
Assert.assertEquals(c.sum(2, 3), 5);
}
}
Please see full example here: https://github.com/haojiwu/testng-retry-example

Related

Mocking/skipping sleep calls in unit tests

What is the general approach in unit tests for methods that involve a sleep() call?
Lets say I have the class:
class Foo {
public bool someMethodWithSleep() {
bool failed = false;
for (int i = 0 : 5) {
failed = trySomething(); // mocked to return false
if (failed) {
TimeUnit.SECONDS.sleep(5);
} else { return true; }
}
return false;
}
}
In test cases for failures that call this function (directly or nested), I dont want it to sleep for 25+ seconds and slowing every test file that calls it. What's the best approach to skip the sleep calls in unit tests?
Replace it with a dependency.
class Foo {
private final Sleeper sleeper;
public Foo(final Sleeper sleeper) {
this.sleeper = sleeper;
}
public Foo() {
this(SecondsSleeper.INSTANCE);
}
public bool someMethodWithSleep() {
bool failed = false;
for (int i = 0 : 5) {
failed = trySomething(); // mocked to return false
if (failed) {
sleeper.sleep(5);
} else { return true; }
}
return false;
}
}
interface Sleeper {
void sleep(int duration);
}
enum SecondsSleeper implements Sleeper {
INSTANCE;
void sleep(final int seconds) {
TimeUnit.SECONDS.sleep(seconds);
}
}
enum TestSleeper implements Sleeper {
INSTANCE;
void sleepSeconds(final int duration) {
// do nothing
}
}
Then in your real application, create Foo as new Foo() and in your test as new Foo(TestSleeper.INSTANCE)

How to run class if test case failed in testNG

I have an issue that I am facing,
I have a class with multiple test cases and I am using test ng with java and selenium.
Is there a possibility that if a test case failed, testNG will run the entire class again?
not the test.
since there are a priority and navigation between pages.
Is there a way to run the entire class that is failed again? I just saw to rerun the test, and it is useless to me.
regards
If I understood you right this is the solution which you needed
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class Retry implements IRetryAnalyzer {
private int count = 0;
private static int maxTry = 3;
public boolean retry(ITestResult iTestResult) {
if (!iTestResult.isSuccess()) {
if (count < maxTry) {
count++;
iTestResult.setStatus(ITestResult.SUCCESS);
return true;
} else {
iTestResult.setStatus(ITestResult.FAILURE);
}
} else {
iTestResult.setStatus(ITestResult.FAILURE);
}
return false;
}
}
Then your all tests methods should be like this:
#Test(retryAnalyzer=Retry.class)
And you should add the BeforeSuite
#BeforeSuite(alwaysRun = true)
public void beforeSuite(ITestContext context) {
for (ITestNGMethod method : context.getAllTestMethods()) {
method.setRetryAnalyzerClass(Retry.class);
}
}

ERROR with java.lang.Exception: Test class should have exactly one public constructor

I am getting an exception when I run my JUnit tests.
java.lang.Exception: Test class should have exactly one public constructor
Below is my code snippet, any ideas?
package com.tests;
import org.junit.Rule;
import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
public class AllTests implements TestRule {
private int retryCount;
private AllTests(int retryCount) {
this.retryCount = retryCount;
}
public Statement apply(Statement base, Description description) {
return statement(base, description);
}
private Statement statement(final Statement base, final Description description) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
Throwable caughtThrowable = null;
// implement retry logic here
for (int i = 0; i < retryCount; i++) {
try {
base.evaluate();
return;
}
catch (Throwable t) {
caughtThrowable = t;
System.err.println(description.getDisplayName() + ": run " + (i + 1) + " failed");
}
}
System.err.println(description.getDisplayName() + ": giving up after " + retryCount + " failures");
throw caughtThrowable;
}
};
}
#Rule
public AllTests allTests = new AllTests(3);
#ClassRule
public static DockerComposeRule docker = wi.getDocker(logs);
#Override
public DockerComposeRule getDocker() {
return docker;
}
#Test
public void myFirstTest() throws Exception {
// Test code here...
}
}
I am running a JUnit test, with Gradle. It is a Java project and it fails straightaway.
I have tried numerous things but to no avail. Happy to provide more details if you want.
The message is clear :
java.lang.Exception: Test class should have exactly one public
constructor
You merged the unit test class with the Rule test class in the same class : AllTests.
You should define your rule in its own class and remove this constructor :
private AllTests(int retryCount) {
this.retryCount = retryCount;
}

Pattern of solution for executing and operation that depends upon multiple conditions

Let us say there are 3 operations ops1(), ops2() and ops3(). The client can request to execute any combination of those 3. For e.g.
perform(1): Should execute ops1()
perform(2): Should execute ops2()
perform(1, 2): Should execute ops1() and if ops1() is successful then execute ops2()
perform(1, 2, 3): Should execute ops1() and if ops1() is successful then execute ops2() and if both ops1() and ops2() are successful then execute ops3()
This can go on for n ops() though for me its just 5.
What is the simple and elegant way of implementing this? Is there a pattern for this?
How about you put your ops in a list, look the operations to perform up in that list, and let the operations throw an exception if they fail? Then the perform method can simply try and perform all methods in the desired order until it's done or an exception occurs.
So
private List<Callable> ops;
public void perform(int... opNums) {
try {
for (int i : opNums) {
ops.get(i-1).call();
}
}
catch(Exception ex) {
}
}
I see a solution somewhat like this:
public void perform(int... ops) {
for(int i : ops) {
switch(i) {
case 1:
//...
// if(taskFailed) return;
break;
case 2:
//...
// if(taskFailed) return;
break;
case 3:
//...
// if(taskFailed) return;
break;
// so on for all 5
}
}
}
It's just the general idea, not tested if the syntax is entirely correct.
the "taskFailed" thing is a pseudocode.
An approach for this would be
Define a common interface for opsX methods and the classes that implement this method.
Define an enum to know which class implementation of this common interface should be called.
Define a class that will work as an orchestrator for these calls.
An implementation of this design may be
interface CommonOps {
boolean ops();
}
class Ops1 implements CommonOps {
#Override
public boolean ops() {
//...
}
}
class Ops2 implements CommonOps {
#Override
public boolean ops() {
//...
}
}
//and on...
enum OpsOrder {
OPS1,
OPS2,
OPS3
//... and on
;
}
class Orchestrator {
public boolean executeOps(OpsOrder order) {
switch (order) {
case OPS1:
return new Ops1().ops();
case OPS2:
return new Ops2().ops();
//...
default:
throw new IllegalArgumentException("Not supported.");
}
throw new UnsupportedOperationException("This exception should never be reached.");
}
public boolean orchestrate(OpsOrder ... orders) {
for (OpsOrder order : orders) {
if (!executeOps(orders)) {
return false;
}
}
return true;
}
}
This can be even more generic by having a factory of CommonOps class implementations so Orchestrator should not know which CommonOps will be called:
final class CommonOpsFactory {
private CommonOpsFactory () { }
public static CommonOps create(OpsOrder order) {
switch (order) {
case OPS1:
return new Ops1();
case OPS2:
return new Ops2();
//...
default:
throw new IllegalArgumentException("Not supported.");
}
}
}
class Orchestrator {
public boolean executeOps(OpsOrder order) {
return CommonOpsFactory.create(order).ops();
}
public boolean orchestrate(OpsOrder ... orders) {
for (OpsOrder order : orders) {
if (!executeOps(orders)) {
return false;
}
}
return true;
}
}
I would use the command pattern in combination with the Decorator for this problem. Your commands, when many, will be wrapping/decorating each others :
public class Command{
private Command subCommand;
public Command(Command subCommand){
this.subCommand=subCommand;
}
public Command(){};
public Command addSubCommand(Command command)
{
subCommand=command;
return command;
}
//A Command class is decorating itself
//by adding a behavior over its subcommand
public void execute() throws CommandExecutionException {
executeImpl();
if(subCommand!=null) subCommand.execute();
}
protected void executeImpl() throws CommandExecutionException {
//To be overiden
}
}
public class CommandA extends Command{
private CommandAExecutor ops1Handler;
protected void executeImpl(){
ops1Handler.ops1();
}
}
//....
public class CommandN extends Command{
private CommandNExecutor opsNHandler;
protected void executeImpl(){
opsNHandler.opsN();
}
}
public class Tester{
public static void main(String[] args){
Command commandA = new CommandA(new CommandAExecutor());
Command commandB = new CommandB(new CommandBExecutor());
Command commandN = new CommandN(new CommandNExecutor());
//The order here is A, N, B
commandA.addSubCommand(commandN).addSubCommand(B);
try{
commandA.execute();
}catch(CommandExecutionException e){
//...failure
}
}
}

Custom BlockJUnit4ClassRunner which runs test suite set number of times

I made the following custom BlockJUnit4ClassRunner
public class RepeatEachTest extends BlockJUnit4ClassRunner {
private int repeats;
public RepeatEachTest(Class<?> klass) throws InitializationError {
super(klass);
Repeat r = klass.getAnnotation(Repeat.class);
if (r == null) {
throw new InitializationError("A #Repeat annonation must also be suplied to class, for example #Repeat(5) to repeat 5 times");
}
repeats = r.value();
}
#Override
protected void runChild(FrameworkMethod method, RunNotifier notifier) {
for (int i = 0; i < repeats; i++) {
super.runChild(method, notifier);
}
}
#Override
public int testCount() {
return repeats * super.testCount();
}
}
and
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.TYPE)
public #interface Repeat {
int value();
}
to execute each test #Repeat.value() number of times. A test run of
#RunWith(RepeatEachTest.class)
#Repeat(2)
public class RepeatEachTestTest {
#Test
public void first() {
System.out.println("ran first");
}
#Test
public void second() {
System.out.println("ran second");
}
}
looks like
ran first
ran first
ran second
ran second
but now I want to implement a second BlockJUnit4ClassRunner which runs the whole test class #Repeat.value() number of times. A run from that setup would look like
ran first
ran second
ran first
ran second
Any thoughts?
That depends on what you want. If you want the #BeforeClass and #AfterClass methods and class rules to be called multiple times, you can override classBlock():
protected Statement classBlock(final RunNotifier notifier) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
for (int i = 0; i < repeats; i++) {
super.classBlock(notifier).evaluate();
}
}
};
}
If you want the #BeforeClass and #AfterClass methods and class rules to be called once, override childrenInvoker() (the code would be similar).
Note, however, that either of these will result in the listeners being notified multiple times that the test has started and completed. Some listeners may not behave correctly in this situation.

Categories

Resources