Develop Reference

Testing an abstract class with Mockito does not give the expected result

I have a class structure similar to the following
public abstract class AbstractStep {
private final Range RANGE;
AbstractStep(AbstractStepBuilder builder) {
RANGE = builder.range;
}
public abstract static class AbstractStepBuilder {
Range range;
public AbstractStepBuilder setRange(int start, end end) {
this.range = new Range(start, end);
return self();
}
abstract AbstractStepBuilder self();
}
public static class Range() {
private final int START;
private final int END;
private Range(int start, int end) {
if(start < 0 || end < 0 || start >= end)
throw new IllegalArgumentException();
START = start;
END = end;
}
}
}
I want to test setRange(int, int) in AbstractStepBuilder to see if the an IllegalArgumentException is thrown. I use TestNG and Mockito, and I have attempted the following using with the help of this.
final class RangeTest {
AbstractStepBuilder builder;
#BeforeSuite
void setup() {
builder = Mockito.mock(AbstractStepBuilder.class);
Mockito.when(builder.self()).thenReturn(null);
}
#Test(expectedExceptions = IllegalArgumentException.class)
final void testCreatingRangeWithNegativeStart() {
builder.setRange(-1, 2);
}
}
This test fails. I have also tried replacing Mockito.mock(AbstractStepBuilder.class) with Mockito.mock(AbstractStepBuilder.class, Mockito.CALLS_REAL_METHODS) as in the top answer of this question.
Note that if I make CodeRange as its own outer class, this test passes, so I do not believe it could be the test itself.
Why is this test failing, and is it possible to fix it without having to use a concrete class in the test instead?

You're calling a method on a mock, that will never throw an Exception until you tell it to. You never mock a class you want to test.
If you want to test the actual class, you'll need to create a subclass of the step builder, create an instance and test that.
I think you can also create a spy (by Mockito.spy(AbstractStepBuilder.class)) to avoid creating a subclass just for the test.

Retry failed tests is not working in TestNg 7.0.0?

If I run the tests on the version testng 6.14.3 , then the dropped tests are restarted.
If I run the tests on the version testng 7.0.0 , then the dropped tests aren't restarted.
public class RetryAnalyzer implements IRetryAnalyzer {
private int count = 0;
private static int maxTry = 2;
#Override public boolean retry(ITestResult iTestResult) {
if (!iTestResult.isSuccess()) {
if (count < maxTry) {
count++;
iTestResult.setStatus(ITestResult.FAILURE);
iTestResult.getTestContext().getFailedTests().removeResult(iTestResult);
return true;
} else {
iTestResult
.setStatus(ITestResult.FAILURE);
}
} else {
iTestResult
.setStatus(ITestResult.SUCCESS);
}
return false;
}
}
public class AnnotationTransformer implements IAnnotationTransformer {
#Override
public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor,
Method testMethod) {
annotation.setRetryAnalyzer(RetryAnalyzer.class);
}
}
My listener:
<listeners>
<listener class-name="package.AnnotationTransformer"/>
</listeners>

I have tried your implementation and it perfectly works for me (used TestNG 7.0.0). Ensure that imports and class-paths are correct (double-check <listener class-name="package.AnnotationTransformer"/>)`.
Also, consider the following, please:
There is no need to override boolean retry() method of IRetryAnalyzer interface. The logic of returning a result (a boolean) dependent on the counter is already defined by TestNG RetryAnalyzerCount class (it also implements IRetryAnalyzer interface). Thus, it is better to re-use the code of RetryAnalyzerCount (with the same logic as yours):
a) by setting the count for maximum retry attempts from within RetryAnalyzer class and
b) by overriding abstract boolean retryMethod(ITestResult var1); from TestNG RetryAnalyzerCount class, which returns a positive boolean result (true) if a test failed.
import java.util.concurrent.atomic.AtomicInteger;
import org.testng.ITestResult;
import org.testng.util.RetryAnalyzerCount;
import lombok.extern.slf4j.Slf4j;
/**
* Used by RetryListener (or AnnotationTransformer) to avoid the declaration of 'retryAnalyzer' in each #Test annotation
*/
#Slf4j
public class RetryAnalyzer extends RetryAnalyzerCount {
private static final int MAX_RETRY_ATTEMPTS = 3;
private AtomicInteger counter = new AtomicInteger(1); //used only for logging purposes
public RetryAnalyzer() {
setCount(MAX_RETRY_ATTEMPTS);
}
#Override
public boolean retryMethod(ITestResult result) {
// log example: [15/04/20 13:31] WARN [RetryAnalyzer] RETRY failed test 'displaySearchResultForValidInput' (1 out of 3 times)
String methodName = result.getMethod().getMethodName();
log.warn("RETRY failed test '{}' ({} out of {} times)",
methodName,
this.counter.getAndIncrement(),
MAX_RETRY_ATTEMPTS);
// enough is only the return statement
return true;
}
}
Leave AnnotationTransformer as it is.
Ensure <listeners> tag in TestNG .xml file includes correct path to the listener (AnnotationTransformer).
OR simplify everything:
a) by deleting AnnotationTransformer and <listeners> from TestNG .xml file and
b) by setting custom RetryAnalyzer for TestNG in #BeforeSuite (!!! not #BeforeMethod, which leads to an infinite loop)
#BeforeSuite
void beforeSuiteSetUp(ITestContext testsContext) {
setCustomRetryAnalyzer(testsContext);
}
private void setCustomRetryAnalyzer(ITestContext testsContext) {
for (ITestNGMethod method : testsContext.getAllTestMethods()) {
method.setRetryAnalyzerClass(RetryAnalyzer.class);
}
}

Cucumber+TestNG+Cucable plugin: How can I pass the appium driver instance that I create in the #beforemethod instance to the page objects?

Here is what I am trying to build:
Cucumber based tests with TestNG Executor
TestNG Executor so that I can re-run failed tests
Cucable plugin - so that the scenarios are split into individual files and runner is auto generated during run time for each of the scenarios. (Enables parallel execution)
Below is the test runner:
#CucumberOptions(
glue = "com.fifa.stepdefs",
features = {"target/parallel/features/[CUCABLE:FEATURE].feature"},
plugin = {"json:target/cucumber-report/[CUCABLE:RUNNER].json"}
)
public class CucableJavaTemplate implements IRetryAnalyzer {
private int count = 0;
private static int maxTry = 3;
#Override
public boolean retry(ITestResult iTestResult) {
if (!iTestResult.isSuccess()) { ;//Check if test not succeed
if (count < maxTry) { //Check if maxtry count is reached
count++; //Increase the maxTry count by 1
iTestResult.setStatus(ITestResult.FAILURE); //Mark test as failed
return true; //Tells TestNG to re-run the test
} else {
iTestResult.setStatus(ITestResult.FAILURE); //If maxCount reached,test marked as failed
}
} else {
iTestResult.setStatus(ITestResult.SUCCESS); //If test passes, TestNG marks it as passed
}
return false;
}
private TestNGCucumberRunner testNGCucumberRunner;
#BeforeClass(alwaysRun = true)
public void setUpClass() throws Exception {
System.out.println("Before Scenario ****");
testNGCucumberRunner = new TestNGCucumberRunner(this.getClass());
}
#Test(groups = "cucumber", description = "Runs Cucumber Scenarios", dataProvider = "scenarios",retryAnalyzer = CucableJavaTemplate.class)
public void scenario(PickleEventWrapper pickleEvent, CucumberFeatureWrapper cucumberFeature) throws Throwable {
testNGCucumberRunner.runScenario(pickleEvent.getPickleEvent());
}
#DataProvider
public Object[][] scenarios() {
return testNGCucumberRunner.provideScenarios();
}
#AfterClass(alwaysRun = true)
public void tearDownClass() throws Exception {
System.out.println("After Scenario ****");
testNGCucumberRunner.finish();
}
}
If I create a Driver instance in the before class: How can I pass it to page object files or step def files?

I gave similar answer on this question:
How to match a specific java file with specific feature file in cucumber
If you have annotated the driver class with #ScenarioScoped then you can easily use Guice to inject this instance in your other classes.

Writing a JUnit Rule: Description doesn't contain my annotation

I'm trying to write a simple JUnit Rule implementation which reruns a test case a given amount of times if not successful.
It works fine as such, but I'd like to make it configurable per method with a custom annotation I attach to the method.
Here's my rule implementation:
public class Retry implements TestRule {
private int retryCount = 10;
#Override
public Statement apply(Statement base, Description description) {
return new Statement() {
public void evaluate() throws Throwable {
RetryCount annotation = description.getAnnotation(RetryCount.class);
// Problem is here, the annotation is always null!
int retries = (annotation != null) ? annotation.retries() : retryCount;
// keep track of the last failure to include it in our failure later
AssertionError lastFailure = null;
for (int i = 0; i < retries; i++) {
try {
// call wrapped statement and return if successful
base.evaluate();
return;
} catch (AssertionError err) {
lastFailure = err;
}
}
// give meaningful message and include last failure for the
// error trace
throw new AssertionError("Gave up after " + retries + " tries", lastFailure);
}
};
}
// the annotation for method-based retries
public static #interface RetryCount {
public int retries() default 1;
}
}
In the line I commented, I don't get the annotation I attach to the method:
public class UnreliableServiceUnitTest {
private UnreliableService sut = new UnreliableService();
#Rule
public Retry retry = new Retry();
#Test
#RetryCount(retries=5) // here it is
public void worksSometimes() {
boolean worked = sut.workSometimes();
assertThat(worked, is(true));
}
}
If I debug into the Rule, the Description annotation list contains the #Test annotation but not the #RetryCount. I also tried adding a #Deprecated which will also get added.
Any idea why?
For completeness, this is the sample SUT:
public class UnreliableService {
private static Random RANDOM = new Random();
// needs at least two calls
private static int COUNTER = RANDOM.nextInt(8) + 2;
public boolean workSometimes() {
if (--COUNTER == 0) {
COUNTER = RANDOM.nextInt(8) + 2;
return true;
}
return false;
}
}

The #Test annotation is a Runtime annotation. Your RetryCount is not defined like that. It should be so you can access it during runtime. Change your code to this:
// the annotation for method-based retries
#Retention(value=RUNTIME)
public static #interface RetryCount {
public int retries() default 1;
}
Using RetentionPolicy Runtime allows you to read the annotations reflectively. See here the Javadoc

How to create own annotation for junit that will skip test if concrete exception was thrown during execution?

My application have several execution modes, and in 1 mode it is normal that some of my tests will throw a concrete exception. I need to annotate this methods with something like #SkipOnFail that will set method as skipped if exception was thrown.
thanks in advance!
#Edit(for my question to be more clear)
#Test(expected=ConcreteException.class)
does not work for me because i need my tests to pass even if ConcreteException.class was not thrown(expected tag in junit will mark my test as failed if this exception won't be thrown), and to be skipped otherwise. In all other cases it should work as always.
#Solution that worked for me(junit v4.7) thx to #axtavt
#Rule
public MethodRule skipRule = new MethodRule() {
public Statement apply(final Statement base, FrameworkMethod method, Object target) {
if(method.getAnnotation(SkipOnFail.class) == null) return base;
return new Statement() {
#Override
public void evaluate() throws Throwable {
try{
base.evaluate();
} catch (ConcreteException e) {
Assume.assumeTrue(false);
}
}
};
}
};
#Thx

I don't think that such a feature is available out of the box, but it should be pretty easy to implement with custom TestRule and Assume, something like this:
#Rule
public TestRule skipRule = new TestRule() {
public Statement apply(final Statement base, Description desc) {
if (desc.getAnnotation(SkipOnFail.class) == null) return base;
return new Statement() {
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (MyExceptoion ex) {
Assume.assumeTrue(false);
}
}
};
}
};

What about using JUnit Extensions?
The following example is taken from their Tutorial.
It provides aditional annotations for Prerequisites (#Prerequisite): Ignore tests based on conditions.
The required approach would be to check this during running tests. So you can simply add a #Prerequisite(requires="") annotation.
public class TestFillDatabase {
#Prerequisite(requires = "databaseIsAvailable")
#Test public void fillData() {
// ...
}
public boolean databaseIsAvailable() {
boolean isAvailable = ...;
return isAvailable;
}
}
public class TestFillDatabase {
#Prerequisite(requires = "databaseIsAvailable")
#Test public void fillData() {
// ...
}
public boolean databaseIsAvailable() {
boolean isAvailable = ...;
return isAvailable ;
}
}
This specified methods with #Prerequisite(requires = "databaseIsAvailable") must be a public method, returning a boolean or Boolean value.
If these methods will be consolidated in helper classes, you can also specify static methods within a class to be called using #Prerequisite(requires = "databaseIsAvailable", callee="DBHelper").
public class TestFillDatabase {
#Prerequisite(requires = "databaseIsAvailable", callee="DBHelper")
#Test public void fillData() {
// ...
}
}
public class DBHelper {
public static boolean databaseIsAvailable() {
boolean isAvailable = ...;
return isAvailable ;
}
}

Also using the Assume class (since jUnit 4.4), you can use assumeNoException():
try{
base.evaluate();
} catch (ConcreteException e) {
Assume.assumeNoException("Concrete exception: skipping test", e);
}

I searched for the docs about JUnit and it appears that from version 4.9 they have introduced what they call test rules (see TestRule). You may start from this.
The ExpectedException class marked as #Rule could be of some help in order to check for exceptions thrown but not mandatory for the test to pass.
For more advanced usage I cannot say for the moment as I've just discovered it.