I came across an answer on how to use Retry in Selenium web driver test. Now I have implemented Retry class under utilities and inside the test annotation given as follows #Test(retryAnalyzer = Retry.class). This is working fine now, but I would like to change this way, can I use the Retry in any other way instead of giving in test annotation as #Test(retryAnalyzer = Retry.class) ? Also I need to comment the #Test(enabled = true), Can someone please advise ?
utilities/Retryclass
public class Retry implements IRetryAnalyzer {
private int retryCount = 0;
private int maxRetryCount = 2;
public boolean retry(ITestResult result) {
if(retryCount < maxRetryCount) {
retryCount ++ ;
return true;
}
return false;
}
}
tests/RepaymentCalculatorTest
public class RepaymentCalculatorTest extends BaseTest {
Retry retryTest = new Retry();
//#Test(enabled = true)
#Test(retryAnalyzer = Retry.class)
public void loanRepaymentCalculator() throws InterruptedException {
// rest of the UI test code added here ....
}
i am using bit different way, no need to add class in #Test annotation
Lets have custom Listener for it
public class RetryFailedTestCases implements IRetryAnalyzer, IAnnotationTransformer{
public void transform(ITestAnnotation testannotation, Class testClass, Constructor testConstructor,
Method testMethod) {
testannotation.setRetryAnalyzer(RetryFailedTestCases.class);
}
int counter = 0;
int retryLimit = 1; //as per need
#Override
public boolean retry(ITestResult result) {
if(counter < retryLimit)
{
counter++;
return true;
}
return false;
}
}
Now, add this in testng.xml file
<listeners>
<listener class-name="package.RetryFailedTestCases"/>
</listeners>
this is simple and helpful when we run as suite.. alternative we can mentioned listener class on top of each class instead of testng.xml file
i am not sure why explicitly mentioned enabled = true ? even if not #Test execute right? if you dont want to execute then mention with false.
Here is what I am trying to build:
Cucumber based tests with TestNG Executor
TestNG Executor so that I can re-run failed tests
Cucable plugin - so that the scenarios are split into individual files and runner is auto generated during run time for each of the scenarios. (Enables parallel execution)
Below is the test runner:
#CucumberOptions(
glue = "com.fifa.stepdefs",
features = {"target/parallel/features/[CUCABLE:FEATURE].feature"},
plugin = {"json:target/cucumber-report/[CUCABLE:RUNNER].json"}
)
public class CucableJavaTemplate implements IRetryAnalyzer {
private int count = 0;
private static int maxTry = 3;
#Override
public boolean retry(ITestResult iTestResult) {
if (!iTestResult.isSuccess()) { ;//Check if test not succeed
if (count < maxTry) { //Check if maxtry count is reached
count++; //Increase the maxTry count by 1
iTestResult.setStatus(ITestResult.FAILURE); //Mark test as failed
return true; //Tells TestNG to re-run the test
} else {
iTestResult.setStatus(ITestResult.FAILURE); //If maxCount reached,test marked as failed
}
} else {
iTestResult.setStatus(ITestResult.SUCCESS); //If test passes, TestNG marks it as passed
}
return false;
}
private TestNGCucumberRunner testNGCucumberRunner;
#BeforeClass(alwaysRun = true)
public void setUpClass() throws Exception {
System.out.println("Before Scenario ****");
testNGCucumberRunner = new TestNGCucumberRunner(this.getClass());
}
#Test(groups = "cucumber", description = "Runs Cucumber Scenarios", dataProvider = "scenarios",retryAnalyzer = CucableJavaTemplate.class)
public void scenario(PickleEventWrapper pickleEvent, CucumberFeatureWrapper cucumberFeature) throws Throwable {
testNGCucumberRunner.runScenario(pickleEvent.getPickleEvent());
}
#DataProvider
public Object[][] scenarios() {
return testNGCucumberRunner.provideScenarios();
}
#AfterClass(alwaysRun = true)
public void tearDownClass() throws Exception {
System.out.println("After Scenario ****");
testNGCucumberRunner.finish();
}
}
If I create a Driver instance in the before class: How can I pass it to page object files or step def files?
I gave similar answer on this question:
How to match a specific java file with specific feature file in cucumber
If you have annotated the driver class with #ScenarioScoped then you can easily use Guice to inject this instance in your other classes.
I'm trying to write a simple JUnit Rule implementation which reruns a test case a given amount of times if not successful.
It works fine as such, but I'd like to make it configurable per method with a custom annotation I attach to the method.
Here's my rule implementation:
public class Retry implements TestRule {
private int retryCount = 10;
#Override
public Statement apply(Statement base, Description description) {
return new Statement() {
public void evaluate() throws Throwable {
RetryCount annotation = description.getAnnotation(RetryCount.class);
// Problem is here, the annotation is always null!
int retries = (annotation != null) ? annotation.retries() : retryCount;
// keep track of the last failure to include it in our failure later
AssertionError lastFailure = null;
for (int i = 0; i < retries; i++) {
try {
// call wrapped statement and return if successful
base.evaluate();
return;
} catch (AssertionError err) {
lastFailure = err;
}
}
// give meaningful message and include last failure for the
// error trace
throw new AssertionError("Gave up after " + retries + " tries", lastFailure);
}
};
}
// the annotation for method-based retries
public static #interface RetryCount {
public int retries() default 1;
}
}
In the line I commented, I don't get the annotation I attach to the method:
public class UnreliableServiceUnitTest {
private UnreliableService sut = new UnreliableService();
#Rule
public Retry retry = new Retry();
#Test
#RetryCount(retries=5) // here it is
public void worksSometimes() {
boolean worked = sut.workSometimes();
assertThat(worked, is(true));
}
}
If I debug into the Rule, the Description annotation list contains the #Test annotation but not the #RetryCount. I also tried adding a #Deprecated which will also get added.
Any idea why?
For completeness, this is the sample SUT:
public class UnreliableService {
private static Random RANDOM = new Random();
// needs at least two calls
private static int COUNTER = RANDOM.nextInt(8) + 2;
public boolean workSometimes() {
if (--COUNTER == 0) {
COUNTER = RANDOM.nextInt(8) + 2;
return true;
}
return false;
}
}
The #Test annotation is a Runtime annotation. Your RetryCount is not defined like that. It should be so you can access it during runtime. Change your code to this:
// the annotation for method-based retries
#Retention(value=RUNTIME)
public static #interface RetryCount {
public int retries() default 1;
}
Using RetentionPolicy Runtime allows you to read the annotations reflectively. See here the Javadoc
My application have several execution modes, and in 1 mode it is normal that some of my tests will throw a concrete exception. I need to annotate this methods with something like #SkipOnFail that will set method as skipped if exception was thrown.
thanks in advance!
#Edit(for my question to be more clear)
#Test(expected=ConcreteException.class)
does not work for me because i need my tests to pass even if ConcreteException.class was not thrown(expected tag in junit will mark my test as failed if this exception won't be thrown), and to be skipped otherwise. In all other cases it should work as always.
#Solution that worked for me(junit v4.7) thx to #axtavt
#Rule
public MethodRule skipRule = new MethodRule() {
public Statement apply(final Statement base, FrameworkMethod method, Object target) {
if(method.getAnnotation(SkipOnFail.class) == null) return base;
return new Statement() {
#Override
public void evaluate() throws Throwable {
try{
base.evaluate();
} catch (ConcreteException e) {
Assume.assumeTrue(false);
}
}
};
}
};
#Thx
I don't think that such a feature is available out of the box, but it should be pretty easy to implement with custom TestRule and Assume, something like this:
#Rule
public TestRule skipRule = new TestRule() {
public Statement apply(final Statement base, Description desc) {
if (desc.getAnnotation(SkipOnFail.class) == null) return base;
return new Statement() {
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (MyExceptoion ex) {
Assume.assumeTrue(false);
}
}
};
}
};
What about using JUnit Extensions?
The following example is taken from their Tutorial.
It provides aditional annotations for Prerequisites (#Prerequisite): Ignore tests based on conditions.
The required approach would be to check this during running tests. So you can simply add a #Prerequisite(requires="") annotation.
public class TestFillDatabase {
#Prerequisite(requires = "databaseIsAvailable")
#Test public void fillData() {
// ...
}
public boolean databaseIsAvailable() {
boolean isAvailable = ...;
return isAvailable;
}
}
public class TestFillDatabase {
#Prerequisite(requires = "databaseIsAvailable")
#Test public void fillData() {
// ...
}
public boolean databaseIsAvailable() {
boolean isAvailable = ...;
return isAvailable ;
}
}
This specified methods with #Prerequisite(requires = "databaseIsAvailable") must be a public method, returning a boolean or Boolean value.
If these methods will be consolidated in helper classes, you can also specify static methods within a class to be called using #Prerequisite(requires = "databaseIsAvailable", callee="DBHelper").
public class TestFillDatabase {
#Prerequisite(requires = "databaseIsAvailable", callee="DBHelper")
#Test public void fillData() {
// ...
}
}
public class DBHelper {
public static boolean databaseIsAvailable() {
boolean isAvailable = ...;
return isAvailable ;
}
}
Also using the Assume class (since jUnit 4.4), you can use assumeNoException():
try{
base.evaluate();
} catch (ConcreteException e) {
Assume.assumeNoException("Concrete exception: skipping test", e);
}
I searched for the docs about JUnit and it appears that from version 4.9 they have introduced what they call test rules (see TestRule). You may start from this.
The ExpectedException class marked as #Rule could be of some help in order to check for exceptions thrown but not mandatory for the test to pass.
For more advanced usage I cannot say for the moment as I've just discovered it.
In JUnit 3, I could get the name of the currently running test like this:
public class MyTest extends TestCase
{
public void testSomething()
{
System.out.println("Current test is " + getName());
...
}
}
which would print "Current test is testSomething".
Is there any out-of-the-box or simple way to do this in JUnit 4?
Background: Obviously, I don't want to just print the name of the test. I want to load test-specific data that is stored in a resource with the same name as the test. You know, convention over configuration and all that.
JUnit 4.7 added this feature it seems using TestName-Rule. Looks like this will get you the method name:
import org.junit.Rule;
public class NameRuleTest {
#Rule public TestName name = new TestName();
#Test public void testA() {
assertEquals("testA", name.getMethodName());
}
#Test public void testB() {
assertEquals("testB", name.getMethodName());
}
}
JUnit 4.9.x and higher
Since JUnit 4.9, the TestWatchman class has been deprecated in favour of the TestWatcher class, which has invocation:
#Rule
public TestRule watcher = new TestWatcher() {
protected void starting(Description description) {
System.out.println("Starting test: " + description.getMethodName());
}
};
Note: The containing class must be declared public.
JUnit 4.7.x - 4.8.x
The following approach will print method names for all tests in a class:
#Rule
public MethodRule watchman = new TestWatchman() {
public void starting(FrameworkMethod method) {
System.out.println("Starting test: " + method.getName());
}
};
JUnit 5 and higher
In JUnit 5 you can inject TestInfo which simplifies test metadata injection to test methods. For example:
#Test
#DisplayName("This is my test")
#Tag("It is my tag")
void test1(TestInfo testInfo) {
assertEquals("This is my test", testInfo.getDisplayName());
assertTrue(testInfo.getTags().contains("It is my tag"));
}
See more: JUnit 5 User guide, TestInfo javadoc.
Try this instead:
public class MyTest {
#Rule
public TestName testName = new TestName();
#Rule
public TestWatcher testWatcher = new TestWatcher() {
#Override
protected void starting(final Description description) {
String methodName = description.getMethodName();
String className = description.getClassName();
className = className.substring(className.lastIndexOf('.') + 1);
System.err.println("Starting JUnit-test: " + className + " " + methodName);
}
};
#Test
public void testA() {
assertEquals("testA", testName.getMethodName());
}
#Test
public void testB() {
assertEquals("testB", testName.getMethodName());
}
}
The output looks like this:
Starting JUnit-test: MyTest testA
Starting JUnit-test: MyTest testB
NOTE: This DOES NOT work if your test is a subclass of TestCase! The test runs but the #Rule code just never runs.
Consider using SLF4J (Simple Logging Facade for Java) provides some neat improvements using parameterized messages. Combining SLF4J with JUnit 4 rule implementations can provide more efficient test class logging techniques.
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.MethodRule;
import org.junit.rules.TestWatchman;
import org.junit.runners.model.FrameworkMethod;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class LoggingTest {
#Rule public MethodRule watchman = new TestWatchman() {
public void starting(FrameworkMethod method) {
logger.info("{} being run...", method.getName());
}
};
final Logger logger =
LoggerFactory.getLogger(LoggingTest.class);
#Test
public void testA() {
}
#Test
public void testB() {
}
}
A convoluted way is to create your own Runner by subclassing org.junit.runners.BlockJUnit4ClassRunner.
You can then do something like this:
public class NameAwareRunner extends BlockJUnit4ClassRunner {
public NameAwareRunner(Class<?> aClass) throws InitializationError {
super(aClass);
}
#Override
protected Statement methodBlock(FrameworkMethod frameworkMethod) {
System.err.println(frameworkMethod.getName());
return super.methodBlock(frameworkMethod);
}
}
Then for each test class, you'll need to add a #RunWith(NameAwareRunner.class) annotation. Alternatively, you could put that annotation on a Test superclass if you don't want to remember it every time. This, of course, limits your selection of runners but that may be acceptable.
Also, it may take a little bit of kung fu to get the current test name out of the Runner and into your framework, but this at least gets you the name.
JUnit 4 does not have any out-of-the-box mechanism for a test case to get it’s own name (including during setup and teardown).
String testName = null;
StackTraceElement[] trace = Thread.currentThread().getStackTrace();
for (int i = trace.length - 1; i > 0; --i) {
StackTraceElement ste = trace[i];
try {
Class<?> cls = Class.forName(ste.getClassName());
Method method = cls.getDeclaredMethod(ste.getMethodName());
Test annotation = method.getAnnotation(Test.class);
if (annotation != null) {
testName = ste.getClassName() + "." + ste.getMethodName();
break;
}
} catch (ClassNotFoundException e) {
} catch (NoSuchMethodException e) {
} catch (SecurityException e) {
}
}
Based on the previous comment and further considering I created an extension of TestWather which you can use in your JUnit test methods with this:
public class ImportUtilsTest {
private static final Logger LOGGER = Logger.getLogger(ImportUtilsTest.class);
#Rule
public TestWatcher testWatcher = new JUnitHelper(LOGGER);
#Test
public test1(){
...
}
}
The test helper class is the next:
public class JUnitHelper extends TestWatcher {
private Logger LOGGER;
public JUnitHelper(Logger LOGGER) {
this.LOGGER = LOGGER;
}
#Override
protected void starting(final Description description) {
LOGGER.info("STARTED " + description.getMethodName());
}
#Override
protected void succeeded(Description description) {
LOGGER.info("SUCCESSFUL " + description.getMethodName());
}
#Override
protected void failed(Throwable e, Description description) {
LOGGER.error("FAILURE " + description.getMethodName());
}
}
Enjoy!
In JUnit 5 TestInfo acts as a drop-in replacement for the TestName rule from JUnit 4.
From the documentation :
TestInfo is used to inject information about the current test or
container into to #Test, #RepeatedTest, #ParameterizedTest,
#TestFactory, #BeforeEach, #AfterEach, #BeforeAll, and #AfterAll
methods.
To retrieve the method name of the current executed test, you have two options : String TestInfo.getDisplayName() and
Method TestInfo.getTestMethod().
To retrieve only the name of the current test method TestInfo.getDisplayName() may not be enough as the test method default display name is methodName(TypeArg1, TypeArg2, ... TypeArg3).
Duplicating method names in #DisplayName("..") is not necessary a good idea.
As alternative you could use
TestInfo.getTestMethod() that returns a Optional<Method> object.
If the retrieval method is used inside a test method, you don't even need to test the Optional wrapped value.
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.TestInfo;
import org.junit.jupiter.api.Test;
#Test
void doThat(TestInfo testInfo) throws Exception {
Assertions.assertEquals("doThat(TestInfo)",testInfo.getDisplayName());
Assertions.assertEquals("doThat",testInfo.getTestMethod().get().getName());
}
JUnit 5 via ExtensionContext
Advantage:
You get to have the added functionalities of ExtensionContext by overriding afterEach(ExtensionContext context).
public abstract class BaseTest {
protected WebDriver driver;
#RegisterExtension
AfterEachExtension afterEachExtension = new AfterEachExtension();
#BeforeEach
public void beforeEach() {
// Initialise driver
}
#AfterEach
public void afterEach() {
afterEachExtension.setDriver(driver);
}
}
public class AfterEachExtension implements AfterEachCallback {
private WebDriver driver;
public void setDriver(WebDriver driver) {
this.driver = driver;
}
#Override
public void afterEach(ExtensionContext context) {
String testMethodName = context.getTestMethod().orElseThrow().getName();
// Attach test steps, attach scsreenshots on failure only, etc.
driver.quit();
}
}
#ClassRule
public static TestRule watchman = new TestWatcher() {
#Override
protected void starting( final Description description ) {
String mN = description.getMethodName();
if ( mN == null ) {
mN = "setUpBeforeClass..";
}
final String s = StringTools.toString( "starting..JUnit-Test: %s.%s", description.getClassName(), mN );
System.err.println( s );
}
};
I usually use something like this:
/** Returns text with test method name
#param offset index of method on call stack to print, 1 for a caller of this method.
*/
static String getName(int offset)
{
Throwable t = new Throwable();
t.fillInStackTrace();
return
t.getStackTrace()[offset].getMethodName()+":"+t.getStackTrace()[offset].getLineNumber();
};
This is exactly what Exception do use when printing stack trace.
Depending on the exact context You may have to figure out correct offset value. It is crude and primitive tough and is not using any fancy modern futures.
I'd suggest you decouple the test method name from your test data set. I would model a DataLoaderFactory class which loads/caches the sets of test data from your resources, and then in your test case cam call some interface method which returns a set of test data for the test case. Having the test data tied to the test method name assumes the test data can only be used once, where in most case i'd suggest that the same test data in uses in multiple tests to verify various aspects of your business logic.
You can achieve this using Slf4j and TestWatcher
private static Logger _log = LoggerFactory.getLogger(SampleTest.class.getName());
#Rule
public TestWatcher watchman = new TestWatcher() {
#Override
public void starting(final Description method) {
_log.info("being run..." + method.getMethodName());
}
};
I have a Junit4 test class that extends TestCase so the example with #Rule didn't work (as mentioned in other answers).
However, if your class extends TestCase you can use getName() to get the current test name so this works:
#Before
public void setUp() {
System.out.println("Start test: " + getName());
}
#After
public void tearDown() {
System.out.println("Finish test: " + getName());
}
A more simpler way is to put this logic in setUp() and tearDown() methods.
Refer below code for better clarity,
import java.lang.reflect.Method;
#BeforeMethod
void setUp(Method method) {
log.info("###############################################");
log.info("Running Test: {}", method.getName());
}
#AfterMethod
void tearDown(Method method) {
log.info("Finished Test: {}", method.getName());
log.info("###############################################");
}
#Test
public void testMethodName() {
// Method logic implementation...
}
Here is the output of above test execution,
#############################################################
Running Test: testMethodName
// Logs related to method execution...
Finished Test: testMethodName
#############################################################