How to capture Junit test case run result - java

I would like to call some methods on the basis of test case run result. My example code snippet is below
testrun ()
{
String Expected, Actual;
assertEquals(Expected, Actual);
}
So, in the above example, if the values of variables Expected and Actual are same, then the test case will pass, else it will fail.
I would like to capture the pass/fail and based upon it to perform some certain actions.
I am new to Junit and looking for some help here.

As answered here, you can use a TestWatcher, like this:
public class WatchTest {
#Rule
public TestRule watcher = new TestWatcher() {
#Override
protected void succeeded(Description description) {
System.out.println("Pass!");
}
#Override
protected void failed(Throwable e, Description description) {
System.out.println("Fail!");
}
};
#Test
public void testRun() throws Exception {
String expected = "asd";
String actual = "qwert";
assertEquals(expected, actual);
}
}

For TestNG (not for JUnit)
Add this code to your test class (see #AfterMethod annotation):
#AfterMethod
public void tearDown(ITestResult r) {
if(r.getStatus()==ITestResult.FAILURE) {
System.out.println(r.getMethod().getMethodName()+ " is failed");
// any more report action what you need
} else {
System.out.println(r.getMethod().getMethodName()+ " is OK");
// any more report action what you need
}
}
How to do it without JUnit/TestNG
You can wrap your test-body into try-catch and do something (call to TestRail etc.) before the test finished. Example (you can run it to see results in Output Panel, it's working):
public class SomeTest {
protected String methodName;
public void setMethodName(Class cl, Exception ex) {
methodName = cl.getName() + "." + ex.getStackTrace()[0].getMethodName() + "()";
}
public int half(Integer v) {
return v/2;
}
#Test
public void willFailed() throws Exception {
setMethodName(this.getClass(), new Exception());
try {
assertEquals(half(null), 2);
System.out.println("Report about good result in " + methodName);
// any more report action what you need
} catch(Exception ex) {
System.out.println("Report about bug in " + methodName); // Report before crash
// any more report action what you need
throw new Exception(ex); // booom
}
}
#Test
public void willPassed() throws Exception {
setMethodName(this.getClass(), new Exception());
try {
assertEquals(half(2), 1);
System.out.println("Report about good result in " + methodName);
// any more report action what you need
} catch(Exception ex) {
System.out.println("Report about bug in " + methodName); // Report before crash
// any more report action what you need
throw new Exception(ex); // booom
}
}
}

Related

How to unit test java main method?

I have a class to which I must pass 2 arguments through its main method, if passed less than 2 args, it displays a system error message. I wrote a unit test for the main method here, when I run the test, it stops at "running" (shows neither pass nor fail). Please suggest.
Example.java
public class Example
{
private static String str1 = null;
private static String str2 = null;
public static void main(String[] args)
{
if( args.length != 2 )
{
call();
}
Example ex = new Example(args[0], args[1]);
ex.getData();
}
public Example(String str1, String str2)
{
Example.str1 = str1;
Example.str2 = str2;
}
public void getData(){
System.out.println("Name is: "+str1);
System.out.println("City is: "+str2);
}
private static void call()
{
System.err.println("Usage: String1 String2");
System.err.println("Where: ");
System.err.println(" String1 - Name");
System.err.println(" String1 - City");
System.exit(1);
}
}
ExampleTest.java
public class ExampleTest {
#Test
public void testPassingWrongNumberOfInputs() {
StringBuffer sb = new StringBuffer();
sb.append("Usage: String1 String2")
.append("Where: ")
.append(" String1 - Name")
.append(" String1 - City");
String expectedErrorMessage = sb.toString();
ByteArrayOutputStream outContent = new ByteArrayOutputStream();
System.setErr(new PrintStream(outContent));
String[] args = {"one"};
Example.main(args);
assertEquals(expectedErrorMessage, outContent.toString());
}
}
How about the following:
class TestingSecurityManager extends SecurityManager {
#Override public void checkExit(int status) {
throw new SecurityException();
}
}
Then in your test...
public class ExampleTest {
#Test
public void testPassingWrongNumberOfInputs() {
StringBuffer sb = new StringBuffer();
sb.append("Usage: String1 String2")
.append("Where: ")
.append(" String1 - Name")
.append(" String1 - City");
String expectedErrorMessage = sb.toString();
ByteArrayOutputStream outContent = new ByteArrayOutputStream();
System.setErr(new PrintStream(outContent));
String[] args = {"one"};
TestSecurityManager sm = new TestSecurityManager ();
System.setSecurityManager(sm);
try {
Example.main(args);
//should throw
fail("Should have thrown exception");
} catch (SecurityException se) {
}
assertEquals(expectedErrorMessage, outContent.toString());
}
}
I finally was able to write the unit test as shown in the following. I only tested if the method is hitting System.exit(1) code or not.
public class ExampleTest {
private SecurityManager m;
private TestSecurityManager sm;
#Before
public void setUp()
{
m = System.getSecurityManager();
sm = new TestSecurityManager ();
System.setSecurityManager(sm);
}
#After
public void tearDown()
{
System.setSecurityManager(m);
}
#Test
public void testPassingWrongNumberOfInputs() {
try {
Example.main(new String[] {"one"});
} catch (SecurityException se) {
assertEquals("1", se.getMessage());
}
}
}
class TestSecurityManager extends SecurityManager {
#Override
public void checkPermission(Permission permission) {
if ("exitVM".equals(permission.getName()))
{
throw new SecurityException("System.exit attempted and blocked.");
}
}
#Override
public void checkExit(int status) {
throw new SecurityException(Integer.toString(status));
}
}
Remove the System.exit(1) call, you don't need it. Your app will exit after main() completes anyway without an unneeded call to explicitly terminate the VM. This call is most likely causing your JUnit to stop executing before you get to your assertEquals statement, because you just told the VM to quit.
Rename the main method, and add a return value, so you can test it. Call this new method from main.

When does onException get triggered in WebDriverEventListener?

I am using a custom WebDriverEventListener in my Selenium tests for logging and screenshot purposes. It works fine:
When for example an element is not found in the browser window an exception is thrown by the webdriver and the onException() method is properly triggerd
#Override
public void onException(Throwable throwable, WebDriver driver) {
// do stuff
}
When I throw an Exception myself like this: throw new WebDriverException("my message"); the event is not triggered.
Can someone explain this behavior?
If you want to do some action when test failed or some exception, you can add to your test a rule(add in class where is #Before setUp()):
#Rule
public TestRule testWatcher = new TestWatcher() {
#Override
public void succeeded(Description test){
for (LogEntry log : driver.manage().logs().get(LogType.DRIVER).getAll()) {
System.out.println("Level:" + log.getLevel().getName());
System.out.println("Message:" + log.getMessage());
System.out.println("Time:" + log.getTimestamp());
System.out.println("-----------------------------------------------");
}
System.out.println();
#Override
public void failed(Throwable t, Description test) {
String testName = test.getClassName();
String subTestName = test.getMethodName();
String screenShotName = String.format("%s\\%s", testName, screenShotName);
if (driver instanceof TakesScreenshot) {
File tempFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
try {
System.out.println(">>>>>>>>>>LOGS: " + yourDirForImages + "\\" + screenShotName + ".png");
FileUtils.copyFile(tempFile, new File(String.format("%s.png", screenShotName)));
} catch (IOException e) {
e.printStackTrace();
}
}
}
Listener can be used for doing some code when its trigger some action, beforeCLick if there is header, logo, or footer.
public class ListenerMethodsImplementation extends AbstractWebDriverEventListener {
public void beforeClickOn(WebElement element, WebDriver myTestDriver) {
assertTrue("No Logo!", myTestDriver.findElements(By.id("logo")) == 1);
}
How to use it:
#Before
public void setUp() {
EventFiringWebDriver myTestDriver = new EventFiringWebDriver(driver);
ListenerMethodsImplementation myListener = new ListenerMethodsImplementation();
myTestDriver.register(myListener);
driver = myTestDriver;
}
How to get driver from listener: ((EventFiringWebDriver) driver).getWrappedDriver()
PS it's only a little portion from my code but i think this will help you.
Now I understand that because the listener is only registered to the WebDriver itelf it won't handle exceptions outsite the WebDriver.
In the abstract test case I did the following as part of a suggestion by Andrian Durlestean.
eventListener = new CustomWebDriverEventListener();
driver = new EventFiringWebDriver(driver).register(eventListener);
#Rule
public TestWatcher watchman = new TestWatcher() {
protected void failed(Throwable e, Description description) {
RuntimeException exception = (RuntimeException) e;
eventListener.onException(e, getDriver());
if (exception instanceof RuntimeException) {
throw exception;
}
};
};

Check that a method throws an exception when applied to any element of a list of values

I want to write unit tests for a parser and would like to check that it correctly throws an exception for all input strings in a list. Now as I understand it, the standard approach with JUnit would be to write a separate test method for each case:
public final class ParseFailureTest1 {
#Test(expected = ParseException.class)
public void testParseFailure1() throws Exception {
Parser.parse("[1 2]"); // Missing comma
}
#Test(expected = ParseException.class)
public void testParseFailure2() throws Exception {
Parser.parse("[1, 2,]"); // Additional commas
}
}
But as I want to apply the same test to 20 or 50 different strings, it seems impractical.
An alternative would be to explicitly check for an exception with a catch block:
public final class ParseFailureTest2 {
#Test
public void testParseFailure() throws Exception {
List<String> documents = Arrays.asList(
"[1 2]", // Missing comma
"[1, 2,]"); // Additional commas
for (String document : documents) {
try {
Parser.parse(document);
throw new AssertionError("Exception was not thrown");
} catch (ParseException e) {
// Expected, do nothing.
}
}
}
}
But this is error prone and I won't get any information about which exception was expected and if a different exception was thrown, it would count as a test error and not a failure.
My solution would be to use a method similar to expectException below:
public final class ParseFailureTest3 {
#Test
public void testParseFailure() throws Exception {
List<String> documents = Arrays.asList(
"[1 2]", // Missing comma
"[1, 2,]"); // Additional commas
for (final String document : documents) {
expectException(ParseException.class, new TestRunnable() {
#Override
public void run() throws Throwable {
Parser.parse(document);
}
});
}
}
public static void expectException(Class<? extends Throwable> expected, TestRunnable test) {
try {
test.run();
} catch (Throwable e) {
if (e.getClass() == expected) {
return; // Expected, do nothing.
} else {
throw new AssertionError(String.format("Wrong exception was thrown: %s instead of %s", e.getClass(), expected), e);
}
}
throw new AssertionError(String.format("Expected exception was not thrown: %s", expected));
}
public interface TestRunnable {
void run() throws Throwable;
}
}
Is there a method for that purpose in the JUnit framework or a related library or would you suggest a different approach (or one of my rejected approaches) to the problem?
Use JUnit4 for Parameterized test feature. The following code should work.
import java.util.Arrays;
import java.util.Collection;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
#RunWith(value = Parameterized.class)
public class ParseTest {
private String parseValue;
public ParseTest(String parseValue) {
this.parseValue = parseValue;
}
#Parameters
public static Collection<Object[]> data() {
Object[][] data = new Object[][] { { "[1 2]" }, { "[1,2,]" } };
return Arrays.asList(data);
}
#Test(expected = ParseException.class)
public void testParseFailure1() throws Exception {
Parse.parse(parseValue);
}
}
For more info refer http://www.mkyong.com/unittest/junit-4-tutorial-6-parameterized-test/
Use the fail() method:
#Test
public void testParseFailure() throws Exception {
List<String> documents = Arrays.asList(
"[1 2]", // Missing comma
"[1, 2,]"); // Additional commas
for (String document : documents) {
try {
Parser.parse(document);
fail("Parsing " + document + " should have thrown a ParseException");
}
catch (ParseException e) {
// Expected, do nothing.
}
}
}
Here's an alternative, keeping the fail idea from the previous answer, for if you care about getting the right message in the exception.
public final class ParseFailureTest {
#Test
public void testParseFailure() throws Exception {
Map<String, String> documents = new LinkedHashMap<String,String>();
documents.put("[1 2]", "Missing comma");
documents.put("[1, 2,]", "Additional commas");
for (Entry<String,String> entry : documents.entrySet()) {
try {
Parser.parse(entry.getKey());
fail("Parsing " + entry.getKey() +
" should have thrown a ParseException");
} catch (ParseException e) {
assertEquals(entry.getValue(), e.getMessage());
}
}
}
}

Retrying Method calls in generic way

My Java application requires a retry logic on remote calls failures.
These remote calls are:
scattered all over the application
pertain to different Remote Service classes.
Also, the retry logic may have varying retry interval and varying retry attempts.
I need a generic retry() implementation which can make appropriate method calls depending on from where it is called. Below is a simple code illustration of I am looking for. I know we can attempt to do this using java reflection, but, is there a framework or an open source available somewhere which is read-to-use?
try {
ClassA objA = remoteServiceA.call(paramA1, paramA2, ...);
} catch (Exception e){
ClassA objA = (ClassA)retry(remoteService, listOfParams, ..); // generic method call
}
..
try {
ClassB objB = remoteServiceB.call(paramB1, paramB2, ...);
} catch (Exception e){
ClassA objB = (ClassB)retry(remoteService, listOfParams, ..); // generic method call
}
As already suggested, you should use AOP and Java annotations. I would recommend a read-made mechanism from jcabi-aspects (I'm a developer):
#RetryOnFailure(attempts = 3, delay = 5)
public String load(URL url) {
return url.openConnection().getContent();
}
Read also this blog post: http://www.yegor256.com/2014/08/15/retry-java-method-on-exception.html
Update: Check RetryFunc from Cactoos.
This is a book example of where aspectj (or aop in general) can be used, see 8.2.7 Example in Spring documentation and 5 Reasons Java Developers Should Learn and Use AspectJ.
Basically an aspect intercepts all calls to given methods (specified using annotation, naming convention, whatever) and retries.
Assume you have a method, that need to retied at every 500ms and upto 5 times.
Current class:
public class RemoteCaller{
Service serviceCaller;
public void remoteCall(String message) {
serviceCaller.updateDetails( this.message);
return null;
}
}
Modified approach:
public class RetriableHelper<T> implements Callable<T> {
private Callable<T> task;
private int numberOfRetries;
private int numberOfTriesLeft;
private long timeToWait;
public RetriableHelper(int numberOfRetries, long timeToWait, Callable<T> task) {
this.numberOfRetries = numberOfRetries;
numberOfTriesLeft = numberOfRetries;
this.timeToWait = timeToWait;
this.task = task;
}
public T call() throws Exception {
while (true) {
try {
return task.call();
} catch (InterruptedException e) {
throw e;
} catch (CancellationException e) {
throw e;
} catch (Exception e) {
numberOfTriesLeft--;
if (numberOfTriesLeft == 0) {
throw e;
}
Thread.sleep(timeToWait);
}
}
}
}
Backend system/remote call class:
public class RemoteCaller{
Service serviceCaller;
public void remoteCall(String message) {
class RemoteCallable implements Callable<Void> {
String message;
public RemoteCallable( String message)
{
this.message = message;
}
public Void call() throws Exception{
serviceCaller.updateDetails( this.message);
return null;
}
}
RetriableHelper<Void> retriableHelper = new RetriableHelper<Void>(5, 500, new RemoteCallable( message));
try {
retriableHelper.call();
} catch (Exception e) {
throw e;
}
}
}
enter link description here Spring has a retry annotation which servers the purpose
Step 1: Add following dependency to your POM
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
<version>1.1.5.RELEASE</version>
</dependency>
Step 2: Enabling Spring Retry
To enable Spring Retry in an application, we need to add the #EnableRetry annotation to our #Configuration class:
Ex:
#Configuration
#EnableRetry
public class AppConfig { ... }
Step 3: To add retry functionality to methods, #Retryable can be used:
Ex:
#Service
public interface MyService {
#Retryable(
value = { SQLException.class },
maxAttempts = 2,
backoff = #Backoff(delay = 5000))
void retryService(String sql) throws SQLException;
...
}
Step 4.The #Recover annotation is used to define a separate recovery method when a #Retryable method fails with a specified exception:
Ex:
#Service
public interface MyService {
...
#Recover
void recover(SQLException e, String sql);
}
See Url for more details : http://www.baeldung.com/spring-retry
where do you get the services from? use a factory to Proxy the service you get from the original factory. The proxy can then implement the retry transparently. See the java Proxy/ProxyGenerators in reflection.
If you are using spring , then better go with Aspects.
Otherwise, below sample solution can work:
public class Test
{
public static void main(String[] args) throws Exception
{
Test test = new Test();
test.toRunFirst("Hello! This is normal invocation");
runWithRetry(test, "toRunFirst", "Hello! This is First, called with retry");
runWithRetry(test, "toRunSecond", "Hello! This is Second, called with retry");
}
public void toRunFirst(String s) {
System.out.println(s);
}
public void toRunSecond(String s) {
System.out.println(s);
}
public static Object runWithRetry(Object obj, String methodName, Object... args) throws Exception
{
Class<?>[] paramClass = new Class<?>[args.length];
for(int i=0; i< args.length; i++) {
paramClass[i] = args[i].getClass();
}
Method method = obj.getClass().getDeclaredMethod(methodName, paramClass);
int retryCount = 2;
for(int i=0; i< retryCount; i++) {
try {
return method.invoke(obj, args);
}
catch (Exception e) {
e.printStackTrace();
}
}
return null;
}
}
I did not find what I needed so there is mine.
The main feature is that it throws the type of Exception you need when maxRetries is reached so you can catch it in the call.
import org.apache.log4j.Logger;
public class TaskUtils {
public static <E extends Throwable> void retry(int maxRetries, Task<E> task) throws E {
retry(maxRetries, 0, null, task);
}
public static <E extends Throwable> void retry(int maxRetries, long waitTimeMs, Logger logger, Task<E> task) throws E {
while (maxRetries > 0) {
maxRetries--;
try {
task.run();
} catch (Exception e) {
if (maxRetries == 0) {
try {
throw e;
} catch (Exception ignored) { // can't happen but just in case we wrap it in
throw new RuntimeException(e);
}
}
if (logger != null)
logger.warn("Attempt " + maxRetries + " failed", e);
try {
Thread.sleep(waitTimeMs);
} catch (InterruptedException ignored) {
}
}
}
}
public interface Task<E extends Throwable> {
void run() throws E;
}
}
Usage :
TaskUtils.retry(3, 500, LOGGER, () -> stmClickhouse.execute(
"ALTER TABLE `" + database + "`.`" + table.getName() + "` ON CLUSTER " + clusterName + allColumnsSql
));
add it into pom.xml
<dependency>
<groupId>org.deking.utils</groupId>
<artifactId>retry</artifactId>
<version>0.0.2-SNAPSHOT</version>
</dependency>
new Retry<String>()
.maxOperationWaitTime(30_000)//Max operation wait time during a single operation
.retryIntervalTime(1_000)//Interval time between two operations
.maxRetryTimes(3)//Retry times when operation failed(or timeout) at the first time
.operation(() -> {
//your operation
return "success!";
})
.judgement(t -> (t == null || t.isEmpty()))//add your judgement whether the operation should be retry(Operation should return a value)
.execute();
If you want add retry config annotation on method,and call it:
class RetryTests{
#RetryConfig( maxRetryTimes=1)
public static String TestAnnotation() {
return "aaa";
}
public static void main(String[] args) {
try {
new Retry<String>()
.of(RetryTest.class.getMethod("TestAnnotation"),null)
.judgement(r -> r.equals("aaa"))
.execute();
} catch (NoSuchMethodException | SecurityException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

JUnit/TestNg: How to simplify/combine multiple { try fail() catch } for same exception?

Example (<Expected Exception> for assert 1 & assert 2 is same) :
#junit.framework.Test // or #org.testng.annotations.Test
public void testCase() {
try {
// assert 1
fail();
} catch (<Expected Exception>) {
}
try {
// assert 2
fail();
} catch (<Expected Exception>) {
}
}
If you're feeling adventurous, you can also try out assertThrows:
https://github.com/dsaff/junit.contrib
Feel free to ask if you have any problems.
If it's too hard to break this up into individual test methods, here's what's worked for me in the past.
Create a method expectsException() that expects a Callback.
interface Callback {
void call() throws Exception;
}
void expectsException(Callback callback) {
try {
callback.call();
fail("ExpectedException was not thrown!");
} catch (Exception e) {
if (!(e instanceof ExpectedException)) {
fail("Expecting ExpectedException, got " + e.getClass());
}
assertEquals("Expected exception message", e.getMessage());
}
}
Then, wrap up the code inside your try {...} catch blocks in the Callback:
#Test
public void testSomething() {
expectsException(new Callback() {
public void call() throws Exception {
// assert 1
}
});
expectsException(new Callback() {
public void call() throws Exception {
// assert 2
}
});
}
Note however, that depending on what you're doing in the catch block, this may or may not end up less verbose than a straightforward try {...} catch.
(When Java gets proper closures, then this approach will make a lot more sense.)
You should probably break your method into two separate methods that will each throw:
#Test(expectedExceptions = NullPointerException.class)
public void testCase1() {
// assert 1
}
#Test(expectedExceptions = NullPointerException.class)
public void testCase2() {
// assert 2
}
catch-exception might help:
public void testCase() {
// assert 1
verifyException(obj, MyException.class).do(1);
// assert 2
verifyException(obj, MyException.class).do(2);
}

Categories

Resources