testng - creating KnownFault and IntermittentFailure annotations - java

I would like to annotate some of my test cases with KnownFault - which would do pretty much what expectedException does plus some magic using YouTrack's REST API. I would also like to have an IntermittentFailure attribute which would mean that I'm aware that the test might occasionally fail with [exception] [message] but I wouldn't want this to block the rest of my build chain.
After some research I found that my test class should implement IHookable, then I could have something like this:
#Override
public void run(IHookCallBack callBack, ITestResult result) {
callBack.runTestMethod(result);
if (result.getThrowable().getCause() instanceof IllegalArgumentException){
System.out.println("This is expected.");
result.setThrowable(null);
}
else{
System.out.println("Unexpected exception");
}
}
The problem with this is the actual implementation of invokeHookable:
final Throwable[] error = new Throwable[1];
IHookCallBack callback = new IHookCallBack() {
#Override
public void runTestMethod(ITestResult tr) {
try {
invokeMethod(thisMethod, testInstance, parameters);
} catch (Throwable t) {
error[0] = t;
tr.setThrowable(t); // make Throwable available to IHookable
}
}
#Override
public Object[] getParameters() {
return parameters;
}
};
hookable.run(callback, testResult);
if (error[0] != null) {
throw error[0];
}
Unfortunately that last line means that my test case is going to throw an exception no matter what as the error array is completely out of my hands in the run method.
So, what would be the proper way of intercepting an exception and handling it the way I want to?

What you are trying to do is really interesting. You should try to propose changes on https://github.com/cbeust/testng/pull/
But maybe IHookable is not the best listener you can use. Did you try IInvokedMethodListener?
void afterInvocation(IInvokedMethod method, ITestResult result) {
if (result.getThrowable().getCause() instanceof IllegalArgumentException) {
System.out.println("This is expected.");
result.setThrowable(null);
result.setStatus(SUCCESS); // If you want to change the status
} else {
System.out.println("Unexpected exception");
}
}

Related

Factoring try catch

I have a Java EE application with dozens of web services using the same pattern:
public Response myWebService1() {
try {
// do something different depending on the web service called
} catch (MyCustomException e1) {
return Response.status(409).build();
} catch (UnauthorizedException e2) {
return Response.status(401).build();
} catch (Exception e3) {
return Response.status(500).build();
}
}
Is that possible to factorize this piece of code?
If this is a JAX-RS environment, see Tunaki's answer, handling this is specifically catered for and wonderfully simple.
If not:
You can have a functional interface accepting a function that can throw exceptions and returns a Response:
#FunctionalInterface
public interface Responder {
Response handleRequest() throws Exception;
}
(As Dici points out, you could make that a generic ThrowingSupplier or similar, since you're allowing it to throw Exception.)
Then have a helper method accepting an instance of it:
private static Response respond(Responder responder) {
try {
return responder.handleRequest();
} catch (MyCustomException e1) {
return Response.status(409).build();
} catch (UnauthorizedException e2) {
return Response.status(401).build();
} catch (Exception e3) {
return Response.status(500).build();
}
}
...and use it via a lambda:
public Response myWebService1() {
return respond(() -> {
// Do stuff here, return a Response or throw / allow throws on error
});
}
Since this is in a JAX-RS context, there is a much better way, that does not rely on catching a lot of different exceptions: use an ExceptionMapper. This is a built-in mechanism of JAX-RS 1.0 that translates an exception type into a proper Response object to send to the client.
In your case, you could have the following classes defined once in your application:
#Provider
public class UnauthorizedExceptionMapper implements ExceptionMapper<UnauthorizedException> {
public Response toResponse(UnauthorizedException e) {
return Response.status(401).build();
}
}
#Provider
public class MyCustomExceptionMapper implements ExceptionMapper<MyCustomException> {
public Response toResponse(MyCustomException e) {
return Response.status(409).build();
}
}
#Provider
public class CatchAllExceptionMapper implements ExceptionMapper<Exception> {
public Response toResponse(Exception e) {
return Response.status(500).build();
}
}
The #Provider annotation tells the JAX-RS runtime to discover this class when scanning. This makes sure that, wherever in your code, if a MyCustomException is thrown (and not explicitly catched), a 409 response will be returned. The code in your application would simply become:
public Response myWebService1() {
// do something, and don't catch anything; just care about the happy path
}
The exception hierarchy is correctly taken into account. If the application code throws a MyCustomExceptionMapper, JAX-RS will look for an exception mapper registered with that type, and will go up the super class if it can't find one: this way, there can be a catch-all exception mapper handling every other case.
If all methods handle exceptions the same way, you can extract the exception handling to an external method :
public static Response exceptionHandler (Exception exc)
{
if (exc instanceof MyCustomException) {
return Response.status(409).build();
} else if (exc instanceof UnauthorizedException) {
return Response.status(401).build();
} else {
return Response.status(500).build();
}
}
public Response myWebService1() {
try {
// do something different depending on the web service called
} catch (Exception exc) {
return exceptionHandler(exc);
}
}
Sure, that is possible. We have a solution that looks like:
} catch (Exception exception) {
exceptionConverter.convertAndThrow(exception);
}
to unify re-throwing of exceptions based on the exception caught.
So that exception converter is the one central place where we "switch" over the exception type and do what needs to be done. Of course, the central element here is: all your classes need the exact same handling for the incoming exceptions.
We even go one step further and allow a wild mix of potential "input causes", but we also have extensive unit tests to ensure that the conversion always gives the expected result.
Please note: my answer is just about refactoring those "catch" cascade. You can still turn to TJs solution; but keep in mind: that approach adds a certain bit of complexity by introducing that Runnable aspect.
#T.J.Crowder's response is perfect. But for those who can't use Java 8, this is how to implement it with earlier version of Java:
The Responder interface:
public interface Responder {
Response handleRequest() throws Exception;
}
The helper method:
private static Response respond(Responder responder) {
try {
return responder.handleRequest();
} catch (MyCustomException e1) {
return Response.status(409).build();
} catch (UnauthorizedException e2) {
return Response.status(401).build();
} catch (Exception e3) {
return Response.status(500).build();
}
}
With an anonymous class instead of a lambda expression:
Response response = respond(new Responder() {
#Override
public Response handleRequest() throws Exception {
...
return Response.ok().build();
}
});

Custom Exception Dispatcher in Spring Boot Application for response logic

I have a Spring Boot application that has the following approximate structure:
project
Api
ApiImpl
Application
Api is an interface that looks like this:
public interface Api {
public String methodOne(...) throws ExceptionOne, ExceptionTwo, ExceptionThree;
...
public int methodN(...) throws ExceptionOne, ExceptionThree, ExceptionFour;
}
ApiImpls is the request controller (in reality there is a second layer, but this should suffice for this example). There, I do something like the following right now:
#Controller
public class ApiImpl {
public String methodOne(...) {
try {
// do stuff that can yield an exception
}
catch(ExceptionOne e) {
// set proper response code and return values
}
catch(ExceptionTwo e) {
// set proper response code and return values
}
catch(ExceptionThree e) {
// set proper response code and return values
}
}
}
Basically, this behaviour yields a lot of repetition (might as well name my exceptions D, R, and Y...), but is otherwise very suited to handling the internal application logic.
My question is: How can I implement a custom Exception Dispatcher that would handle this in Java? Ideally, I would want something like this answer here, but unfortunately simply throwing the current exception like in that C++ code is not possible in Java, as far as I know. For brevity, what I would like to accomplish is something like the following:
#Controller
public class ApiImpl {
public String methodOne(...) {
try {
// do stuff that can yield an exception
}
catch(ExceptionOne e) {
handle()
}
}
private void handle() { // maybe Throwable or Exception subclass as parameter
// handle the correct exception type, set correct response code, etc.
}
}
Are there any good approaches to doing this so as to minimize code repetition?
Here is a preliminary attempt I tried to get this working:
public class Thrower {
public Thrower(int e) throws ExceptionOne, ExceptionTwo, ExceptionThree {
if(e == 0) {
throw new ExceptionOne();
}
if(e == 1) {
throw new ExceptionTwo();
}
if(e == 2) {
throw new ExceptionThree();
}
}
}
class ExceptionOne extends Exception {}
class ExceptionTwo extends Exception {}
class ExceptionThree extends Exception {}
public class ExceptionHandler {
private void handle(Exception ex) throws Exception {
try {
throw ex;
}
catch(ExceptionOne e) {
e.printStackTrace();
System.out.println("Exception one");
}
catch(ExceptionTwo e) {
e.printStackTrace();
System.out.println("Exception two");
}
catch(ExceptionThree e) {
e.printStackTrace();
System.out.println("Exception three");
}
}
public void causesException(int which) throws Throwable {
try {
Thrower t = new Thrower(which);
}
catch(Exception e) {
handle(e);
}
}
public static void main(String[] args) throws Throwable {
ExceptionHandler eh = new ExceptionHandler();
eh.causesException(0);
eh.causesException(1);
eh.causesException(2);
}
}
This works as expected, and I can handle the different exception types as needed (shown here using a constructor, but the principle would be the same). However, this feels extremely clunky.
If you are looking for globally handling all Controller Layer exceptions (in Spring MVC architecture), you can do that at one place for all controllers (option1 below) by using #ExceptionHandler methods which is a ControllerAdvice from Spring.
Option(1): Configure Exceptions in Separate Class
#ControllerAdvice
class MyProjectExceptionHandler {
#ExceptionHandler(value = ExceptionOne.class)
public R exceptionOne(ExceptionOne exe) {
//set proper response code and return values
}
#ExceptionHandler(value = ExceptionTwo.class)
public R exceptionTwo(ExceptionTwo exe) {
//set proper response code and return values
}
}
Option(2): Configure Exceptions in Controller Class itself
If you are looking for handling the exceptions within the Controller class itself, then you can do that as below:
#Controller
public class ApiImpl {
public String methodOne(...) {
}
#ExceptionHandler(ExceptionOne.class)
public R exceptionTwo(ExceptionOne exe) {
//set proper response code and return values
}
//other exceptions
}
You can look more on this at here

Should assert be placed in the test case or verification method?

For regression testing (not unit testing), where we have elaborate scenarios written in TestNG, is there a proper place the Assert checks should be done? Does it matter or not if it's in the test case, or in a calling method? For example:
This test case calls a validation method that contains the asserts:
#Test
public void test1() {
validateResponse();
}
public void validateResponse() {
Assert.assertEquals(a, "123");
Assert.assertEquals(b, "455");
Assert.assertEquals(c, "5678");
Assert.assertEquals(d, "3333");
}
This test case asserts based on the return value of the verification method:
#Test
public void test1() {
Assert.assertTrue(validateResponse());
}
public boolean void validateResponse() throws Exception {
try {
if (!a.equals("123")) throw new Exception();
if (!b.equals("455")) throw new Exception();
if (!c.equals("5678")) throw new Exception();
if (!d.equals("3333")) throw new Exception();
} catch (Exception e) {
e.printStackTrace();
return false;
}
return true;
}
Your assert should be as specific and granular as possible to help the developer quickly identify the problem. e.g.
#Test
public void testResponseFields(){
// create response to be tested
// JUnit style
Assert.assertEquals("Response 'alpha' should be '123'", 123, response.getAlpha());
// TestNG style
Assert.assertEquals(response.getAlpha(), 123, "Response 'alpha' should be '123'");
}
Once you set a failure message in the Assert.assertXX call, it becomes more of a moot point as to where the Assert is called as you will have a message explaining the problem and a stack trace to see where and when it failed.

Checking for 2 expected values in Junit

I have a java program which throws an exception with 2 different messages for 2 different scenarios and I want the Junit test case to check for equality for both of these messages. As an example -
public void amethod() {
// do some processing
if(scenario1 == true) {
throw new MySystemException("An error occured due to case 1 being incorrect.");
}
else if(scenario2 == true) {
throw new MySystemException("An error occured as case 2 could not be found");
}
}
Now the JUnit for this would be something like-
public void testAMethod() {
// do something
assertEquals("Expected", "Actual");
}
As I understand, in this above example, if I use the Scenario1 exception message the junit will fail when an exception is thrown for Scenario2 and vice versa.
I would like to know if there is any other way provided in Junit by which I can use this one test method and check for both the messages for the test to pass?
Something like an OR, if possible to provide the "Expected" value with both these expected message.
I hope my query is clear enough.
Thanks
UPDATE
Sorry for the delayed response, had got caught up with some other urgent matter.
Thank you all for the very nice suggestions, it certainly has helped me to understand a bit better now.
Eventually, to keep it rather simple I decided to implement a somewhat similar solution suggested by Don Roby. So created a new test class which looks like -
public void testAMethodScenario1() {
// do the necessary
assertEquals("Expected Exception Message 1", "Actual");
}
public void testAMethodScenario2() {
// do the necessary
assertEquals("Expected Exception Message 2", "Actual");
}
Thank you all again for your responses.
I think you need to manually catch the exception (for each scenario) and individually check the message:
try {
// trigger scenario 1
fail("An exception should have been thrown here !");
} catch (MySystemException e1) {
assertEquals("Wrong error message", m1, e1.getMessage());
}
try {
// trigger scenario 2
fail("An exception should have been thrown here !");
} catch (MySystemException e2) {
assertEquals("Wrong error message", m2, e2.getMessage());
}
Of course, you can have these scenarios defined as enum constants and simply iterate through them and check each of them within a loop, since the "copy/paste design pattern" is pretty obvious in the above code. :)
You seem to be asking two things here, how to test an exception and how to assert that a value matches either of two possible expected values.
To test for an exception, you can either use a JUnit4 annotation:
#Test(expected=MySystemException.class)
public void testException() {
amethod();
}
or use a try-catch in your test:
#Test
public void testException() {
try {
amethod();
fail("MySystemException expected");
}
catch (MySystemException e) {
// Success!
}
}
And if you have only one message, in the try-catch version you can assert that you got it with an AssertEquals in the catch block.
The best testing would have separate tests for your two scenarios, and expect the correct single message. Better code might in fact have distinct exceptions for the two situations.
But the need for a more complex assertion than simple equality does come up anyway, and there's an elegant solution for it in Hamcrest matchers.
Using that for this situation, you could write something like (untested - don't trust my syntax completely):
#Test
public void testException() {
try {
amethod();
fail("MySystemException expected");
}
catch (MySystemException e) {
String expectedMessage1 = "An error occured due to case 1 being incorrect.";
String expectedMessage2 = "An error occured as case 2 could not be found";
assertThat(e.getMessage(),
anyOf(equalTo(expectedMessage1), equalTo(expectedMessage2)));
}
}
Can you predict which scenario will occur? If so, Costi's answer is correct. If not, because there's some randomness or whatever, you can write:
#Test
public void testAmethodThrowsException() {
try {
amethod();
fail("amethod() should have thrown an exception");
}
catch (MySystemException e) {
String msg = e.getMessage();
assertTrue("bad message: " + msg, msg.equals("An error occured due to case 1 being incorrect.") || msg.equals("An error occured as case 2 could not be found"));
}
}
The declared types of exception thrown bya method are part of its API. If you really want to distinguish different failure modes, you should declare a different exception type for each failure mode.
So, something like this:
/**
* Do something.
* #throws MySystemException1 in case 1.
* #throws MySystemException2 if Foo not found.
*/
public void amethod() {
// do some processing
if(scenario1 == true) {
throw new MySystemException1("Case 1.");
}
else if(scenario2 == true) {
throw new MySystemException2("Foo not found");
}
}
#Rule solution in JUnit4:
public class ExceptionRule implements MethodRule {
#Override
public Statement apply(final Statement base, final FrameworkMethod method, Object target) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
try {
base.evaluate();
Assert.fail();
} catch (MySystemException e) {
if(scenario1)
assertEquals("Expected error message1", e1.getMessage();
if(scenario2)
assertEquals("Expected error message2",e1.getMessage();
}
}
};
}
}
In your testcase, use the Rule:
#Rule public ExceptionRule rule = new ExceptionRule();
JUnit 4 provides (Expected Exception.class)
#Test(expected= MySystemException.class) public void empty() {
// what ever you want
}
Google: Expected Exceptions JUnit for more info.
BDD Style Solution with Catch Exception
#Test
public void testAMethodScenario1() {
//given scenario 1
when(foo).amethod();
then(caughtException())
.isInstanceOf(MySystemException.class)
.hasMessage("An error occured due to case 1 being incorrect.");
}
#Test
public void testAMethodScenario2() {
//given scenario 2
when(foo).amethod();
then(caughtException())
.isInstanceOf(MySystemException.class)
.hasMessage("An error occured as case 2 could not be found");
}
Source code
https://gist.github.com/mariuszs/7490875
Dependencies
com.googlecode.catch-exception:catch-exception:1.2.0
A better solution with #Rule, you can assert both exception and expection message as well.
#Rule
public ExpectedException expectedException = ExpectedException.none();
#Test
public void aMethod_Scenario1True_ThrowsException() {
expectedException.expect(MySystemException.class);
expectedExcepion.expectMessage("An error occured due to case 1 being incorrect.");
//when().thenReturn();
//handle the repositories, static methods and other sub methods, if needed
amethod();
}
#Rule is the more elegant way to write the exception.

Java exception handling in non sequential tasks (pattern/good practice)

There are some task that should't be done in parallel, (for example opening a file, reading, writing, and closing, there is an order on that...)
But... Some task are more like a shoping list, I mean they could have a desirable order but it's not a must..example in communication or loading independient drivers etc..
For that kind of tasks,
I would like to know a java best practice or pattern for manage exceptions..
The java simple way is:
getUFO {
try {
loadSoundDriver();
loadUsbDriver();
loadAlienDetectorDriver();
loadKeyboardDriver();
} catch (loadSoundDriverFailed) {
doSomethingA;
} catch (loadUsbDriverFailed) {
doSomethingB;
} catch (loadAlienDetectorDriverFailed) {
doSomethingC;
} catch (loadKeyboardDriverFailed) {
doSomethingD;
}
}
But what about having an exception in one of the actions but wanting to
try with the next ones??
I've thought this approach, but don't seem to be a good use for exceptions
I don't know if it works, doesn't matter, it's really awful!!
getUFO {
Exception ex=null;
try {
try{ loadSoundDriver();
}catch (Exception e) { ex=e; }
try{ loadUsbDriver();
}catch (Exception e) { ex=e; }
try{ loadAlienDetectorDriver();
}catch (Exception e) { ex=e; }
try{ loadKeyboardDriver()
}catch (Exception e) { ex=e; }
if(ex!=null)
{ throw ex;
}
} catch (loadSoundDriverFailed) {
doSomethingA;
} catch (loadUsbDriverFailed) {
doSomethingB;
} catch (loadAlienDetectorDriverFailed) {
doSomethingC;
} catch (loadKeyboardDriverFailed) {
doSomethingD;
}
}
seems not complicated to find a better practice for doing that.. I still didn't
thanks for any advice
Consider the execute around idiom.
Another option (which isn't really all that different, it just decouples them more) is to do each task in a separate thread.
Edit:
Here is the kind of thing I have in mind:
public interface LoadableDriver {
public String getName();
public void loadDriver() throws DriverException;
public void onError(Throwable e);
}
public class DriverLoader {
private Map<String, Exception> errors = new HashMap<String, Exception>();
public void load(LoadableDriver driver) {
try {
driver.loadDriver();
} catch (DriverException e) {
errors.put(driver.getName(), e);
driver.onError(e);
}
}
public Map<String, Exception> getErrors() { return errors; }
}
public class Main {
public void loadDrivers() {
DriverLoader loader = new DriverLoader();
loader.loadDriver(new LoadableDriver(){
public String getName() { return "SoundDriver"; }
public void loadDriver() { loadSoundDriver(); }
public void onError(Throwable e) { doSomethingA(); }
});
//etc. Or in the alternative make a real class that implements the interface for each driver.
Map<String, Exception> errors = loader.getErrors();
//react to any specific drivers that were not loaded and try again.
}
}
Edit: This is what a clean Java version would ultimately look like if you implemented the drivers as classes (which is what the Java OO paradigm would expect here IMHO). The Main.loadDrivers() method would change like this:
public void loadDrivers(LoadableDriver... drivers) {
DriverLoader loader = ...
for(LoadableDriver driver : drivers) {
loader.load(driver);
}
//retry code if you want.
Set<LoadableDriver> failures = loader.getErrors();
if(failures.size() > 0 && tries++ > MAX_TRIES) {
//log retrying and then:
loadDrivers(drivers.toArray(new LoadableDriver[0]));
}
}
Of course I no longer use a map because the objects would be self-sufficient (you could get rid of the getName() method as well, but probably should override toString()), so the errors are just returned in a set to retry. You could make the retry code even simpler if each driver was responsible for knowing how often it should it retry.
Java won't look as nice as a well done C++ template, but that is the Java language design choice - prefer simplicity over complex language features that can make code hard to maintain over time if not done properly.
Try this:
protected void loadDrivers() {
loadSoundDriver();
loadUsbDriver();
loadAlienDetectorDriver();
loadKeyboardDriver();
}
Then:
protected void loadSoundDriver() {
try {
// original code ...
}
catch( Exception e ) {
soundDriverFailed( e );
}
}
protected void soundDriverFailed( Exception e ) {
log( e );
}
This gives subclasses a chance to change the behaviour. For example, a subclass could implement loading each driver in a separate thread. The main class need not care about how the drivers are loaded, nor should any users of the main class.
IMO, for your case, if the exception is "ignorable" it's best if the "loadSoundDriver" method catches the exception and simply returns an error.
Then in the function that loads stuff, you can record all the errors and at the end of the sequence, decide what to do with them.
[edit]
Something like this:
// init
MyError soundErr = loadSoundDriver();
MyError otherErr = loadOtherDriver();
if(soundErr!=null || otherErr !=null){
// handle the error(s)
}
Just surround every single load operation with its own try / catch block.
try {
loadSoundDriver();
} catch (loadSoundDriverFailed) {
doSomethingA;
}
try {
loadUsbDriver();
} catch (loadUsbDriverFailed) {
doSomethingB;
}
// ...
So you can handle every exception by itself and continue processing the oder operations.

Categories

Resources