All of my tests for my Groovy code look like this
public void testButtons() {
try {
page.getButtons();
} catch (Exception e) {
throw org.codehaus.groovy.runtime.StackTraceUtils.sanitize(e);
}
}
because I need to sanitize any possible StackTrace that appears (otherwise it's very hard to read since it's got all the Groovy meta-code). Is there any way to specify that all JUnit tests get wrapped in particular way (like an error handler)?
Note: I am running these in Eclipse, but if there's a way to do this in IntelliJ or Netbeans, that would be good to know.
Yes, use a Rule. Basically you have to have a class which implements the MethodRule interface that handles the exception handling in the apply method by substituting its own Statement implementation that has the try/catch in it.
To use a rule you define a field in the test class like so:
#Rule public MethodRule exceptionCleanser = new ExceptionCleanser();
A first cut implementation would probably look something like this:
public class ExceptionCleanser implements MethodRule {
public Statement apply(final Statement base, FrameworkMethod method, Object target) {
return new Statement() {
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Exception e) {
throw org.codehaus.groovy.runtime.StackTraceUtils.sanitize(e);
}
}
};
}
}
The above is totally untested, but you should be able to get the idea. The #Rule annotation was introduced in JUnit 4.7, so you may need to update to use it.
Related
I'm looking for a way to cause a succeed through an custom exception without expecting it all the time in junit4. Is this possible with a testrule or something, without touching every single testcase?
I know these options exist but then the exception is expected and the test fails, if no exception is thrown. I want the test to continue even if no exception is thrown and just use the exception to end the test in some special cases through aspectj.
#Test(TestSuccessException.class)
public void testCase() {
...
}
public class TestClass{
#Rule
public ExpectedException thrown = ExpectedException.none();
#Test
public void someTest() {
thrown.expect(MyRuntimeException.class);
...
}
}
As far as the junit4 source code looks, there isn't a way to achieve this.
The only way I found is by customizing the runner itself.
So something like this:
public class CustomTestRunner extends Runner {
private Class testClass;
public CustomTestRunner(Class testClass) {
super();
this.testClass = testClass;
}
#Override
public Description getDescription() {
return Description.EMPTY;
}
#Override
public void run(RunNotifier notifier) {
// Load all methods with #Test annotation from the given class and fire the execution
try {
Object testObject = testClass.getConstructor().newInstance();
for (Method method : testClass.getMethods()) {
if (method.isAnnotationPresent(Test.class)) {
fire(notifier, testObject, method);
}
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private void fire(RunNotifier notifier, Object testObject, Method method) throws IllegalAccessException, InvocationTargetException {
notifier.fireTestStarted(Description
.createTestDescription(testClass, method.getName()));
try {
// Call the test method
method.invoke(testObject);
} catch (InvocationTargetException e) {
// method.invoke wraps the original exception with InvocationTargetException
// The original exception is accessible via getCause()
// Check if the type of the original exception is the custom "early exist" exception
// If it doesn't match, throw the exception again; otherwise, ignore and mark the test as successful
if (!(e.getCause() instanceof EndTestEarlyException)) {
throw e;
}
}
notifier.fireTestFinished(Description
.createTestDescription(testClass, method.getName()));
}
}
You can use this by annotating the Test class as follows:
#RunWith(CustomTestRunner.class)
class MyIntegrationTest {
...
}
Note: Runner is the most generic Runner possible.
You could also attempt overriding a more specific runner if you already use one.
Edit:
As you are working with legacy, I intentionally tried not to use newer language features, like generics (Class<?>).
The solution is based on this baeldung article.
Junit5
Last but not least:
This is probably not relevant in your particular case but might be interesting for future readers.
If you manage to upgrade to Junit5, you could handle this within an extension.
You could implement a custom extension like this:
public class IgnoreEndTestEarlyException implements TestExecutionExceptionHandler {
#Override
public void handleTestExecutionException(ExtensionContext context,
Throwable throwable) throws Throwable {
if (throwable instanceof EndTestEarlyException ) {
return;
}
throw throwable;
}
}
And use it like this:
#ExtendWith(IgnoreEndTestEarlyException.class)
public class MyIntegrationTest
I tend to create another annotation (something like #IntegrationTest), put the #ExtendsWith on there, and use the new annotation.
It would be cleaner and easier to add multiple extensions.
You can run Junit4 and Junit5 within the same module, but you must replace all annotations within your integration test suit.
It might not be worth the effort for multiple thousand tests.
For Junit4 I found a better solution for my usecase. Just override the runChild Method from BlockJUnit4ClassRunner and add a try catch for the EndTestEarlyException.
#Override
protected void runChild(final FrameworkMethod method, RunNotifier notifier) {
Description description = describeChild(method);
if (isIgnored(method)) {
notifier.fireTestIgnored(description);
} else {
Statement statement = new Statement() {
#Override
public void evaluate() throws Throwable {
try {
methodBlock(method).evaluate();
} catch (EndTestEarlyException e) {
System.out.println("EndTestEarlyException - ignore");
}
}
};
runLeaf(statement, description, notifier);
}
}
I use exceptions to flow control a lot, but I have a strange feeling that I am doing something wrong. Is it a good practice to write something like code shown bellow?
public static void main(String[] args)
{
try
{
methodA();
}
catch (Exception ex)
{
handleException(ex);
}
}
public static methodA()
{
methodB();
}
public static methodB()
{
if(someConditionIsNotMet)
{
throw new RuntimeException("Some Condition Is Not Met");
}
}
I use exceptions to flow control a lot
Throwing specific and functional exceptions to indicate a functional issue during the workflow is not bad itself.
It is not the single way to do it but it is a valid way.
Another way relies on methods that returns boolean and testing the returned value.
Personally, I don't use this way as I found it rather verbose, error-prone (we have not to forget to test the returned boolean) and less expressive (it has only two values: true and false) than an exception (it may have as many as required).
Suppose that method B has to check something and that if the check fails the processing should be stopped and the client notified of the issue, it would be totally valid to use exceptions for this purpose.
Now, it would make more sense to make the exception a specific exception rather than Exception.
Otherwise how the client could interpret the exception meaning ?
The exception could be a workflow exception but it could be also any exception thrown at runtime for another reason such as a NullPointerException.
You want to handle workflow exceptions in a specific way while you will not apply a specific processing to other thrown exceptions.
For example you could write something as :
public static methodA()
{
methodB();
}
public static methodB(){
if (!expectedDataFound()){
throw new DataNotFoundException("data xxx was not found");
}
if (!hasRights()){
throw new PermissionException("user xxx has not the rights for yyy");
}
}
Then from the client side, you have two ways.
Catching each exception individually or catching them in a common way (that is possible only if they make part of the same hierarchy).
Catching each exception individually :
public static void main(String[] args)
{
try
{
methodA();
}
catch (DataNotFoundException ex)
{
handleDataNotFoundException(ex);
}
catch (PermissionException ex)
{
handlePermissionException(ex);
}
}
Catching exception globally:
public static void main(String[] args)
{
try
{
methodA();
}
catch (WorkflowException ex)
{
handleWorkflowException(ex);
}
}
I think you are too harsh on yourself with saying that you "use exceptions to control flow". It is an antipattern to use exceptions for control flow, but in your example you do not.
Let's say that you have a method that sets the age for the user, and of course if the caller provided negative number, you should not complete the action. So a very reasonable way to ensure that, would be:
public void setAge(int age) {
if(age <0) {
throw new InvalidArgumentException("Age has to be zero or positive number");
}
}
If you prefer not to use exceptions maybe you can use features of the language such as Optionals or create response structure that handles both success and errors. For example, lets say you have a method that retrieves employees
public EmployeesOverview getEmployees() { ... }
Your response class could look like this:
public class EmployeesOverview {
private Ok ok;
private Error error;
class Ok {
private Set<Employee> employees;
}
class Error {
private String errorMessage;
}
}
So without throwing exception your method will provide clients with results or if there is a problem, the client will be informed about it.
I would separate flow control and exception handling. Flow control is meant for making sure statements are executed in correct sequence and under correct conditions. This must be determined at design time. Exception handling is meant to handle unforeseen situations at run time. Exceptions are almost always due to external factors: time outs, no disk space, data errors...
Just my two cents.
How do I use Hamcrest to test for an exception? According to a comment in https://code.google.com/p/hamcrest/wiki/Tutorial, "Exception handling is provided by Junit 4 using the expected attribute."
So I tried this and found that it worked:
public class MyObjectifyUtilTest {
#Test
public void shouldFindFieldByName() throws MyObjectifyNoSuchFieldException {
String fieldName = "status";
String field = MyObjectifyUtil.getField(DownloadTask.class, fieldName);
assertThat(field, equalTo(fieldName));
}
#Test(expected=MyObjectifyNoSuchFieldException.class)
public void shouldThrowExceptionBecauseFieldDoesNotExist() throws MyObjectifyNoSuchFieldException {
String fieldName = "someMissingField";
String field = MyObjectifyUtil.getField(DownloadTask.class, fieldName);
assertThat(field, equalTo(fieldName));
}
}
Does Hamcrest provide any additional functionality above and beyond the #Test(expected=...) annotation from JUnit?
While someone asked about this in Groovy (How to use Hamcrest to test for exception?), my question is for unit tests written in Java.
Do you really need to use the Hamcrest library?
If not, here's how you do it with Junit's support for exception testing. The ExpectedException class has a lot of methods that you can use to do what you want beyond checking the type of the thrown Exception.
You can use the Hamcrest matchers in combination with this to assert something specific, but it's better to let Junit expect the thrown exceptions.
public class MyObjectifyUtilTest {
// create a rule for an exception grabber that you can use across
// the methods in this test class
#Rule
public ExpectedException exceptionGrabber = ExpectedException.none();
#Test
public void shouldThrowExceptionBecauseFieldDoesNotExist() throws MyObjectifyNoSuchFieldException {
String fieldName = "someMissingField";
// a method capable of throwing MyObjectifyNoSuchFieldException too
doSomething();
// assuming the MyObjectifyUtil.getField would throw the exception,
// I'm expecting an exception to be thrown just before that method call
exceptionGrabber.expect(MyObjectifyNoSuchFieldException.class);
MyObjectifyUtil.getField(DownloadTask.class, fieldName);
...
}
}
This approach better than the
#Test (expected=...) approach because #Test (expected=...) only
tests if the method execution halts by throwing the given exception,
not if the call you wanted to throw the exception threw one. For example, the test will succeed even if doSomething method threw the MyObjectifyNoSuchFieldException exception which may not be desirable
You get to test more than just the type of the exception being thrown. For example, you could check for a particular exception instance or exception message and so on
The try/catch block approach, because of readability and conciseness.
I couldn't implement it in a nice way if counting assertion error descriptions (probably this is why Hamcrest does not provide such a feature), but if you're playing well with Java 8 then you might want something like this (however I don't think it would be ever used because of the issues described below):
IThrowingRunnable
This interface is used to wrap code that could potentially throw exceptions. Callable<E> might be used as well, but the latter requires a value to be returned, so I think that a runnable ("void-callable") is more convenient.
#FunctionalInterface
public interface IThrowingRunnable<E extends Throwable> {
void run()
throws E;
}
FailsWithMatcher
This class implements a matcher that requires the given callback to throw an exception. A disadvantage of this implementation is that having a callback throwing an unexpected exception (or even not throwing a single) does not describe what's wrong and you'd see totally obscure error messages.
public final class FailsWithMatcher<EX extends Throwable>
extends TypeSafeMatcher<IThrowingRunnable<EX>> {
private final Matcher<? super EX> matcher;
private FailsWithMatcher(final Matcher<? super EX> matcher) {
this.matcher = matcher;
}
public static <EX extends Throwable> Matcher<IThrowingRunnable<EX>> failsWith(final Class<EX> throwableType) {
return new FailsWithMatcher<>(instanceOf(throwableType));
}
public static <EX extends Throwable> Matcher<IThrowingRunnable<EX>> failsWith(final Class<EX> throwableType, final Matcher<? super EX> throwableMatcher) {
return new FailsWithMatcher<>(allOf(instanceOf(throwableType), throwableMatcher));
}
#Override
protected boolean matchesSafely(final IThrowingRunnable<EX> runnable) {
try {
runnable.run();
return false;
} catch ( final Throwable ex ) {
return matcher.matches(ex);
}
}
#Override
public void describeTo(final Description description) {
description.appendText("fails with ").appendDescriptionOf(matcher);
}
}
ExceptionMessageMatcher
This is a sample matcher to make a simple check for the thrown exception message.
public final class ExceptionMessageMatcher<EX extends Throwable>
extends TypeSafeMatcher<EX> {
private final Matcher<? super String> matcher;
private ExceptionMessageMatcher(final Matcher<String> matcher) {
this.matcher = matcher;
}
public static <EX extends Throwable> Matcher<EX> exceptionMessage(final String message) {
return new ExceptionMessageMatcher<>(is(message));
}
#Override
protected boolean matchesSafely(final EX ex) {
return matcher.matches(ex.getMessage());
}
#Override
public void describeTo(final Description description) {
description.appendDescriptionOf(matcher);
}
}
And the test sample itself
#Test
public void test() {
assertThat(() -> emptyList().get(0), failsWith(IndexOutOfBoundsException.class, exceptionMessage("Index: 0")));
assertThat(() -> emptyList().set(0, null), failsWith(UnsupportedOperationException.class));
}
Note that this approach:
... is test-runner-independent
... allows to specify multiple assertions in a single test
And the worst thing, a typical fail will look like
java.lang.AssertionError:
Expected: fails with (an instance of java.lang.IndexOutOfBoundsException and is "Index: 0001")
but: was <foo.bar.baz.FailsWithMatcherTest$$Lambda$1/127618319#6b143ee9>
Maybe using a custom implementation of the assertThat() method could fix it.
I suppose the cleanest way is to define a function like
public static Throwable exceptionOf(Callable<?> callable) {
try {
callable.call();
return null;
} catch (Throwable t) {
return t;
}
}
somewhere and then e.g. call
assertThat(exceptionOf(() -> callSomethingThatShouldThrow()),
instanceOf(TheExpectedException.class));
perhaps also using something like the ExceptionMessageMatcher of this answer.
Since junit 4.13 you can use its Assert.assertThrows, like this:
import static org.junit.Assert.assertThrows;
...
MyObjectifyNoSuchFieldException ex = assertThrows(MyObjectifyNoSuchFieldException.class, () -> MyObjectifyUtil.getField(DownloadTask.class, fieldName));
// now you can go further and assert things about the exception ex
// if MyObjectifyUtil.getField(...) does not throw exception, the test will fail right at assertThrows
In my opinion this sort of exceptions asserting is superior to #Test(expected=MyObjectifyNoSuchFieldException.class) because you can:
further assert things about the exception itself;
assert things about side effects (in your mocks, for example);
continue your test case.
You should use junit-utils, which does contain an ExceptionMatcher that can be used together with Hamcrest's assertThat() method.
Example 1:
assertThat(() -> MyObjectifyNoSuchFieldException.class,
throwsException(MyObjectifyNoSuchFieldException.class));
Example 2:
assertThat(() -> myObject.doStuff(null),
throwsException(MyObjectifyNoSuchFieldException.class)
.withMessageContaining("ERR-120008"));
Additional details here: obvj.net/junit-utils
In addition to the above.
if you change the interfaces to ... extends Exception, you can throw an Error like this:
#Override
protected boolean matchesSafely(final IThrowingRunnable<EX> runnable) {
try {
runnable.run();
throw new Error("Did not throw Exception");
} catch (final Exception ex) {
return matcher.matches(ex);
}
}
trace will look like this:
java.lang.Error: Did not throw Exception
at de.test.test.FailsWithMatcher.matchesSafely(FailsWithMatcher.java:31)
at de.test.test.FailsWithMatcher.matchesSafely(FailsWithMatcher.java:1)
at org.hamcrest.TypeSafeMatcher.matches(TypeSafeMatcher.java:65)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:12)
at org.junit.Assert.assertThat(Assert.java:956)
at org.junit.Assert.assertThat(Assert.java:923)
at
...
I'll try to provide a hackneyed, useless example that reduces the problem nicely :-)
I have a GenericException, and a MoreSpecificException which extends GenericException.
I need to test that SomeService.doThis() throws a MoreSpecificException. JUnit lets me do this elegantly like so.
#Test(expected = MoreSpecificException.class)
public void testDoThis() throws GenericException {
new SomeService().doThis();
}
However, I also need to test that SomeService.doThat() throws a GenericException, so I tried this.
#Test(expected = GenericException.class)
public void testDoThat() throws GenericException {
new SomeService().doThat();
}
However, I found that if doThat() actually throws a MoreSpecificException then the second test still passes. I assume this is because MoreSpecificException is a GenericException and the annotation is implemented to respect that relationship.
While this is a sensible default behaviour, I don't want this. I want to test that doThat() throws a GenericException and only a GenericException. If it throws a MoreSpecificException or any other subclass of GenericException, I want the test to fail.
Reading the docs it doesn't seem I can do anything with the annotation to change this behaviour, so looks like I'll have to use another solution.
At the moment I'm resorting to the following ugly solution - EDIT made significantly less ugly by Nathan Hughes' answer :-)
#Test
public void testDoThat() {
try {
new SomeService().doThat();
Assert.fail();
} catch(GenericException ex) {
Assert.assertEquals(GenericException.class, ex.getClass());
}
}
Is there a more elegant way to achieve what I want within the JUnit framework?
BDD Style Solution
JUnit 4 + Catch Exception + AssertJ
The most elegant solution ;) Readable, without boilerplate code.
#Test
public void testDoThat() {
when(new SomeService()).doThat();
then(caughtException()).isExactlyInstanceOf(GenericException.class);
}
The code is identical for FEST Assertions 2 + Catch-Exceptions.
Source code
https://gist.github.com/mariuszs/7489706
Dependencies
org.assertj:assertj-core:1.4.0
com.googlecode.catch-exception:catch-exception:1.2.0
You can assert that the class of the Exception is what you expect:
#Test
public void testDoThat() {
try {
new SomeService().doThat();
Assert.fail();
} catch(GenericException ex) {
assertEquals(GenericException.class, ex.getClass());
}
}
Also got rid of the flag, instead having the test fail if no exception is thrown.
You can use the ExpectedException rule and a custom Hamcrest matcher that specifies which class can be thrown.
The following test will print that you expected an instance of RuntimeException, but got an IllegalArgumentException.
#Rule
public ExpectedException thrown = ExpectedException.none();
#Test
public void testThrows() {
thrown.expect(isClass(RuntimeException.class));
throw new IllegalArgumentException("FAKE");
}
public class ClassMatchMatcher extends BaseMatcher<Object> {
private final Class<?> expectedClass;
private ClassMatchMatcher(Class<?> expectedClass) {
this.expectedClass = expectedClass;
}
#Override
public boolean matches(Object item) {
return expectedClass.equals(item.getClass());
}
#Override
public void describeTo(Description description) {
description.appendText("an instance of ")
.appendText(expectedClass.getName());
}
}
public class ExtraMatchers {
public static Matcher<Object> isClass(Class<?> aClass) {
return new ClassMatchMatcher(aClass);
}
}
Edit: Added a static factory method to make the test code cleaner.
I have a couple of classes following the "Template Method" pattern. Abstract class A, and concrete extensions, B and C. Like this:
public abstract class A
{
protected abstract String getData() throws SomeException;
public void doWork() throws OtherException
{
try
{
// business logic ...
String data = this.getData();
// more business logic ...
}
catch(SomeException e)
{
log("...", e);
throw new OtherException("...", e);
}
}
}
public Class B extends A
{
protected String getData() throws SomeException
{
// complicated logic relying on lots of dependencies
}
}
public Class C extends A
{
protected String getData() throws SomeException
{
// different but equally complicated logic relying on lots of dependencies
}
}
I want to write a test to verify when getData() throws SomeException that OtherException is thrown. I really want to avoid mocking up all of the complicated dependencies that would be required to force getData() to throw. I don't care how getData() throws, I just want it to throw. So I think a partial mock is what I want. This is what I have:
import static org.easymock.EasyMock.*;
....
#Test(expected = OtherException.class)
public void testSomethingOrAnother() throws Exception
{
B target = createMockBuilder(B.class).addMockedMethod("getData").createMock();
expect(target.getData()).andThrow(SomeException.class).once();
replay(target)
try
{
target.doWork(); // expect this to throw OtherException;
}
finally
{
verify(target);
}
}
The test looks good to me, but when I run it I get this:
java.lang.Exception: Unexpected exception, expected<OtherException> but was<java.lang.RuntimeException>
... deleted for brevity ...
Caused by: java.lang.RuntimeException: Ambiguous name: More than one method are named getData
at org.easymock.internal.ReflectionUtils.findMethod(ReflectionUtils.java:96)
at org.easymock.internal.ReflectionUtils.findMethod(ReflectionUtils.java:64)
at org.easymock.internal.MockBuilder.addMockedMethod(MockBuilder.java:73)
at org.easymock.internal.MockBuilder.addMockedMethods(MockBuilder.java:92)
at com.mycompany.more.packages.BTest(BTest.java:83)
... deleted for brevity ...
... 16 more
To be clear: There is NOT an overload of the getData() method anywhere in the hierarchy.
Is EasyMock able to do what I'm trying to do here? What am I missing?
relevant versions numbers:
EasyMock 3.0
JUnit 4.4
Java 1.6
I think your problem may be the use of the addMockedMethod(String). Not sure why EasyMock is complaining about an ambiguous method name if there are no overloads. But the following worked for me:
#Test
public void testSomethingOrAnother() {
B target = null;
try {
target = EasyMock.createMockBuilder(B.class).addMockedMethod(B.class.getDeclaredMethod("getData")).createMock();
EasyMock.expect(target.getData()).andThrow(new SomeException());
EasyMock.replay(target);
} catch (NoSuchMethodException e) {
fail(e.getMessage());
} catch (SomeException e) {
fail(e.getMessage());
}
try {
target.doWork();
fail("doWork should have thrown an exception");
} catch (OtherException e) {
//pass
}
}
With Easymock:3.2 you are able to specify types of the parameters of the method. Take a look IMockBuilder#addMockedMethod(String methodName,Class<?>... parameterTypes)
Thanks.