I'll try to provide a hackneyed, useless example that reduces the problem nicely :-)
I have a GenericException, and a MoreSpecificException which extends GenericException.
I need to test that SomeService.doThis() throws a MoreSpecificException. JUnit lets me do this elegantly like so.
#Test(expected = MoreSpecificException.class)
public void testDoThis() throws GenericException {
new SomeService().doThis();
}
However, I also need to test that SomeService.doThat() throws a GenericException, so I tried this.
#Test(expected = GenericException.class)
public void testDoThat() throws GenericException {
new SomeService().doThat();
}
However, I found that if doThat() actually throws a MoreSpecificException then the second test still passes. I assume this is because MoreSpecificException is a GenericException and the annotation is implemented to respect that relationship.
While this is a sensible default behaviour, I don't want this. I want to test that doThat() throws a GenericException and only a GenericException. If it throws a MoreSpecificException or any other subclass of GenericException, I want the test to fail.
Reading the docs it doesn't seem I can do anything with the annotation to change this behaviour, so looks like I'll have to use another solution.
At the moment I'm resorting to the following ugly solution - EDIT made significantly less ugly by Nathan Hughes' answer :-)
#Test
public void testDoThat() {
try {
new SomeService().doThat();
Assert.fail();
} catch(GenericException ex) {
Assert.assertEquals(GenericException.class, ex.getClass());
}
}
Is there a more elegant way to achieve what I want within the JUnit framework?
BDD Style Solution
JUnit 4 + Catch Exception + AssertJ
The most elegant solution ;) Readable, without boilerplate code.
#Test
public void testDoThat() {
when(new SomeService()).doThat();
then(caughtException()).isExactlyInstanceOf(GenericException.class);
}
The code is identical for FEST Assertions 2 + Catch-Exceptions.
Source code
https://gist.github.com/mariuszs/7489706
Dependencies
org.assertj:assertj-core:1.4.0
com.googlecode.catch-exception:catch-exception:1.2.0
You can assert that the class of the Exception is what you expect:
#Test
public void testDoThat() {
try {
new SomeService().doThat();
Assert.fail();
} catch(GenericException ex) {
assertEquals(GenericException.class, ex.getClass());
}
}
Also got rid of the flag, instead having the test fail if no exception is thrown.
You can use the ExpectedException rule and a custom Hamcrest matcher that specifies which class can be thrown.
The following test will print that you expected an instance of RuntimeException, but got an IllegalArgumentException.
#Rule
public ExpectedException thrown = ExpectedException.none();
#Test
public void testThrows() {
thrown.expect(isClass(RuntimeException.class));
throw new IllegalArgumentException("FAKE");
}
public class ClassMatchMatcher extends BaseMatcher<Object> {
private final Class<?> expectedClass;
private ClassMatchMatcher(Class<?> expectedClass) {
this.expectedClass = expectedClass;
}
#Override
public boolean matches(Object item) {
return expectedClass.equals(item.getClass());
}
#Override
public void describeTo(Description description) {
description.appendText("an instance of ")
.appendText(expectedClass.getName());
}
}
public class ExtraMatchers {
public static Matcher<Object> isClass(Class<?> aClass) {
return new ClassMatchMatcher(aClass);
}
}
Edit: Added a static factory method to make the test code cleaner.
Related
I'm looking for a way to cause a succeed through an custom exception without expecting it all the time in junit4. Is this possible with a testrule or something, without touching every single testcase?
I know these options exist but then the exception is expected and the test fails, if no exception is thrown. I want the test to continue even if no exception is thrown and just use the exception to end the test in some special cases through aspectj.
#Test(TestSuccessException.class)
public void testCase() {
...
}
public class TestClass{
#Rule
public ExpectedException thrown = ExpectedException.none();
#Test
public void someTest() {
thrown.expect(MyRuntimeException.class);
...
}
}
As far as the junit4 source code looks, there isn't a way to achieve this.
The only way I found is by customizing the runner itself.
So something like this:
public class CustomTestRunner extends Runner {
private Class testClass;
public CustomTestRunner(Class testClass) {
super();
this.testClass = testClass;
}
#Override
public Description getDescription() {
return Description.EMPTY;
}
#Override
public void run(RunNotifier notifier) {
// Load all methods with #Test annotation from the given class and fire the execution
try {
Object testObject = testClass.getConstructor().newInstance();
for (Method method : testClass.getMethods()) {
if (method.isAnnotationPresent(Test.class)) {
fire(notifier, testObject, method);
}
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private void fire(RunNotifier notifier, Object testObject, Method method) throws IllegalAccessException, InvocationTargetException {
notifier.fireTestStarted(Description
.createTestDescription(testClass, method.getName()));
try {
// Call the test method
method.invoke(testObject);
} catch (InvocationTargetException e) {
// method.invoke wraps the original exception with InvocationTargetException
// The original exception is accessible via getCause()
// Check if the type of the original exception is the custom "early exist" exception
// If it doesn't match, throw the exception again; otherwise, ignore and mark the test as successful
if (!(e.getCause() instanceof EndTestEarlyException)) {
throw e;
}
}
notifier.fireTestFinished(Description
.createTestDescription(testClass, method.getName()));
}
}
You can use this by annotating the Test class as follows:
#RunWith(CustomTestRunner.class)
class MyIntegrationTest {
...
}
Note: Runner is the most generic Runner possible.
You could also attempt overriding a more specific runner if you already use one.
Edit:
As you are working with legacy, I intentionally tried not to use newer language features, like generics (Class<?>).
The solution is based on this baeldung article.
Junit5
Last but not least:
This is probably not relevant in your particular case but might be interesting for future readers.
If you manage to upgrade to Junit5, you could handle this within an extension.
You could implement a custom extension like this:
public class IgnoreEndTestEarlyException implements TestExecutionExceptionHandler {
#Override
public void handleTestExecutionException(ExtensionContext context,
Throwable throwable) throws Throwable {
if (throwable instanceof EndTestEarlyException ) {
return;
}
throw throwable;
}
}
And use it like this:
#ExtendWith(IgnoreEndTestEarlyException.class)
public class MyIntegrationTest
I tend to create another annotation (something like #IntegrationTest), put the #ExtendsWith on there, and use the new annotation.
It would be cleaner and easier to add multiple extensions.
You can run Junit4 and Junit5 within the same module, but you must replace all annotations within your integration test suit.
It might not be worth the effort for multiple thousand tests.
For Junit4 I found a better solution for my usecase. Just override the runChild Method from BlockJUnit4ClassRunner and add a try catch for the EndTestEarlyException.
#Override
protected void runChild(final FrameworkMethod method, RunNotifier notifier) {
Description description = describeChild(method);
if (isIgnored(method)) {
notifier.fireTestIgnored(description);
} else {
Statement statement = new Statement() {
#Override
public void evaluate() throws Throwable {
try {
methodBlock(method).evaluate();
} catch (EndTestEarlyException e) {
System.out.println("EndTestEarlyException - ignore");
}
}
};
runLeaf(statement, description, notifier);
}
}
I'm trying to figure out a way to build a method that will test whether a checked exception has indeed been thrown. As I was building the following minimal working example (CustomThrowablexxx are all custom types declared on their own files for readability):
package demos.exceptions;
public class TestExceptions {
// This method will check whether the provided method throws exceptions
// of the type provided.
private static boolean throwsParticularThrowable(Runnable method,
Class<Throwable> cls){
try {
method.run();
} catch(Throwable t){
if(t.getClass().equals(cls))
return true;
}
return false;
}
private static void methodOne() throws CustomThrowableOne {
throw new CustomThrowableOne("methodOne() throws");
}
private static void methodTwo() throws CustomThrowableTwo {
throw new CustomThrowableTwo("methodTwo() throws");
}
private static void methodThree() throws CustomThrowableThree {
throw new CustomThrowableThree("methodThree() throws");
}
public static void main(String[] args){
if(!throwsParticularThrowable(TestExceptions::methodOne,
CustomThrowableOne.class))
System.out.println("Nope!");
}
}
I unfortunately noticed that the access to TestExceptions::methodOne was not safe, because the compiler complained that I'm not checking for the throwing of methodOne, which I guess makes sense.
Is there any way I can automate this instead of copying and pasting the code inside throwsParticularThrowable every time?
I don't know what you are looking for but it's easier to test if exceptions are thrown using JUnit ExpectedException
https://junit.org/junit4/javadoc/4.12/org/junit/rules/ExpectedException.html
How do I use Hamcrest to test for an exception? According to a comment in https://code.google.com/p/hamcrest/wiki/Tutorial, "Exception handling is provided by Junit 4 using the expected attribute."
So I tried this and found that it worked:
public class MyObjectifyUtilTest {
#Test
public void shouldFindFieldByName() throws MyObjectifyNoSuchFieldException {
String fieldName = "status";
String field = MyObjectifyUtil.getField(DownloadTask.class, fieldName);
assertThat(field, equalTo(fieldName));
}
#Test(expected=MyObjectifyNoSuchFieldException.class)
public void shouldThrowExceptionBecauseFieldDoesNotExist() throws MyObjectifyNoSuchFieldException {
String fieldName = "someMissingField";
String field = MyObjectifyUtil.getField(DownloadTask.class, fieldName);
assertThat(field, equalTo(fieldName));
}
}
Does Hamcrest provide any additional functionality above and beyond the #Test(expected=...) annotation from JUnit?
While someone asked about this in Groovy (How to use Hamcrest to test for exception?), my question is for unit tests written in Java.
Do you really need to use the Hamcrest library?
If not, here's how you do it with Junit's support for exception testing. The ExpectedException class has a lot of methods that you can use to do what you want beyond checking the type of the thrown Exception.
You can use the Hamcrest matchers in combination with this to assert something specific, but it's better to let Junit expect the thrown exceptions.
public class MyObjectifyUtilTest {
// create a rule for an exception grabber that you can use across
// the methods in this test class
#Rule
public ExpectedException exceptionGrabber = ExpectedException.none();
#Test
public void shouldThrowExceptionBecauseFieldDoesNotExist() throws MyObjectifyNoSuchFieldException {
String fieldName = "someMissingField";
// a method capable of throwing MyObjectifyNoSuchFieldException too
doSomething();
// assuming the MyObjectifyUtil.getField would throw the exception,
// I'm expecting an exception to be thrown just before that method call
exceptionGrabber.expect(MyObjectifyNoSuchFieldException.class);
MyObjectifyUtil.getField(DownloadTask.class, fieldName);
...
}
}
This approach better than the
#Test (expected=...) approach because #Test (expected=...) only
tests if the method execution halts by throwing the given exception,
not if the call you wanted to throw the exception threw one. For example, the test will succeed even if doSomething method threw the MyObjectifyNoSuchFieldException exception which may not be desirable
You get to test more than just the type of the exception being thrown. For example, you could check for a particular exception instance or exception message and so on
The try/catch block approach, because of readability and conciseness.
I couldn't implement it in a nice way if counting assertion error descriptions (probably this is why Hamcrest does not provide such a feature), but if you're playing well with Java 8 then you might want something like this (however I don't think it would be ever used because of the issues described below):
IThrowingRunnable
This interface is used to wrap code that could potentially throw exceptions. Callable<E> might be used as well, but the latter requires a value to be returned, so I think that a runnable ("void-callable") is more convenient.
#FunctionalInterface
public interface IThrowingRunnable<E extends Throwable> {
void run()
throws E;
}
FailsWithMatcher
This class implements a matcher that requires the given callback to throw an exception. A disadvantage of this implementation is that having a callback throwing an unexpected exception (or even not throwing a single) does not describe what's wrong and you'd see totally obscure error messages.
public final class FailsWithMatcher<EX extends Throwable>
extends TypeSafeMatcher<IThrowingRunnable<EX>> {
private final Matcher<? super EX> matcher;
private FailsWithMatcher(final Matcher<? super EX> matcher) {
this.matcher = matcher;
}
public static <EX extends Throwable> Matcher<IThrowingRunnable<EX>> failsWith(final Class<EX> throwableType) {
return new FailsWithMatcher<>(instanceOf(throwableType));
}
public static <EX extends Throwable> Matcher<IThrowingRunnable<EX>> failsWith(final Class<EX> throwableType, final Matcher<? super EX> throwableMatcher) {
return new FailsWithMatcher<>(allOf(instanceOf(throwableType), throwableMatcher));
}
#Override
protected boolean matchesSafely(final IThrowingRunnable<EX> runnable) {
try {
runnable.run();
return false;
} catch ( final Throwable ex ) {
return matcher.matches(ex);
}
}
#Override
public void describeTo(final Description description) {
description.appendText("fails with ").appendDescriptionOf(matcher);
}
}
ExceptionMessageMatcher
This is a sample matcher to make a simple check for the thrown exception message.
public final class ExceptionMessageMatcher<EX extends Throwable>
extends TypeSafeMatcher<EX> {
private final Matcher<? super String> matcher;
private ExceptionMessageMatcher(final Matcher<String> matcher) {
this.matcher = matcher;
}
public static <EX extends Throwable> Matcher<EX> exceptionMessage(final String message) {
return new ExceptionMessageMatcher<>(is(message));
}
#Override
protected boolean matchesSafely(final EX ex) {
return matcher.matches(ex.getMessage());
}
#Override
public void describeTo(final Description description) {
description.appendDescriptionOf(matcher);
}
}
And the test sample itself
#Test
public void test() {
assertThat(() -> emptyList().get(0), failsWith(IndexOutOfBoundsException.class, exceptionMessage("Index: 0")));
assertThat(() -> emptyList().set(0, null), failsWith(UnsupportedOperationException.class));
}
Note that this approach:
... is test-runner-independent
... allows to specify multiple assertions in a single test
And the worst thing, a typical fail will look like
java.lang.AssertionError:
Expected: fails with (an instance of java.lang.IndexOutOfBoundsException and is "Index: 0001")
but: was <foo.bar.baz.FailsWithMatcherTest$$Lambda$1/127618319#6b143ee9>
Maybe using a custom implementation of the assertThat() method could fix it.
I suppose the cleanest way is to define a function like
public static Throwable exceptionOf(Callable<?> callable) {
try {
callable.call();
return null;
} catch (Throwable t) {
return t;
}
}
somewhere and then e.g. call
assertThat(exceptionOf(() -> callSomethingThatShouldThrow()),
instanceOf(TheExpectedException.class));
perhaps also using something like the ExceptionMessageMatcher of this answer.
Since junit 4.13 you can use its Assert.assertThrows, like this:
import static org.junit.Assert.assertThrows;
...
MyObjectifyNoSuchFieldException ex = assertThrows(MyObjectifyNoSuchFieldException.class, () -> MyObjectifyUtil.getField(DownloadTask.class, fieldName));
// now you can go further and assert things about the exception ex
// if MyObjectifyUtil.getField(...) does not throw exception, the test will fail right at assertThrows
In my opinion this sort of exceptions asserting is superior to #Test(expected=MyObjectifyNoSuchFieldException.class) because you can:
further assert things about the exception itself;
assert things about side effects (in your mocks, for example);
continue your test case.
You should use junit-utils, which does contain an ExceptionMatcher that can be used together with Hamcrest's assertThat() method.
Example 1:
assertThat(() -> MyObjectifyNoSuchFieldException.class,
throwsException(MyObjectifyNoSuchFieldException.class));
Example 2:
assertThat(() -> myObject.doStuff(null),
throwsException(MyObjectifyNoSuchFieldException.class)
.withMessageContaining("ERR-120008"));
Additional details here: obvj.net/junit-utils
In addition to the above.
if you change the interfaces to ... extends Exception, you can throw an Error like this:
#Override
protected boolean matchesSafely(final IThrowingRunnable<EX> runnable) {
try {
runnable.run();
throw new Error("Did not throw Exception");
} catch (final Exception ex) {
return matcher.matches(ex);
}
}
trace will look like this:
java.lang.Error: Did not throw Exception
at de.test.test.FailsWithMatcher.matchesSafely(FailsWithMatcher.java:31)
at de.test.test.FailsWithMatcher.matchesSafely(FailsWithMatcher.java:1)
at org.hamcrest.TypeSafeMatcher.matches(TypeSafeMatcher.java:65)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:12)
at org.junit.Assert.assertThat(Assert.java:956)
at org.junit.Assert.assertThat(Assert.java:923)
at
...
I have a JUnit test class that has two test methods:
#Test
public void test1() {
// Setup: Create foo1.m
// Exercise
// Tear Down: Delete foo1.m
}
#Test
public void test2() {
// Setup: Create foo2.m
// Exercise
// Tear Down: Delete foo2.m
}
For each method, I would like to make sure that, if the Exercise section fails for any reason, the Tear Down will still run. Note that the Setup and Tear Down code for both test methods are different, so I don't think I can use JUnit's #Before and #After annotations to do what I want.
I could put TRY-CATCH blocks into each test method:
#Test
public void test2() {
// Setup: Create foo2.m
try {
// Exercise
} finally {
// Tear Down: Delete foo2.m
}
}
but that seems ugly. Is there a way to make sure the test-method-specific tear down code in each test method is executed, without using a TRY-CATCH block?
If the setup and teardown are different, you are essentially cramming two different test fixtures into a single file. The sensible answer is to put them in separate files and use the normal annotations. If they have anything in common separate that out into a common abstract class.
Adding multiple setups in the same file can easily result in a situation where it's not clear which instance members are used in which tests, so that maintaining the tests becomes a lot harder than it needs to be.
Update:
I found a better solution, so I include here, the original answer can be found below. I think JUnit 4 rules can be used here:
class PrepareFile implements org.junit.rules.TestRule {
#Retention(RetentionPolicy.RUNTIME)
public #interface FileName {
String value() default "";
}
#Override
public Statement apply(final Statement statement, final Description description) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
String fileName = description.getAnnotation(FileName.class).value();
File file = new File(fileName);
try {
file.createNewFile();
statement.evaluate();
} finally {
file.delete();
}
}
};
}
}
Using it in the test:
#Rule
public PrepareFile prepareFile = new PrepareFile();
#Test
#PrepareFile.FileName("foo1.m")
public void test1() {
// Exercise
}
#Test
#PrepareFile.FileName("foo2.m")
public void test2() {
// Exercise
}
Here comes my original answer:
You may try to use the #BeforeClass and #AfterClass annotations.
#BeforeClass
public static void setUp() {
// Setup1: Create foo1.m
// Setup2: Create foo2.m
}
#AfterClass
public static void tearDown() {
// Tear Down1: Delete foo1.m
// Tear Down2: Delete foo2.m
}
#Test
public void test1() {
// Exercise
}
#Test
public void test2() {
// Exercise
}
This way you can setup and tear down all test cases once and the framework ensures that teadDown() is called in case of errors as well.
Good time!
I need to substitute the class' private void method with a mock implementation, and can't to figure out how to do this. I've tried to use such a construction:
Test test = PowerMock.createPartialMock(Test.class, "setId");
PowerMock.expectPrivate(test , "setId", EasyMock.anyLong()).andAnswer(
new IAnswer<Void>() {
#Override
public Void answer() throws Throwable {
return null;
}
});
PowerMock.replay(test);
but the internal PowerMock's class called WhiteBox invokes my "setId" method which is wrong for my task. Could someone, please, suggest, how to avoid the method invokation and possibly to replace the method body with a custom one?
Finally. I've got the solution.
The problem was I missed the following annotations:
#RunWith(PowerMockRunner.class)
#PrepareForTest(Test.class)
Anyway, it seems rather confusing that to make the PowerMock working I need to add some annotations. If that wasn't a legacy code I'd prefer Mockito.
Not quite sure that I get the question.
But for me code below works perfect and just string "Invoked!" is getting printed
and if I remove test.setS(33L); test will fail with exception:
#RunWith(PowerMockRunner.class)
#PrepareForTest(MainTest.Test2.class)
public class MainTest {
#Test
public void testName() throws Exception {
Test2 test = PowerMock.createPartialMock(Test2.class, "setS");
PowerMock.expectPrivate(test , "setS", EasyMock.anyLong()).andAnswer(
new IAnswer<Void>() {
#Override
public Void answer() throws Throwable {
System.out.println("Invoked!");
return null;
}
}).atLeastOnce();
PowerMock.replay(test);
test.setS(33L);
PowerMock.verify(test);
}
class Test2 {
long s;
private long getS() {
return s;
}
private void setS(long s) {
this.s = s;
System.out.println("Not this!");
}
}
}