Develop Reference

Java test with expected = exception fails with assertion error [duplicate]

This question already has answers here:
Why won't this Expect Exception Junit Test Work?
(2 answers)
Closed 6 years ago.
I'm having an issue with a test case.
The method being tested has a try / catch that catches a MalformedURLException but during testing I get a failure due to a Junit AssertionError that expects a MalformedURLException. But I can't find out what it is actually throwing! Here is my code (created as a MWE in eclipse).
My method that I want to test
public void throwMalFormedURLException(){
String s = new String("www.google.com");
try{
url = new URL(s);
}catch (Exception e){
e.printStackTrace();
e.getClass();
e.getCause();
}
}
the test method
#Test (expected = MalformedURLException.class)
public void testThrowMalFormedURLException() {
MWE testClass = new MWE();
testClass.throwMalFormedURLException();
System.out.println("End of test");
}
This is the output in the console
End of test error details are: java.net.MalformedURLException: no protocol: www.google.com
at java.net.URL.(URL.java:593)
at java.net.URL.(URL.java:490)
at java.net.URL.(URL.java:439)
at MWE.throwMalFormedURLException(MWE.java:12)
at testMWE.testThrowMalFormedURLException(testMWE.java:12)
In the Junit console it says :
java.lang.AssertionError: Expected exception: Java.net.MalformedURLException
But the Junit is reporting failure, even though the console is telling me I've got a MalformedURLException.
What am I doing wrong with this test ?
Thanks for your thoughts.
David

You are catching the exception and therefore it is not being thrown.
If your intent is to test that you are capturing the exception and relaying it back to the 'user' properly, you should create tests for that specifically. You probably don't want UI elements in your unit tests* so this is a place where abstraction and DI have a lot of value. One simple approach is to create a mock object that will listen for the error message and mark a flag when the message is received. Your unit test trigger the error and then pass if the flag is set. You should also have a negative test or assert that the flag is not set prior to throwing the exception.
*Testing the UI is also a good idea but it can be a little slow. There are various tools for automating that. It generally falls in to a different phase of testing. You really want the unit tests to be really fast so that you can run them very frequently.

You have written production code that simply isn't testable. And more specifically: this code doesn't have any "programmatically" observable side effects in the first place.
Meaning: when you write "production code", the code within a method can do three things that could be observed:
Make method calls on objects that are fields of the class under test
Return some value
Throw an exception
For each of these options, you might be able to write testing code:
Using dependency injection, you can put a mocked object into your class under test. And then you can check that the expected methods are invoked on your mock object; with the parameter that you would expect.
You compare the result you get from calling that method with some expected value
You use expected to ensure that a specific exception was thrown
When you look at your code; you can see: it does none of that. It only operates on objects that are created within the method. It doesn't return any value. And, most importantly: it doesn't throw an exception!
Long story short: a caught exception isn't "leaving" the method. It is caught there, and the method ends normally. The fact that you print details about the caught exception doesn't change that.
So, the first thing you have to do: remove the whole try/catch from your production code!
And, if you want to have a more specific test, you can do something like:
#Test
public void testException() {
try {
new MWE().throwMalFormedURLException();
fail("should have thrown!");
} catch ( MalFormedURLException me ) {
assertThat(me.getMessage(), containsString("whatever"));
}
The above:
fails when no exception is thrown
fails when any other exception than MalFormedURLException is thrown
allows you to check further properties of the thrown exception

This is a valid test failure. The test asserts that calling throwMalFormedURLException() will throw MalformedURLException, but since you're catching the exception, it doesn't throw it, so the test fails.

Managing checked exceptions in different JUnit tests

I am writing a Java Unit test for one of my method. The method declaration is like this:
public int convertToInteger() throws InvalidRomanNumberException
{
int result=0;
BaseRomanNumeral num1, num2;
int i=0;
if(!validOperation())
throw new InvalidRomanNumberException();
}
Now I am trying to write two unit tests. One is to test if the right exception is thrown. Another one is to make sure that that the write conversion happens. This is how my test case looks
#Test
public void testRomanNumberConversion() {
String romanValue="MCMII";
RomanNumber num=new RomanNumber(romanValue);
assertEquals(1903,num.convertToInteger());
}
#Test(expected = InvalidRomanNumberException.class)
public void testInvalidRomanNumberExceptionThrown() {
String romanValue="MCMIIII";
RomanNumber num=new RomanNumber(romanValue);
num.convertToInteger();
}
For both these test cases I am getting an error saying Unhandled InvalidRomanNumberException. This is resolved only when I add throws InvalidRomanNumberException to each method definition. But I don't think that is the right way. Just want to check with the rest of you, what is the norm here? How should I resolve this unhandled exception message

Since it looks like InvalidRomanNumberException is a checked exception, you have to either surround it with a try-catch or declare that the method throws InvalidRomanNumberException. JUnit or not, this is the norm.
That being said, the test case method that you expect will throw a InvalidRomanNumberException should ideally declare that it throws one since there is no point suppressing it with a try-catch as your test case will fail. On the other hand, the test case method that you expect will not throw an exception can use a try-catch around the convertToInteger method and regardless of whether an exception is thrown, this test case should have an assert on the expected result from convertToInteger method.
The end result of a JUnit test case should be whether the test passed or failed. An exception at runtime would indicate neither. A JUnit test case must not crash.

This feels more like it should be an unchecked exception as opposed to a checked exception.
Recall the difference between the two: a checked exception is meant to be something that's reasonably recoverable from, like a missing file or a malformed URL. An unchecked/run time exception is meant to be something that is irrecoverable, like dividing by zero.
If a user enters in an invalid Roman numeral, it might not make sense to say that they can recover and try again - the conversion layer shouldn't be responsible for that. It sounds more like a thing should be decided at instantiation time.
If you instead make your custom exception extend RuntimeException, then you won't need to declare it to be thrown (and if you did, it wouldn't have any effect), and you won't have to deal with it in your tests.
The alternative would be to declare it to be thrown in your tests instead. This has the advantage of allowing you to keep these exceptions as checked and ensuring that the tests won't complain about you not handling the potential exception from being uncaught or unthrown.
#Test
public void testRomanNumberConversion() throws InvalidRomanNumberException {
String romanValue = "MCMII";
RomanNumber num = new RomanNumber(romanValue);
assertEquals(1903, num.convertToInteger());
}
#Test(expected = InvalidRomanNumberException.class)
public void testInvalidRomanNumberExceptionThrown() throws InvalidRomanNumberException {
String romanValue = "MCMIIII";
RomanNumber num = new RomanNumber(romanValue);
num.convertToInteger();
}

Is "throws Throwable" good practice

In the past I'd read tons of code with methods like:
public Object doSomething() throws Throwable {
...
}
Is it common practice to do that?
What are pros & cons?
throws Trowable seemed to me like the "Agent Orange" way of getting the Exception- matter done
EDIT
Handle expected Exceptions in the Method
Throw unexpected Exceptions (one by one)
Don't care of Errors
Is that the way to go?

You should not throw Throwable. Here's why.
Throwable is the top of the hierarchy of things that can be thrown and is made up of Exceptions and Errors. Since Errors by definition arise from unsalvagable conditions, it is pointless to include them in your method declaration. That leaves just Exception.
You should declare your method with throws Exception instead.
Note that the narrower the range of throws the better.
Declaring your method to be throws Exception is ok if your method doesn't generate the exceptions, but instead calls other code that is declared as throws Exception and you want exceptions to percolate up the call stack.
If your method is the generating the exception, then declare a narrower range, eg throws IOException, MyProcessingException, etc

That's a loaded question. This isn't so much about exception handling as it is about code readability.
It depends where you get your code samples from. Professionals prefer to be more specific when throwing out of a method. The main reason is that it keeps your APIs more readable. For example, if your method throws Throwable, that basically means anything could happen and your method doesn't want to deal with it, no matter what. But really, only a limited number of things could happen:
Whatever checked exceptions resulting from other calls you are making in your method
Whatever checked exceptions you are throwing on purpose based on your own assertions
Whatever unchecked exception you didn't plan for
Errors (java.lang.Error) that are more global to the JVM and the environment
By specifically stating the exceptions you want to throw, you are telling the users of your API about what they should beware of. For example, when you use InputStream, you'll notice most methods throw at least java.io.IOException, which gives you some useful information about what you should watch for.
When coding, as a general rule, you want to try to keep your APIs as expressive as possible. You've got essentially one line of code to show the public API of a method (i.e. its signature, annotations too I guess), so you want it completely expressive (return type, name, parameters, but also the thrown exceptions).
As far as catching the throwables and printing the stack trace, I'd say that you should not catch the exception unless you can do something about it. Instead, let it roll up the call stack until some class catches it to do something about it. Sometimes, it may roll all the way up to your main class, which I guess would have to catch it and print the stack trace as last resort. Basically, if you can't act upon the exception, then let it go up the call stack. Also it is extremely rare that you find yourself in a situation where you should silence an exception (i.e. catch it but do nothing about it). That's usually inviting problems when comes time to troubleshoot issues.
Here is a fun but interesting article around misuse of exception handling in general.

In some rare cases it is acceptable to throw Throwables. For example, #Around advices in Spring AOP are usually declared to throw a Throwable.
The following example is copied verbatim from Spring AOP docs:
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.ProceedingJoinPoint;
#Aspect
public class AroundExample {
#Around("com.xyz.myapp.SystemArchitecture.businessService()")
public Object doBasicProfiling(ProceedingJoinPoint pjp) throws Throwable {
// start stopwatch
Object retVal = pjp.proceed();
// stop stopwatch
return retVal;
}
}
Why is doBasicProfiling declared to throw a Throwable? Because the original method (i.e. the execution join point), might throw an Error, RuntimeException, or a checked exception. So it only makes sense to declare doBasicProfiling to throw a Throwable.

Functionally, it is equivalent with throws Exception, since errors are unchecked.
I see no reason to declare a method to throw Throwable. However, this doesn't mean that catch and printStackTrace is a good alternative.
Usually, you want to catch throwables where you can do something sensible with them.
Code that throws a throwable you don't expect should explode gloriously, so you can see the error and fix the bug.

Is it common practice to do that?
In the JDK it is rare. This is mostly used when it is not clear how to handle checked exceptions.
What are pros & cons?
The pros is that you get your code to compile without worrying about checked exception.s
The cons is that exception you should be handling are being ignored.
Isn't it better to catch and printStackTrace()?
Unhandled exception are usually printed anyway so catching them doesn't help much.
You should catch an exception when you can add some value by doing so and add the exception to the throws clause when you can't.

It is really debatable matter.
Having method throwing too many exceptions will result in lot of error handling code. Some times it is not intended.
But because I don't like too many exception in signature does not mean that Lets use Parent of all exceptions and we are done!! It will not work.
What one can do is categorise exceptions such as BusinessException,ServiceException so that if you have a business rule which says that minimum balance in account can not be less than say 100$ then InsufficientBalance exception will be generated which will be child of BusinessException
so you method will be like
public Object doSomething() throws BusinessException {
if(!hasMinimumbalance())
{
throw new InsufficientBalance(ErrorCode);
}
}
What this will do is club related exceptions together and whenever API user wants to detect exception specific error then he can do it, else generic error handling is possible.
The core point here is on the UI you should display to the user that You have run out of balance and you can not withdraw money
You can say on the larger aspect to display human readable form of error it is really necessary to have separation of exceptions.

Are you asking about Throwable specifically? If so, then it's not good practice. It doesn't provide any useful information to class (method) user.

Throwing (and catching) Throwable (or Exception) is generally bad practice because it 'blankets' any specific exceptions you might want to catch. Then you would have to resort to ugliness like below:
public void myMethod() throws Throwable {
if (x) {
throw new MyException1();
}
if (y) {
throw new MyException2();
}
}
public void callingMethod() {
try {
myMethod();
}
catch(Throwable t) {
if (t instanceof MyException1) {
// handle exception 1
}
else if (t instanceof MyException2) {
// handle exception 2
}
else {
// handle other exceptions
}
}
}
Which is error prone (and flagged by CheckStyle as a code violation). It is much preferrable to have code like this:
public void myMethod() throws MyException1, MyException2 {
if (x) {
throw new MyException1();
}
if (y) {
throw new MyException2();
}
}
public void callingMethod() {
try {
myMethod();
}
catch(MyException1 e) {
// handle exception 1
}
catch(MyException2 e) {
// handle exception 2
}
}
Handling an exception just by calling printStackTrace() is usually not a good idea. printStackTrace() sends the stacktrace to standard error, which may not be read at all. A better option is to use the application's logging facility (like log4j) to report the exception. Even then, just logging it might no be enough.
My rule of thumb is:
If you can handle an exception locally, do so. For example when parsing a String as an Integer you could catch the NumberFormatException and return a default value:
prvate int parseAmount(String amountValue) {
int amount;
try {
amount = Integer.parseInt(amountValue);
}
catch(NumberFormatException e) {
// default amount
amount = 0;
}
return amount;
}
If you cannot handle an exception locally, consider if you should expose the exception type that is being thrown. If this type is some obscure (implementation-dependent) type, then wrapping it in your own generic exception type is probably a good idea:
private Customer getCustomer(int customerId) throws ServiceException {
try {
return customerService.getCustomer(customerId);
}
catch(CustomerServiceSpaghettiTangledException e) {
throw new ServiceException("Error calling the customer service", e);
}
}
Here 'ServiceException' is a subclass of Exception created by you. Spring also offers an exception hierarchy specifically for this purpose.
By wrapping the exception you hide the implementation details, making your service layer much simpler to use.
If you decide to throw an exception from your method, you will need to handle it 'higher up' in the callstack. This can be a generic error page in your web application stating that something went wrong and possibly providing an error message or code. In some cases the higher level code can attempt a retry or possibly an alternative way to obtain the required result.

The only use case I can think of would be for test code like unit tests. But Adam's counterpoint still stands "If so, then it's not good practice. It doesn't provide any useful information to class (method) user."

Testing Exceptions of a method with EasyMock

I am newbie to unit testing. I am using TestNG with MyEclipse to develop unit test cases for my application. While doing it I am facing some problems with EasyMock. Here is my code (Name of the class, method names and return types are changed for security reasons but you will get a clear idea what I am trying to achieve here).
public MyClass
{
// This is a method in my class which calls a collaborator which I
// want to mock in my test case
public SomeObject findSomething(SomeOtherObject param) throws Exception
{
SomeOtherObject param a = myCollaborator.doSomething(param);
// Do something with the object and then return it
return a;
}
}
Now here is my test. Now what I actually want to achieve in my test
case is that I want to check that my function (findSomething) properly
throws exception in case some exception is thrown. In future some
other developer can change the signature (throws Exception isn't
really part of method signature) of the method and remove the throws
Exception from my method. So how can I make sure that nobody changes
it?
#Test(dataProvider="mydataProvider", expectedExceptions=Exception.class)
public void MyTest(SomeOtherObject param) throws Exception {
{
EasyMock.expect(myCollaboratorMock.doSomething(param)).andThrow(new Exception());
EasyMock.replay(myCollaboratorMock);
}
I am getting exception
"java.lang.IllegalArgumentException: last
method called on mock cannot throw java.lang.Exception"
What I am
doing wrong here? Can someone shed some light on how to write a test
case for my particular scenario?

The collaborator's doSomething() method doesn't declare that it may throw Exception, and you're telling its mock to throw one. It's not possible.
Exception is a checked exception. It can only be thrown if it's declared in the method signature. If the method has no throws clause, all it can do is throwing runtime exceptions (i.e. RuntimeException or any descendant class).

JUnit4 fail() is here, but where is pass()?

There is a fail() method in JUnit4 library. I like it, but experiencing a lack of pass() method which is not present in the library. Why is it so?
I've found out that I can use assertTrue(true) instead but still looks unlogical.
#Test
public void testSetterForeignWord(){
try {
card.setForeignWord("");
fail();
} catch (IncorrectArgumentForSetter ex){
}
// assertTrue(true);
}

Call return statement anytime your test is finished and passed.

As long as the test doesn't throw an exception, it passes, unless your #Test annotation specifies an expected exception. I suppose a pass() could throw a special exception that JUnit always interprets as passing, so as to short circuit the test, but that would go against the usual design of tests (i.e. assume success and only fail if an assertion fails) and, if people got the idea that it was preferable to use pass(), it would significantly slow down a large suite of passing tests (due to the overhead of exception creation). Failing tests should not be the norm, so it's not a big deal if they have that overhead.
Note that your example could be rewritten like this:
#Test(expected=IncorrectArgumentForSetter.class)
public void testSetterForeignWord("") throws Exception {
card.setForeignWord("");
}
Also, you should favor the use of standard Java exceptions. Your IncorrectArgumentForSetter should probably be an IllegalArgumentException.

I think this question needs an updated answer, since most of the answers here are fairly outdated.
Firstly to the OP's question:
I think its pretty well accepted that introducing the "expected excepetion" concept into JUnit was a bad move, since that exception could be raised anywhere, and it will pass the test. It works if your throwing (and asserting on) very domain specific exceptions, but I only throw those kinds of exceptions when I'm working on code that needs to be absolutely immaculate, --most APIS will simply throw the built in exceptions like IllegalArgumentException or IllegalStateException. If two calls your making could potentitally throw these exceptions, then the #ExpectedException annotation will green-bar your test even if its the wrong line that throws the exception!
For this situation I've written a class that I'm sure many others here have written, that's an assertThrows method:
public class Exceptions {
private Exceptions(){}
public static void assertThrows(Class<? extends Exception> expectedException, Runnable actionThatShouldThrow){
try{
actionThatShouldThrow.run();
fail("expected action to throw " + expectedException.getSimpleName() + " but it did not.");
}
catch(Exception e){
if ( ! expectedException.isInstance(e)) {
throw e;
}
}
}
}
this method simply returns if the exception is thrown, allowing you to do further assertions/verification in your test.
with java 8 syntax your test looks really nice. Below is one of the simpler tests on our model that uses the method:
#Test
public void when_input_lower_bound_is_greater_than_upper_bound_axis_should_throw_illegal_arg() {
//setup
AxisRange range = new AxisRange(0,100);
//act
Runnable act = () -> range.setLowerBound(200);
//assert
assertThrows(IllegalArgumentException.class, act);
}
these tests are a little wonky because the "act" step doesn't actually perform any action, but I think the meaning is still fairly clear.
there's also a tiny little library on maven called catch-exception that uses the mockito-style syntax to verify that exceptions get thrown. It looks pretty, but I'm not a fan of dynamic proxies. That said, there syntax is so slick it remains tempting:
// given: an empty list
List myList = new ArrayList();
// when: we try to get the first element of the list
// then: catch the exception if any is thrown
catchException(myList).get(1);
// then: we expect an IndexOutOfBoundsException
assert caughtException() instanceof IndexOutOfBoundsException;
Lastly, for the situation that I ran into to get to this thread, there is a way to ignore tests if some conidition is met.
Right now I'm working on getting some DLLs called through a java native-library-loading-library called JNA, but our build server is in ubuntu. I like to try to drive this kind of development with JUnit tests --even though they're far from "units" at this point--. What I want to do is run the test if I'm on a local machine, but ignore the test if we're on ubuntu. JUnit 4 does have a provision for this, called Assume:
#Test
public void when_asking_JNA_to_load_a_dll() throws URISyntaxException {
//this line will cause the test to be branded as "ignored" when "isCircleCI"
//(the machine running ubuntu is running this test) is true.
Assume.assumeFalse(BootstrappingUtilities.isCircleCI());
//an ignored test will typically result in some qualifier being put on the results,
//but will also not typically prevent a green-ton most platforms.
//setup
URL url = DLLTestFixture.class.getResource("USERDLL.dll");
String path = url.toURI().getPath();
path = path.substring(0, path.lastIndexOf("/"));
//act
NativeLibrary.addSearchPath("USERDLL", path);
Object dll = Native.loadLibrary("USERDLL", NativeCallbacks.EmptyInterface.class);
//assert
assertThat(dll).isNotNull();
}

I was looking for pass method for JUnit as well, so that I could short-circuit some tests that were not applicable in some scenarios (there are integration tests, rather than pure unit tests). So too bad it is not there.
Fortunately, there is a way to have a test ignored conditionally, which actually fits even better in my case using assumeTrue method:
Assume.assumeTrue(isTestApplicable);
So here the test will be executed only if isTestApplicable is true, otherwise test will be ignored.

There is no need for the pass method because when no AssertionFailedException is thrown from the test code the unit test case will pass.
The fail() method actually throws an AssertionFailedException to fail the testCase if control comes to that point.

I think that this question is a result of a little misunderstanding of the test execution process. In JUnit (and other testing tools) results are counted per method, not per assert call. There is not a counter, which keeps track of how many passed/failured assertX was executed.
JUnit executes each test method separately. If the method returns successfully, then the test registered as "passed". If an exception occurs, then the test registered as "failed". In the latter case two subcase are possible: 1) a JUnit assertion exception, 2) any other kind of exceptions. Status will be "failed" in the first case, and "error" in the second case.
In the Assert class many shorthand methods are avaiable for throwing assertion exceptions. In other words, Assert is an abstraction layer over JUnit's exceptions.
For example, this is the source code of assertEquals on GitHub:
/**
* Asserts that two Strings are equal.
*/
static public void assertEquals(String message, String expected, String actual) {
if (expected == null && actual == null) {
return;
}
if (expected != null && expected.equals(actual)) {
return;
}
String cleanMessage = message == null ? "" : message;
throw new ComparisonFailure(cleanMessage, expected, actual);
}
As you can see, in case of equality nothing happens, otherwise an excepion will be thrown.
So:
assertEqual("Oh!", "Some string", "Another string!");
simply throws a ComparisonFailure exception, which will be catched by JUnit, and
assertEqual("Oh?", "Same string", "Same string");
does NOTHING.
In sum, something like pass() would not make any sense, because it did not do anything.