Is there any reason to group multiple assertions:
public void shouldTellIfPrime(){
Assertions.assertAll(
() -> assertTrue(isPrime(2)),
() -> assertFalse(isPrime(4))
);
}
instead of doing this:
public void shouldTellIfPrime(){
Assertions.assertTrue(isPrime(2));
Assertions.assertFalse(isPrime(4));
}
The interesting thing about assertAll is that it always checks all of the assertions that are passed to it, no matter how many fail. If all pass, all is fine - if at least one fails you get a detailed result of all that went wrong (and right for that matter).
It is best used for asserting a set of properties that belong together conceptionally. Something where your first instinct would be, "I want to assert this as one".
Example
Your specific example is not an optimal use case for assertAll because checking isPrime with a prime and a non-prime is independent of each other - so much so that I would recommend writing two test methods for that.
But assume you have a simple class like an address with fields city, street, number and would like to assert that those are what you expect them to be:
Address address = unitUnderTest.methodUnderTest();
assertEquals("Redwood Shores", address.getCity());
assertEquals("Oracle Parkway", address.getStreet());
assertEquals("500", address.getNumber());
Now, as soon as the first assertion fails, you will never see the results of the second, which can be quite annoying. There are many ways around this and JUnit Jupiter's assertAll is one of them:
Address address = unitUnderTest.methodUnderTest();
assertAll("Should return address of Oracle's headquarter",
() -> assertEquals("Redwood Shores", address.getCity()),
() -> assertEquals("Oracle Parkway", address.getStreet()),
() -> assertEquals("500", address.getNumber())
);
If the method under test returns the wrong address, this is the error you get:
org.opentest4j.MultipleFailuresError:
Should return address of Oracle's headquarter (3 failures)
expected: <Redwood Shores> but was: <Walldorf>
expected: <Oracle Parkway> but was: <Dietmar-Hopp-Allee>
expected: <500> but was: <16>
According to documentation here
Asserts that all supplied executables do not throw an AssertionError.
If any supplied Executable throws an AssertionError, all remaining
executables will still be executed, and all failures will be
aggregated and reported in a MultipleFailuresError. However, if an
executable throws an exception that is not an AssertionError,
execution will halt immediately, and the exception will be rethrown as
is but masked as an unchecked exception.
So main difference is that the assertAll will allow all the asserts to execute without breaking the flow while the others like assertTrue and the lot will stop the test with the AssertionError
So in your first example both assertions will execute regardless of pass to fail, while in the second example test will stop if first assertion fails.
Is there any reason to group multiple assertions
If you want all assertions exercised in the unit test.
assert and assertAll, both methods are designed to validate expected output vs actual output.
In simple assert, if the first assertion fails, it fails the entire test case and doesn't validate the rest of asserts. assertAll validates all test cases.
If some assertions fail, then also it will continue the rest of the assertions and return the validation result for all failed assertion.
For example:
public Apple addApple(int appleId, String appleName) {
Apple apple = new Apple(appleId, appleName);
return apple;
}
Test case:
#Test
void addAppleAssertTest() {
System.out.println("AppleCalculatorTest.addAppleTest");
AppleCalculator appleCalculator = new AppleCalculator();
Apple apple = appleCalculator.addApple(1, "apple");
assertNotNull(apple, "apple object should not be null");
assertEquals(11, apple.getAppleId(), "appleId should be 1");
assertEquals("apple1", apple.getAppleName(), "appleName should be apple");
}
#Test
void addAppleAssertAllTest() {
System.out.println("AppleCalculatorTest.addAppleTest");
AppleCalculator appleCalculator = new AppleCalculator();
Apple apple = appleCalculator.addApple(1, "apple");
assertAll(() -> assertNotNull(apple, "apple object should not be null"),
() -> assertEquals(11, apple.getAppleId(), "appleId should be 1"),
() -> assertEquals("apple1", apple.getAppleName(), "appleName should be apple"));
}
Related
I have a logic that does something like this that I want to test out:
public void doSomething(int num) {
var list = service.method1(num);
if (!list.isEmpty()) {
// Flow 1
LOG.info("List exists for {}", num);
doAnotherThing(num);
} else {
// Flow 2
LOG.info("No list found for {}", num);
}
}
public void doAnotherThing(int num) {
Optional<Foo> optionalFoo = anotherService.get(num);
optionalFoo.ifPresentOrElse(
foo -> {
if (!foo.type().equals("no")) {
// Flow 3
anotherService.filter(foo.getFilter());
} else {
// Flow 4
LOG.info("Foo is type {} - skipping", foo.type());
}
},
// Flow 5
() -> LOG.info("No foo found for {} - skipping", num));
}
For each test that'll test out different flow, my first thought was to Mockito.verify() to see if they were called or not. So to test out Flow 1, I would verify that anotherService.get() was called inside doAnotherThing(). And to test out Flow 2, I would verify that anotherService.get() was never called. This would've been fine except for Flow 4 and Flow 5. They would both invoke anotherService.get() once but not anything else.
Because of that, I've created a class to capture logs in tests. This would check to see if certain logs were logged and I would be able to see by it which flow it landed on. But I wanted to ask: is this a bad practice? I would combine this with verify() so that on flows that can be reached by verify() will take that as higher precedence.
One downside to this would be that the tests would rely on the log messages being correct so it would be a bit unstable. To account for that issue, I thought about taking some of these log messages out as a protected static variable that the tests can also use so the message would remain the same between the methods and the respective tests. This way, only the flow would be tested.
If the answer is that it is a bad practice, I would appreciate any tips on how to test out Flow 4 and Flow 5.
Log statements are usually not part of the logic to test but just a tool for ops. They should be adjusted to optimize ops (not too much info, not too few), so that you can quickly find out if and where something went wrong, if something went wrong. The exact text, the log levels and the number of log statements should not be considered as something stable to rely your tests on. Otherwise it will make it harder to change the logging concept.
I'm developing a project. The subject of this project, companies send message to users. Each company has a message limit and the system throws an exception based on the language chosen by the company when the message limit is exceeded.
I wrote unit test for the exception.
// given
Company company = new Company("Comp1", 2); // constructor (company name, language) ** 2 -> EN
User user = new User("User1");
Email email = new Email("Email Test", "Test");
int emailLimit = company.getEmailLimit();
// when
for (int i = 0; i < emailLimit; i++) {
company.SendEmail(email, user);
}
Throwable throwable = catchThrowable(() -> company.SendEmail(email, user));
// then
assertThat(throwable).isInstanceOf(MessageLimitException.class);
I also want to check the message content.
There is a class named "ErrorMessages" that manages the content of the error message.
public class ErrorMessages {
private static String[] messageLimitErrorMessage = {
"Message Limit Error", // 0 -> default
"Mesaj limiti aşıldı", // 1 -> TR
"Message limit exceeded" // 2 -> EN
};
public static String messageLimitException(int languageIndex) {
return messageLimitErrorMessage[languageIndex];
};
}
Which one should I prefer as the expected value?
// Option 1
assertThat(throwable).hasMessage(ErrorMessages.messageLimitException(company.getLanguage()));
// or
// Option 2
assertThat(throwable).hasMessage("Message limit exceeded");
Both are correct but which one should I prefer for the accuracy of the test, Option 1 or 2 ?
Thanks for your answer in advance.
There is no definitive answer to this question. It depends what you're trying to achieve. If the exact error message that is returned is important (e.g. part of the spec) then you should choose Option 2. If you don't want to hard code the message into the test (e.g. because it may change) then you can choose Option 1.
In general a test should focus on one specific thing that it's trying to test. Testing the exact error message might be better off done in a separate unit test (e.g. where you could test all of the different messages). Personally, I don't usually bother to write tests for error messages, unless there is something special about them (e.g. they have some kind of variability within the message itself). You only have so much time and it's probably better spent elsewhere.
You should also consider using Java's built-in support for internationalized message bundles. It lets you hold Locale-specific messages in properties files and loads them in for you.
"As the tests become more specific, the code becomes more generic."
Writing the test with a very specific expectation, decouples it from the implementation, and allows the code to become more generic over time.
This should tell you that you should probably use option 2.
Here's Robert Martin's take on it.
JUnit specification provides devs with following signature when displaying overloaded asserEquals methods:
assertEquals([TYPE] expected, [TYPE] actual)
If one logically changes the order of expected parameters in a business context, for example consider:
// Class provides correct surname based on name or age etc.
class SurnameProvider {
static String getSurnameForName(String name) {
// implementation
}
}
#org.junit.Test
public void test() {
String retrievedSurname = SurnameProvider.getSurnameForName("Vilde");
String expectedSurname = "Hansen";
assertEquals(retrievedSurname, expectedSurname); // here expected value is second
}
Does order ( actual first and expected second) cause any risks or is highly inappropriate ?
The order affects the error message generated by the JUnit framework when the assertion fails. See the following unit test code:
int numberOfHoursPerDay = 2+6*9-5;
Assertions.assertEquals(24, numberOfHoursPerDay);
This will generate the following error message:
expected: <24> but was: <51>
So you can easily see and check: "Hmm, it should be 24, why is it returning 51?". If you change the order like this:
int numberOfHoursPerDay = 2+6*9-5;
Assertions.assertEquals(numberOfHoursPerDay, 24);
You will get the following message:
expected: <51> but was: <24>
And now you are thinking: "Hmm, 24 sounds about right, but why should it be 51?" and when you try to fix it you get "Wait, why should it be 38 now? Shouldn't it be 51 as previously? No, wait, it should be 24. What's going on???".
So, changing the order of the values will only confuses you and everyone running/using your unit test.
I don't know there is any risk about that. And NullPointerException will only happens when you call a method when de-reference a null object, so it should not happen in your example.
However, the code convention is put the expected value before the actual value. This is basically the standard the industry is following.
So I'm writing unit tests in which I'm testing capability to blacklist and unblacklist users (which is a feature in my code that is itself working fine).
Here's a sample command that works as expected:
assertThrows(ExecutionException.class, () -> onlineStore.lookup("533"));
If I blacklist user "533", and then run the above command, it works fine, because an ExecutionException is raised (because you're trying to lookup a user who is blacklisted). Similarly, if I had NOT blacklisted user "533" but still ran the above command, the test would fail, which is expected too for similar reason (i.e. no exception is now thrown as you're NOT fetching a blacklisted user).
However if I have a List of user IDs called userIds (which user "533" is now part of) and I blacklist them all (funtionality which I know is working fine), and then run the command below:
userIds.stream().map(id -> assertDoesNotThrow(() -> onlineStore.lookup(id)));
... the test passes, even through it should have FAILED. Why ? Because all users are now blacklisted, so when fetching these users, ExecutionExceptions should have been thrown ..
If I now, replace the streams command above with either of the following, they work as expected:
assertThrows(ExecutionException.class, () -> onlineStore.lookup("533"));
assertDoesNotThrow(() -> onlineStore.lookup("533"));
So this all leads me to believe that for some reason, when going through Java Streams, thrown ExecutionExceptions aren't getting caught.
Any explanation for this behavior ?
You're not calling any terminal operation on the stream, so your assertion is never executed.
You're abusing map(), which is supposed to create a new stream by transforming every element. What you actually want to do is to execute a method which has a side effect on every element. That's what forEach is for (and it's also a terminal operation which actually consumes the stream):
userIds.stream().forEach(id -> assertDoesNotThrow(() -> onlineStore.lookup(id)));
testLogicalDoc = new LogicalDocumentImpl(-4);
assertTrue(testLogicalDoc==null);
In my code above, I have an assert condition with which I want to make sure I don't create my object with negative size. It is a stringBuilder beneath the covers which throws NegativeArrayBoundsException for a size less than zero. But my junit test fails here. I don't know any other way of making sure an object is not created with a negative size.
Any thoughts on how this could be tested ? or should it be a Junit test at all ??
Many thanks,
-Pan
EDIT:
#Test(expected=NegativeArraySizeException.class)
public void testCreate4b()
{
LogicalDocumentImpl testLogicalDoc = new LogicalDocumentImpl(-4);
}
I'm catching the exception in the LogicalDocumentImpl class but still this test fails with an assertion error but only succeeds when I do a try catch on assertion error ..why is that so ??
if you are throwing NegativeArrayBoundsException your test case could check like this
#Test(expected= NegativeArrayBoundsException.class)
That means your test should throw the exception NegativeArrayBoundsException.
Alternatively you can use fail('should never come here for negative values..')
testLogicalDoc = new LogicalDocumentImpl(-4);
fail('should never come here for negative values..');
Catch AssertionError and fail otherwise:
try {
LogicalDocumentImpl testLogicalDoc = new LogicalDocumentImpl(-4);
fail("should throw");
}
catch (AssertionError e) {
}
Usually Junit test cases are meant to test that the behavior of your code in certain cases is what you expect. Therefore, for this case you expect that an exception be thrown.
Looking at the JUnit faq (http://junit.sourceforge.net/doc/faq/faq.htm#tests_7) you want to use something like the following:
#Test(expected=NegativeArrayBoundsException.class)