What is the difference between following two blocks of code?
#Test
public void getObjectTest() throws Exception {
Object object;
//Some code
Assert.assertNotNull(object);
}
AND
#Test
public void getObjectTest() throws Exception {
Object object;
//Some code
assert object!=null;
}
I do understand that Assert.AssertNotNull is a function call from TestNG and assert is a key word of Java ( introduced in Java 1.4 ). Are there any other differences in these two ? e.g. working, performance etc
Well the first case uses the Assert class from your testing framework and is the right way to go, since TestNG will report the error in an intelligent way.
You can also add your own message to the test:
Assert.assertNotNull(object, "This object should not be null");
The second case uses the assert keyword - it will give you a failure but the stack trace may or may not be understandable at a glance. You also need to be aware that assertions may NOT be enabled.
Talking about Performance:
assert object!=null; is a single java statement but Assert.assertNotNull(object) will result in calling of Multiple functions of TestNG, so w.r.t. performance assert object!=null; will be slightly better.
Yes, there is: the assert keyword needs to be enabled using the -ea flag, like
java -ea MyClass
The assert can therefor be turned on and off without any changes to the code.
The Assert class will always work instead. So, if you're doing testing, use the Assert class, never the assert keyword. This might give you the idea all your tests are passing, and while actually they don't assert anything. In my 10+ years of coding, I've never seen anyone enable assertions except Jetbrains, so use Assert instead. Or better, use Hamcrest.
The difference is basically the lack of assert keyword.
This is the source from testng:
static public void assertNotNull(Object object) {
assertNotNull(object, null);
}
static public void assertNotNull(Object object, String message) {
assertTrue(object != null, message);
}
Please note that the OP used the testng tag!
This is not JUnit's Assert!
Some info about java's assert keyword: assert
Related
Here is a snippet of my code. I want to force call the catch block with WakeupException.
public void run() {
try {
try {
while (true) {
LOGGER.logp(Level.INFO, CLASS_NAME, "run()", "Attempting to Poll");
ConsumerRecords<String, String> records = consumer.poll(10000);
if (records.count() == 0) {
LOGGER.logp(Level.INFO, CLASS_NAME, "run()", "No Response. Invalid Topic");
break;
}
else if(records.count()>0) {
LOGGER.logp(Level.INFO, CLASS_NAME, "run()", "Response Received");
}
}
}
} catch (WakeupException e) {
consumer.close();
}
}
Here is what I tried:
#Test(expected = WakeupException.class)
public void failRun() throws WakeupException, IOException {
KafkaConsumerForTests consumerThread3;
consumerThread3 = Mockito.mock(KafkaConsumerForTests.class);
doThrow(new WakeupException()).when(consumerThread3).run();
//Mockito.when(consumerThread2.run()).thenThrow(new WakeupException());
consumerThread3.run();
}
I just want to call the WakeupException so that I get line coverage for that block of code. What would I do. This is a void method by the way. I'm open to suggestions involving PowerMock as well.
After seeing the code, I am quite sure that the call we want to mock is consumer.poll(...). I am not an expert in using Kafka so take everything from here with a grain of salt. Seeing that consumer is an attribute of the class under test, it should be possible to inject a mocked instance into the class under test and throw the WakeupException we need. Instead of (or additional to - your decision) the class under test, we create a(n additional) mock of the consumer and mock its poll(...)-method to throw the desired WakeupException when called. Instead of mocking the call to consumerThread3.run(), we mock the call to consumer.poll(...).
A remark on your question: "I just want to call the WakeupException so that I get line coverage" - This should never be the reason to write a test. A test should test behaviour. If there is no behaviour to test (which is rarely the case), do not write a test.
OP edited the question and added some additional information. I am quite confident that the first paragraph of this post should answer the question. The other paragraphs were written before OP added the relevant code in the try-block. They are written on a more abstract level. The interested reader may read them, but this is not necessary to understand the answer.
お楽しみください! - Please enjoy!
We want to verify the behaviour of the catch-block. In productive code, something in the try-block would throw the corresponding Exception triggering the catch-block. Thus, in order to test the catch-block, we should mock something in the try-block to throw said Exception.
If mocking a call within the block seems impossible, that may be due to the fact that the code was not developed test-driven. You see, an upside of Test-Driven Development is that you intrinsically generate testable code. If we are stuck with untestable/hard to test code, w ehave two (or maybe three) options:
Leave it as is, do not test it. This can be a valid answer if there is no behaviour to test.
Rewrite the code, make it testable. Depending on the structure of your project this could take from five minutes up to 2 weeks or more. Hard to say without knowing the codebase
Use unconventional tools. Normal mocking frameworks like Mockito have certain limittations, e.g. for Mockito mocking of static or final methods is not supported. Other tools, like PowerMock, aim to eliminate those limitations. But be warned: PowerMock operates on bytecode level. This means that
we are not necessarily testing the bytecode we use in production
this can screw with other tools, e.g. JaCoCo.
Those tools should be your last resort only and used sparsely.
I have a method like this one:
public void foo(#Nonnull String value) {...}
I would like to write a unit test to make sure foo() throws an NPE when value is null but I can't since the compiler refuses to compile the unit test when static null pointer flow analysis is enabled in IDE.
How do I make this test compile (in Eclipse with "Enable annotation-based null analysis" enabled):
#Test(expected = NullPointerException.class)
public void test() {
T inst = ...
inst.foo(null);
}
Note: In theory the static null pointer of the compiler should prevent cases like that. But there is nothing stopping someone from writing another module with the static flow analysis turned off and calling the method with null.
Common case: Big messy old project without flow analysis. I start with annotating some utility module. In that case, I'll have existing or new unit tests which check how the code behaves for all the modules which don't use flow analysis yet.
My guess is that I have to move those tests into an unchecked module and move them around as I spread flow analysis. That would work and fit well into the philosophy but it would be a lot of manual work.
To put it another way: I can't easily write a test which says "success when code doesn't compile" (I'd have to put code pieces into files, invoke the compiler from unit tests, check the output for errors ... not pretty). So how can I test easily that the code fails as it should when callers ignore #Nonnull?
Hiding null within a method does the trick:
public void foo(#NonNull String bar) {
Objects.requireNonNull(bar);
}
/** Trick the Java flow analysis to allow passing <code>null</code>
* for #Nonnull parameters.
*/
#SuppressWarnings("null")
public static <T> T giveNull() {
return null;
}
#Test(expected = NullPointerException.class)
public void testFoo() {
foo(giveNull());
}
The above compiles fine (and yes, double-checked - when using foo(null) my IDE gives me a compile error - so "null checking" is enabled).
In contrast to the solution given via comments, the above has the nice side effect to work for any kind of parameter type (but might probably require Java8 to get the type inference correct always).
And yes, the test passes (as written above), and fails when commenting out the Objects.requireNonNull() line.
Why not just use plain old reflection?
try {
YourClass.getMethod("foo", String.class).invoke(someInstance, null);
fail("Expected InvocationException with nested NPE");
} catch(InvocationException e) {
if (e.getCause() instanceof NullPointerException) {
return; // success
}
throw e; // let the test fail
}
Note that this can break unexpectedly when refactoring (you rename the method, change the order of method parameters, move method to new type).
Using assertThrows from Jupiter assertions I was able to test this:
public MethodName(#NonNull final param1 dao) {....
assertThrows(IllegalArgumentException.class, () -> new MethodName(null));
Here design by contract comes to picture. You can not provide null value parameter to a method annotated with notNull argument.
You can use a field which you initialize and then set to null in a set up method:
private String nullValue = ""; // set to null in clearNullValue()
#Before
public void clearNullValue() {
nullValue = null;
}
#Test(expected = NullPointerException.class)
public void test() {
T inst = ...
inst.foo(nullValue);
}
As in GhostCat's answer, the compiler is unable to know whether and when clearNullValue() is called and has to assume that the field is not null.
Is there any junit assertion, with which i can force a line to be executed?
For example:
doAnswer(new Answer<Void>() {
#SuppressWarnings("unchecked")
#Override
public Void answer(final InvocationOnMock invocation) throws Throwable {
Object[] arguments = invocation.getArguments();
Map<String, String> fieldMapActual = (Map<String, String>) arguments[0];
assertEquals(fieldMap, fieldMapActual);
**assertFailIfThisLineIsNotExecuted();**
return null;
}
}).when(x).myMethod(xxx);
As i simulate the behaviour of myMethod, the method answer from the anonymous inner type will be executed at runtime of the myMethod (not at runtime of junit test), if myMethod will be called with the intended value/parameter.
In order to assert that the method is called, i must additionally define a verify (otherwise my test would still run even if the method is not called).
verify(x).myMethod(xxx);
If i had a chance to write sth like assertFailIfThisLineIsNotExecuted in the answer method, i would not have to define an extra verify. So again, Is there any junit assertion, with which i can force a line to be executed? Opposite of fail() so to speak, without immediately defining the method as "successful".
If you want to make sure that a certain line of your test is being executed, use a boolean flag:
final boolean[] wasExecuted = { false };
...
wasExecuted[0] = true;
...
assertTrue("Some code wasn't executed", wasExecuted);
But my gut feeling is that you're trying to solve a different problem.
The verify says "This method must have been called". It doesn't matter if you mocked an answer or not. So this is the approach that you should use.
I'm using my flag approach only when I can't create a mock for some reason. If that happens, I extend the class under test in my test code and add the flag.
The advantage of the flag over verify is that the flag documents in which place I expect the code to be (you can have the IDE search all places where the flag is used). verify() is not that easy to locate when it fails.
verify(x).myMethod(xxx); should be what you want. It also expresses intent.
assertFailIfThisLineIsNotExecuted() would also be a single line of code (so how would it be "better" than verify?), it's not supported by JUnit, you would have to write code to get a good error message, etc.
Background
I am working with a Selenium/Junit test environment and I want to implement a class to perform "soft asserts": meaning that I want it to record whether or not the assert passed, but not actually fail the test case until I explicitly tell it to validate the Asserts. This way I can check multiple fields on a page an record all of the ones which do not match.
Current Code
My "verify" methods appear as such (similar ones exist for assertTrue/assertFalse):
public static void verifyEquals(Object expected, Object actual) {
try {
assertEquals(expected, actual);
} catch (Throwable e) {
verificationFailuresList.add(e);
}
}
Once all the fields have been verified, I call the following method:
public static void checkAllPassed() {
if (!verificationFailuresList.isEmpty()) {
for (Throwable failureThrowable : verificationFailuresList) {
log.error("Verification failure:" + failureThrowable.getMessage(), failureThrowable);
// assertTrue(false);
}
}
}
Question
At the moment, I am currently just using assertTrue(false) as a way to quickly fail the test case; however, this clutters the log with a nonsense failure and pushes the real problem further up. Is there a cleaner way to purposefully fail a JUnit testcase? If not, is there a better solution to implement soft asserts? I know of an article which has a very well done implementation, but to my knowledge JUnit has no equivalent to the IInvokedMethodListener class
In case you want to fail a JUnit test on purpose you should use org.junit.Assert.fail()
Other option is to switch to TestNG framework which already has a SoftAssert class in it's latest version.
You can use JUnit's ErrorCollector rule.
There is a fail() method in JUnit4 library. I like it, but experiencing a lack of pass() method which is not present in the library. Why is it so?
I've found out that I can use assertTrue(true) instead but still looks unlogical.
#Test
public void testSetterForeignWord(){
try {
card.setForeignWord("");
fail();
} catch (IncorrectArgumentForSetter ex){
}
// assertTrue(true);
}
Call return statement anytime your test is finished and passed.
As long as the test doesn't throw an exception, it passes, unless your #Test annotation specifies an expected exception. I suppose a pass() could throw a special exception that JUnit always interprets as passing, so as to short circuit the test, but that would go against the usual design of tests (i.e. assume success and only fail if an assertion fails) and, if people got the idea that it was preferable to use pass(), it would significantly slow down a large suite of passing tests (due to the overhead of exception creation). Failing tests should not be the norm, so it's not a big deal if they have that overhead.
Note that your example could be rewritten like this:
#Test(expected=IncorrectArgumentForSetter.class)
public void testSetterForeignWord("") throws Exception {
card.setForeignWord("");
}
Also, you should favor the use of standard Java exceptions. Your IncorrectArgumentForSetter should probably be an IllegalArgumentException.
I think this question needs an updated answer, since most of the answers here are fairly outdated.
Firstly to the OP's question:
I think its pretty well accepted that introducing the "expected excepetion" concept into JUnit was a bad move, since that exception could be raised anywhere, and it will pass the test. It works if your throwing (and asserting on) very domain specific exceptions, but I only throw those kinds of exceptions when I'm working on code that needs to be absolutely immaculate, --most APIS will simply throw the built in exceptions like IllegalArgumentException or IllegalStateException. If two calls your making could potentitally throw these exceptions, then the #ExpectedException annotation will green-bar your test even if its the wrong line that throws the exception!
For this situation I've written a class that I'm sure many others here have written, that's an assertThrows method:
public class Exceptions {
private Exceptions(){}
public static void assertThrows(Class<? extends Exception> expectedException, Runnable actionThatShouldThrow){
try{
actionThatShouldThrow.run();
fail("expected action to throw " + expectedException.getSimpleName() + " but it did not.");
}
catch(Exception e){
if ( ! expectedException.isInstance(e)) {
throw e;
}
}
}
}
this method simply returns if the exception is thrown, allowing you to do further assertions/verification in your test.
with java 8 syntax your test looks really nice. Below is one of the simpler tests on our model that uses the method:
#Test
public void when_input_lower_bound_is_greater_than_upper_bound_axis_should_throw_illegal_arg() {
//setup
AxisRange range = new AxisRange(0,100);
//act
Runnable act = () -> range.setLowerBound(200);
//assert
assertThrows(IllegalArgumentException.class, act);
}
these tests are a little wonky because the "act" step doesn't actually perform any action, but I think the meaning is still fairly clear.
there's also a tiny little library on maven called catch-exception that uses the mockito-style syntax to verify that exceptions get thrown. It looks pretty, but I'm not a fan of dynamic proxies. That said, there syntax is so slick it remains tempting:
// given: an empty list
List myList = new ArrayList();
// when: we try to get the first element of the list
// then: catch the exception if any is thrown
catchException(myList).get(1);
// then: we expect an IndexOutOfBoundsException
assert caughtException() instanceof IndexOutOfBoundsException;
Lastly, for the situation that I ran into to get to this thread, there is a way to ignore tests if some conidition is met.
Right now I'm working on getting some DLLs called through a java native-library-loading-library called JNA, but our build server is in ubuntu. I like to try to drive this kind of development with JUnit tests --even though they're far from "units" at this point--. What I want to do is run the test if I'm on a local machine, but ignore the test if we're on ubuntu. JUnit 4 does have a provision for this, called Assume:
#Test
public void when_asking_JNA_to_load_a_dll() throws URISyntaxException {
//this line will cause the test to be branded as "ignored" when "isCircleCI"
//(the machine running ubuntu is running this test) is true.
Assume.assumeFalse(BootstrappingUtilities.isCircleCI());
//an ignored test will typically result in some qualifier being put on the results,
//but will also not typically prevent a green-ton most platforms.
//setup
URL url = DLLTestFixture.class.getResource("USERDLL.dll");
String path = url.toURI().getPath();
path = path.substring(0, path.lastIndexOf("/"));
//act
NativeLibrary.addSearchPath("USERDLL", path);
Object dll = Native.loadLibrary("USERDLL", NativeCallbacks.EmptyInterface.class);
//assert
assertThat(dll).isNotNull();
}
I was looking for pass method for JUnit as well, so that I could short-circuit some tests that were not applicable in some scenarios (there are integration tests, rather than pure unit tests). So too bad it is not there.
Fortunately, there is a way to have a test ignored conditionally, which actually fits even better in my case using assumeTrue method:
Assume.assumeTrue(isTestApplicable);
So here the test will be executed only if isTestApplicable is true, otherwise test will be ignored.
There is no need for the pass method because when no AssertionFailedException is thrown from the test code the unit test case will pass.
The fail() method actually throws an AssertionFailedException to fail the testCase if control comes to that point.
I think that this question is a result of a little misunderstanding of the test execution process. In JUnit (and other testing tools) results are counted per method, not per assert call. There is not a counter, which keeps track of how many passed/failured assertX was executed.
JUnit executes each test method separately. If the method returns successfully, then the test registered as "passed". If an exception occurs, then the test registered as "failed". In the latter case two subcase are possible: 1) a JUnit assertion exception, 2) any other kind of exceptions. Status will be "failed" in the first case, and "error" in the second case.
In the Assert class many shorthand methods are avaiable for throwing assertion exceptions. In other words, Assert is an abstraction layer over JUnit's exceptions.
For example, this is the source code of assertEquals on GitHub:
/**
* Asserts that two Strings are equal.
*/
static public void assertEquals(String message, String expected, String actual) {
if (expected == null && actual == null) {
return;
}
if (expected != null && expected.equals(actual)) {
return;
}
String cleanMessage = message == null ? "" : message;
throw new ComparisonFailure(cleanMessage, expected, actual);
}
As you can see, in case of equality nothing happens, otherwise an excepion will be thrown.
So:
assertEqual("Oh!", "Some string", "Another string!");
simply throws a ComparisonFailure exception, which will be catched by JUnit, and
assertEqual("Oh?", "Same string", "Same string");
does NOTHING.
In sum, something like pass() would not make any sense, because it did not do anything.