I'm a bit confused.
How can I get TestNG to report an error for a test?
// ...
#DataProvider(name = "foo")
public Object[][] provideData () {
throw new SomeRuntimeException("Some error occurred. The test configuration "
+ "is somehow incorrect.");
}
This will just lead to test skipping. The exception doesn't even get logged.
Moving this to a constructor will just get the exception logged but that's not enough...
I want a big fat error message.
At the moment, using a dedicated (self) test method does the job which at leasts shows some test failure...
Anyway, it would be nice to know how testNG's definition of an error looks like.
Thank you for any hints!
Here is the article which proposes fair set of alternatives with detailed enough explanations:
Fail instead of Skip a Test when TestNG’s DataProvider throws an Exception
For me best way happened to add test for data providers and here is brief illustration of idea:
public class MyClass {
#DataProvider(name = "my-data-provider")
private Object [][] myProvider() {
// ...
}
#Test
public void testDataProviders() {
Assert.assertTrue(myProvider().length > 0);
}
#Test
// ... Real tests.
}
Related
I'm trying to test this method:
public void deleteCurrentlyLoggedInUser(Principal principal) {
if (findLoggedInUser(principal) == null) {
throw new UserAlreadyDeletedException();
}
userRepository.delete(findLoggedInUser(principal));
}
Here is findLoggedInUser:
User findLoggedInUser(Principal principal) {
return userRepository.findByUsername(principal.getName());
}
And here is my test so far:
#Test
public void shouldThrowExceptionWhenUserNotFound() {
// given
when(sut.findLoggedInUser(principalStub)).thenReturn(null);
// when
sut.deleteCurrentlyLoggedInUser(principalStub);
// then
catchException
verify(userRepositoryMock, never()).delete(any(User.class));
}
So how do I catch exception using catch-exception here? Method that I'm testing returns void and I just can't seem to find a way to assert that exception was found.
EDIT: I know I could use: #Test(expected = UserAlreadyDeletedException.class) but I want to switch my whole project to catch-exception because it's much better and using expected in #Test is not very reasonable.
I've never heard of catch-exception, but it doesn't exactly seem like an up-to-date library: the last update to the main source code (at the time of writing) was on May 3 2015.
If you're using Java 8, and can use JUnit 4.13 or later, you can use assertThrows:
assertThrows(
UserAlreadyDeletedException.class,
() -> sut.deleteCurrentlyLoggedInUser(principalStub));
If you're going to migrate all of your code to something, this seems like a better long-term bet.
It might be that using Rules is something that could work for you?
Rules allow very flexible addition or redefinition of the behavior of each test method in a test class. Testers can reuse or extend one of the provided Rules below, or write their own.
You can read more about this neat feature of junit4 here:
https://github.com/junit-team/junit4/wiki/Rules
Example:
public static class HasExpectedException {
#Rule
public final ExpectedException thrown = ExpectedException.none();
#Test
public void throwsNullPointerException() {
thrown.expect(NullPointerException.class);
throw new NullPointerException();
}
}
I am trying to report error using JUnit error collector. Although my assertion is failing, error is not reported in JUnit. But I am getting the "error" message in console.
#Rule
public ErrorCollector errcol = new ErrorCollector();
#Then("^Business alert message on the screen$")
public void Business_alert_message_on_the_screen(Result_Update) throws Throwable {
if (userType.equals("Admin")) {
try {
Assert.assertEquals("Update button is not present for the admin user", true, Result_Update);
} catch (Throwable t) {
errcol.addError(t);
System.out.println("Error");
}
}
}
tl;dr : Make sure your test class doesn't extend TestCase.
I had a similar problem when I was using JUnit 4 with IntelliJ IDEA. I naïvely selected a base class of TestCase in the dialog, which was the default for JUnit 3, because I figured "it'd be nice to have those handy this#assert* methods" (the default for JUnit 4 is null). The bad code (which didn't work) is below:
public class SassCompilerTest extends TestCase {
#Rule
public ErrorCollector collector = new ErrorCollector();
#Test
public void testCompiler() throws IOException {
collector.checkThat(true, CoreMatchers.equalTo(false));
}
}
However, in JUnit 4, that prevented a lot of features from working. Removing the parent class fixed the test:
public class SassCompilerTest {
#Rule
public ErrorCollector collector = new ErrorCollector();
#Test
public void testCompiler() throws IOException {
collector.checkThat(true, CoreMatchers.equalTo(false));
}
}
The solution was suggested to me by a comment in the issue with Cucumber mentioned by #StefanBirkner in another answer. After reading that, I tried extending ErrorCollector to make the ErrorCollector#verify public and call it from an #After method, but the #After method wasn't getting called, which made me realize something was either wrong with the TestRunner (which was IntelliJ's default) or the Test itself.
According to JUnit:
The ErrorCollector rule allows execution of a test to continue after
the first problem is found
errcol.addError(t);//only adds the error to the ErrorCollector
This means that the test continues after collecting the error.
You should add:
errcol.checkThat(...); //will pass/fail the test
See examples:
https://junit.org/junit4/javadoc/4.12/org/junit/rules/ErrorCollector.html (Updated)
https://gist.github.com/cb372/2419626
The Cucumber runner does not support #Rule because it extends ParentRunner and not BlockJUnit4ClassRunner (see source code of the runner). There is already an issue for supporting ErrorCollector.
Here is the code that I am working with. In this test I want to verify that the log method is being called when an exception is caught.
public class SuperClass(){
public void log()
{
do some logging;
}
}
public class ClassUnderTest extends SuperClass(){
public String methodbeingtested(Object param)
{
try
{
String a = SomeObject.
methodthatthrowsexception(param);//static method, throws JAXB/NPE
}
catch(Exception exp)
{
log("log msg",exp);//inherited method
}
}
}
public class ClassUnderTestTest {
#Test
public testmethodbeingtested(){
ClassUnderTest cut = new ClassUnderTest()
ClassUnderTest cutspy = Mockito.spy(cut);
cutspy.methodbeingtested(param);
Mockito.verify(cutspy).log("log msg", new Exception()); // exp is needed to here.
}
}
After looking at several samples, the above was the closest I could get. This testcase forces an exception. But it fails to verify the log method call as Mockito.verify requires the exact exception (exp) that is caught, which the test case does not have access to.
Is there any other way to test this scenario?
Mockito's verify method can be used with argument matchers. If you want to verify that log was called, with any Exception at all as the second argument, you can just write
verify(cutspy).log(eq("log msg"), any(Exception.class));
I've assumed that you have the right static imports for verify, eq and any.
As an aside, this test does not need PowerMock. Your line PowerMock.expectLastCall().once(); is both redundant and confusing, and should probably be removed, along with the #PrepareForTest annotation.
Instead of spying on ClassUnderTest, you should mock the logging framework, inject it into the class and then verify that the log method gets called. Id' also mock the SomeObject class and have it throw exception.
As an aside, you should really evaluate if you need to verify your log statements. Perhaps you have a valid reason to do so but typically, asserting/verifying to this extent is not required and will only make your tests brittle.
what's wrong with this. I have junit 4
You can declare on the #Test annotation that, for the test to pass, it must throw these exceptions:
#Test(expected = NullPointerException.class)
public void testSynapseOne() {
// test
}
#Test(expected = IllegalStateException.class)
public void testSynapseTwo() {
// test
}
Of course, you have to be sure you're testing the right thing - currently, your tests don't make use of the constructor, which is the critical piece you want to test.
Oh - you don't want to have your tests extend TestCase unless you need compatibility with JUnit3.x.
You can use the annotation #Test(expected = TheClassException.class) to write a test which is supposed to throws the exception of class TheClassException
OK, so the #Ignore annotation is good for marking that a test case shouldn't be run.
However, sometimes I want to ignore a test based on runtime information. An example might be if I have a concurrency test that needs to be run on a machine with a certain number of cores. If this test were run on a uniprocessor machine, I don't think it would be correct to just pass the test (since it hasn't been run), and it certainly wouldn't be right to fail the test and break the build.
So I want to be able to ignore tests at runtime, as this seems like the right outcome (since the test framework will allow the build to pass but record that the tests weren't run). I'm fairly sure that the annotation won't give me this flexibility, and suspect that I'll need to manually create the test suite for the class in question. However, the documentation doesn't mention anything about this and looking through the API it's also not clear how this would be done programmatically (i.e. how do I programatically create an instance of Test or similar that is equivalent to that created by the #Ignore annotation?).
If anyone has done something similar in the past, or has a bright idea of how else I could go about this, I'd be happy to hear about it.
The JUnit way is to do this at run-time is org.junit.Assume.
#Before
public void beforeMethod() {
org.junit.Assume.assumeTrue(someCondition());
// rest of setup.
}
You can do it in a #Before method or in the test itself, but not in an #After method. If you do it in the test itself, your #Before method will get run. You can also do it within #BeforeClass to prevent class initialization.
An assumption failure causes the test to be ignored.
Edit: To compare with the #RunIf annotation from junit-ext, their sample code would look like this:
#Test
public void calculateTotalSalary() {
assumeThat(Database.connect(), is(notNull()));
//test code below.
}
Not to mention that it is much easier to capture and use the connection from the Database.connect() method this way.
You should checkout Junit-ext project. They have RunIf annotation that performs conditional tests, like:
#Test
#RunIf(DatabaseIsConnected.class)
public void calculateTotalSalary() {
//your code there
}
class DatabaseIsConnected implements Checker {
public boolean satisify() {
return Database.connect() != null;
}
}
[Code sample taken from their tutorial]
In JUnit 4, another option for you may be to create an annotation to denote that the test needs to meet your custom criteria, then extend the default runner with your own and using reflection, base your decision on the custom criteria. It may look something like this:
public class CustomRunner extends BlockJUnit4ClassRunner {
public CTRunner(Class<?> klass) throws initializationError {
super(klass);
}
#Override
protected boolean isIgnored(FrameworkMethod child) {
if(shouldIgnore()) {
return true;
}
return super.isIgnored(child);
}
private boolean shouldIgnore(class) {
/* some custom criteria */
}
}
Additionally to the answer of #tkruse and #Yishai:
I do this way to conditionally skip test methods especially for Parameterized tests, if a test method should only run for some test data records.
public class MyTest {
// get current test method
#Rule public TestName testName = new TestName();
#Before
public void setUp() {
org.junit.Assume.assumeTrue(new Function<String, Boolean>() {
#Override
public Boolean apply(String testMethod) {
if (testMethod.startsWith("testMyMethod")) {
return <some condition>;
}
return true;
}
}.apply(testName.getMethodName()));
... continue setup ...
}
}
A quick note: Assume.assumeTrue(condition) ignores rest of the steps but passes the test.
To fail the test, use org.junit.Assert.fail() inside the conditional statement. Works same like Assume.assumeTrue() but fails the test.