In JUnit4 you can write parameterized unit tests by providing parameters collection in one method, which will be passed to the constructor of the test and testing in another method. If I have a parameter for which I expect an exception to be thrown, how do I specify that?
this is how i use junit parameterized test with expected exceptions:
#RunWith(Parameterized.class)
public class CalcDivTest {
#Parameter(0)
public int num1;
#Parameter(1)
public int num2;
#Parameter(2)
public int expectedResult;
#Parameter(3)
public Class<? extends Exception> expectedException;
#Parameter(4)
public String expectedExceptionMsg;
#Rule
public ExpectedException thrown = ExpectedException.none();
#Parameters
public static Iterable<Object[]> data() {
return Arrays.asList(new Object[][] {
// calculation scenarios:
{ 120, 10, 12, null, null }, // simple div
{ 120, 0, -1, ArithmeticException.class, "/ by zero" }, // div by zero
});
}
#Test
public void testDiv() throws CCalculationException {
//setup expected exception
if (expectedException != null) {
thrown.expect(expectedException);
thrown.expectMessage(expectedExceptionMsg);
}
assertEquals("calculation result is not as", expectedResult, div(num1, num2) );
}
private int div(int a, int b) {
return a/b;
}
}
In contrast to what other suggest, I would not introduce any kind of logic to tests - even simple ifs!
What you should have are two testing methods:
first one takes valid parameters (and expects some output)
second takes invalid parameters (and expects exceptions)
Not sure if JUnit with its constructor-based parametrized testing is able to do this. Probably you would have to create two test classes for this. Go with JUnit Params or TestNG which offer much more convenient solution.
I agree with Tomek, and would go with two tests. The first tests for cases where no exceptions are expected. The second tests for values that should result in exceptions being thrown (i.e., and fails if they are not thrown).
Below is a simple example, where the implementation of ExceptionThrower.throwAnInstanceException(int) simply throws an IllegalArgumentException when the supplied int is less-than-1. In your implementation, all supplied values should trigger the exception.
#ParameterizedTest
#ValueSource(ints = {0, 1})
public void parameterizedIntExceptionTest(int testValue) {
ExceptionThrower exceptionThrower = new ExceptionThrower();
assertThrows(IllegalArgumentException.class, () -> {
exceptionThrower.throwAnInstanceException(testValue);
});
}
If you wanted to supply multiple arguments, then you'd be looking at using a MethodSource vice a ValueSource for the test.
if (parameter == EXCEPTION_EXPECTED) {
try {
method(parameter);
fail("didn't throw an exception!");
} catch (ExpectedException ee) {
// Test succeded!
}
}
Gabriel, please look at TestWatcher rule (since JUnit 4.9). Here is the sample code quoted from http://junit-team.github.io/junit/javadoc/4.11/org/junit/rules/TestWatcher.html:
public static class WatchmanTest {
private static String watchedLog;
#Rule
public TestWatcher watchman= new TestWatcher() {
#Override
protected void failed(Throwable e, Description description) {
watchedLog+= description + "\n";
}
#Override
protected void succeeded(Description description) {
watchedLog+= description + " " + "success!\n";
}
};
#Test
public void fails() {
fail();
}
#Test
public void succeeds() {
}
}
Another approach would be to use ErrorCollector from JUnit 4.7:
#Rule
public ExpectedException thrown = ExpectedException.none();
#Test
public void testCollectingErrors() {
thrown.handleAssertionErrors();
thrown.expect(MultipleFailureException.class); // or #expectMessage()/#expectCause()
collector.checkThat("a", equalTo("b"));
//...
}
If you used catch-exception instead of the corresponding annotations and rules of JUnit4, then your code would look like this:
catchException(obj).method(parameter);
if (parameter != EXCEPTION_EXPECTED) {
assert caughtException() instanceof ExpectedException;
}
// more assertions
#Test(expected = Exception.class)
#Parameters(value = { "invalidInput1", "invalidInput2" })
public void shouldThrowOnInvalidInput(String input) {
ClassToTest.methodToTest(input);
}
Using junitparams.Parameters from junitparams library.
Related
I want to verify that an expected exception meets certain criteria. Take this as a starting point:
class MyException extends RuntimeException {
int n;
public MyException(String message, int n) {
super(message);
this.n = n;
}
}
public class HowDoIDoThis {
#Rule
public ExpectedException thrown = ExpectedException.none();
#Test
public void test1() {
thrown.expect(MyException.class);
throw new MyException("x", 10);
}
}
How do I assert, for example, that the thrown exception has n > 1 and message contains only lowercase letters? I was thinking of using thrown.expect(Matcher) but can't figure out how to get a Hamcrest matcher to check arbitrary fields of an object.
A consise and neat alternative way is to use AssertJ instead of the ExpectedException rule.
assertThatThrownBy(() -> {
throw new MyException("x", 10);
})
.matches(e -> e.getMessage().equals(e.getMessage().toLower()), "message is lowercase")
.matches(e -> ((CustomException) e).n > 10, "n > 10");
You can use TypeSafeMatcher where you can provide your MyException class, and then an IntPredicate to check the n value against a condition:
public class MyExceptionMatcher extends TypeSafeMatcher<MyException> {
private final IntPredicate predicate;
public MyExceptionMatcher(IntPredicate predicate) {
this.predicate = predicate;
}
#Override
protected boolean matchesSafely(MyException item) {
return predicate.test(item.n);
}
#Override
public void describeTo(Description description) {
description.appendText("my exception which matches predicate");
}
}
Then you can expect like so:
thrown.expect(new MyExceptionMatcher(i -> i > 1));
There's also FeatureMatcher in Hamcrest that is great for creating matchers for nested "features" of objects. So in your example, you could structure it using FeatureMatcher in the following way (this is the pattern I tend to follow when creating matchers for nested fields):
public final class MyExceptionMatchers {
public static Matcher<MyException> withNthat(Matcher<Integer> nMatcher) {
return new FeatureMatcher<MyException, Integer>(nMatcher, "n", "n") {
#Override
protected Integer featureValueOf(MyException actual) {
return actual.n;
}
}
};
}
And in your test:
import static x.y.z.MyExceptionMatchers.withNthat;
import static org.hamcrest.Matchers.greaterThan;
...
thrown.expect(withNThat(greaterThan(1)));
With this layout it's very easy to add more matchers for MyException and it feels like a more "canonical" approach to build composable matchers that allow you to construct the exact matcher you want for your test case.
To validate exception message I can use ExpectedMessage object but what if I want to validate concreate exception and it's details? For example I have exception class TerribleWindException which has some additional methods like getWindSpeed() and in junit test method I want to check the getWindSpeed() result. Look at this example:
// pseudo implementation
public class TerribleWindException extends Exception {
private Integer windSpeed = 0;
public TerribleWindException(final Integer windSpeed) {
this.windSpeed = windSpeed;
}
}
public class WalkService {
private Integer windSpeed = 0;
public WalkService(final Integer windSpeed) {
this.windSpeed = windSpeed;
}
public void goForAWalk() throws TerribleWindException {
if (windSpeed>10) {
throw new TerribleWindException(windSpeed);
}
}
}
// test
public class WalkServiceTest {
#Test
public void testGoForAWalkWhenSpeedIsToPowerfulShouldThrowTerribleWindException throws TerribleWindException {
WalkService ws = new WalkService(100);
goForAWalk(); // this will throw TerribleWindException. The only way to check it's exception details is to use try {} catch() {} ?
}
}
The only way to check exception details is to use try {} catch() {} ?
You could use JUnit's ExpectedException rule together with Hamcrest matchers.
public class WalkServiceTest {
#Rule
public final ExpectedException thrown = ExpectedException.none();
#Test
public void testGoForAWalkWhenSpeedIsToPowerfulShouldThrowTerribleWindException throws TerribleWindException {
WalkService ws = new WalkService(100);
thrown.expect(TerribleWindException.class);
thrown.expect(Matchers.hasProperty("windSpeed", Matchers.equalTo("expected speed")));
ws.goForAWalk(); // this will throw TerribleWindException. The only way to check it's exception details is to use try {} catch() {} ?
}
}
If you're using Java 8 then you can use the Vallado library together with Hamcrest.
public class WalkServiceTest {
#Test
public void testGoForAWalkWhenSpeedIsToPowerfulShouldThrowTerribleWindException throws TerribleWindException {
WalkService ws = new WalkService(100);
when(() -> ws.goForAWalk())
.thenA(TerribleWindException.class)
.that(hasProperty("windSpeed", equalTo("expected speed")))
.isThrown();
}
}
For such case i suggest you to use catch-exception library from Google, code sample (from their homepage):
import static com.googlecode.catchexception.CatchException.*;
import static com.googlecode.catchexception.apis.CatchExceptionBdd.*;
// given: an empty list
List myList = new ArrayList();
// when: we try to get the first element of the list
when(myList).get(1);
// then: we expect an IndexOutOfBoundsException
then(caughtException())
.isInstanceOf(IndexOutOfBoundsException.class)
.hasMessage("Index: 1, Size: 0")
.hasNoCause();
See also catch-exception AssertJ api if above example is not enough.
Yes. You can use an #Test(expected=Exception.class) annotation to test the class of the exception thrown, but to test the details you will need to catch and interrogate the exception yourself.
Don't forget to fail() if the exception isn't thrown. e.g.
try {
operation();
fail("Expected an exception");
}
catch (MyExpectedException e) {
// assertions...
}
If you use Java 8 there is actually a more flexible way to achieve what you want, which uses Lambdas.
The following steps are necessary:
Create a getter for the windSpeed-attribute of your Exception
Create a functional interface for a lambda which takes no parameters and returns nothing and which might throw an exception, like this (Runnable is not suitable, because it does not throw an exception and Callable has a return value):
#FunctionalInterface
public interface TestCommand {
public void exec() throws Exception;
}
Create your own assertion for exceptions, like this:
public <T extends Exception> void myAssertThrows(TestCommand testAction,
Predicate<T> p) {
try {
testAction.exec();
fail("must throw exception");
} catch (Exception exception) {
assertTrue(p.test((T) exception));
}
}
Use it in a test:
#Test
public void testGoForAWalkWhenSpeedIsToPowerfulShouldThrowTerribleWindException(){
WalkService ws = new WalkService(10);
myAssertThrows(() -> {
ws.goForAWalk();
}, (TerribleWindException e) -> e.getWindSpeed() == 100);
}
So how does it work? ws.goForAWalk(); is packed into a lambda of type TestCommand so it is not immediately executed. It is the first parameter of myAssertThrows and executed in this method. The second parameter is a predicate on Exception, which is simply a lambda which takes one exception as parameter, returns a boolean and can be used to check properties of the exception, which is thrown. The assert method uses the well-known try-catch-style.
You could actually also use the assert method in Java 7, but the call would be less readable, because it would need anonymous classes.
This one makes more sense than my previous answer, but I am not editing my previous answer because it serves most of the use cases.
The ExpectedException rule allows you to specify, within your test, what exception you are expecting and even what the exception message is
import static org.hamcrest.Matchers.*;
import static org.junit.Assert.*;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;
public class CloudTest {
#Rule
public ExpectedException exception = ExpectedException.none();
#Test
public void testExpectedException() {
exception.expect(CustomException.class);
exception.expectMessage(containsString('Invalid cloud type'));
new Cloud("Rainy","With thunder", "big");
}
}
You could use Junit's Rules Junit Rules, which includes an ExpectedException rule, which can be used to also test messages of exceptions.
EDIT:
If you implemented TerribleWindException the following way:
public class TerribleWindException extends Exception {
private Integer windSpeed = 0;
public TerribleWindException(final Integer windSpeed) {
// use Exception constructor, which passes a message
super("windSpeed: " + windSpeed);
this.windSpeed = windSpeed;
}
}
You could achieve what you want using this test code:
#Rule
public ExpectedException thrown = ExpectedException.none();
#Test
public void shouldTestExceptionMessage() throws IndexOutOfBoundsException {
List<Object> list = new ArrayList<Object>();
thrown.expect(TerribleWindException .class);
thrown.expectMessage("windSpeed: 100");
WalkService ws = new WalkService(100);
ws.goForAWalk();
}
I often use a scenario like the following:
#Test
public void testGoForAWalkWhenSpeedIsToPowerfulShouldThrowTerribleWindException throws TerribleWindException {
WalkService ws = new WalkService(100);
try {
goForAWalk();
fail();
catch(TerribleWindException twe) {
}
}
No you can catch specific exception by using the following template
#Test(expected= TerribleWindException.class)
My application have several execution modes, and in 1 mode it is normal that some of my tests will throw a concrete exception. I need to annotate this methods with something like #SkipOnFail that will set method as skipped if exception was thrown.
thanks in advance!
#Edit(for my question to be more clear)
#Test(expected=ConcreteException.class)
does not work for me because i need my tests to pass even if ConcreteException.class was not thrown(expected tag in junit will mark my test as failed if this exception won't be thrown), and to be skipped otherwise. In all other cases it should work as always.
#Solution that worked for me(junit v4.7) thx to #axtavt
#Rule
public MethodRule skipRule = new MethodRule() {
public Statement apply(final Statement base, FrameworkMethod method, Object target) {
if(method.getAnnotation(SkipOnFail.class) == null) return base;
return new Statement() {
#Override
public void evaluate() throws Throwable {
try{
base.evaluate();
} catch (ConcreteException e) {
Assume.assumeTrue(false);
}
}
};
}
};
#Thx
I don't think that such a feature is available out of the box, but it should be pretty easy to implement with custom TestRule and Assume, something like this:
#Rule
public TestRule skipRule = new TestRule() {
public Statement apply(final Statement base, Description desc) {
if (desc.getAnnotation(SkipOnFail.class) == null) return base;
return new Statement() {
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (MyExceptoion ex) {
Assume.assumeTrue(false);
}
}
};
}
};
What about using JUnit Extensions?
The following example is taken from their Tutorial.
It provides aditional annotations for Prerequisites (#Prerequisite): Ignore tests based on conditions.
The required approach would be to check this during running tests. So you can simply add a #Prerequisite(requires="") annotation.
public class TestFillDatabase {
#Prerequisite(requires = "databaseIsAvailable")
#Test public void fillData() {
// ...
}
public boolean databaseIsAvailable() {
boolean isAvailable = ...;
return isAvailable;
}
}
public class TestFillDatabase {
#Prerequisite(requires = "databaseIsAvailable")
#Test public void fillData() {
// ...
}
public boolean databaseIsAvailable() {
boolean isAvailable = ...;
return isAvailable ;
}
}
This specified methods with #Prerequisite(requires = "databaseIsAvailable") must be a public method, returning a boolean or Boolean value.
If these methods will be consolidated in helper classes, you can also specify static methods within a class to be called using #Prerequisite(requires = "databaseIsAvailable", callee="DBHelper").
public class TestFillDatabase {
#Prerequisite(requires = "databaseIsAvailable", callee="DBHelper")
#Test public void fillData() {
// ...
}
}
public class DBHelper {
public static boolean databaseIsAvailable() {
boolean isAvailable = ...;
return isAvailable ;
}
}
Also using the Assume class (since jUnit 4.4), you can use assumeNoException():
try{
base.evaluate();
} catch (ConcreteException e) {
Assume.assumeNoException("Concrete exception: skipping test", e);
}
I searched for the docs about JUnit and it appears that from version 4.9 they have introduced what they call test rules (see TestRule). You may start from this.
The ExpectedException class marked as #Rule could be of some help in order to check for exceptions thrown but not mandatory for the test to pass.
For more advanced usage I cannot say for the moment as I've just discovered it.
Is there a way to set my own custom test case names when using parameterized tests in JUnit4?
I'd like to change the default — [Test class].runTest[n] — to something meaningful.
This feature has made it into JUnit 4.11.
To use change the name of parameterized tests, you say:
#Parameters(name="namestring")
namestring is a string, which can have the following special placeholders:
{index} - the index of this set of arguments. The default namestring is {index}.
{0} - the first parameter value from this invocation of the test.
{1} - the second parameter value
and so on
The final name of the test will be the name of the test method, followed by the namestring in brackets, as shown below.
For example (adapted from the unit test for the Parameterized annotation):
#RunWith(Parameterized.class)
static public class FibonacciTest {
#Parameters( name = "{index}: fib({0})={1}" )
public static Iterable<Object[]> data() {
return Arrays.asList(new Object[][] { { 0, 0 }, { 1, 1 }, { 2, 1 },
{ 3, 2 }, { 4, 3 }, { 5, 5 }, { 6, 8 } });
}
private final int fInput;
private final int fExpected;
public FibonacciTest(int input, int expected) {
fInput= input;
fExpected= expected;
}
#Test
public void testFib() {
assertEquals(fExpected, fib(fInput));
}
private int fib(int x) {
// TODO: actually calculate Fibonacci numbers
return 0;
}
}
will give names like testFib[1: fib(1)=1] and testFib[4: fib(4)=3]. (The testFib part of the name is the method name of the #Test).
Looking at JUnit 4.5, its runner clearly doesn't support that, as that logic is buried inside a private class inside the Parameterized class. You could not use the JUnit Parameterized runner, and create your own instead which would understand the concept of names (which leads to the question of how you might set a name ...).
From a JUnit perspective, it would be nice if instead of (or in addition to) just passing an increment, they would pass the comma delimited arguments. TestNG does this. If the feature is important to you, you can comment on the yahoo mailing list referenced at www.junit.org.
I recently came across the same problem when using JUnit 4.3.1. I implemented a new class which extends Parameterized called LabelledParameterized. It has been tested using JUnit 4.3.1, 4.4 and 4.5. It reconstructs the Description instance using the String representation of the first argument of each parameter array from the #Parameters method. You can see the code for this at:
http://code.google.com/p/migen/source/browse/trunk/java/src/.../LabelledParameterized.java?r=3789
and an example of its use at:
http://code.google.com/p/migen/source/browse/trunk/java/src/.../ServerBuilderTest.java?r=3789
The test description formats nicely in Eclipse which is what I wanted since this makes failed tests a lot easier to find! I will probably further refine and document the classes over the next few days/weeks. Drop the '?' part of the URLs if you want the bleeding edge. :-)
To use it, all you have to do is copy that class (GPL v3), and change #RunWith(Parameterized.class) to #RunWith(LabelledParameterized.class) assuming the first element of your parameter list is a sensible label.
I don't know if any later releases of JUnit address this issue but even if they did, I can't update JUnit since all my co-developers would have to update too and we have higher priorities than re-tooling. Hence the work in the class to be compilable by multiple versions of JUnit.
Note: there is some reflection jiggery-pokery so that it runs across the different JUnit versions as listed above. The version specifically for JUnit 4.3.1 can be found here and, for JUnit 4.4 and 4.5, here.
With Parameterized as a model, I wrote my own custom test runner / suite -- only took about half an hour. It's slightly different from darrenp's LabelledParameterized in that it lets you specify a name explicitly rather than relying on the first parameter's toString().
It also doesn't use arrays because I hate arrays. :)
public class PolySuite extends Suite {
// //////////////////////////////
// Public helper interfaces
/**
* Annotation for a method which returns a {#link Configuration}
* to be injected into the test class constructor
*/
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.METHOD)
public static #interface Config {
}
public static interface Configuration {
int size();
Object getTestValue(int index);
String getTestName(int index);
}
// //////////////////////////////
// Fields
private final List<Runner> runners;
// //////////////////////////////
// Constructor
/**
* Only called reflectively. Do not use programmatically.
* #param c the test class
* #throws Throwable if something bad happens
*/
public PolySuite(Class<?> c) throws Throwable {
super(c, Collections.<Runner>emptyList());
TestClass testClass = getTestClass();
Class<?> jTestClass = testClass.getJavaClass();
Configuration configuration = getConfiguration(testClass);
List<Runner> runners = new ArrayList<Runner>();
for (int i = 0, size = configuration.size(); i < size; i++) {
SingleRunner runner = new SingleRunner(jTestClass, configuration.getTestValue(i), configuration.getTestName(i));
runners.add(runner);
}
this.runners = runners;
}
// //////////////////////////////
// Overrides
#Override
protected List<Runner> getChildren() {
return runners;
}
// //////////////////////////////
// Private
private Configuration getConfiguration(TestClass testClass) throws Throwable {
return (Configuration) getConfigMethod(testClass).invokeExplosively(null);
}
private FrameworkMethod getConfigMethod(TestClass testClass) {
List<FrameworkMethod> methods = testClass.getAnnotatedMethods(Config.class);
if (methods.isEmpty()) {
throw new IllegalStateException("#" + Config.class.getSimpleName() + " method not found");
}
if (methods.size() > 1) {
throw new IllegalStateException("Too many #" + Config.class.getSimpleName() + " methods");
}
FrameworkMethod method = methods.get(0);
int modifiers = method.getMethod().getModifiers();
if (!(Modifier.isStatic(modifiers) && Modifier.isPublic(modifiers))) {
throw new IllegalStateException("#" + Config.class.getSimpleName() + " method \"" + method.getName() + "\" must be public static");
}
return method;
}
// //////////////////////////////
// Helper classes
private static class SingleRunner extends BlockJUnit4ClassRunner {
private final Object testVal;
private final String testName;
SingleRunner(Class<?> testClass, Object testVal, String testName) throws InitializationError {
super(testClass);
this.testVal = testVal;
this.testName = testName;
}
#Override
protected Object createTest() throws Exception {
return getTestClass().getOnlyConstructor().newInstance(testVal);
}
#Override
protected String getName() {
return testName;
}
#Override
protected String testName(FrameworkMethod method) {
return testName + ": " + method.getName();
}
#Override
protected void validateConstructor(List<Throwable> errors) {
validateOnlyOneConstructor(errors);
}
#Override
protected Statement classBlock(RunNotifier notifier) {
return childrenInvoker(notifier);
}
}
}
And an example:
#RunWith(PolySuite.class)
public class PolySuiteExample {
// //////////////////////////////
// Fixture
#Config
public static Configuration getConfig() {
return new Configuration() {
#Override
public int size() {
return 10;
}
#Override
public Integer getTestValue(int index) {
return index * 2;
}
#Override
public String getTestName(int index) {
return "test" + index;
}
};
}
// //////////////////////////////
// Fields
private final int testVal;
// //////////////////////////////
// Constructor
public PolySuiteExample(int testVal) {
this.testVal = testVal;
}
// //////////////////////////////
// Test
#Ignore
#Test
public void odd() {
assertFalse(testVal % 2 == 0);
}
#Test
public void even() {
assertTrue(testVal % 2 == 0);
}
}
You may also want to try JUnitParams: https://github.com/Pragmatists/JUnitParams
from junit4.8.2, you can create your own MyParameterized class by simply copy Parameterized class. change the getName() and testName() methods in TestClassRunnerForParameters.
None of it was working for me, so I got the source for Parameterized and modified it create a a new test runner. I didn't have to change much but IT WORKS!!!
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import org.junit.Assert;
import org.junit.internal.runners.ClassRoadie;
import org.junit.internal.runners.CompositeRunner;
import org.junit.internal.runners.InitializationError;
import org.junit.internal.runners.JUnit4ClassRunner;
import org.junit.internal.runners.MethodValidator;
import org.junit.internal.runners.TestClass;
import org.junit.runner.notification.RunNotifier;
public class LabelledParameterized extends CompositeRunner {
static class TestClassRunnerForParameters extends JUnit4ClassRunner {
private final Object[] fParameters;
private final String fParameterFirstValue;
private final Constructor<?> fConstructor;
TestClassRunnerForParameters(TestClass testClass, Object[] parameters, int i) throws InitializationError {
super(testClass.getJavaClass()); // todo
fParameters = parameters;
if (parameters != null) {
fParameterFirstValue = Arrays.asList(parameters).toString();
} else {
fParameterFirstValue = String.valueOf(i);
}
fConstructor = getOnlyConstructor();
}
#Override
protected Object createTest() throws Exception {
return fConstructor.newInstance(fParameters);
}
#Override
protected String getName() {
return String.format("%s", fParameterFirstValue);
}
#Override
protected String testName(final Method method) {
return String.format("%s%s", method.getName(), fParameterFirstValue);
}
private Constructor<?> getOnlyConstructor() {
Constructor<?>[] constructors = getTestClass().getJavaClass().getConstructors();
Assert.assertEquals(1, constructors.length);
return constructors[0];
}
#Override
protected void validate() throws InitializationError {
// do nothing: validated before.
}
#Override
public void run(RunNotifier notifier) {
runMethods(notifier);
}
}
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.METHOD)
public static #interface Parameters {
}
private final TestClass fTestClass;
public LabelledParameterized(Class<?> klass) throws Exception {
super(klass.getName());
fTestClass = new TestClass(klass);
MethodValidator methodValidator = new MethodValidator(fTestClass);
methodValidator.validateStaticMethods();
methodValidator.validateInstanceMethods();
methodValidator.assertValid();
int i = 0;
for (final Object each : getParametersList()) {
if (each instanceof Object[])
add(new TestClassRunnerForParameters(fTestClass, (Object[]) each, i++));
else
throw new Exception(String.format("%s.%s() must return a Collection of arrays.", fTestClass.getName(), getParametersMethod().getName()));
}
}
#Override
public void run(final RunNotifier notifier) {
new ClassRoadie(notifier, fTestClass, getDescription(), new Runnable() {
public void run() {
runChildren(notifier);
}
}).runProtected();
}
private Collection<?> getParametersList() throws IllegalAccessException, InvocationTargetException, Exception {
return (Collection<?>) getParametersMethod().invoke(null);
}
private Method getParametersMethod() throws Exception {
List<Method> methods = fTestClass.getAnnotatedMethods(Parameters.class);
for (Method each : methods) {
int modifiers = each.getModifiers();
if (Modifier.isStatic(modifiers) && Modifier.isPublic(modifiers))
return each;
}
throw new Exception("No public static parameters method on class " + getName());
}
public static Collection<Object[]> eachOne(Object... params) {
List<Object[]> results = new ArrayList<Object[]>();
for (Object param : params)
results.add(new Object[] { param });
return results;
}
}
You can create a method like
#Test
public void name() {
Assert.assertEquals("", inboundFileName);
}
While I wouldn't use it all the time it would be useful to figure out exactly which test number 143 is.
I make extensive use of static import for Assert and friends, so it is easy for me to redefine assertion:
private <T> void assertThat(final T actual, final Matcher<T> expected) {
Assert.assertThat(editThisToDisplaySomethingForYourDatum, actual, expected);
}
For example, you could add a "name" field to your test class, initialized in the constructor, and display that on test failure. Just pass it in as the first elements of your parameters array for each test. This also helps label the data:
public ExampleTest(final String testLabel, final int one, final int two) {
this.testLabel = testLabel;
// ...
}
#Parameters
public static Collection<Object[]> data() {
return asList(new Object[][]{
{"first test", 3, 4},
{"second test", 5, 6}
});
}
A workaround would be to catch and nest all Throwables into a new Throwable with a custom message that contains all information about the parameters. The message would appear in the stack trace.
This works whenever a test fails for all assertions, errors and exceptions as they are all subclasses of Throwable.
My code looks like this:
#RunWith(Parameterized.class)
public class ParameterizedTest {
int parameter;
public ParameterizedTest(int parameter) {
super();
this.parameter = parameter;
}
#Parameters
public static Collection<Object[]> data() {
return Arrays.asList(new Object[][] { {1}, {2} });
}
#Test
public void test() throws Throwable {
try {
assertTrue(parameter%2==0);
}
catch(Throwable thrown) {
throw new Throwable("parameter="+parameter, thrown);
}
}
}
The stack trace of the failed test is:
java.lang.Throwable: parameter=1
at sample.ParameterizedTest.test(ParameterizedTest.java:34)
Caused by: java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at sample.ParameterizedTest.test(ParameterizedTest.java:31)
... 31 more
When you want the parameter values in test name then you can do something like -
#ParameterizedTest(name="{index} {arguments} then return false" )
#ValueSource(strings = {"false","FALSE"," ","123","abc"})
#DisplayName("When Feature JVM argument is ")
void test_Feature_JVM_Argument_Is_Empty_Or_Blank_Strings_Or_False(String params) {
System.setProperty("FeatureName", params);
assertFalse(Boolean.parseBoolean(System.getProperty("FeatureName")));
}
Test name will look like -
JUnit Test image
Check out JUnitParams as dsaff mentioned, works using ant to build parameterized test method descriptions in the html report.
This was after trying LabelledParameterized and finding that it although it works with eclipse it does not work with ant as far as the html report is concerned.
Cheers,
Since the parameter accessed (e.g. with "{0}" always returns the toString() representation, one workaround would be to make an anonymous implementation and override toString() in each case. For example:
public static Iterable<? extends Object> data() {
return Arrays.asList(
new MyObject(myParams...) {public String toString(){return "my custom test name";}},
new MyObject(myParams...) {public String toString(){return "my other custom test name";}},
//etc...
);
}
Parameterized test is calling toString() internally.
If you create an object wrapper overiding toString(), it will change the names of the test.
Here is an example, I answered in other post.
https://stackoverflow.com/a/67023556/1839360
For a more complex object you may do the following (example with JUnit 4):
#RunWith(Parameterized.class)
public class MainTest {
private static Object[] makeSample(String[] array, int expectedLength) {
return new Object[]{array, expectedLength, Arrays.toString(array)};
}
#Parameterized.Parameters(name = "for input {2} length should equal {1}")
public static Collection<Object[]> data() {
return Arrays.asList(
makeSample(new String[]{"a"}, 1),
makeSample(new String[]{"a", "b"}, 2)
);
}
private final int expectedLength;
private final String[] array;
public MainTest(String[] array, int expectedLength, String strArray) {
this.array = array;
this.expectedLength = expectedLength;
}
#Test
public void should_have_expected_length() {
assertEquals(expectedLength, array.length);
}
}
The trick here is to use one input parameter as a string describing either some part of input or the whole test case.
Before adding third parameter it looked like this
And after like this
Is there a way to set my own custom test case names when using parameterized tests in JUnit4?
I'd like to change the default — [Test class].runTest[n] — to something meaningful.
This feature has made it into JUnit 4.11.
To use change the name of parameterized tests, you say:
#Parameters(name="namestring")
namestring is a string, which can have the following special placeholders:
{index} - the index of this set of arguments. The default namestring is {index}.
{0} - the first parameter value from this invocation of the test.
{1} - the second parameter value
and so on
The final name of the test will be the name of the test method, followed by the namestring in brackets, as shown below.
For example (adapted from the unit test for the Parameterized annotation):
#RunWith(Parameterized.class)
static public class FibonacciTest {
#Parameters( name = "{index}: fib({0})={1}" )
public static Iterable<Object[]> data() {
return Arrays.asList(new Object[][] { { 0, 0 }, { 1, 1 }, { 2, 1 },
{ 3, 2 }, { 4, 3 }, { 5, 5 }, { 6, 8 } });
}
private final int fInput;
private final int fExpected;
public FibonacciTest(int input, int expected) {
fInput= input;
fExpected= expected;
}
#Test
public void testFib() {
assertEquals(fExpected, fib(fInput));
}
private int fib(int x) {
// TODO: actually calculate Fibonacci numbers
return 0;
}
}
will give names like testFib[1: fib(1)=1] and testFib[4: fib(4)=3]. (The testFib part of the name is the method name of the #Test).
Looking at JUnit 4.5, its runner clearly doesn't support that, as that logic is buried inside a private class inside the Parameterized class. You could not use the JUnit Parameterized runner, and create your own instead which would understand the concept of names (which leads to the question of how you might set a name ...).
From a JUnit perspective, it would be nice if instead of (or in addition to) just passing an increment, they would pass the comma delimited arguments. TestNG does this. If the feature is important to you, you can comment on the yahoo mailing list referenced at www.junit.org.
I recently came across the same problem when using JUnit 4.3.1. I implemented a new class which extends Parameterized called LabelledParameterized. It has been tested using JUnit 4.3.1, 4.4 and 4.5. It reconstructs the Description instance using the String representation of the first argument of each parameter array from the #Parameters method. You can see the code for this at:
http://code.google.com/p/migen/source/browse/trunk/java/src/.../LabelledParameterized.java?r=3789
and an example of its use at:
http://code.google.com/p/migen/source/browse/trunk/java/src/.../ServerBuilderTest.java?r=3789
The test description formats nicely in Eclipse which is what I wanted since this makes failed tests a lot easier to find! I will probably further refine and document the classes over the next few days/weeks. Drop the '?' part of the URLs if you want the bleeding edge. :-)
To use it, all you have to do is copy that class (GPL v3), and change #RunWith(Parameterized.class) to #RunWith(LabelledParameterized.class) assuming the first element of your parameter list is a sensible label.
I don't know if any later releases of JUnit address this issue but even if they did, I can't update JUnit since all my co-developers would have to update too and we have higher priorities than re-tooling. Hence the work in the class to be compilable by multiple versions of JUnit.
Note: there is some reflection jiggery-pokery so that it runs across the different JUnit versions as listed above. The version specifically for JUnit 4.3.1 can be found here and, for JUnit 4.4 and 4.5, here.
With Parameterized as a model, I wrote my own custom test runner / suite -- only took about half an hour. It's slightly different from darrenp's LabelledParameterized in that it lets you specify a name explicitly rather than relying on the first parameter's toString().
It also doesn't use arrays because I hate arrays. :)
public class PolySuite extends Suite {
// //////////////////////////////
// Public helper interfaces
/**
* Annotation for a method which returns a {#link Configuration}
* to be injected into the test class constructor
*/
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.METHOD)
public static #interface Config {
}
public static interface Configuration {
int size();
Object getTestValue(int index);
String getTestName(int index);
}
// //////////////////////////////
// Fields
private final List<Runner> runners;
// //////////////////////////////
// Constructor
/**
* Only called reflectively. Do not use programmatically.
* #param c the test class
* #throws Throwable if something bad happens
*/
public PolySuite(Class<?> c) throws Throwable {
super(c, Collections.<Runner>emptyList());
TestClass testClass = getTestClass();
Class<?> jTestClass = testClass.getJavaClass();
Configuration configuration = getConfiguration(testClass);
List<Runner> runners = new ArrayList<Runner>();
for (int i = 0, size = configuration.size(); i < size; i++) {
SingleRunner runner = new SingleRunner(jTestClass, configuration.getTestValue(i), configuration.getTestName(i));
runners.add(runner);
}
this.runners = runners;
}
// //////////////////////////////
// Overrides
#Override
protected List<Runner> getChildren() {
return runners;
}
// //////////////////////////////
// Private
private Configuration getConfiguration(TestClass testClass) throws Throwable {
return (Configuration) getConfigMethod(testClass).invokeExplosively(null);
}
private FrameworkMethod getConfigMethod(TestClass testClass) {
List<FrameworkMethod> methods = testClass.getAnnotatedMethods(Config.class);
if (methods.isEmpty()) {
throw new IllegalStateException("#" + Config.class.getSimpleName() + " method not found");
}
if (methods.size() > 1) {
throw new IllegalStateException("Too many #" + Config.class.getSimpleName() + " methods");
}
FrameworkMethod method = methods.get(0);
int modifiers = method.getMethod().getModifiers();
if (!(Modifier.isStatic(modifiers) && Modifier.isPublic(modifiers))) {
throw new IllegalStateException("#" + Config.class.getSimpleName() + " method \"" + method.getName() + "\" must be public static");
}
return method;
}
// //////////////////////////////
// Helper classes
private static class SingleRunner extends BlockJUnit4ClassRunner {
private final Object testVal;
private final String testName;
SingleRunner(Class<?> testClass, Object testVal, String testName) throws InitializationError {
super(testClass);
this.testVal = testVal;
this.testName = testName;
}
#Override
protected Object createTest() throws Exception {
return getTestClass().getOnlyConstructor().newInstance(testVal);
}
#Override
protected String getName() {
return testName;
}
#Override
protected String testName(FrameworkMethod method) {
return testName + ": " + method.getName();
}
#Override
protected void validateConstructor(List<Throwable> errors) {
validateOnlyOneConstructor(errors);
}
#Override
protected Statement classBlock(RunNotifier notifier) {
return childrenInvoker(notifier);
}
}
}
And an example:
#RunWith(PolySuite.class)
public class PolySuiteExample {
// //////////////////////////////
// Fixture
#Config
public static Configuration getConfig() {
return new Configuration() {
#Override
public int size() {
return 10;
}
#Override
public Integer getTestValue(int index) {
return index * 2;
}
#Override
public String getTestName(int index) {
return "test" + index;
}
};
}
// //////////////////////////////
// Fields
private final int testVal;
// //////////////////////////////
// Constructor
public PolySuiteExample(int testVal) {
this.testVal = testVal;
}
// //////////////////////////////
// Test
#Ignore
#Test
public void odd() {
assertFalse(testVal % 2 == 0);
}
#Test
public void even() {
assertTrue(testVal % 2 == 0);
}
}
You may also want to try JUnitParams: https://github.com/Pragmatists/JUnitParams
from junit4.8.2, you can create your own MyParameterized class by simply copy Parameterized class. change the getName() and testName() methods in TestClassRunnerForParameters.
None of it was working for me, so I got the source for Parameterized and modified it create a a new test runner. I didn't have to change much but IT WORKS!!!
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import org.junit.Assert;
import org.junit.internal.runners.ClassRoadie;
import org.junit.internal.runners.CompositeRunner;
import org.junit.internal.runners.InitializationError;
import org.junit.internal.runners.JUnit4ClassRunner;
import org.junit.internal.runners.MethodValidator;
import org.junit.internal.runners.TestClass;
import org.junit.runner.notification.RunNotifier;
public class LabelledParameterized extends CompositeRunner {
static class TestClassRunnerForParameters extends JUnit4ClassRunner {
private final Object[] fParameters;
private final String fParameterFirstValue;
private final Constructor<?> fConstructor;
TestClassRunnerForParameters(TestClass testClass, Object[] parameters, int i) throws InitializationError {
super(testClass.getJavaClass()); // todo
fParameters = parameters;
if (parameters != null) {
fParameterFirstValue = Arrays.asList(parameters).toString();
} else {
fParameterFirstValue = String.valueOf(i);
}
fConstructor = getOnlyConstructor();
}
#Override
protected Object createTest() throws Exception {
return fConstructor.newInstance(fParameters);
}
#Override
protected String getName() {
return String.format("%s", fParameterFirstValue);
}
#Override
protected String testName(final Method method) {
return String.format("%s%s", method.getName(), fParameterFirstValue);
}
private Constructor<?> getOnlyConstructor() {
Constructor<?>[] constructors = getTestClass().getJavaClass().getConstructors();
Assert.assertEquals(1, constructors.length);
return constructors[0];
}
#Override
protected void validate() throws InitializationError {
// do nothing: validated before.
}
#Override
public void run(RunNotifier notifier) {
runMethods(notifier);
}
}
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.METHOD)
public static #interface Parameters {
}
private final TestClass fTestClass;
public LabelledParameterized(Class<?> klass) throws Exception {
super(klass.getName());
fTestClass = new TestClass(klass);
MethodValidator methodValidator = new MethodValidator(fTestClass);
methodValidator.validateStaticMethods();
methodValidator.validateInstanceMethods();
methodValidator.assertValid();
int i = 0;
for (final Object each : getParametersList()) {
if (each instanceof Object[])
add(new TestClassRunnerForParameters(fTestClass, (Object[]) each, i++));
else
throw new Exception(String.format("%s.%s() must return a Collection of arrays.", fTestClass.getName(), getParametersMethod().getName()));
}
}
#Override
public void run(final RunNotifier notifier) {
new ClassRoadie(notifier, fTestClass, getDescription(), new Runnable() {
public void run() {
runChildren(notifier);
}
}).runProtected();
}
private Collection<?> getParametersList() throws IllegalAccessException, InvocationTargetException, Exception {
return (Collection<?>) getParametersMethod().invoke(null);
}
private Method getParametersMethod() throws Exception {
List<Method> methods = fTestClass.getAnnotatedMethods(Parameters.class);
for (Method each : methods) {
int modifiers = each.getModifiers();
if (Modifier.isStatic(modifiers) && Modifier.isPublic(modifiers))
return each;
}
throw new Exception("No public static parameters method on class " + getName());
}
public static Collection<Object[]> eachOne(Object... params) {
List<Object[]> results = new ArrayList<Object[]>();
for (Object param : params)
results.add(new Object[] { param });
return results;
}
}
You can create a method like
#Test
public void name() {
Assert.assertEquals("", inboundFileName);
}
While I wouldn't use it all the time it would be useful to figure out exactly which test number 143 is.
I make extensive use of static import for Assert and friends, so it is easy for me to redefine assertion:
private <T> void assertThat(final T actual, final Matcher<T> expected) {
Assert.assertThat(editThisToDisplaySomethingForYourDatum, actual, expected);
}
For example, you could add a "name" field to your test class, initialized in the constructor, and display that on test failure. Just pass it in as the first elements of your parameters array for each test. This also helps label the data:
public ExampleTest(final String testLabel, final int one, final int two) {
this.testLabel = testLabel;
// ...
}
#Parameters
public static Collection<Object[]> data() {
return asList(new Object[][]{
{"first test", 3, 4},
{"second test", 5, 6}
});
}
A workaround would be to catch and nest all Throwables into a new Throwable with a custom message that contains all information about the parameters. The message would appear in the stack trace.
This works whenever a test fails for all assertions, errors and exceptions as they are all subclasses of Throwable.
My code looks like this:
#RunWith(Parameterized.class)
public class ParameterizedTest {
int parameter;
public ParameterizedTest(int parameter) {
super();
this.parameter = parameter;
}
#Parameters
public static Collection<Object[]> data() {
return Arrays.asList(new Object[][] { {1}, {2} });
}
#Test
public void test() throws Throwable {
try {
assertTrue(parameter%2==0);
}
catch(Throwable thrown) {
throw new Throwable("parameter="+parameter, thrown);
}
}
}
The stack trace of the failed test is:
java.lang.Throwable: parameter=1
at sample.ParameterizedTest.test(ParameterizedTest.java:34)
Caused by: java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at sample.ParameterizedTest.test(ParameterizedTest.java:31)
... 31 more
When you want the parameter values in test name then you can do something like -
#ParameterizedTest(name="{index} {arguments} then return false" )
#ValueSource(strings = {"false","FALSE"," ","123","abc"})
#DisplayName("When Feature JVM argument is ")
void test_Feature_JVM_Argument_Is_Empty_Or_Blank_Strings_Or_False(String params) {
System.setProperty("FeatureName", params);
assertFalse(Boolean.parseBoolean(System.getProperty("FeatureName")));
}
Test name will look like -
JUnit Test image
Check out JUnitParams as dsaff mentioned, works using ant to build parameterized test method descriptions in the html report.
This was after trying LabelledParameterized and finding that it although it works with eclipse it does not work with ant as far as the html report is concerned.
Cheers,
Since the parameter accessed (e.g. with "{0}" always returns the toString() representation, one workaround would be to make an anonymous implementation and override toString() in each case. For example:
public static Iterable<? extends Object> data() {
return Arrays.asList(
new MyObject(myParams...) {public String toString(){return "my custom test name";}},
new MyObject(myParams...) {public String toString(){return "my other custom test name";}},
//etc...
);
}
Parameterized test is calling toString() internally.
If you create an object wrapper overiding toString(), it will change the names of the test.
Here is an example, I answered in other post.
https://stackoverflow.com/a/67023556/1839360
For a more complex object you may do the following (example with JUnit 4):
#RunWith(Parameterized.class)
public class MainTest {
private static Object[] makeSample(String[] array, int expectedLength) {
return new Object[]{array, expectedLength, Arrays.toString(array)};
}
#Parameterized.Parameters(name = "for input {2} length should equal {1}")
public static Collection<Object[]> data() {
return Arrays.asList(
makeSample(new String[]{"a"}, 1),
makeSample(new String[]{"a", "b"}, 2)
);
}
private final int expectedLength;
private final String[] array;
public MainTest(String[] array, int expectedLength, String strArray) {
this.array = array;
this.expectedLength = expectedLength;
}
#Test
public void should_have_expected_length() {
assertEquals(expectedLength, array.length);
}
}
The trick here is to use one input parameter as a string describing either some part of input or the whole test case.
Before adding third parameter it looked like this
And after like this