Develop Reference

How to get annotations of JUnit test suite class?

I have such an example of test suite class to run many test classes at once in JUnit 4.13.
#RunWith(Suite.class)
#SuiteClasses({
FirstTest.class,
SecondTest.class
})
#TestSuiteAnnotation
public class TestSuite {
}
These are my test classes.
#FirstAnnotation
public class FirstTest extends ExtTest {
#Test
public void test() {
}
}
#SecondAnnotation
public class SecondTest extends ExtTest {
#Test
public void test() {
}
}
public class ExtTest {
#Before
public void beforeMethod() {
System.out.println("Annotations from " + this.getClass());
Arrays.asList(this.getClass().getAnnotations()).forEach(System.out::println);
}
}
When I run test from TestSuite.class, the console output is:
Annotations from class FirstTest
#FirstAnnotation()
Annotations from class SecondTest
#SecondAnnotation()
Currently, this.getClass().getAnnotations() returns annotations from test classes (i.e. FirstTest.class, SecondTest.class). I want to obtain annotation #TestSuiteAnnotation, when I run tests from TestSuite.class.
The expected output should be:
Annotations from class FirstTest
#FirstAnnotation()
#TestSuiteAnnotation()
Annotations from class SecondTest
#SecondAnnotation()
#TestSuiteAnnotation()
Can I somehow obtain annotation #TestSuiteAnnotation, when I run tests from TestSuite.class?

You have multiple options:
JUnit 4 run listener
On JUnit 4, you can register a RunListener, like #nrainer said. If you build with Maven, it is easy to register a run listener like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<includes>
<include>org.acme.TestSuite</include>
</includes>
<properties>
<property>
<name>listener</name>
<value>org.acme.SuiteRunListener</value>
</property>
</properties>
</configuration>
</plugin>
The run listener can override the events testSuiteStarted and testSuiteFinished and either directly log the annotations you are interested in or assign them to a static thread-local variable like private static ThreadLocal<List<Annotation>> currentSuiteAnnotations in testSuiteStarted, then unassign it again in testSuiteFinished.
This works nicely from Maven, I tested it. Unfortunately, there is no direct support for running tests with run listeners from IDEs like IntelliJ IDEA or Eclipse. So if you want to avoid running the tests manually from a class with a main method as shown here, because it would take away all the nice IDE test reporting with drill-down from suite to test class to test method, this is not an option.
JUnit 5 test execution listener
Similar to JUnit 4's run listener, you can register a TestExecutionListener for your JUnit 5 tests. The advantage in JUnit 5 is that you can register it globally via Java's s ServiceLoader mechanism, i.e. it will be picked up when bootstrapping JUnit and should also work in IDEs. I did something similar with another type of extension, and it worked nicely in IntelliJ IDEA and of course also in Maven.
JUnit 4 with custom suite runner
Coming back to JUnit 4, we can extend the first approach with the run listener by declaring a special type of suite. You simply use that suite instead of org.junit.runners.Suite and can enjoy the working run listener in both Maven and the IDE. It works like that, see also my MCVE on GitHub for your convenience:
package org.acme;
import org.junit.runner.Description;
import org.junit.runner.RunWith;
import org.junit.runner.notification.RunListener;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
import java.lang.annotation.Annotation;
import java.util.List;
import java.util.stream.Collectors;
public class SuiteRunListener extends RunListener {
private static ThreadLocal<String> currentSuiteName = new ThreadLocal<String>();
private static ThreadLocal<List<Annotation>> currentSuiteAnnotations = new ThreadLocal<>();
#Override
public void testSuiteStarted(Description description) throws Exception {
super.testSuiteStarted(description);
final RunWith runWith = description.getAnnotation(RunWith.class);
if (runWith != null && runWith.value().equals(SuiteWithListener.class)) {
currentSuiteName.set(description.getDisplayName());
currentSuiteAnnotations.set(
description.getAnnotations().stream()
.filter(annotation -> {
final Class<? extends Annotation> annotationType = annotation.annotationType();
return !(annotationType.equals(RunWith.class) || annotationType.equals(SuiteClasses.class));
})
.collect(Collectors.toList())
);
}
}
#Override
public void testSuiteFinished(Description description) throws Exception {
super.testSuiteFinished(description);
final RunWith runWith = description.getAnnotation(RunWith.class);
if (runWith != null && runWith.value().equals(SuiteWithListener.class)) {
currentSuiteName.set(null);
currentSuiteAnnotations.set(null);
}
}
public static String getCurrentSuiteName() {
return currentSuiteName.get();
}
public static List<Annotation> getCurrentSuiteAnnotations() {
return currentSuiteAnnotations.get();
}
}
package org.acme;
import org.junit.runner.Runner;
import org.junit.runner.notification.RunNotifier;
import org.junit.runners.Suite;
import org.junit.runners.model.InitializationError;
import org.junit.runners.model.RunnerBuilder;
import java.util.List;
public class SuiteWithListener extends Suite {
public SuiteWithListener(Class<?> klass, RunnerBuilder builder) throws InitializationError {
super(klass, builder);
}
public SuiteWithListener(RunnerBuilder builder, Class<?>[] classes) throws InitializationError {
super(builder, classes);
}
protected SuiteWithListener(Class<?> klass, Class<?>[] suiteClasses) throws InitializationError {
super(klass, suiteClasses);
}
protected SuiteWithListener(RunnerBuilder builder, Class<?> klass, Class<?>[] suiteClasses) throws InitializationError {
super(builder, klass, suiteClasses);
}
protected SuiteWithListener(Class<?> klass, List<Runner> runners) throws InitializationError {
super(klass, runners);
}
#Override
public void run(RunNotifier notifier) {
notifier.addListener(new SuiteRunListener()); // !!!
super.run(notifier);
}
}
package org.acme;
import org.junit.runner.RunWith;
import org.junit.runners.Suite.SuiteClasses;
#RunWith(SuiteWithListener.class) // !!!
#SuiteClasses({
FirstTest.class,
SecondTest.class
})
#TestSuiteAnnotation
public class TestSuite {}
package org.acme;
import org.junit.Before;
import java.util.Arrays;
public class ExtTest {
#Before
public void beforeMethod() {
String currentSuiteName = SuiteRunListener.getCurrentSuiteName();
if (currentSuiteName != null) {
System.out.println("Annotations from suite " + currentSuiteName);
SuiteRunListener.getCurrentSuiteAnnotations().forEach(System.out::println);
}
System.out.println("Annotations from class " + this.getClass());
Arrays.asList(this.getClass().getAnnotations()).forEach(System.out::println);
System.out.println();
}
}
Now when running your suite, you should see output like this:
Annotations from suite org.acme.TestSuite
#org.acme.TestSuiteAnnotation()
Annotations from class class org.acme.FirstTest
#org.acme.FirstAnnotation()
Annotations from suite org.acme.TestSuite
#org.acme.TestSuiteAnnotation()
Annotations from class class org.acme.SecondTest
#org.acme.SecondAnnotation()
Please note: I was assuming that you really need access to the current suite from each single test method, not just at the test class or suite level. If you do not need that and it is enough to let the run listener do something when a suite is started and/or finished, of course you do not need the getter methods for current suite name and suite annotations. I just extended your own example.

The Junit test fails in Netbeans 11.3 IDE "Could not find or load main class"

I just installed Netbeans 11.3, on windows 10 running JDK 14. I tried to follow the simple 10 minutes learning tutorial provided at https://netbeans.apache.org/kb/docs/java/javase-intro.html just to get my feet wet. Everything worked fine until I reached the section under "Creating JUnits Tests". The IDE does a good job in generating the code for the JUnit test class, however, the test fails with Could not find or load main class error. Here is the stack trace from the output window:
compile-test:
Created dir: C:\Users\....\Documents\NetBeansProjects\MyLib\build\test\results
Testsuite: org.me.mylib.LibClassTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.152 sec
Testcase: initializationError(org.me.mylib.LibClassTest): Caused an ERROR
No runnable methods
java.lang.Exception: No runnable methods
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
Test org.me.mylib.LibClassTest FAILED
The exception seems to take place during the test initialization, internal to the JUnit framework.
Here is the code for the test class (LibClassTest from the file LibClassTest.java) generated by the IDE per tutorial example:
package org.me.mylib;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
/**
* #author hrh74
*/
public class LibClassTest {
public LibClassTest() {
System.out.println("Creating LibClassTest");
}
#BeforeAll
public static void setUpClass() {
}
#AfterAll
public static void tearDownClass() {
}
#BeforeEach
public void setUp() {
}
#AfterEach
public void tearDown() {
}
/**
* Test of acrostic method, of class LibClass.
*/
#Test
public void testAcrostic() {
System.err.println("Running testAcrostic...");
String result = LibClass.acrostic(new String[]{"fnord", "polly", "tropism"});
assertEquals("Correct value", "foo", result);
}
}
Here is the code for the class to be tested by the JUnit per the tutorial example:
package org.me.mylib;
/**
*
* #author hrh74
*/
public class LibClass {
public static String acrostic(String[] args) {
StringBuffer b = new StringBuffer();
for (int i = 0; i < args.length; i++) {
if (args[i].length() > i) {
b.append(args[i].charAt(i));
} else {
b.append('?');
}
}
System.out.println("The content of the buffer " + b);
return b.toString();
}
}
Any insights would be great.
Thanks in advance.

Well, it turned out that the putative "JUnit 5" feature in Netbeans 11.3 is not working. The support from the Netbeans mailing list was able to reproduce the bug. The solution is to change the test class. The import statements having org.junit.jupiter should be supplanted by their corresponding org.junit statements. In addition, the JUnit 5 annotations need to change to their corresponding older version. Here is a working example:
package org.me.mylib;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import static org.junit.Assert.*;
public class LibClassTest {
public LibClassTest() {
}
#BeforeClass
public static void setUpClass() {
}
#AfterClass
public static void tearDownClass() {
}
#Before
public void setUp() {
}
#After
public void tearDown() {
}
#Test
public void testAcrostic() {
System.err.println("Running testAcrostic...");
String result = LibClass.acrostic(new String[]{"fnord", "polly", "tropism"});
assertEquals("Correct value", "foo", result);
}
}

Verifiyng behavior of mock type without instance controlling

I'm developing custom runner of JUnit for internal purposes and, for instance, I've introduced custom annotation for test methods which on applying should make my runner to run method with this annotation after all other test methods without this annotation.
I want to write junit test to verify behavior of my custom runner.
Test class:
public class TestClass {
#Test
#CustomAnnotation
public void test1() {
System.out.println("test1");
}
#Test
public void test2() {
System.out.println("test2");
}
}
An abstract code that will test my runner:
public class MyCustomRunnerTest {
#Test
public void order() throws InitializationError {
// Arrange
// Some code of mocking library might be placed here
// Act
MyCustomRunner runner = new MyCustomRunner(TestClass.class);
runner.run(new RunNotifier());
// Assert
// Here I want to verify that method test1() has been called
// after method test2()
}
}
Is there any mocking libraries that will allow me to perform such verification? Or may be is there any other way to check that?

Why do you not extract the logic that determines the run order of test methods into a separate class or method? This method should return a list of test method names (or other descriptors) in order in which they will run. Then your testing will come down to passing it the test class and asserting that the output is { "test2", "test1" }. No mocking required.
A Better Solution
Use RunListener to log test methods as they are being run by your runner. You of course will have your own MyCustomRunner class, but the rest of the code can stay as in the example below:
import static org.hamcrest.Matchers.contains;
import static org.junit.Assert.assertThat;
import java.util.ArrayList;
import java.util.Collection;
import org.junit.Test;
import org.junit.runner.Description;
import org.junit.runner.RunWith;
import org.junit.runner.notification.RunListener;
import org.junit.runner.notification.RunNotifier;
import org.junit.runners.BlockJUnit4ClassRunner;
import org.junit.runners.model.InitializationError;
#RunWith(BlockJUnit4ClassRunner.class)
public class RunnerTest {
// dummy "custom" test runner
public static class MyCustomRunner extends BlockJUnit4ClassRunner {
public MyCustomRunner(Class<?> klass) throws InitializationError {
super(klass);
}
}
public static class TestClass {
#Test
public void test1() {}
#Test
public void test2() {}
}
#Test
public void myCustomRunnerExecutesTestsInOrder() throws InitializationError {
RunNotifier notifier = new RunNotifier();
Collection<String> runTestMethods = new ArrayList<>();
notifier.addListener(new RunListener() {
#Override
public void testStarted(Description description) throws Exception {
runTestMethods.add(description.getMethodName());
}
});
new MyCustomRunner(TestClass.class).run(notifier);
// assert that the collection contains methods names in the specified order
assertThat(runTestMethods, contains("test1", "test2"));
}
}

How to run entire JUnit Test Suite from within code

In eclipse, with JUnit 4, you can right click a project or package and click Run as JUnit Test, and it will run all the JUnit tests within that grouping. Is there a way to do this same thing from within the code?

You can use packages in junit such as JUnitCore like this:
public static void main(String[] args){
List tests = new ArrayList();
tests.add(TestOne.class);
tests.add(TestTwo.class);
for (Class test : tests){
runTests(test);
}
}
private static void runTests(Class test){
Result result = JUnitCore.runClasses(test);
for (Failure failure : result.getFailures()){
System.out.println(failure.toString());
}
}

Use JUnit Suite:
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
#RunWith(Suite.class)
// Put your Test Case Class here
#Suite.SuiteClasses({
JunitTest1.class,
JunitTest2.class,
JunitTest3.class
})
public class JunitTestSuite {}
Then create a main method to run it.
import org.junit.runner.JUnitCore;
import org.junit.runner.Result;
import org.junit.runner.notification.Failure;
public class JunitTestSuiteRunner {
public static void main(String[] args) {
Result result = JUnitCore.runClasses(JunitTestSuite.class);
for (Failure fail : result.getFailures()) {
System.out.println(fail.toString());
}
if (result.wasSuccessful()) {
System.out.println("All tests finished successfully...");
}
}}

JUnit provides the test Suite. Give that a try.
[...]
public class TestCaseA {
#Test
public void testA1() {
// omitted
}
}
[...]
public class TestCaseB {
#Test
public void testB1() {
// omitted
}
}
[...]
#RunWith(value=Suite.class)
#SuiteClasses(value = {TestCaseA.class})
public class TestSuiteA {
}
[...]
#RunWith(value=Suite.class)
#SuiteClasses(value = {TestCaseB.class})
public class TestSuiteB {
}
[...]
#RunWith(value = Suite.class )
#SuiteClasses(value = {TestSuiteA.class, TestSuiteB.class})
public class MasterTestSuite{
}

JUnit 4.x : why is #Before never executed?

When I create an and run unit-test, (in Eclipse (Galileo) with JUnit 4.5 or 4.82),
the #Before is never executed (?).
Below is some sample-code. I would expect the output to be :
initialize
testGetFour
But it is just :
testGetFour
#BeforeClass and #AfterClass are never executed either.
Can someone tell me how come ?
public class SomeClass
{
public static int getFour()
{
return 4;
}
}
//---
import org.junit.Before;
import org.junit.Test;
import junit.framework.TestCase;
public class TestSomeClass extends TestCase
{
#Before
public void initialize() // This method will never execute (?!).
{
System.err.println("initialize");
}
#Test
public void testGetFour()
{
System.err.println("testGetFour");
assertEquals(4, SomeClass.getFour());
}
}

Because you're extending TestCase. This is a JUnit3 class, and so Eclipse treats it as such. JUnit 4 does not require the test class to extend anything.
Remove that, it should work fine.

You should not extend from TestCase (JUnit 3 way of using JUnit) and it will work.