Develop Reference

Is there a way to get the list of scenarios that are to be run in the #BeforeClass annotation in cucumber JVM

I have got a requirement to get the list of all the scenarios that are to be executed based on the tag I provided in cucumber Test runner. However I have to get this list before tests start execution.
I know there is a tag called "#BeforeClass" but I am not sure if I can use to get the list of all the scenarios that are going to be run. For example something like this
#BeforeClass
public void intialize(Scenario[] scenario) throws Exception { }
Below is the code for me test runner class
package com.automation.cucumber;
import com.automation.Utils;
import io.cucumber.java.Scenario;
import io.cucumber.testng.*;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.DataProvider;
import java.io.File;
#CucumberOptions(features = "features/amazon"
,glue="com.automation.cucumber"
,tags = "#tt"
,dryRun = true
, plugin = {"json:target/cucumber-reports/cucumber.json"})
public class CucumberTestRunner extends AbstractTestNGCucumberTests {
static String resultFolder;
#DataProvider(parallel = true)
public Object[][] scenarios() {
return super.scenarios();
}
#BeforeClass
public void intialize() throws Exception {
resultFolder = Utils.createTestReportFolder();
if(resultFolder==null)
{
throw new Exception("Unable to create a result folder");
}
System.out.println();
}
}

You may have to implement EventListener class to get that information and do dryRun = true in your Runner class in #CucumberOptions
Quoting from a question that can help you achieve what you need
public class DryRunPlugin implements EventListener {
#Override
public void setEventPublisher(EventPublisher publisher) {
publisher.registerHandlerFor(TestCaseStarted.class, this::handleCaseStarted);
}
private void handleCaseStarted(TestCaseStarted event) {
System.out.println(event.getTestCase().getUri());
System.out.println(event.getTestCase().getName());
System.out.println(event.getTestCase().getScenarioDesignation());
event.getTestCase().getTags().stream().forEach(t ->
System.out.println(t.getName()));
}
}

Verifiyng behavior of mock type without instance controlling

I'm developing custom runner of JUnit for internal purposes and, for instance, I've introduced custom annotation for test methods which on applying should make my runner to run method with this annotation after all other test methods without this annotation.
I want to write junit test to verify behavior of my custom runner.
Test class:
public class TestClass {
#Test
#CustomAnnotation
public void test1() {
System.out.println("test1");
}
#Test
public void test2() {
System.out.println("test2");
}
}
An abstract code that will test my runner:
public class MyCustomRunnerTest {
#Test
public void order() throws InitializationError {
// Arrange
// Some code of mocking library might be placed here
// Act
MyCustomRunner runner = new MyCustomRunner(TestClass.class);
runner.run(new RunNotifier());
// Assert
// Here I want to verify that method test1() has been called
// after method test2()
}
}
Is there any mocking libraries that will allow me to perform such verification? Or may be is there any other way to check that?

Why do you not extract the logic that determines the run order of test methods into a separate class or method? This method should return a list of test method names (or other descriptors) in order in which they will run. Then your testing will come down to passing it the test class and asserting that the output is { "test2", "test1" }. No mocking required.
A Better Solution
Use RunListener to log test methods as they are being run by your runner. You of course will have your own MyCustomRunner class, but the rest of the code can stay as in the example below:
import static org.hamcrest.Matchers.contains;
import static org.junit.Assert.assertThat;
import java.util.ArrayList;
import java.util.Collection;
import org.junit.Test;
import org.junit.runner.Description;
import org.junit.runner.RunWith;
import org.junit.runner.notification.RunListener;
import org.junit.runner.notification.RunNotifier;
import org.junit.runners.BlockJUnit4ClassRunner;
import org.junit.runners.model.InitializationError;
#RunWith(BlockJUnit4ClassRunner.class)
public class RunnerTest {
// dummy "custom" test runner
public static class MyCustomRunner extends BlockJUnit4ClassRunner {
public MyCustomRunner(Class<?> klass) throws InitializationError {
super(klass);
}
}
public static class TestClass {
#Test
public void test1() {}
#Test
public void test2() {}
}
#Test
public void myCustomRunnerExecutesTestsInOrder() throws InitializationError {
RunNotifier notifier = new RunNotifier();
Collection<String> runTestMethods = new ArrayList<>();
notifier.addListener(new RunListener() {
#Override
public void testStarted(Description description) throws Exception {
runTestMethods.add(description.getMethodName());
}
});
new MyCustomRunner(TestClass.class).run(notifier);
// assert that the collection contains methods names in the specified order
assertThat(runTestMethods, contains("test1", "test2"));
}
}

No runnable methods found Running multiple JUnit4 testSuite

I have the following testSuite
package com.swaserver.junit;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
import junit.framework.TestSuite;
public class MyTestSuite extends TestSuite
{
#RunWith(Suite.class)
#SuiteClasses( { BinarySearchTest.class })
public class RunTestSuite
{
}
}
However, It tells me I have no runnable methods. My Unit test itself runs fine in my IDE using Junit4 annotations so there is defo a valid JUnit test contained in it.
However the Test Suite above complains about no Runnable Methods when run using the IDE and ANT
package com.swaserver.junit;
import org.junit.Test;
public class BinarySearchTest
{
#Test
public void test()
{
}
}
I was referencing this online example
http://selftechy.com/2011/04/16/junit-4-executing-multiple-test-cases

MyTestSuite should have the annotations defined at the class level, like so:
#RunWith(Suite.class)
#SuiteClasses( { BinarySearchTest.class })
public class MyTestSuite
{
}
If you're using the annotation based approach you shouldn't extend TestSuite, also you don't need the constructor.

JUnit 4.x : why is #Before never executed?

When I create an and run unit-test, (in Eclipse (Galileo) with JUnit 4.5 or 4.82),
the #Before is never executed (?).
Below is some sample-code. I would expect the output to be :
initialize
testGetFour
But it is just :
testGetFour
#BeforeClass and #AfterClass are never executed either.
Can someone tell me how come ?
public class SomeClass
{
public static int getFour()
{
return 4;
}
}
//---
import org.junit.Before;
import org.junit.Test;
import junit.framework.TestCase;
public class TestSomeClass extends TestCase
{
#Before
public void initialize() // This method will never execute (?!).
{
System.err.println("initialize");
}
#Test
public void testGetFour()
{
System.err.println("testGetFour");
assertEquals(4, SomeClass.getFour());
}
}

Because you're extending TestCase. This is a JUnit3 class, and so Eclipse treats it as such. JUnit 4 does not require the test class to extend anything.
Remove that, it should work fine.

You should not extend from TestCase (JUnit 3 way of using JUnit) and it will work.

How do I Dynamically create a Test Suite in JUnit 4?

I would like to create a junit test suite using JUnit 4 where the names of the test classes to be included are not known until the test suite is run.
In JUnit 3 I could do this:
public final class MasterTester extends TestCase
{
/**
* Used by junit to specify what TestCases to run.
*
* #return a suite containing what TestCases to run
*/
public static TestSuite suite() {
TestSuite suite = new TestSuite();
for(Class<?> klass : gatherTestClasses()) {
suite.addTestSuite(klass);
}
return suite;
}
}
and let the gatherTestClasses() method deal with figuring out what test classes to run.
In JUnit 4, the documentation says to use an annotation: #SuiteClasses({TestClass1.class, TestClass2.class...}) to build up my test suite. There are numerous SO answers showing how to do this. Unfortunately the examples I see do not seem to allow for passing a dynamically generated list of TestClasses.
This SO answer suggested I would have to subclass BlockJUnit4ClassRunner which I do not want to do.
Dynamically specified test suites seem like something that must be in JUnit 4 somewhere. Does anyone know where?

To create a dynamic test suite, you need to use the #RunWith annotation. There are two common ways to use it:
#RunWith(Suite.class)
This allows you to specify, which classes compose the test suite in question. This is equivalent to the JUnit 3 style:
import junit.framework.TestSuite;
import junit.framework.TestCase;
public final class MasterTester extends TestCase {
public static TestSuite suite() {
TestSuite suite = new TestSuite();
suite.addTestSuite(TestClass1.class);
suite.addTestSuite(TestClass2.class);
// etc...
return suite;
}
}
The equivalent JUnit 4 class will be:
import org.junit.runners.Suite;
#RunWith(Suite.class)
#SuiteClasses({TestClass1.class, TestClass2.class})
public final class MasterTester {
}
#RunWith(AllTests.class)
This allows you to dynamically specify the tests, which compose the test suite. If your tests are not known until runtime, you cannot specify them in the annotations. You can use this construction instead. So, if the JUnit 3 code is:
import junit.framework.TestCase;
import junit.framework.TestSuite;
import junit.framework.Test;
public final class MasterTester extends TestCase {
public static TestSuite suite() {
TestSuite suite = new TestSuite();
for (Test test : findAllTestCasesRuntime()) {
suite.addTest(test);
}
return suite;
}
}
The equivalent JUnit 4 code will be:
import org.junit.runners.AllTests;
import junit.framework.TestSuite;
import junit.framework.Test;
#RunWith(AllTests.class)
public final class MasterTester {
public static TestSuite suite() {
TestSuite suite = new TestSuite();
for (Test test : findAllTestCasesRuntime()) {
suite.addTest(test);
}
return suite;
}
}

I've tried this using JUnit 4.8 and it works:
#RunWith(AllTests.class)
public class SomeTests
{
public static TestSuite suite()
{
TestSuite suite = new TestSuite();
suite.addTest(new JUnit4TestAdapter(Test1.class));
suite.addTest(new JUnit4TestAdapter(Test2.class));
return suite;
}
}

I found Classpath suite quite useful when used with a naming convention on my test classes.
https://github.com/takari/takari-cpsuite
Here is an example:
import org.junit.extensions.cpsuite.ClasspathSuite;
import org.junit.runner.RunWith;
#RunWith(ClasspathSuite.class)
#ClassnameFilters({".*UnitTest"})
public class MySuite {
}

I'm not sure what gatherTestClasses() does, but let's say it returns some tests when the OS is Linux and different tests when the OS is Windows. You can replicate that in JUnit 4.4 with assumptions:
#Test
public void onlyOnLinux() {
assumeThat(getOS(), is(OperatingSystem.LINUX));
// rest of test
}
#Test
public void onlyOnWindows() {
assumeThat(getOS(), is(OperatingSystem.WINDOWS));
// rest of test
}
#Test
public void anyOperatingSystem() {
// just don't call assumeThat(..)
}
The implementation of getOS() and OperatingSystem being your custom code.

Here is a Complete example how to implement that. it combines of two testCase classes and one suite.
ExampleInstrumentedTest:
import android.support.test.rule.ActivityTestRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
#RunWith(JUnit4.class)
public class ExampleInstrumentedTest {
#Rule
public ActivityTestRule<MainActivity> mActivityTestRule = new ActivityTestRule<>(MainActivity.class);
#Test
public void checkInputs() throws Exception {
}
}
ExampleInstrumentedTest2:
import android.support.test.rule.ActivityTestRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
#RunWith(JUnit4.class)
public class ExampleInstrumentedTest2 {
#Rule
public ActivityTestRule<MainActivity> mActivityTestRule = new ActivityTestRule<>(MainActivity.class);
#Test
public void checkInputs() throws Exception {
}
}
ExampleInstrumentedSuite:
import junit.framework.TestSuite;
import org.junit.runner.RunWith;
import org.junit.runners.AllTests;
#RunWith(AllTests.class)
public class ExampleInstrumentedSuite {
public static TestSuite suite() {
TestSuite suite = new TestSuite();
suite.addTest(new junit.framework.JUnit4TestAdapter(ExampleInstrumentedTest.class));
suite.addTest(new junit.framework.JUnit4TestAdapter(ExampleInstrumentedTest2.class));
return suite;
}
}
Note that you should use #RunWith(JUnit4.class) instead of default #RunWith(AndroidJUnit4.class) in testCase Class

public class MyTestCase extends TestCase {
#Override
public void runTest() {
// define assertion here <===
assertEquals("yes", "yes");
}
}
#RunWith(AllTests.class)
public class DynamicTestSuite {
public static TestSuite suite() {
TestSuite suite = new TestSuite();
// dynamically create your test case here <====
suite.addTest(new MyTestCase());
return suite;
}
}

Expanding on #kissLife's answer, here's a something you can paste and run that creates multiple tests on the fly:
import junit.framework.TestCase;
import junit.framework.TestSuite;
public final class TestJunit4DynamicConstruction {
public static TestSuite suite() {
TestSuite suite = new TestSuite();
suite.addTest(new CompareInts(1, 1));
suite.addTest(new CompareInts(2, 2));
suite.addTest(new CompareInts(2, 1)); // huh, for some reason, 2 != 1
suite.addTest(new CompareInts(1, 1));
return suite;
}
static public class CompareInts extends TestCase {
private final int got;
private final int expected;
CompareInts(int got, int expected) {
super(Integer.toString(got) + ":" + Integer.toString(expected));
this.got = got;
this.expected = expected;
}
#Override
public void runTest() {
assertEquals(got, expected);
}
}
}
You'll run these tests:
TestJunit4DynamicConstruction$CompareInts.1:1
TestJunit4DynamicConstruction$CompareInts.2:2
TestJunit4DynamicConstruction$CompareInts.2:1
TestJunit4DynamicConstruction$CompareInts.1:1
TestJunit4DynamicConstruction$CompareInts
and get this error:
junit.framework.AssertionFailedError:
Expected :2
Actual :1
...
TestJunit4DynamicConstruction$CompareInts.runTest(TestJunit4DynamicConstruction.java:26)
...
Process finished with exit code 255