Develop Reference

TestNg is running a test in a class after commenting #Test annotation

I have a class with list of tests annotated with #Test. I commented #Test annotation for one of the test however when I am running the suit the commented test is also running. Any idea on what's happening?
Example:
public class TestCasesClass {
#BeforeSuite
public void testSetup() throws Exception
{
super.testSetup();
}
#Test
public void test1() {
//Some test code
}
#Test
public void test12() {
//Some test code
}
//#Test
public void test3() {
//Some test code
}
}
When running the test suit all the test are running, including test3.
I tried #Test(enable-false) then also test running in the suit.

Is there anything like Spring TestExecutionListener for TestSuite?

Currently for tests I'm using TestExecutionListener and it works just perfect
public class ExecutionManager extends AbstractTestExecutionListener {
#Override
public void beforeTestClass(TestContext testContext) throws Exception {
System.out.println("beforeClass");
}
#Override
public void afterTestClass(TestContext testContext) throws Exception {
System.out.println("afterClass");
}
}
Test classes:
#RunWith(SpringJUnit4ClassRunner.class)
#TestExecutionListeners(ExecutionManager.class)
public final class TC_001 {
#Test
public void test() {
System.out.println("Test_001");
}
}
#RunWith(SpringJUnit4ClassRunner.class)
#TestExecutionListeners(ExecutionManager.class)
public final class TC_002 {
#Test
public void test() {
System.out.println("Test_002");
}
}
When I include those tests in test suite, beforeTestClass(TestContext testContext) and afterTestClass(TestContext testContext) methods are executed for each test class, what is quite logical:
#RunWith(Suite.class)
#Suite.SuiteClasses({
TC_001.class,
TC_002.class
})
public final class TS_001 {
}
Is there anything like SuiteExecutionListener (TestExecutionListener for suites)?
Basically I need non-static #BeforeClass and #AfterClass for suite
OR
In ExecutionListener I need to find out what class has been launched: case or suite. For this purpose I can:
analyze StackTrace and get calling class
use Reflection.getCallerClass(int i) (which is deprecated)
pass caller class to ExecutionManager (By the way, how can I do that? Is it possible to put Object into TestContext like in Android Bundle?)
But I don't really like those solutions. SuiteExecutionListener is much more preferable
Thank you

No, there is unfortunately no such thing as a SuiteExecutionListener in the Spring TestContext Framework (TCF).
The TCF does not integrate with JUnit 4 at the suite level.
If you want to store something in the TestContext, that's not a problem. TestContext implements org.springframework.core.AttributeAccessor, so you can store attributes in the TestContext. Note, however, that the lifecycle of a given TestContext is tied to a test class.

running #Test one by one using reflection

I’m trying to run a package with many unit tests (one by one, not as a class) using reflection,
So when I get all the #Test methods that needs to be run I try to do
Result result = new JUnitCore().run(Request.method(Class
.forName(packageAndClass),getTestName()));
But the class returned in packageAndClass has
#Before, #BeforeClass methods (that also might be in its superclass)
So when running the code above I get all the tests running and fail(because some of their values are assigned in the #Before and #BeforeClass methods)
But when running it from eclipse (select the test method name -> right click -> run as -> Junit test)
They all pass (runing together or one by one)
Is there an api of Request that will run the before methods?

Why are you doing that? JUnit is supposed to run the tests for you!

I ran the following test with junit 4.9 :
public class RunOneTest {
public static void main(final String[] args) {
final Result result = new JUnitCore().run(Request.method(RunOneTest.class, "oneTest"));
System.out.println("result " + result.wasSuccessful());
}
#Test
public void oneTest() throws Exception {
System.out.println("oneTest");
}
#Test
public void anotherTest() throws Exception {
System.out.println("anotherTest");
}
#Before
public void before() {
System.out.println("before");
}
#BeforeClass
public static void beforeClass() {
System.out.println("beforeClass");
}
#After
public void after() {
System.out.println("after");
}
#AfterClass
public static void afterClass() {
System.out.println("afterClass");
}
}
and the output was :
beforeClass
before
oneTest
after
afterClass
result true
Are you really sure that the methods are not run ?

Stopping JUnit suite if particular test fails

I have a JUnit test suite in the form:
#RunWith(Suite.class)
#Suite.SuiteClasses( { xx.class, yy.cass })
public class AllTests {
public static Test suite() {
TestSuite suite = new TestSuite(AllTests.class.getName());
//$JUnit-BEGIN$
//$JUnit-END$
return suite;
}
}
This then calls vanilla tests like this:
public class xxx {
#Test
public void test () throws {
...
I have a situation where I'd like to stop the rest of the test suite running if there's an error or fail in the first test. But errors / fails in the others are ok and the suite should complete as many other tests as it can. Basically the first test failing would indicate it isn't safe to run the rest.
Is this possible?

First you need junit RunListener:
import org.junit.runner.notification.Failure;
import org.junit.runner.notification.RunListener;
import org.junit.runner.notification.RunNotifier;
public class FailureListener extends RunListener {
private RunNotifier runNotifier;
public FailureListener(RunNotifier runNotifier) {
super();
this.runNotifier=runNotifier;
}
#Override
public void testFailure(Failure failure) throws Exception {
super.testFailure(failure);
this.runNotifier.pleaseStop();
}
}
Then prepare a suite:
public class StopOnFailureSuite extends Suite {
public StopOnFailureSuite(Class<?> klass, Class<?>[] suiteClasses) throws InitializationError {
super(klass, suiteClasses);
}
public StopOnFailureSuite(Class<?> klass) throws InitializationError {
super(klass, klass.getAnnotation(SuiteClasses.class).value());
}
#Override
public void run(RunNotifier runNotifier) {
runNotifier.addListener(new FailureListener(runNotifier));
super.run(runNotifier);
}
}
And run your suite:
#RunWith(StopOnFailureSuite.class)
#Suite.SuiteClasses({
FirstTestClass.class,
SecondTestClass.class,
...
})

What's wrong with calling System.exit()?

If it's first test then consider moving its validation to #BeforeClass and throw exception if it fails. Then only #AfterClass method would run in case of this exception.
Of course, that way you lack all the fixture artifacts created in test setup method(s).

Like your answer but using #Before in an integration test, I did something like this:
public class FooTest {
private static boolean bar;
#BeforeClass
public static void setUpBeforeClass() throws Exception {
bar = false;
}
#Before
public void setUp() throws Exception {
assertTrue(bar);
}
#Test
public void test() {
System.out.println("something");
assertTrue(true);
}
#Test
public void test1() {
System.out.println("Something2");
assertTrue(true);
}
}
Regards!

Based on the answer from Hiro2k (thanks!) I've used the following solution. It's a bit of a hack but it works.
The test which can prevent other tests running goes at the top of the #Suite.SuiteClasses list. That test then has the following:
private static boolean shouldStopRestOfSuite = false;
#Test
public void test () throws Throwable {
try {
... run some test code...
}
catch (Throwable e) {
shouldStopRestOfSuite = true;
throw e;
}
}
Note the above does need to catch Throwable (not exception) so it catches assertion errors. It also re-throws the error so it's logged by JUnit for analysis.
Then there's another test method:
#Test
public void testKillIfNeeded () throws Exception {
if (!shouldStopRestOfSuite) {
return;
}
System.out.println ("Test suite killed due to dangerous error / failure");
System.exit(1);
}
The above is run second and will kill the JUnit process.
Using this method the JUnit test won't end on fail / error if there's an issue but the fail / error is logged for analysis by JUnit and no further tests will run.
Not too pretty but it does the job :)

Firstly you should catch an error and check the same before you run the 2nd test.
#Rule
public ErrorCollector collector = new ErrorCollector();
1. Add Error.
collector.addError(new Throwable("first thing went wrong"));
2. Check before the dependent run.
collector.checkThat(getResult(), not(containsString("ERROR!")));
Reference - ErrorCollector

Are you running tests using ant?
You could write a custom test listener. You can set this in ant http://ant.apache.org/manual/Tasks/junit.html ( enableTestListenerEvents).

I find it troubling that this functionality is so tedious to implement in such a mature library. If you're using JUnit 5 / Jupiter you can use an extension called JUnit Pioneer (https://junit-pioneer.org).
With JUnit Pioneer you can simply add a #DisableIfTestFails annotation to your test class to make all tests stop when one fails.

Run External Command Before JUnit Tests in Eclipse

Is it possible to run an external command before running tests in a given JUnit file? I run my tests using the Eclipse's Run command. Using JUnit 4.
Thanks.

Very vague question. Specifically, you didn't mention how you are running your JUnit tests. Also you mentioned 'file', and a file can contain several JUnit tests. Do you want to run the external command before each of those tests, or before any of them are executed?
But more on topic:
If you are using JUnit 4 or greater then you can tag a method with the #Before annotation and the method will be executed before each of your tagged #Test methods. Alternatively, tagging a static void method with #BeforeClass will cause it to be run before any of the #Test methods in the class are run.
public class MyTestClass {
#BeforeClass
public static void calledBeforeAnyTestIsRun() {
// Do something
}
#Before
public void calledBeforeEachTest() {
// Do something
}
#Test
public void testAccountCRUD() throws Exception {
}
}
If you are using a version of JUnit earlier than 4, then you can override the setUp() and setUpBeforeClass() methods as replacements for #Before and #BeforeClass.
public class MyTestClass extends TestCase {
public static void setUpBeforeClass() {
// Do something
}
public void setUp() {
// Do something
}
public void testAccountCRUD() throws Exception {
}
}

Assuming you are using JUnit 4.0, you could do the following:
#Test
public void shouldDoStuff(){
Process p = Runtime.getRuntime().exec("application agrument");
// Run the rest of the unit test...
}
If you want to run the external command for every unit test, then you should do it in the #Before setup method.