I have a class with list of tests annotated with #Test. I commented #Test annotation for one of the test however when I am running the suit the commented test is also running. Any idea on what's happening?
Example:
public class TestCasesClass {
#BeforeSuite
public void testSetup() throws Exception
{
super.testSetup();
}
#Test
public void test1() {
//Some test code
}
#Test
public void test12() {
//Some test code
}
//#Test
public void test3() {
//Some test code
}
}
When running the test suit all the test are running, including test3.
I tried #Test(enable-false) then also test running in the suit.
i have encoutered a problem where cobertura report does not cover all test cases, and i could not find a satisfactory answer on internet.
i have a main class:
public class SomeClass {
public Somereturn method1(){
//Something doing here
}
public Somereturn method2(){
//Something doing here
}
public Somereturn method3(){
//Something doing here
}
public Somereturn method4(){
//Something doing here
}
}
when I am preparing test case for first or second method cobertura will identify its coverage :
public testclass {
#Test
Public void test1mehtod1(){
// testing mothod1 for some scenario
}
#Test
Public void test2mehtod1(){
// testing mothod1 again for some other scenario
}
#Test
Public void test1mehtod2(){
// testing mothod2 some other scenario
}
}
this is identified by cobertura lets say 50 % coverage.
but if I choose last or second last method:
public testclass {
#Test
Public void test1mehtod3(){
// testing mothod3 for some scenario
}
#Test
Public void test1mehtod4(){
// testing mothod4 for some other scenario
}
}
It refused to identify coverage and shows only 0 % Coverage.
if I talk about reality, for one controller class I have covered 100% preparing all test cases. but cobertura only shows 22% coverage .
I am working on a sprintboot application.
We use a custom Guice scope, #TestScoped, for some of our JUnit tests that lasts for a single test method, and a JUnit #Rule to enter and exit the scope appropriately. It looks like this:
public class MyJUnitTest {
#Rule public CustomRule customRule = new CustomRule(MyModule.class);
#Inject private Thing thing;
#Test
public void test1() {
// Use "thing"
}
#Test
public void test2() {
// Assuming "Thing" is #TestScoped, we'll have a new instance
}
}
We're starting to use TestNG for some of our tests in other projects, and we'd like to have a similar pattern. So far we've come up with this:
#Listeners(CustomTestNGListener.class)
#Guice(modules = MyModule.class)
public class MyTestNGTest {
#Inject private Provider<Thing> thingProvider;
#Test
public void test1() {
Thing thing = thingProvider.get();
// Use "thing"
}
#Test
public void test2() {
Thing thing = thingProvider.get();
// Assuming "Thing" is #TestScoped, we'll have a new instance
}
}
public class CustomTestNGListener implements IHookable {
#Override
public void run(IHookCallBack callBack, ITestResult testResult) {
TestScope.INSTANCE.enter();
try {
callBack.runTestMethod(testResult);
} finally {
TestScope.INSTANCE.exit();
}
}
}
There are a couple issues with this design:
Unlike JUnit, TestNG uses the same instance of the test class for each method. That means we have to inject Provider<Thing> instead of just Thing, which is awkward.
For some reason, CustomTestNGListener is running on all of our tests, even ones that don't have that #Listeners(CustomTestNGListener.class) annotation. I've worked around this by explicitly checking for that annotation in the listener itself, but it feels like a hack (though I do see that MockitoTestNGListener does the same thing).
Does someone with more familiarity with TestNG have any suggestions for dealing with these issues?
Instead of
public class MyTestNGTest {
#Inject private Provider<Thing> thingProvider;
#Test
public void test1() {
Thing thing = thingProvider.get();
In TestNG you can used
public class MyTestNGTest {
#Inject
private Thing thingInjected;
private Thing thing;
#BeforeTest
public void doBeforeTest() {
thing = thingInjected.clone();
}
Or just call thingProvider.get() in doBeforeTest(), it's better in you have a lot of # Test
public class MyTestNGTest {
#Inject private Provider<Thing> thingProvider;
private Thing thing;
#BeforeTest
public void doBeforeTest() {
thing = thingProvider.get();
}
i got a question if it is possible to create a Runner for this:
#RunWith(MyRunner.class)
public class Class1Test {
#Test public void test2() {
doSomething();
assertTrue(something);
doSomethingElse();
assertTrue(somethingElse);
}
}
The Runner should make a log after every call / assertion in the test.
public class MyRunner extends BlockJUnit4ClassRunner {
public MyRunner(Class<?> klass) throws InitializationError {
super(klass);
}
/* Here should be an method that just executed the calls/assertions but log everything */
}
That's not possible because the test methods aka Statements are the smallest units visible to JUnit.
You can log the calls to JUnit's Assertclass by wrapping it and doing the logging in your wrapper.
I have a JUnit test suite in the form:
#RunWith(Suite.class)
#Suite.SuiteClasses( { xx.class, yy.cass })
public class AllTests {
public static Test suite() {
TestSuite suite = new TestSuite(AllTests.class.getName());
//$JUnit-BEGIN$
//$JUnit-END$
return suite;
}
}
This then calls vanilla tests like this:
public class xxx {
#Test
public void test () throws {
...
I have a situation where I'd like to stop the rest of the test suite running if there's an error or fail in the first test. But errors / fails in the others are ok and the suite should complete as many other tests as it can. Basically the first test failing would indicate it isn't safe to run the rest.
Is this possible?
First you need junit RunListener:
import org.junit.runner.notification.Failure;
import org.junit.runner.notification.RunListener;
import org.junit.runner.notification.RunNotifier;
public class FailureListener extends RunListener {
private RunNotifier runNotifier;
public FailureListener(RunNotifier runNotifier) {
super();
this.runNotifier=runNotifier;
}
#Override
public void testFailure(Failure failure) throws Exception {
super.testFailure(failure);
this.runNotifier.pleaseStop();
}
}
Then prepare a suite:
public class StopOnFailureSuite extends Suite {
public StopOnFailureSuite(Class<?> klass, Class<?>[] suiteClasses) throws InitializationError {
super(klass, suiteClasses);
}
public StopOnFailureSuite(Class<?> klass) throws InitializationError {
super(klass, klass.getAnnotation(SuiteClasses.class).value());
}
#Override
public void run(RunNotifier runNotifier) {
runNotifier.addListener(new FailureListener(runNotifier));
super.run(runNotifier);
}
}
And run your suite:
#RunWith(StopOnFailureSuite.class)
#Suite.SuiteClasses({
FirstTestClass.class,
SecondTestClass.class,
...
})
What's wrong with calling System.exit()?
If it's first test then consider moving its validation to #BeforeClass and throw exception if it fails. Then only #AfterClass method would run in case of this exception.
Of course, that way you lack all the fixture artifacts created in test setup method(s).
Like your answer but using #Before in an integration test, I did something like this:
public class FooTest {
private static boolean bar;
#BeforeClass
public static void setUpBeforeClass() throws Exception {
bar = false;
}
#Before
public void setUp() throws Exception {
assertTrue(bar);
}
#Test
public void test() {
System.out.println("something");
assertTrue(true);
}
#Test
public void test1() {
System.out.println("Something2");
assertTrue(true);
}
}
Regards!
Based on the answer from Hiro2k (thanks!) I've used the following solution. It's a bit of a hack but it works.
The test which can prevent other tests running goes at the top of the #Suite.SuiteClasses list. That test then has the following:
private static boolean shouldStopRestOfSuite = false;
#Test
public void test () throws Throwable {
try {
... run some test code...
}
catch (Throwable e) {
shouldStopRestOfSuite = true;
throw e;
}
}
Note the above does need to catch Throwable (not exception) so it catches assertion errors. It also re-throws the error so it's logged by JUnit for analysis.
Then there's another test method:
#Test
public void testKillIfNeeded () throws Exception {
if (!shouldStopRestOfSuite) {
return;
}
System.out.println ("Test suite killed due to dangerous error / failure");
System.exit(1);
}
The above is run second and will kill the JUnit process.
Using this method the JUnit test won't end on fail / error if there's an issue but the fail / error is logged for analysis by JUnit and no further tests will run.
Not too pretty but it does the job :)
Firstly you should catch an error and check the same before you run the 2nd test.
#Rule
public ErrorCollector collector = new ErrorCollector();
1. Add Error.
collector.addError(new Throwable("first thing went wrong"));
2. Check before the dependent run.
collector.checkThat(getResult(), not(containsString("ERROR!")));
Reference - ErrorCollector
Are you running tests using ant?
You could write a custom test listener. You can set this in ant http://ant.apache.org/manual/Tasks/junit.html ( enableTestListenerEvents).
I find it troubling that this functionality is so tedious to implement in such a mature library. If you're using JUnit 5 / Jupiter you can use an extension called JUnit Pioneer (https://junit-pioneer.org).
With JUnit Pioneer you can simply add a #DisableIfTestFails annotation to your test class to make all tests stop when one fails.