What can cause JUnit to disregard #Ignore annotations? - java

I just used MyEclipse to automatically generate some JUnit test cases. One of the generated methods looks like this:
#Ignore("Ignored") #Test
public void testCreateRevision()
{
fail("Not yet implemented"); // TODO
}
I added the #Ignore annotation manually. However, when I run the test, JUnit lists that method, and others like it, under "failures," rather than ignoring them (related: What's the difference between failure and error in JUnit?). And it displays the "Not yet implemented" message instead of the "Ignored" message. Clearly, fail() must be getting called, and therefore, the #Ignore assertion is not working.
What's going on here? Is there a setting I need to enable for this to work?
EDIT :
Things I have considered/tried so far:
I am using JUnit 4, so it's not a version problem.
I am importing org.junit.Ignore, so it's not a case of the wrong Ignore being used.
I have tried using #Ignore alone, #Ignore #Test and #Ignore("message") #Test; all fail.
EDIT 2 :
I created the tests with MyEclipse, via New > Other; Java > JUnit > JUnit Test Case; New JUnit 4 test, and the library in my build path is JUnit 4. I'm building with ant and actually running the case with MyEclipse.


Make sure you are importing the right #Ignore. To be sure use #org.junit.Ignore explicitly.
Double check if your test is being executed by JUnit 4, not 3. The easiest way to do this is to either change the test name so it is not prefixed by test (now it shouldn't be executed at all and JUnit 4 does not need this prefix anyway) or examine your test case inheritance hierarchy: it shouldn't extend directly or indirectly from junit.framework.TestCase (Junit 3 requirement).


I had this problem also even though JUnit 3 was not on my classpath. I believe that compatibility mode on Junit 4 picks up on the 'test' prefix in your testname and thus operates as JUnit 3 would rather than picking up the #Ignore. The solution is to rename your test.


In JUnit5 use #Disabled or #Disabled("Reason text")


Are you sure the test classes were recompiled?
It's a quite common problem, that the recompilation fails because there was typo somewhere in the sources (like a missing semicolon), and the IDE does not tell you that compiling failed.
Try deleting the target/test-classes folder.


In my case I found my IDE was executing the test disregarding the #Ignore annotation. When I ran mvn install (or any other maven phase) the test was skipped and that's what I was actually aiming for (see attached ilustration).
Ilustration of ways of execution


I think its just #Ignore that will skip the test
JUnit Ignore


Ensuring the test class is also imported fixed the issue for me i.e.
import org.junit.Ignore;
import org.junit.Test;

Related

Run an initializer in the very beginning of `gradle test` for JUnit 4

In our project, we have a lot of JUnit 4 tests, and they are executed from Gradle.
Now, we wanted to insert an initializer for the entire "gradle test" execution. The initializer is a class-loading tweak, so it should run just once in the very beginning.
I have looked into some:
JUnit's RunListener
RunListener does not run on Gradle.
https://github.com/gradle/gradle/issues/1330
Similar with junit-foundation
I was trying this, but it didn't work for unsure reasons yet.
It might be because of its complicated classloading structure.
Still worth going deeper into it.
JUnit's RunWith
We need to specify all the test class names.
We already have a lot of test classes.
And, when we add a new test class, we have to add the test class there, too. We easily forget.
JUnit 4: Set up things in a test suite before tests are run (like a test's #BeforeClass method, just for a test suite)
We wanted all the test classed to be looked up automatically (like normal gradle test).
How do I Dynamically create a Test Suite in JUnit 4?
ClasspathSuite may help to auto-lookup test classes?
https://github.com/takari/takari-cpsuite
But, gradle tests --tests="..." would not work as intended.
Static initializer in test classes, to run just once through the entire test run?
Need to add so many static {} blocks in so many existing tests?
Easily to forget, too.
Does anyone have any more ideas to solve this? (If we have a way in JUnit 5, we'll consider that.)

java.lang.Exception: No tests found matching Method using Intellij IDEA

I am experiencing a strange behavior of Intellij IDEA 2016.3. Having a class with method foo and a JUnit test for the method when I get java.lang.Exception: No tests found matching Method foo when running the test. After I do mvn test it succeeds and then running the unit test right after executing mvn command it suddenly runs green. Seems like IDEA does not compile automatically. How can I fix this?
P.S. No settings were altered after upgrading to v. 2016.3

If you're using a theory testing framework like Junit's or Robolectric's, make sure to run the class containing the test you want, instead the test itself. Since these frameworks use the test methods as instance methods instead of static methods, any testing framework looking for a normal public static test won't find anything.

The same issue i got with Gradle (4.5+) + new Build Cache feature
Sometimes it's unable to find new test methods and throws exception (like you mentioned in topic)
Solution: clean .gradle, build and out directories and try again ;)

Well, after "playing" a bit with run configurations of each unit test I noticed that each Run Config has a Build goal preset in the Before Launch option (See pic below):
After changing Build to Build Project the tests run fine.

If you originally run a test named "foo", and then rename it to "fooBar", you must subsequently run "fooBar" with a new Run Configuration.
If you use the same original Run Configuration for "foo" to run "fooBar", it still looks for a test named "foo" which it does not find (thus the Exception) because it was renamed to "fooBar". The new Run Configuration would correctly look for "fooBar" test.
I made this mistake unknowingly because I renamed a test, but then just clicked the green run button in IntelliJ: Doing that runs the last Run Configuration, which in this scenario has the old "foo" name.

Deleting Intellij's out directory fixed this issue for me.

In addition to the other answers here: the error can also happen when you forget #Test before your test method declaration. IntelliJ (2018.1) will still show you the green "Play-Button" for test execution, but that public method in your Test-Class will not be an actual test.

Make sure that your #test methods as well as the test class are public.

Since you got your answer and for others searching for solution,
Look if your test class is extending TestCase abstract class which is a JUnit 3 related. In order to fix this you have to start your method name with "test".
For example public void testFoo().
If JUnit 3 is not the case, you're missing #Test annotation from your JUnit 4 test method.
Note: If you're extending from TestCase and test methods are annotated with #Test and also your methods' names start with "test", then probably you're mixing JUnit 3 with JUnit 4. Don't do that. It will lead to other errors such as methods that annotated with #Ignore will not be ignored if those methods' names start with "test".

This situation can also occur if you do not place #Test annotation above the test method.

Maybe you just give a wrong name for test method.
I met this problem because I used '—' instead of '_' which intelliJ cannot represent.

I had to add test before the test Method Name.
methodtest() does not work but testMethod() worked

Make sure #org.junit.Test is added on top of your method. I forgot them and that fixed it for me!

Make sure you've correct runner mentioned above your class.
I was getting this weird message when I was using runner CucumberWithSerenity.class. When I changed to SerenityRunner.class it got fixed.
#RunWith(SerenityRunner.class)
//#RunWith(CucumberWithSerenity.class)
public class WordPressAppTest {
I'm using Serenity framework for web automation and use below runner class
import net.serenitybdd.cucumber.CucumberWithSerenity;
import net.serenitybdd.junit.runners.SerenityRunner;
import org.junit.runner.RunWith;
I feel IDEA ( 2017.2.6 ) can show some better error message than this

You may check that Run/Debug Configurations in the top of the IntelliJ screen.
Delete all of then with the "minus button" and hit "run" green button again to run the test.
You may reload the project on the maven tab on the right as well.

In my spring mvc project. I solved this problem by adding
#RunWith(SpringJUnit4ClassRunner.class)
to my test class.

In my case, I copied a test from another class and modified it, but while running the test it was still pointing to the previous one.
Build > Clean Project solved the problem

Spock unit tests in grails for individual test does not work in intellij

I am trying to run individual spock unit tests using intellij idea.
Consider:
// rest of code
def "Test Something"() {
// test code below
}
In above test, when I goto the test body and right context menu, I get two kinds of tests for Test Something. One is the grails test and other is the junit test.
Referring to this question, the accepted answer recommends using the jUnit runner. But using it, the code simply does not compile(probably because certain plugins and other classes are not available).
(I am not sure though as this is the desired behavior because I am just running a single test and not all tests. So wonder why is it compiling all classes ,including plugin classes not required by the test target class.)
Using the grails runner, I check the configuration and here is the screenshot:
So nothing looks wrong with the command there.
But the test on running gives Test framework quit unexpectedly error.
I try running same command from grails console(CMD windows) and it runs without any error message.
But on checking the output html files(in target/test-reports) I see that none of the tests actually ran!
So what is going on here and why are not individual tests running?
PS:
When I run All tests using test-app command, tests run as expected. Only individual (unit)tests are not running.

Part of the price paid for Spock's nice test naming, is that you can't specify an individual test to run anymore.
Here are some articles about it. The first seems pretty on-point:
Run a specific test in a single test class with Spock and Maven
This one isn't about running a single test, but has some relevance and talks about Spock's test-name conversions, plus Peter Niederwieser chimes in with comments:
Can TestNG see my Spock (JUnit) test results?
A workaround for this could be the #IgnoreRest annotation. Simply annotate the test you want to run with #IgnoreRest, and then specify that test class to run, and only the annotated test will run. http://spockframework.github.io/spock/javadoc/1.0/spock/lang/IgnoreRest.html

Try using the grails unit test and add the following in the command line part:
-Dgrails.env=development
This will run the test as we change the running environment to development . Hope this will help to everyone facing such problems.

Why are JUnit tests run twice from within Eclipse?

A similar question has already been asked here.
One (unaccepted) answer states:
the test class will always be started directly and then through the
"link" in the suite. This is as expected.
Can someone explain what this actually means and whether or not it is possible to prevent the tests running twice.
When I run the tests from the command line using mvn test they only run once.
UPDATE
I have a test suite defined as follows:
#RunWith(Suite.class)
#SuiteClasses({ TestCase1.class, TestCase2.class })
public class MyTestSuite
{
}

When you run tests in Eclipse on project level (or package level), Eclipse searches all project's source folders for JUnit classes (or selected package). These are all classes with #Test annotations and all classes with #RunWith (probably some more too). Then for all these classes it runs them as tests.
As a result of this behavior, if you have a suite class that references tests classes in the same project, these tests will run twice. If you had another suite that did the same, they would run three times and so on. To understand this behavior try running a suite that contains one test case twice, for instance:
#RunWith(Suite.class)
#SuiteClasses({ TestCase1.class, TestCase1.class })
public class TestSuite {}
Accepted strategy here is to define a suite or suites for a project an run them exclusively. Do not start tests on a project level but run selected suites only.
As far as Maven is concerned, I suspect that its default configuration only picks out suite class and omits test cases. Had it been configured differently, it would behave the same as Eclipse.

Elipse tests 2 classes and give you 2 results.
Maven tests 2 classes and give you one result with 2 sub results.
I think is somethink like this, but still most important thing is that result are
positive! :)
Regards!

Same as this question https://github.com/spring-projects/spring-boot/issues/13750
Just exclude individual test cases and include the suite test cases.

How to make ignored junit steps in cucumber-jvm cause test scenario to fail

I am using cucumber-jvm with appium using eclipse and junit.
Some of my tests stop working halfway through. They do not overtly fail the junit tests but instead stop working and ignore the remaining steps.
When I look at these steps in junit (through eclipse) they appear to have passed, until I drill into them and see steps have been ignored.
Is there a way to mark any test scenarios with ignored steps as failures rather than as passes?

I presume you have a JUnit test case with an #CucumberOptions annotation on it. If you have this, you should be able to make ignored tests fail the build by setting strict=true. e.g.
#RunWith(Cucumber.class)
#CucumberOptions(strict = true)
public class CucumberRunnerTest {
}

Categories

Resources