Test suites in Junit4 run nicely, but there's a snag here:
#RunWith(Suite.class)
#Suite.SuiteClasses({ A.class, B.class, ...})
If somebody develops a unit test and forgets to include it in Suite.SuiteClasses, that's obviously a problem.
(that's not a burning problem since Ant will catch that later but still)
So I wondered: if you have say "test" folder in Eclipse project and there are some packages with classes in it - is there a way to include them all automatically in junit4 test suite somehow?
(yes you can right-click "test" folder and do Run as Junit but that sometimes fails individual tests for some reason while they individually pass so I do not have much trust in this solution, plus test suites are nice toys to play with ;-)).
I suggest ClasspathSuite
import org.junit.extensions.cpsuite.ClasspathSuite;
import org.junit.runner.RunWith;
#RunWith(ClasspathSuite.class)
public class MySuite {}
Not an answer but more then comment:
[...] Run as Junit but that sometimes fails individual tests for some reason while they individually pass [...]
The reason is that some tests don't clean up correctly. This should always put you on alert. Try to identify a pair of tests that can't be executed in one "run" and have a close look at the first test. From my own experience: fix those issues as soon as possible (aka: NOW!), otherwise you may run in very deep problems later (typically: complaints from QA guys, like "the tests fail on my environment")
i don't agree with Andreas_D. it's not because tests don't clean after themselves. good unit tests don't have to clean after themselves. it's because your some of your tests depends on result of another. you need better tests and/or fixtures
however i agree with part 'fix them now!'. you have a serious problem when results of your tests are not reproducible
Related
In our project, we have a lot of JUnit 4 tests, and they are executed from Gradle.
Now, we wanted to insert an initializer for the entire "gradle test" execution. The initializer is a class-loading tweak, so it should run just once in the very beginning.
I have looked into some:
JUnit's RunListener
RunListener does not run on Gradle.
https://github.com/gradle/gradle/issues/1330
Similar with junit-foundation
I was trying this, but it didn't work for unsure reasons yet.
It might be because of its complicated classloading structure.
Still worth going deeper into it.
JUnit's RunWith
We need to specify all the test class names.
We already have a lot of test classes.
And, when we add a new test class, we have to add the test class there, too. We easily forget.
JUnit 4: Set up things in a test suite before tests are run (like a test's #BeforeClass method, just for a test suite)
We wanted all the test classed to be looked up automatically (like normal gradle test).
How do I Dynamically create a Test Suite in JUnit 4?
ClasspathSuite may help to auto-lookup test classes?
https://github.com/takari/takari-cpsuite
But, gradle tests --tests="..." would not work as intended.
Static initializer in test classes, to run just once through the entire test run?
Need to add so many static {} blocks in so many existing tests?
Easily to forget, too.
Does anyone have any more ideas to solve this? (If we have a way in JUnit 5, we'll consider that.)
I am trying to run individual spock unit tests using intellij idea.
Consider:
// rest of code
def "Test Something"() {
// test code below
}
In above test, when I goto the test body and right context menu, I get two kinds of tests for Test Something. One is the grails test and other is the junit test.
Referring to this question, the accepted answer recommends using the jUnit runner. But using it, the code simply does not compile(probably because certain plugins and other classes are not available).
(I am not sure though as this is the desired behavior because I am just running a single test and not all tests. So wonder why is it compiling all classes ,including plugin classes not required by the test target class.)
Using the grails runner, I check the configuration and here is the screenshot:
So nothing looks wrong with the command there.
But the test on running gives Test framework quit unexpectedly error.
I try running same command from grails console(CMD windows) and it runs without any error message.
But on checking the output html files(in target/test-reports) I see that none of the tests actually ran!
So what is going on here and why are not individual tests running?
PS:
When I run All tests using test-app command, tests run as expected. Only individual (unit)tests are not running.
Part of the price paid for Spock's nice test naming, is that you can't specify an individual test to run anymore.
Here are some articles about it. The first seems pretty on-point:
Run a specific test in a single test class with Spock and Maven
This one isn't about running a single test, but has some relevance and talks about Spock's test-name conversions, plus Peter Niederwieser chimes in with comments:
Can TestNG see my Spock (JUnit) test results?
A workaround for this could be the #IgnoreRest annotation. Simply annotate the test you want to run with #IgnoreRest, and then specify that test class to run, and only the annotated test will run. http://spockframework.github.io/spock/javadoc/1.0/spock/lang/IgnoreRest.html
Try using the grails unit test and add the following in the command line part:
-Dgrails.env=development
This will run the test as we change the running environment to development . Hope this will help to everyone facing such problems.
A similar question has already been asked here.
One (unaccepted) answer states:
the test class will always be started directly and then through the
"link" in the suite. This is as expected.
Can someone explain what this actually means and whether or not it is possible to prevent the tests running twice.
When I run the tests from the command line using mvn test they only run once.
UPDATE
I have a test suite defined as follows:
#RunWith(Suite.class)
#SuiteClasses({ TestCase1.class, TestCase2.class })
public class MyTestSuite
{
}
When you run tests in Eclipse on project level (or package level), Eclipse searches all project's source folders for JUnit classes (or selected package). These are all classes with #Test annotations and all classes with #RunWith (probably some more too). Then for all these classes it runs them as tests.
As a result of this behavior, if you have a suite class that references tests classes in the same project, these tests will run twice. If you had another suite that did the same, they would run three times and so on. To understand this behavior try running a suite that contains one test case twice, for instance:
#RunWith(Suite.class)
#SuiteClasses({ TestCase1.class, TestCase1.class })
public class TestSuite {}
Accepted strategy here is to define a suite or suites for a project an run them exclusively. Do not start tests on a project level but run selected suites only.
As far as Maven is concerned, I suspect that its default configuration only picks out suite class and omits test cases. Had it been configured differently, it would behave the same as Eclipse.
Elipse tests 2 classes and give you 2 results.
Maven tests 2 classes and give you one result with 2 sub results.
I think is somethink like this, but still most important thing is that result are
positive! :)
Regards!
Same as this question https://github.com/spring-projects/spring-boot/issues/13750
Just exclude individual test cases and include the suite test cases.
It's been 3 hours now and I still didn't find a solution, even though I seem to have read all related questions already.
I am building an Android application and I just want to create a couple of simple Unit Tests that test my basic functions. I don't need to test any Android related logic or activity features.
So I have created a new directory in my solution in which I have created a new JUnit Test Case.
To keep things simple my test methods are not testing much yet, but even when doing a Right Click > Run As > JUnit Test, it's not doing anything.
As you can see in my screenshot the JUnit pane on the left shows my test is terminated but does not show any test that has been executed.
I have created a simple Unit Test in a new Java Project and then it's working. If I repeat the same steps in a new Android Application Project it's not working.
What do I need to do to run my simple Unit Tests?!
Thanks!
(My Compiler Compliance Level is 1.6)
Go to Window -> Show View -> Error Log to see what the actual error is.
For my case it was No test found with test runner 'Junit 5'.
Then one can google for respective solution.
You either don't have JUnit on the build path or you don't have the library (jar) at hand. Make sure both are in place.
I think you will need an unit test suite if all else fails:
#RunWith(Suite.class)
#SuiteClasses({ GpsLocationTest.class })
public class AllTests {
}
There is one more thing you can do: check whether you import the right #Test annotation. Restart Eclipse and clean your project if the problem persists.
You may want to refer to the vogella guide about unit testing.
You can use AndroidTestCase, which inherits from junit.framework.TestCase, not org.junit.Test.
This is related to Memory Issue.
Simply add these line in VM argument:
right click on Junit Test -> Run as -> Run Configuration -> Arguments -> add "-XX:MaxPermSize=512m" under VM argument
The java Build path of the project has Junit4.jar lets say and the Run configuration for the test you are running has Junit5 - then it causes to terminate and nothing happens.
I just used MyEclipse to automatically generate some JUnit test cases. One of the generated methods looks like this:
#Ignore("Ignored") #Test
public void testCreateRevision()
{
fail("Not yet implemented"); // TODO
}
I added the #Ignore annotation manually. However, when I run the test, JUnit lists that method, and others like it, under "failures," rather than ignoring them (related: What's the difference between failure and error in JUnit?). And it displays the "Not yet implemented" message instead of the "Ignored" message. Clearly, fail() must be getting called, and therefore, the #Ignore assertion is not working.
What's going on here? Is there a setting I need to enable for this to work?
EDIT :
Things I have considered/tried so far:
I am using JUnit 4, so it's not a version problem.
I am importing org.junit.Ignore, so it's not a case of the wrong Ignore being used.
I have tried using #Ignore alone, #Ignore #Test and #Ignore("message") #Test; all fail.
EDIT 2 :
I created the tests with MyEclipse, via New > Other; Java > JUnit > JUnit Test Case; New JUnit 4 test, and the library in my build path is JUnit 4. I'm building with ant and actually running the case with MyEclipse.
Make sure you are importing the right #Ignore. To be sure use #org.junit.Ignore explicitly.
Double check if your test is being executed by JUnit 4, not 3. The easiest way to do this is to either change the test name so it is not prefixed by test (now it shouldn't be executed at all and JUnit 4 does not need this prefix anyway) or examine your test case inheritance hierarchy: it shouldn't extend directly or indirectly from junit.framework.TestCase (Junit 3 requirement).
I had this problem also even though JUnit 3 was not on my classpath. I believe that compatibility mode on Junit 4 picks up on the 'test' prefix in your testname and thus operates as JUnit 3 would rather than picking up the #Ignore. The solution is to rename your test.
In JUnit5 use #Disabled or #Disabled("Reason text")
Are you sure the test classes were recompiled?
It's a quite common problem, that the recompilation fails because there was typo somewhere in the sources (like a missing semicolon), and the IDE does not tell you that compiling failed.
Try deleting the target/test-classes folder.
In my case I found my IDE was executing the test disregarding the #Ignore annotation. When I ran mvn install (or any other maven phase) the test was skipped and that's what I was actually aiming for (see attached ilustration).
Ilustration of ways of execution
I think its just #Ignore that will skip the test
JUnit Ignore
Ensuring the test class is also imported fixed the issue for me i.e.
import org.junit.Ignore;
import org.junit.Test;