Hi – I wanted to automatically sweep JUnit tests into suites as part of my continuous builds, so I derived a runner from JUnit's Suite which finds all test classes in a package. The runner works just fine, but the results display is less than expected.
I have one class in my testing support package with a #RunWith annotation for my runner. The runner works by reading a property to get the package under test. Set the property and tell JUnit to run the annotated class, and all tests in that package are executed. The name of the suite is reported as the name of the class which has the #RunWith annotation, in both Ant and IntelliJ. My runner has an override for ParentRunner.getName() which returns the name of the package under test. I verified that the string gets into the runner's Description object. What am I missing?
Environment:
JUnit: 4.5
Ant: 1.7.0
IntelliJ IDEA: 8.1
Thanks for whatever direction you can provide.
This is because ANT and IntelliJ use their own runners, so they are building the name based on the test, and not getting the name from your runner. In other words, the runner is delegated to for the purpose of running the test, but not for the purpose of describing it.
I had a similar problem just a few weeks ago and created an open source project because of it.
You may include it via maven
<dependency>
<groupId>com.github.cschoell</groupId>
<artifactId>junit-dynamicsuite</artifactId>
<version>0.2.0</version>
<scope>test</scope>
</dependency>
or download it from the github page, where you will find its documentation as well.
https://github.com/cschoell/Junit-DynamicSuite
There is a Junit runner included which allows to scan either a directory or the classpath for unit tests and filter them by implementing a simple interface.
Related
I created 2 Test classes (1 with 30 testcases, the other one with 10 testcases) in Apache Camel with Spring and Maven.
One of the Test class works fine, and runs if I build with Maven.
mvn test <- Works with one Test class
mvn package <- Works also with one Test class
But the other test class don't run. Both are in src/test/java
Also both test classes are in target/test-classes/...
One different is in target/test-classes/... the not working class have an second file with ...Test$1.class in the same folder. But I cannot open it.
And one second different have the classes: The not working class extends from CamelTestSupport and is a junit4. The working test-class is from junit5.
I think the maven-surefire-plugin don't find the junit4 test class. But how can I make maven-surefire-plugin finding junit4 test classes?
The JUnit 4 tests have to be executed with the Vintage engine of JUnit and the JUnit 5 tests with the Jupiter engine.
See this answer how to configure the POM to use both engines depending on the version of the surefire plugin.
I am trying to run individual spock unit tests using intellij idea.
Consider:
// rest of code
def "Test Something"() {
// test code below
}
In above test, when I goto the test body and right context menu, I get two kinds of tests for Test Something. One is the grails test and other is the junit test.
Referring to this question, the accepted answer recommends using the jUnit runner. But using it, the code simply does not compile(probably because certain plugins and other classes are not available).
(I am not sure though as this is the desired behavior because I am just running a single test and not all tests. So wonder why is it compiling all classes ,including plugin classes not required by the test target class.)
Using the grails runner, I check the configuration and here is the screenshot:
So nothing looks wrong with the command there.
But the test on running gives Test framework quit unexpectedly error.
I try running same command from grails console(CMD windows) and it runs without any error message.
But on checking the output html files(in target/test-reports) I see that none of the tests actually ran!
So what is going on here and why are not individual tests running?
PS:
When I run All tests using test-app command, tests run as expected. Only individual (unit)tests are not running.
Part of the price paid for Spock's nice test naming, is that you can't specify an individual test to run anymore.
Here are some articles about it. The first seems pretty on-point:
Run a specific test in a single test class with Spock and Maven
This one isn't about running a single test, but has some relevance and talks about Spock's test-name conversions, plus Peter Niederwieser chimes in with comments:
Can TestNG see my Spock (JUnit) test results?
A workaround for this could be the #IgnoreRest annotation. Simply annotate the test you want to run with #IgnoreRest, and then specify that test class to run, and only the annotated test will run. http://spockframework.github.io/spock/javadoc/1.0/spock/lang/IgnoreRest.html
Try using the grails unit test and add the following in the command line part:
-Dgrails.env=development
This will run the test as we change the running environment to development . Hope this will help to everyone facing such problems.
A similar question has already been asked here.
One (unaccepted) answer states:
the test class will always be started directly and then through the
"link" in the suite. This is as expected.
Can someone explain what this actually means and whether or not it is possible to prevent the tests running twice.
When I run the tests from the command line using mvn test they only run once.
UPDATE
I have a test suite defined as follows:
#RunWith(Suite.class)
#SuiteClasses({ TestCase1.class, TestCase2.class })
public class MyTestSuite
{
}
When you run tests in Eclipse on project level (or package level), Eclipse searches all project's source folders for JUnit classes (or selected package). These are all classes with #Test annotations and all classes with #RunWith (probably some more too). Then for all these classes it runs them as tests.
As a result of this behavior, if you have a suite class that references tests classes in the same project, these tests will run twice. If you had another suite that did the same, they would run three times and so on. To understand this behavior try running a suite that contains one test case twice, for instance:
#RunWith(Suite.class)
#SuiteClasses({ TestCase1.class, TestCase1.class })
public class TestSuite {}
Accepted strategy here is to define a suite or suites for a project an run them exclusively. Do not start tests on a project level but run selected suites only.
As far as Maven is concerned, I suspect that its default configuration only picks out suite class and omits test cases. Had it been configured differently, it would behave the same as Eclipse.
Elipse tests 2 classes and give you 2 results.
Maven tests 2 classes and give you one result with 2 sub results.
I think is somethink like this, but still most important thing is that result are
positive! :)
Regards!
Same as this question https://github.com/spring-projects/spring-boot/issues/13750
Just exclude individual test cases and include the suite test cases.
I converted a dynamically generated JUnit test suite to TestNG using a #Factory annotation. The tests are generated by scanning a directory that contains several test specification files (written in a DSL) and by loading them in an test class that knows how to execute them. The test class has a single #Test method named test, implements org.testng.ITest and overrides getTestName() as recommended.
However, in the TestNG Eclipse UI, only the factory and a single execution of test is shown (although it is executed several times, as expected). With JUnit, I had the name of each spec listed as a separate test. This is very important because several test specs could fail and I would need to see all failures (which does not work in the TestNG Eclipse UI for me).
How can I achieve something similar with TestNG? I use the Eclipse TestNG plugin 6.7.0 and testng 6.7.
Try extending XmlTest in your test class and call setName(). I extended XmlSuite and used setName to name my "test suite" which is a class with multiple tests defined in it and that worked for me. In fact I just tried what I suggested on the same class and now it's called a test with the correct name. I'm not sure why the ITest interface is being ignored as I also tried that approach without success.
As carlin.scott suggested, extending XmlTest works for this problem. However, I overrode toString() to return the test name (instead of calling setName). This has the advantage that the test name in the TestNG view is easier to read. By default, it contains additional information such as parameters and metagroups.
I know that it's possible to run a specific test class with -Dtest=MyTest. But is it possible to run a specific test within that class?
I.e. if MyTest defines testFoo() and testBar(), is there a way to specify that only testfoo() should be run?
I'm aware that it's trivially easy to do this in an IDE, but I occasionally need to run tests on the command line on another server.
From Running a Single Test Using Maven Surefire Plugin
With version 2.7.3, you can run only n tests in a single Test Class.
NOTE : it's supported for junit 4.x and TestNG.
You must use the following syntax
mvn -Dtest=TestCircle#mytest test
You can use patterns too
mvn -Dtest=TestCircle#test* test
It will be available as of Surefire 2.8, see SUREFIRE-577
Don't think its available. You can work around it by passing some system properties & ignore execution of tests based on the property value. However it does not seem to add a great value add. There is also TestNG which offers additional features.
http://maven.apache.org/plugins/maven-surefire-plugin/examples/testng.html
To execute one Test at a time, run mvn test
mvn -Dtest=MyUnitlTest test
To execute one Test at a time and a specific method from it:
mvn -Dtest=MyUnitTest#method test
where MyUnitTest is the name of your test and #method is the name of your method.
Execute tests with surefire:
mvn surefire:test