I am new to JUnit. Following is my class hierarchy where classes at top and bottom of the hierarchy has #Test methods. When I execute whole bunch, ConverterBaseEnquiryTest which is at middle of the hierarchy is also getting executed. (That's my guess because I see exception in surefire reports with this name).
And I am getting initialization errors from ConverterBaseEnquiryTest because it wasn't supposed to be executed. Is there any way to figure out why this is happening?
Following is my hierarchy:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {
"classpath:spring/test-default-context.xml"
})
#TransactionConfiguration( transactionManager="profile.transactionManager", defaultRollback=false )
#Ignore
public class BaseCommonTest extends AbstractTransactionalJUnit4SpringContextTests {
abstract public class ConverterBaseTest<F,T> extends BaseCommonTest {
// ...
}
public class ConverterBaseEnquiryTest<F, T> extends ConverterBaseTest<Enquiry, T> {
// ...
}
public class EnquiryToDvsMedicareRequestConverterTest extends ConverterBaseEnquiryTest<Enquiry, DvsMedicareRequest> {
// ...
}
I think this is because #Ignore annotation is not inheritable, so JUnit will execute tests in subclasses. (Although posting the stacktrace would help us).
This means you will also need to place #Ignore on all subclasses.
Alternatively, to save you from that, you could perhaps use #IfProfileValue (which is inherited) on the base-class to disable the tests. This is a Spring annotation supported by the SpringJUnit4ClassRunner.
Related
I am aware that I can make a TestSuite enumerating all the classes that I want, for example:
#RunWith(Suite.class)
#SuiteClasses({SQLServerTests1.class, SQLServerTest2.class, ... })
public class AllSQLServerTests {}
However I have almost 100+ classes and I don't want to have to remember to include any new one in the #SuiteClasses annotation.
As my classes have a naming convention (starting with "SQLServer" for example) I am searching for a way to do something like this:
#RunWith(Suite.class)
#SuiteClasses(prefix="SQLServer")
public class AllSQLServerTests {}
is it possible with plain JUnit? with spring or any other framework?
Tag'em
You can add many tags to each test or test class:
#Test
#Tag("red")
#Tag("production")
public void testWithColour() {...}
#RunWith(JUnitPlatform.class)
#IncludeTags("red & !production")
public class JUnit5Example {
//...
}
You can also use #ExcludeTags but it cannot co-exist with #IncludeTags
Run all in test package
#RunWith(JUnitPlatform.class)
#SelectPackages("com.acme.megaproduct.slowtests")
public class JUnit5Example {
//...
}
Write custom Test Runner
Perhaps none of the above can acommodate your needs, in which case you can add custom filtering by writing your own runner.
See here for step by step how to do it.
Then you just use it like:
#RunWith(MyCustomRunner.class)
public class CustomTestSuite {
//...
}
I need to know the suite class that included the currently running JUnit test. For example, if you have
#SuiteClasses({SubSuite.class})
class ParentSuite { }
#SuiteClasses({TestCase.class})
class SubSuite { }
class TestCase {
#Test
public void testMethod() { }
}
and execute ParentSuite with JUnit, then I want to get a reference to ParentSuite.class. Is this possible?
There is the TestWatcher rule that gets you an instance of Description, which is kind of in the right direction, but that does not include the suite class.
I know this is probably not the best way to write unit tests. My original problem is to run a validation on all classes that are in the project under test, but not those of the dependencies. The TestCase will be in a dependency and included by test suites in other projects. The only solution I could think of is filter those classes with the same source location as that of the top-most-suite. To make that more clear:
BaseLibrary
* contains TestCase and TestSuite
* has classes that should not be validated
ConsumerProject
* has a test-scoped and test-classified dependency to BaseLibrary
* contains ParentSuite
* has classes that should be validated
You probably want to use JUnit #Category https://github.com/junit-team/junit/wiki/Categories
Here's a blog on how to make Categories https://weblogs.java.net/blog/johnsmart/archive/2010/04/25/grouping-tests-using-junit-categories-0
This way you can make Categories for each of your projects.
public interface ConsumerProject {}
Then you can use the #Category annotation to mark test classes (or even test methods) as being in particular categories:
#Category(ConsumerProject.class)
public class TestCase { ... }
Then your suites can be set up to run all the tests for particular categories
#RunWith(Categories.class)
#IncludeCategory(ConsumerProject.class)
#SuiteClasses( { ....})
public class ConsumerProjectSuite { }
You can even have the same test get marked with multiple categories
#Category({ConsumerProject.class, OtherProject.class})
public class CommonTests { }
I'm having some problems when defining inner classes in a Test class inherited from TestCase, for JUnit 3. Scenario is like following:
Foo.java
public class Foo {
public void method() { ... }
}
FooTest.java
public class FooTest extends TestCase {
public class Bar extends Foo {
public void method() { ... }
}
public void testMethod() { ... }
}
Now, if I run this from Eclipse, the tests run ok, but if I try to run from an Ant task it fails:
[junit] junit.framework.AssertionFailedError: Class Foo$Bar has no public constructor TestCase(String name) or TestCase()
Bar is NOT a Test class, it's just a subclass of Foo overriding some method that I don't need to do the real stuff when testing.
I'm quite lost at the moment and I don't know how to approach this problem. Is the only way to create the subclasses as standalone?
This is because you included a nested class into junit fileset.
Add an "excludes" property to your build.xml.
For example:
<target name="test" depends="test-compile">
<junit>
<batchtest todir="${test.build.dir}" unless="testcase">
<fileset dir="${test.build.classes}"
includes = "**/Test*.class"
excludes = "**/*$*.class"/>
</batchtest>
</junit>
</target>
You could try defining the Bar class as static:
public class FooTest extends TestCase {
public static class Bar extends Foo {
public void method() { ... }
}
public void testMethod() { ... }
}
... but the fact that it works in one environment but not in another suggests one of two things:
Java version
Classpath
[Edit: as suggested by Jim below] Different versions of junit.jar
I'm feeling like a necrposter, but the thing is that I've ran into similar problem with maven today.
Usual mvn test runs well but when I want run tests from specific package like mvn test -Dtest=com.test.* - initializationError is thrown. This "works" for both Junit 3 and 4.
I found the reason for my maven-case, this may be the same for ant.
The thing is: by default maven's test plugin (surefire that is) considers only specific subset of all classes as "test-classes", namely searching them by name, like *Test and so on (you can read about this at surefire's home page).When we define test property we completely override default behavior. This means that with -Dtest=com.test.* surefire will pick up not only com.test.MyTestClass but also com.test.MyTestClass.InnerClass and even com.test.MyTestClass$1 (i.e. anonymous classes).
So in order to run e.g. classes from some package you should use something like -Dtest=com.test.*Test (if you use suffixes for identifying test-classes of course).
You can also annotate the nested class #Ignore if you don't want to exclude all inner classes.
Is there any way to group tests in JUnit, so that I can run only some groups?
Or is it possible to annotate some tests and then globally disable them?
I'm using JUnit 4, I can't use TestNG.
edit: #RunWith and #SuiteClasses works great. But is it possible to annotate like this only some tests in test class? Or do I have to annotate whole test class?
JUnit 4.8 supports grouping:
public interface SlowTests {}
public interface IntegrationTests extends SlowTests {}
public interface PerformanceTests extends SlowTests {}
And then...
public class AccountTest {
#Test
#Category(IntegrationTests.class)
public void thisTestWillTakeSomeTime() {
...
}
#Test
#Category(IntegrationTests.class)
public void thisTestWillTakeEvenLonger() {
...
}
#Test
public void thisOneIsRealFast() {
...
}
}
And lastly,
#RunWith(Categories.class)
#ExcludeCategory(SlowTests.class)
#SuiteClasses( { AccountTest.class, ClientTest.class })
public class UnitTestSuite {}
Taken from here: https://community.oracle.com/blogs/johnsmart/2010/04/25/grouping-tests-using-junit-categories-0
Also, Arquillian itself supports grouping:
https://github.com/weld/core/blob/master/tests-arquillian/src/test/java/org/jboss/weld/tests/Categories.java
Do you want to group tests inside a test class or do you want to group test classes? I am going to assume the latter.
It depends on how you are running your tests. If you run them by Maven, it is possible to specify exactly what tests you want to include. See the Maven surefire documentation for this.
More generally, though, what I do is that I have a tree of test suites. A test suite in JUnit 4 looks something like:
#RunWith(Suite.class)
#SuiteClasses({SomeUnitTest1.class, SomeUnitTest2.class})
public class UnitTestsSuite {
}
So, maybe I have a FunctionTestsSuite and a UnitTestsSuite, and then an AllTestsSuite which includes the other two. If you run them in Eclipse you get a very nice hierarchical view.
The problem with this approach is that it's kind of tedious if you want to slice tests in more than one different way. But it's still possible (you can for example have one set of suites that slice based on module, then another slicing on the type of test).
To handle the globally disabling them, JUnit (4.5+) has two ways One is to use the new method assumeThat. If you put that in the #BeforeClass (or the #Before) of a test class, and if the condition fails, it will ignore the test. In the condition you can put a system property or something else that can be globally set on or off.
The other alternative is to create a custom runner which understands the global property and delegates to the appropriate runner. This approach is a lot more brittle (since the JUnit4 internal runners are unstable and can be changed from release to release), but it has the advantage of being able to be inherited down a class hierarchy and be overridden in a subclass. It is also the only realistic way to do this if you have to support legacy JUnit38 classes.
Here is some code to do the custom Runner. Regarding what getAppropriateRunnerForClass might do, the way I implemented it was to have a separate annotation that tells the custom runner what to run with. The only alternative was some very brittle copy paste from the JUnit code.
private class CustomRunner implements Runner
private Runner runner;
public CustomRunner(Class<?> klass, RunnerBuilder builder) throws Throwable {
if (!isRunCustomTests()) {
runner = new IgnoredClassRunner(klass);
} else {
runner = getAppropriateRunnerForClass(klass, builder);
}
public Description getDescription() {
return runner.getDescription();
}
public void run(RunNotifier notifier) {
runner.run(notifier);
}
}
EDIT: The #RunWith tag only works for a whole class. One way to work around that limiation is to move the test methods into a static inner class and annotate that. That way you have the advantage of the annotation with the organization of the class. But, doing that won't help with any #Before or #BeforeClass tags, you will have to recreate those in the inner class. It can call the outer class's method, but it would have to have its own method as a hook.
In JUnit 5 you can declare #Tag for filtering tests, either at the class or method level; analogous to test groups in TestNG or Categories in JUnit 4
From the javadoc :
tags are used to filter which tests are executed for a given test
plan. For example, a development team may tag tests with values such
as "fast", "slow", "ci-server", etc. and then supply a list of tags to
be used for the current test plan, potentially dependent on the
current environment.
For example you could declare a test class with a "slow" #Tag that will be inherited for all methods and override it for some methods if required :
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
#Tag("slow")
public class FooTest{
//
#Test
void loadManyThings(){
...
}
#Test
void loadManyManyThings(){
...
}
#Test
#Tag("fast")
void loadFewThings(){
...
}
}
You could apply the same logic for other test classes.
In this way test classes (and methods too) belongs to a specific tag.
As a good practice instead of copying and pasting #Tag("fast") and #Tag("slow") throughout the test classes, you can create custom composed annotations.
For example :
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import org.junit.jupiter.api.Tag;
#Target({ ElementType.TYPE, ElementType.METHOD })
#Retention(RetentionPolicy.RUNTIME)
#Tag("slow")
public #interface Slow {
}
and use it as :
#Test
#Slow
void slowProcessing(){
...
}
To enable or disable test marked with a specific tag during the text execution you can rely on the maven-surefire-plugin documentation :
To include tags or tag expressions, use groups.
To exclude tags or tag expressions, use either excludedGroups.
Just configure in your pom.xml the plugin according to your requirement (example of the doc) :
<build>
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<groups>acceptance | !feature-a</groups>
<excludedGroups>integration, regression</excludedGroups>
</configuration>
</plugin>
</plugins>
</build>
For information the test goal documentation is not updated.
Try JUnit Test Groups. From documentation :
#TestGroup("integration")
public class MyIntegrationTest {
#ClassRule
public static TestGroupRule rule = new TestGroupRule();
...
}
Execute a simple test group: -Dtestgroup=integration
Execute multiple test groups: -Dtestgroup=group1,group2
Execute all test groups: -Dtestgroup=all
You can create test Suite objects that contain groups of tests. Alternatively, your IDE (like Eclipse) may have support for running all the tests contained in a given package.
You can Use Test Suite(http://qaautomated.blogspot.in/2016/09/junit-test-suits-and-test-execution.html) or you can Junit Categories(http://qaautomated.blogspot.in/2016/09/junit-categories.html) for grouping your test cases effectively.
I have a base class that represents a database test in TestNG, and I want to specify that all classes extending from this class are of a group "db-test", however I have found that this doesn't seem possible. I have tried the #Test annotation:
#Test(groups = { "db-test" })
public class DBTestBase {
}
However, this doesn't work because the #Test annotation will try to make a bunch of methods into tests, and warnings/errors pop up in eclipse when the tests are run.
So I tried disabling the test, so at least the groups are assigned:
#Test(enabled = false, groups = { "db-test" })
public class DBTestBase {
}
but then any #BeforeTest (and other similar annotations) ALSO get disabled... which is of course not what I want.
I would like some way to annotate a class as being of a particular type of group, but it doesn't quite seem possible in TestNG. Does anyone have any other ideas?
TestNG will run all the public methods from a class with a #Test annotation. Maybe you could change the methods you don't want TestNG to run to be non-public
The answer is through a custom org.testng.IMethodSelector:
Its includeMethod() can exclude any method we want, like a public not-annotated method.
However, to register a custom Java MethodSelector, you must add it to the XMLTest instance managed by any TestRunner, which means you need your own custom TestRunner.
But, to build a custom TestRunner, you need to register a TestRunnerFactory, through the -testrunfactory option.
BUT that -testrunfactory is NEVER taken into account by TestNG class... so you need also to define a custom TestNG class :
in order to override the configure(Map) method,
so you can actually set the TestRunnerFactory
TestRunnerFactory which will build you a custom TestRunner,
TestRunner which will set to the XMLTest instance a custom XMLMethodSelector
XMLMethodSelector which will build a custom IMethodSelector
IMethodSelector which will exclude any TestNG methods of your choosing!
Ok... it's a nightmare. But it is also a code-challenge, so it must be a little challenging ;)
All the code is available at DZone snippets.
As usual for a code challenge:
one java class (and quite a few inner classes)
copy-paste the class in a 'source/test' directory (since the package is 'test')
run it (no arguments needed)
Update from Mike Stone:
I'm going to accept this because it sounds pretty close to what I ended up doing, but I figured I would add what I did as well.
Basically, I created a Groups annotation that behaves like the groups property of the Test (and other) annotations.
Then, I created a GroupsAnnotationTransformer, which uses IAnnotationTransformer to look at all tests and test classes being defined, then modifies the test to add the groups, which works perfectly with group exclusion and inclusion.
Modify the build to use the new annotation transformer, and it all works perfectly!
Well... the one caveat is that it doesn't add the groups to non-test methods... because at the time I did this, there was another annotation transformer that lets you transform ANYTHING, but it somehow wasn't included in the TestNG I was using for some reason... so it is a good idea to make your before/after annotated methods to alwaysRun=true... which is sufficient for me.
The end result is I can do:
#Groups({ "myGroup1", "myGroup2"})
public class MyTestCase {
#Test
#Groups("aMethodLevelGroup")
public void myTest() {
}
}
And I made the transformer work with subclassing and everything.
It would seem to me as the following code-challenge (community wiki post):
How to be able to execute all test methods of Extended class from the group 'aGlobalGroup' without:
specifying the 'aGlobalGroup' group on the Extended class itself ?
testing non-annotated public methods of Extended class ?
The first answer is easy:
add a class TestNG(groups = { "aGlobalGroup" }) on the Base class level
That group will apply to all public methods of both Base class and Extended class.
BUT: even non-testng public methods (with no TestNG annotation) will be included in that group.
CHALLENGE: avoid including those non-TestNG methods.
#Test(groups = { "aGlobalGroup" })
public class Base {
/**
*
*/
#BeforeClass
public final void setUp() {
System.out.println("Base class: #BeforeClass");
}
/**
* Test not part a 'aGlobalGroup', but still included in that group due to the class annotation. <br />
* Will be executed even if the TestNG class tested is a sub-class.
*/
#Test(groups = { "aLocalGroup" })
public final void aFastTest() {
System.out.println("Base class: Fast test");
}
/**
* Test not part a 'aGlobalGroup', but still included in that group due to the class annotation. <br />
* Will be executed even if the TestNG class tested is a sub-class.
*/
#Test(groups = { "aLocalGroup" })
public final void aSlowTest() {
System.out.println("Base class: Slow test");
//throw new IllegalArgumentException("oups");
}
/**
* Should not be executed. <br />
* Yet the global annotation Test on the class would include it in the TestNG methods...
*/
public final void notATest() {
System.out.println("Base class: NOT a test");
}
/**
* SubClass of a TestNG class. Some of its methods are TestNG methods, other are not. <br />
* The goal is to check if a group specify in the super-class will include methods of this class. <br />
* And to avoid including too much methods, such as public methods not intended to be TestNG methods.
* #author VonC
*/
public static class Extended extends Base
{
/**
* Test not part a 'aGlobalGroup', but still included in that group due to the super-class annotation. <br />
* Will be executed even if the TestNG class tested is a sub-class.
*/
#Test
public final void anExtendedTest() {
System.out.println("Extended class: An Extended test");
}
/**
* Should not be executed. <br />
* Yet the global annotation Test on the class would include it in the TestNG methods...
*/
public final void notAnExtendedTest() {
System.out.println("Extended class: NOT an Extended test");
}
}
I'm not sure how the annotation inheritance works for TestNG but this article may be of some use.
Actually, this may help better, look at inheritGroups.
You can specify the #Test annotation at method level that allows for maximum flexibility.
public class DBTestBase {
#BeforeTest(groups = "db-test")
public void beforeTest() {
System.out.println("Running before test");
}
public void method1() {
Assert.fail(); // this does not run. It does not belong to 'db-test' group.
}
#Test(groups = "db-test")
public void testMethod1() {
Assert.assertTrue(true);
}
}
Does this works for you or I am missing something from your question.