I have been setting up some contract JUnit 4 tests (following advice in answer to this question) - e.g. if I have an interface (I've included generics in the example, since I'm using them in my project) - but, this is a significantly simplified example:
public interface MyInterface<T> {
public T myMethod();
}
I can create an abstract contract test - e.g.:
public abstract class AbstractMyInterfaceTest<T> {
private final MyInterface<T> _impl;
// plus, store stuff to test against;
public AbstractMyInterfaceTest(MyInterface<T> impl, ... stuff to test against){
_impl = impl;
// stuff to test against...
}
#Test
public void testMyMethod(){
// test some stuff...
}
}
...and extend this abstract test class for each implementation (which I am running with #RunWith(Parameterized.class)). I'm also passing some variables to test the implementation of the interface against into the abstract test class.
However, in actuality, some interfaces return objects that are implementations of other interfaces, and I would like to be able to test this implementation using the abstract contract class for that interface (...and against the relevant variables). For example, a method in MyInterface might return an Iterator - which is an interface. I have a separate AbstractIteratorTest, which runs a contract test for Iterator against a set of values, and I want to check that the Iterator returned by the implementations of MyInterface passes the contract tests in AbstractIteratorTest with the expected set of values.
At the moment, I'm adding the following to the abstract test classes:
// N.b. not setting #Test - so doesn't run all tests twice...
public void runTests(){
testMyMethod();
// etc.
}
However, this doesn't run each test independently (e.g. running setUp() before each test method), so there are problems if those contract tests expect the implementation to be in a particular state for each test. (A specific example would be testing implementations of the Iterator interface, where each test method can change the state of the tested Iterator)
Is there was a better way of doing this, perhaps making use of features in JUnit 4?
I've seen stuff about the #RunWith(Suite.class) annotation, but that seems to be about running multiple test classes from a single test class, and doesn't allow you to pass variables into the different test classes that are being run. There may be a way to use this Suite to solve my problem, but I'm not sure what it is...
Firstly, if you're subclassing your abstract test class, you don't need to use Parameterized. You could use Parameterized instead of subclassing, but you're unlikely to need both. Unless you want to test multiple instances of each implementation, which i will demonstrate below!
One way to do this would be to make the tests for returned instances simply be more subclasses or parameterisations of AbstractAnotherInterfaceTest.
Say you have:
public abstract class AbstractAnotherInterfaceTest {
private AnotherInterface instance;
protected AbstractAnotherInterfaceTest(AnotherInterface instance) {
this.instance = instance;
}
#Test
public void everythingIsOkay() {...}
}
And you had:
public class Pony implements AnotherInterface { ... }
public class PonyProducingMyInterface implements MyInterface<Pony> { ... }
You could write:
#RunWith(Parameterized.class)
public class PonyTest extends AbstractAnotherInterfaceTest {
#Parameterized.Parameters
public static Collection<Pony> ponies() {
return Arrays.asList(
new Pony(), // base case
new PonyProducingMyInterface().myMethod() // testing the factory
);
}
public PonyTest(Pony instance) {
super(pony);
}
}
This is admittedly a bit weird, though, because the new PonyProducingMyInterface().myMethod() case is really a test for PonyProducingMyInterface, but is classified under AbstractAnotherInterfaceTest.
The only approach i can think of that would make use of the AbstractAnotherInterfaceTest tests in some subclass of AbstractMyInterfaceTest would be to write a custom ParentRunner which knew how to set up instances of AbstractAnotherInterfaceTest as children of instances of AbstractMyInterfaceTest. You'd probably want to make that a bit generic, using annotations to guide the process.
Related
Had a quick look at past questions, couldn't see something similar so here goes:
I also made a mistake in choosing dummy names for this example to illustrate my point, I'll rename them.
I have a class which has a JUnit test:
public class CheckFilter {
#Test
public void Run_filter_test() {
//some code
}
}
And then another class:
public class CheckVideoPlays {
#Test
public void Play_video_in_full() {
//some more code here etc
}
}
Finally, how do I call these two tests from another class, obviously you can't extend multiple classes.
public class RunAllTests {
//How do i call both
//eg
//
//Run_filter_test();
//Play_video_in_full();
}
Note: I don't want to call the class. Don't want to run as:
#RunWith(Suite.class)
#Suite.SuiteClasses({
CheckFilter.class,
CheckVideoPlays.class
})
A few things.
Change the name of Sanatize_all_inputs to the (java standard form) camel case, perhaps sanitizeAllImports. When using Java, obey Java.
It seems likely that you will sanitize inputs once per test,
which, to me, indicates that you want a class level variable of type ConvertAll in your jUnit test class.
Either use composition (another class level variable of type BaseTestBlammy) or inheritance (extend class BaseTestBlammy) to acquire access to the BaseTestBlammy methods.
Here is an example:
public MyJunitTestKapow
extends BaseTestBlammy
{
private final ConvertAll convertAll;
public MyJunitTestKapow()
{
convertAll = new ConvertAll();
}
#Test
public void someTest()
{
convertAll.sanitizeAllInputs(...);
... // do the rest of the test here.
}
}
You could make static method in ConvertAll.sanitize then call this method in both ConvertAll.Sanitise_all_inputs test and CheckFilter.Run_filter_test
As better (more maintainable and powerful) solution you may create Sanitiser class with sanitise() method (method may be static or not). Then each class that requires sanitise functionality will call Sanitiser.sanitise. This soltion may be better in long run- you may pass paramethers to sanitise method (or to Sanitiser constructor), Sanitiser may have some internal state, etc
Side note: you may consider migrating to Junit5 (basically it is just change of imported packages). Junit5 has #DisplayName annotation that declares nice test method names (with spaces). So your test methods will respect Java naming convention.
#Test
#DiplayName("Sanitize all inputs")
public void sanitiseAllInputs() {
//some more code here etc
}
I have a test method belonging to multiple groups, I want to ensure that one #BeforeGroups method (setupHomePage) runs before another (setupUniversalSearchPage). The order must be defined within java only as I am supposed to use any testng xml.
Although in similar case in another test class the order of execution is as desired.
public abstract class HomePageTest extends TestBase
{
#BeforeGroups("homePageTests")
public void setupHomePage()
{
loadHomePage();
}
}
public abstract class UniversalSearchPageTest extends HomePageTest
{
#BeforeGroups("universalPageTests")
public void setupUniversalSearchPage()
{
navigateToSearchPage();
}
}
public class UniversalSearchPageBasicTest extends UniversalPageTest
{
#Test(groups = {"homePageTests","universalPageTests")
public void searchVerificationTest()
{
//test code here
}
}
So far I have tried following :
1. reversing the order of groups in #Test annotation of searchVerificationTest
2. adding #Test(groups = "homePageTests") above declaration of UniversalSearchPageBasicTest class.
I would like to know how the order #BeforeGroups method is determined as it is different in one class than other.
If it is of any relation I am using maven in eclipse and Appium
I am new to testng , let me know if I am missing anything basic here
#BeforeMethod methods will run in inheritance order - the highest superclass first, then going down the inheritance chain. #AfterMethod methods run in reverse order (up the inheritance chain).
Note, however, that the ordering of multiple annotated methods within one class is not guaranteed (so it's best to avoid that).
Also there is a online documentation available, checkout http://testng.org/doc/documentation-main.html#annotations
I'm designing UI Tests for a web application with Selenium in JUnit. I have a base test class with something like this from which I inherit my tests:
public class BaseTest {
protected TestSteps test;
protected Assertions assertion;
// set everything up...
}
and the tests then only look like this:
public class TestX extends BaseTest {
#Test
public testFeature1() {
test.clickSomething().enterSomething(); // method chaining
assertion.assertSomething();
//...
}
}
The problem I'm having: There are different modules in the web app, and Assertions/TestSteps methods that only apply to one module clutter the interface of the Assertions/TestSteps class for the other modules.
Thus I tried to split the Assertions/TestSteps up.
The problem is, the method chaining returns instances of TestSteps. Of course, when I have Module1TestSteps with method doSomethingSpecific() then I would expect test.clickSomething().doSomethingSpecific() to work, but it does not, because clickSomething() would return a TestSteps instance, not a Module1TestSteps instance.
I "solved" this by making an AbstractTestSteps<T extends AbstractTestSteps<T> class (which contains all the base TestSteps methods) protected abstract T getThis();.
I then extend this class like this:
public class BaseTestSteps extends AbstractTestSteps<BaseTestSteps> {
// Constructors
protected BaseTestSteps getThis() {
return this;
}
// that's it, the "base methods" are all inherited from AbstractTestSteps...
}
for the base TestSteps and
public class Module1TestSteps extends AbstractTestSteps<Module1TestSteps> {
// same constructors...
protected Module1TestSteps getThis() {
return this;
}
public Module1TestSteps doSomeThingSpecific() {
// do something
return getThis();
}
}
for my specialized TestSteps. It works for now, but I don't like it because of the following reasons:
All the general methods are in the AbstractTestSteps class, but they are used through an instance of BaseTestSteps
What if I have a submodule of Module1? I can't inherit from Module1TestSteps, only from AbstractTestSteps.
I think it's not trivial to understand the relation of these classes when one of my colleagues tries to add a new TestSteps class.
How can this be made better?
Use the Page Object pattern. That is, create an API for each page so that your tests describe navigating and interacting with pages in a way that describes the user's experience.
It has a few benefits that address your concerns:
It uses composition, not inheritance
It is easy to understand and explain to people maintaining the tests because the tests read like a description of somebody using the application
Assume that interface for is defined
interface Foo {
int getBaz();
void doBar();
}
Further assume that the contract states that everytime doBar is called baz is incremented. (Ok this is a contrived bit of code but stick with me here)
Now I want to provide a unit test that I can provide to Foo implementers so that they can verify that they meet all the contract conditions.
class FooTest {
protected Foo fooImpl;
#Test
public void int testBazIncrement()
{
int b = fooImpl getBaz();
fooImpl.doBar();
Assert.assertEquals( b+1, fooImpl.getBaz();
}
}
What are the best practices for making the test available to the implemnters of Foo? It seems to me that there needs to be a mechanism for them to call the FooTest and provide a Foo or a FooFactory to construct Foo instances. Furthermore, if there are many tests (think big interface) then I want to put all those tests in one FooTest class.
Are there any best practices for how to implement such tests?
This is a textbook example of Dependency Injection. If you use Spring as the DI container, you can wire in the fooImpl
#Inject
protected Foo fooImpl;
Your test needs to be annotated with #RunWith(SpringJUnit4ClassRunner.class), and it's up to the interface provider to configure Spring with their implementation.
Without a DI container, you can just create an abstract test class with all the tests in it and an abstract method to provide the implementation object.
protected abstract Foo createFoo();
It's up to the implementation provider to subclass the test and implement the abstract method.
class FooImplTest extends FooTestSuper {
#Override
protected Foo createFoo() {
return new FooImpl();
}
If you have multiple tests, consider JUnit's #Suite annotation. It's compatible with the Spring test runner.
You can implement a testDataFactory where you istance your objects, or use GSon for create your objects (personally, I like GSon, is clear and fast, you learn it in a few time).
For the test implementation, I suggest to write more tests and not a single one.
In this way, unit test can be indipendent and you can segregate your problems in a closed structure.
Sonar
Sonar is a tool that help you a lot, making analisys of your code. You can see from sonar front-end how you application is tested:
sonar unit test
as you can see, Sonar can show you where your code is tested or not
Why not just have, say, a InterfaceTester that gets called from the unit tests InterfaceImplATest, InterfaceImplBTest, etc?
e.g.
#Test
public void testSerialisation()
{
MyObject a = new MyObject();
...
serialisationTester.testSimpleRoundTrip(a);
serialisationTester.testEdgeCases(a);
...
}
After much pondering and some dead-ends I have begun using the following pattern:
In the following:
[INTERFACE] refers to the interface being tested.
[CLASS] refers to the implementation of the interface being tested.
Interface tests are built so that developers may test that implementations meet the contract
set out in the interface and accompanying documentation.
The major items under test use an instance of an [INTERFACE]ProducerInterface to create the instance of the object being tested. An implementation of [INTERFACE]ProducerInterface must track all the instances created during the test and close all of them when requested. There is an Abstract[INTERFACE]Producer that handles most of that functionality but requires a createNewINTERFACE implementation.
TESTS
Interface tests are noted as Abstract[INTERFACE]Test. Tests generally extend the Abstract[INTERFACE]ProducerUser class. This class handles cleaning up all the graphs at the end of the tests and provides a hook for implementers to plug in their [INTERFACE]ProducerInterface implementation.
In general to implement a test requires a few lines of code as is noted in the example below
where the new Foo graph implementation is being tested.
public class FooGraphTest extends AbstractGraphTest {
// the graph producer to use while running
GraphProducerInterface graphProducer = new FooGraphTest.GraphProducer();
#Override
protected GraphProducerInterface getGraphProducer() {
return graphProducer;
}
// the implementation of the graph producer.
public static class GraphProducer extends AbstractGraphProducer {
#Override
protected Graph createNewGraph() {
return new FooGraph();
}
}
}
SUITES
Test suites are named as Abstract[INTERFACE]Suite. Suites contain several tests that excersize all of the tests for components of the object under test. For example if the Foo.getBar() returned an instance of the Bar interface the Foo suite includes tests for the Foo iteself as well as running the Bar tests the Bar. Running the suites is a bit more complicated then running the tests.
Suites are created using the JUnit 4 #RunWith(Suite.class) and #Suite.SuiteClasses({ })
annotations. This has several effects that the developer should know about:
The suite class does not get instantiated during the run.
The test class names must be known at coding time (not run time) as they are listed in the annotation.
Configuration of the tests has to occur during the static initialization phase of class loading.
To meet these requirements the Abstract[INTERFACE]Suite has a static variable that holds the instance of the [INTERFACE]ProducerInterface and a number of local static implementations of the Abstract tests that implement the "get[INTERFACE]Producer()" method by returning the static instance. The names of the local tests are then used in the #Suite.SuiteClasses annotation. This makes creating an instance of the Abstract[INTERFACE]Suite for an [INTERFACE] implementation fairly simple as is noted below.
public class FooGraphSuite extends AbstractGraphSuite {
#BeforeClass
public static void beforeClass() {
setGraphProducer(new GraphProducer());
}
public static class GraphProducer extends AbstractGraphProducer {
#Override
protected Graph createNewGraph() {
return new FooGraph();
}
}
}
Note that the beforeClass() method is annotated with #BeforeClass. the #BeforeClass causes it to be run once before any of the test methods in the class. This will set the static
instance of the graph producer before the suite is run so that it is provided to the enclosed tests.
FUTURE
I expect that further simplification and removal duplicate code can be achieved through the use of java generics, but I have not gotten to that point yet.
Here are some of my thoughts about how to make a qualified unit test:
First of all, try to make your a implementer class fully tested, which means all of its methods should be covered by the UT. Doing this will save you a lot of time when you need to refactor your code. For your case, it could be :
class FooTest {
protected Foo fooImpl;
#Test
public void testGetBaz() {
...
}
#Test
public void testDoBar() {
...
}
}
You will find you need to check the internal state of your class and there is nothing wrong it for UT should be a kind of white-box test.
Second, all methods should be tested separately and not depend on each other. In my opinion, for your code posted above, it looks like more than a function test or an integration test, but it's also necessary.
Third, I don't think it's a good practice to use spring or other container to assemble the target object for you, which will make your test running relatively slow, especially when there a load of tests to run. And your class should intrinsically be pojo and you can complete the assembling in another method in your test class if your target object is really complex.
Fourth, if the parent interface of some class is really big, Dividing the test methods into several groups is something you should do. Here is more info.
Suppose I have interface and implementation class that implements it and I want to write unit-test for this. What should I test interface or Impl?
Here is an example:
public interface HelloInterface {
public void sayHello();
}
public class HelloInterfaceImpl implements HelloInterface {
private PrintStream target = System.out;
#Override
public void sayHello() {
target.print("Hello World");
}
public void setTarget(PrintStream target){
this.target = target;
}
}
So, I have HelloInterface and HelloInterfaceImpl that implements it. What is unit-under-test interface or Impl?
I think it should be HelloInterface. Consider following sketch of JUnit test:
public class HelloInterfaceTest {
private HelloInterface hi;
#Before
public void setUp() {
hi = new HelloInterfaceImpl();
}
#Test
public void testDefaultBehaviourEndsNormally() {
hi.sayHello();
// no NullPointerException here
}
#Test
public void testCheckHelloWorld() throws Exception {
ByteArrayOutputStream out = new ByteArrayOutputStream();
PrintStream target = new PrintStream(out);
PrivilegedAccessor.setValue(hi, "target", target);
//You can use ReflectionTestUtils in place of PrivilegedAccessor
//really it is DI
//((HelloInterfaceImpl)hi).setTarget(target);
hi.sayHello();
String result = out.toString();
assertEquals("Hello World", result);
}
}
The main line is actually one that I commented out.
((HelloInterfaceImpl)hi).setTarget(target);
Method setTarget() is not part of my public interface, so I don't want to accidentally call it. If I really want to call it, I should take a moment and think about it. It helps me, for example, to discover that what I'm really trying to do is dependency injection. It opens for me the whole world of new opportunities. I can use some existing dependency injection mechanism (Spring's, for example), I can simulate it myself as I actually did in my code or to take totally different approach. Take a closer look, preparation of PrintSream wasn't that easy, maybe I should use mock object instead?
EDIT:
I think I should always focus on the interface. From my point of view setTarget() is not part of the "contract" of the impl class neither, it serves sally for dependency injection. I think any public method of Impl class should be considered as private from the testing perspective. It doesn't mean that I ignore the implementation details, though.
See also Should Private/Protected methods be under unit test?
EDIT-2 In the case of multiple implementations\multiple interfaces, I would test all of the implementations, but when I declare a variable in my setUp() method I would definitely use interface.
The implementation is the unit that needs to be tested. That is of course what you are instantiating and what contains the program/business logic.
If you had a critical interface and you wanted to make sure every implementation adhered to it properly, then you may write a test suite that focuses on the interface and requires an instance be passed in (agnostic of any implementation type).
Yes, it would probably be easier to use Mockito for PrintStream, it may not always be possible to avoid using a mock object like you did in this specific example.
I would test to the interface.
I think the mistake was writing the implemenation in such a way that it was hard-wired to write to System.out; you gave yourself no way to override with another PrintStream. I would have used a constructor instead of a setter. There's no need for a mock or casting that way.
This is a simple case. I'd imagine that a more complex one would have a factory for creating different, more complex implementations of an interface. Hopefully you wouldn't design that in such a way that you'd be boxed in.
Sticking to the interface in your tests makes mocking a lot easier, too.
public class HelloInterfaceImpl implements HelloInterface {
private PrintStream target;
public HelloInterfaceImpl() {
this(System.out);
}
public HelloInterfaceImpl(PrintStream ps) {
this.target = ps;
}
#Override
public void sayHello() {
target.print("Hello World");
}
}
Here's the test:
public class HelloInterfaceTest {
#Test
public void testDefaultBehaviourEndsNormally() {
HelloInterface hi = new HelloInterfaceImpl();
hi.sayHello();
}
#Test
public void testCheckHelloWorld() throws Exception {
ByteArrayOutputStream out = new ByteArrayOutputStream();
PrintStream target = new PrintStream(out);
HelloInterface hi = new HelloInterfaceImpl(target);
hi.sayHello();
String result = out.toString();
assertEquals("Hello World", result);
}
}
I always test implementations - one class can implement several interfaces and also one interface can be implemented by several classes - each of them should be covered by tests.
The requirement of invoking setter in the unit test (casting the interface to the implementation):
((HelloInterfaceImpl)hi).setTarget(target);
means that you actually test the implementation. It's not the part of the contract, but this is the important part making the implementation to work and should be tested properly.
Let's take an example from JDK. You have interface List and two implementations: ArrayList and LinkedList. Additionally LinkedList implements Deque interface. If you write test for List interface what would you cover? Array or linked list? What's more in case of LinkedList, what interface would you choose to test? Deque or List? As you see, when you test implementations you don't have such problems.
For me, personally, casting interface to implementation in the unit test is obvious sign that something is going wrong ;)
I would say it depends on the implementation and what it does beyond the contract of the interface. Many implementations only implement the functionality provided in the interface, in others, the interface is only a small portion of the classes functionality. It may implement multiple interfaces.
Ultimately you are testing the implementation.
In a simple case like you have defined, I say six of one and half a dozen of the other. Write your test cases to the interface or the implementation, as long as it tests the implementation sufficiently, the results are the same.
Take a different example where I would have a class that collects statistics on a communication channel by decorating the real reader and writer. My class may now implement those interfaces, but it also collects statistics, which has nothing to do with either contract. I could certainly still write tests based on those interfaces, but it will not fully test this class.