How do you manage dummy data used for tests? Keep them with their respective entities? In a separate test project? Load them with a Serializer from external resources? Or just recreate them wherever needed?
We have an application stack with several modules depending on another with each containing entities. Each module has its own tests and needs dummy data to run with.
Now a module that has a lot of dependencies will need a lot of dummy data from the other modules. Those however do not publish their dummy objects because they are part of the test resources so all modules have to setup all dummy objects they need again and again.
Also: most fields in our entities are not nullable so even running transactions against the object layer requires them to contain some value, most of the time with further limitations like uniqueness, length, etc.
Is there a best practice way out of this or are all solutions compromises?
More Detail
Our stack looks something like this:
One Module:
src/main/java --> gets jared (.../entities/*.java contains the entities)
src/main/resources --> gets jared
src/test/java --> contains dummy object setup, will NOT get jared
src/test/resources --> not jared
We use Maven to handle dependencies.
module example:
Module A has some dummy objects
Module B needs its own objects AND the same as Module A
Option a)
A Test module T can hold all dummy objects and provide them in a test scope (so the loaded dependencies don't get jared) to all tests in all Modules. Will that work? Meaning: If I load T in A and run install on A will it NOT contain references introduced by T especially not B? Then however A will know about B's datamodel.
Option b)
Module A provides the dummy objects somewhere in src/main/java../entities/dummy allowing B to get them while A does not know about B's dummy data
Option c)
Every module contains external resources which are serialized dummy objects. They can be deserialized by the test environment that needs them because it has the dependency to the module to which they belong. This will require every module to create and serialize its dummy objects though and how would one do that? If with another unit test it introduces dependencies between unit tests which should never happen or with a script it'll be hard to debug and not flexible.
Option d)
Use a mock framework and assign the required fields manually for each test as needed. The problem here is that most fields in our entities are not nullable and thus will require setters or constructors to be called which would end us up at the start again.
What we don't want
We don't want to set up a static database with static data as the required objects' structure will constantly change. A lot right now, a little later. So we want hibernate to set up all tables and columns and fill those with data at unit testing time. Also a static data base would introduce a lot of potential errors and test interdependencies.
Are my thoughts going in the right direction? What's the best practice to deal with tests that require a lot of data? We'll have several interdependent modules that will require objects filled with some kind of data from several other modules.
EDIT
Some more info on how we're doing it right now in response to the second answer:
So for simplicity, we have three modules: Person, Product, Order.
Person will test some manager methods using a MockPerson object:
(in person/src/test/java:)
public class MockPerson {
public Person mockPerson(parameters...) {
return mockedPerson;
}
}
public class TestPerson() {
#Inject
private MockPerson mockPerson;
public testCreate() {
Person person = mockPerson.mockPerson(...);
// Asserts...
}
}
The MockPerson class will not be packaged.
The same applies for the Product Tests:
(in product/src/test/java:)
public class MockProduct() { ... }
public class TestProduct {
#Inject
private MockProduct mockProduct;
// ...
}
MockProduct is needed but will not be packaged.
Now the Order Tests will require MockPerson and MockProduct, so now we currently need to create both as well as MockOrder to test Order.
(in order/src/test/java:)
These are duplicates and will need to be changed every time Person or Product changes
public class MockProduct() { ... }
public class MockPerson() { ... }
This is the only class that should be here:
public class MockOrder() { ... }
public class TestOrder() {
#Inject
private order.MockPerson mockPerson;
#Inject
private order.MockProduct mockProduct;
#Inject
private order.MockOrder mockOrder;
public testCreate() {
Order order = mockOrder.mockOrder(mockPerson.mockPerson(), mockProduct.mockProduct());
// Asserts...
}
}
The problem is, that now we have to update person.MockPerson and order.MockPerson whenever Person is changed.
Isn't it better to just publish the Mocks with the jar so that every other test that has the dependency anyway can just call Mock.mock and get a nicely setup object? Or is this the dark side - the easy way?
This may or may not apply - I'm curious to see an example of your dummy objects and the setup code related. (To get a better idea of whether it applies to your situation.) But what I've done in the past is not even introduce this kind of code into the tests at all. As you describe, it's hard to produce, debug, and especially package and maintain.
What I've usaully done (and AFAIKT in Java this is the best practice) is try to use the Test Data Builder pattern, as described by Nat Pryce in his Test Data Builders post.
If you think this is somewhat relevant, check these out:
Does a framework like Factory Girl exist for Java?
make-it-easy, Nat's framework that implements this pattern.
Well, I read carefully all evaluations so far, and it is very good question. I see following approaches to the problem:
Set up (static) test data base;
Each test has it's own set up data that creates (dynamic) test data prior to running unit tests;
Use dummy or mock object. All modules know all dummy objects, this way there is no duplicates;
Reduce the scope of the unit test;
First option is pretty straight forward and has many drawbacks, somebody has to reproduce it's once in while, when unit tests "mess it up", if there are changes in the data-module, somebody has to introduce corresponding changes to the test data, a lot of maintenance overhead. Not to say that generation of this data on the first hand maybe tricky. See aslo second option.
Second option, you write your test code that prior to the testing invokes some of your "core" business methods that creates your entity. Ideally, your test code should be independent from the production code, but in this case, you will end up with duplicate code, that you should support twice. Sometimes, it is good to split your production business method in order to have entry point for your unit test (I makes such methods private and use Reflection to invoke them, also some remark on the method is needed, refactoring is now a bit tricky). The main drawback that if you must change your "core" business methods it suddenly effects all of your unit test and you can't test. So, developers should be aware of it and not make partials commits to the "core" business methods, unless they works. Also, with any change in this area, you should keep in your mind "how it will affect my unit test". Sometimes also, it is impossible to reproduce all the required data dynamically (usually, it is because of the third-parties API, for example, you call another application with it's own DB from which you required to use some keys. This keys (with the associated data) is created manually through third-party application. In such a case, this data and only this data, should be created statically. For example, your created 10000 keys starting from 300000.
Third option should be good. Options a) and d) sounds for me pretty good. For your dummy object you can use the mock framework or you can not to use it. Mock Framework is here only to help you. I don't see problem that all of your unit know all your entities.
Fourth option means that you redefine what is "unit" in your unit test. When you have couple of modules with interdependence than it can be difficult to test each module in isolation. This approach says, that what we originally tested was integration test and not unit test. So, we split our methods, extract small "units of works" that receives all it's interdependences to another modules as parameters. This parameters can be (hopefully) easily mocked up. The main drawback of this approach, that you don't test all of your code, but only so to say, the "focal points". You need to make integration test separately (usually by QA team).
I'm wondering if you couldn't solve your problem by changing your testing approach.
Unit Testing a module which depends on other modules and, because of that, on the test data of other modules is not a real unit test!
What if you would inject a mock for all of the dependencies of your module under test so you can test it in complete isolation. Then you don't need to setup a complete environment where each depending module has the data it needs, you only setup the data for the module your actually testing.
If you imagine a pyramid, then the base would be your unit tests, above that you have functional tests and at the top you have some scenario tests (or as Google calls them, small, medium and big tests).
You will have a huge amount of Unit Tests that can test every code path because the mocked dependencies are completely configurable. Then you can trust in your individual parts and the only thing that your functional and scenario tests will do is test if each module is wired correctly to other modules.
This means that your module test data is not shared by all your tests but only by a few that are grouped together.
The Builder Pattern as mentioned by cwash will definitely help in your functional tests.
We are using a .NET Builder that is configured to build a complete object tree and generate default values for each property so when we save this to the database all required data is present.
Related
I am very new to Python world and learning my way in it. I have Java and C# background and in Java/C# unit testing approach is very clear. We have a class under test and we inject all the mocked dependency with expectations to test code.
like:
Car class with a service fuelLevel
Test case: Car is showing low fuel indication, we can write a test.
FuelLevelService fsMock = mock(FuelLevelService .class);
when(fsMock.getLevel()).thenReturn(LOW);
Car car = Car(fsMock)
car.start()
assertTrue(car.warningIndicator());
assertTrue(car.warning().contains("Fuel level low Warning"));
In python i have read we can use monkey patching to set the dependent objects and we can test.
#mock.patch('models.fuel_service.getLevel')
def test_main(self, fuel_service):
fuel_service.return_value = 'LOW'
Car car = Car(fuel_service)
car.start()
self.assertEqual(car.warningIndicator(), True)
I was going to use this approach. But my colleagues have rejected this idea and are proposing to instantiate the actual objects (FuellevelService) and set its value. I find it to be very cumbersome and not needed as in mock also we can do the same with just our expectations.
I am currently struggling to understand if i am wrong or in python my colleagues approach is regarded better pattern.
*Above i have given a simple example. In our project to create an instance object and setting its value is 10-25 lines of code [don't ask me why but that is how it is :)] To create this object and set a value so it returns that same value looks extra maintenance to me. For this approach to create dependent object we will have many py files in module.
Another point is we are here only testing class under test (not dependency service class), class under test which will react on the returned value of the dependency service. We can can have both negative and positive tests to cover class under test.
For dependency service class in same way it should have its own tests to validate all its scenarios separability. So it will be class under test there. By doing this we will have clear separation and test will fail where there is bug.
You only need to use mocks if creating the real objects are inconvenient/cumbersome to create (see here).
If, as you allude to in your question, you can create the real instance cheaply, and set its value easily, there's no real need for a mock.
Additionally, you don't have any guarantee that the mock behaves like the real implementation, so your test might test... something, but not necessarily how your code will really behave in production. Using the production class is preferable.
I've been struggling with understanding the minefield that is classes supported by the jmockit Mocking API. I found what I thought was a bug with File, but the issue was simply closed saying "Don't mock a File or Path" which no explanation. Can someone on here help me understand why certain classes should not be mocked and how I'm supposed to work around that. I'm trying to contribute to the library with bug reports, but each one I file just leaves me more confused. Pardon my ignorance, but if anyone can point me to the rationale for forbidden mocks etc. I'd greatly appreciate it.
I'm not certain that there's a fool proof list of rules. There's always a certain degree of knowledge and taste in these matters. Dogma is usually not a good idea.
I voted to close this question because I think a lot of it is opinion.
But with that said, I'll make an attempt.
Unit tests should be about testing individual classes, not interactions between classes. Those are called integration tests.
If your object is calling out to other remote objects, like services, you should mock those to return the data needed for your test. The idea is that services and their clients should also be tested individually in their own unit tests. You need not repeat those when testing a class that depends on them.
One exception to this rule, in my opinion, are data access objects. There is no sense in testing one of those without connecting to a remote database. Your test needs to prove the proper operation of your code. That requires a database connection in the case of data access objects. These should be written to be transactional: seed the database, perform the test, and reverse the actions of the test. The database should be in the same state when you're done.
Once your data access objects are certified as working correctly, all clients that use them should mock them. No need to re-test.
You should not be mocking classes in the JVM.
You asked for a why about File or Stream in particular - here's one. Take it or ignore it.
You don't have to test JVM classes because Sun/Oracle have already done that. You know they work. You want your class to use those classes because a failing test will expose the fact that the necessary file isn't available. A mock won't tell me that I've neglected to put a required file in my CLASSPATH. I want to find out during testing, not in production.
Another reason is that unit tests are also documentation. It's a live demonstration for others to show how to properly use your class.
In unit test you have to test that your code is doing the right thing. You can mock out any external pieces of code that is not the direct code being tested. This is a strict definition of unit test and it assumes there is another form of testing called integration testing that will be done later on. In integration testing you test how your code interacts with external elements, like a DB or another web service, or the network, or the hard drive.
If I have a piece of code that interacts with an object, like a File, and my code does 3 things to that file, then in my unit test I am going to test that my code has done those three things.
For example:
public void processFile(File f) {
if (f.exists()) {
//perform some tasks
} else {
//perform some other tasks
}
}
To properly unit-test the code above I would run at least two unit tests. One to test if the file exists, and the other to test that my code does the correct thing when the file does not exist. Because unit testing, IMHO, is only testing my code and not doing integration tests, then it is perfectly fine to mock File so that you can test both branches of this method.
During integration testing you can then test with a real File as your application will be interacting with its surroundings.
Try Mockito:
I do not know why jmockit does not allow you to mock the File class. In Mockito it can be done. Example below.
import java.io.File;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class NewMain {
public static void main(String[] args) {
File f = mock(File.class);
when(f.exists()).thenReturn(true);
System.out.println("f.exists = " + f.exists());
}
}
In our project which has huge sized classes and sub relations we have been developing for months. Also, we have been developing junit test cases.
Automatic unit tests are good in general, but in real life it is not easy as we think. Managing the unit test architecture is easier than creating mock objects and stubs etc. Also, we are testing the dao and service layers.
The problem is that our classes have so many attributes. (I know it's not a good object oriendted design but it's legacy architecture.)
For example; customer class has 58 attributes and it's related to address, marsaccounts and etc. Totally if you want to test this class you have to create inputs, inputs with 90 or more attributes.
Our architecture has many business rule on Customer, so that we have to create more than 50 customer inputs to test every rule, method or flow.
In short, you have to create 4500 (90 x 50) attributes for all, but less for a reliable test (only necessary attributes).
Preparing the test inputs are painful and annoying. Imagine, 2 columns added to Customer object and they store critical values. it seems easy, but refactoring the test inputs are soul-destroying.
How can I manage the test stub and How can I overcome the huge input set ?
Regards.
it looks like you have only two options: make less attributes relevant or learn to easily set-up tests with large number of attributes.
less attributes: refactor. and no one will give you much more detailed help because we don't know your business. try to find smaller groups of attributes that control some logic and test that logic using only those attributes.
easy test set-up: you can use customer builders. by default they create customer with some standard/most common settings and you tune the result however you want like
customer = makeCustomer().withActiveStatus(false).withDebit(3000).build()
Then, when new attribute appears you just have to change makeCustomer() in one place.
you can also use parameterized tests to define a test case as a one liner or load data from spreadsheet which may be easier to maintain.
Often when new attribute appears it's not that it changes everything completely. usually it just adds new behaviour in one spot when that attribute is non-standard. therefore usually it's just adding the default attribute to the builder and a few tests that override that attribute
another way to make tests easy is to do property testing (QuickCheck family) although not always it's easy to use it in business logic
Use mocked customer. Setup only attributes that relevant for rule that you want test.
As mentioned in the other answers, your best bet is to refactor those huge classes towards more fine-grained classes with a single responsibility, but in order to get your legacy code under test it might be worth looking into the test data builder pattern.
In essence, test data builders provide an abstraction that hides away object construction. For example, they can provide default values for all constructor arguments, allowing you to specify the relevant ones in each of your tests (often through a fluent API which make your tests very readable).
new AddressBuilder()
.withName("Sherlock Holmes")
.withStreet("221b Baker Street")
.withCity("London")
.withPostCode("NW1", "3RX")
.build();
In short, you have to create 4500 (90 x 50) attributes for a reliable
test.
No, you don't. You're not thinking about this right. You might need 4500 attributes for an exhaustive test, but not for a reliable test.
Mocking is an entirely legitimate and "real" - you're word - means of testing, and it's the appropriate answer here.
There are several ways to make your code and tests more manageable.
customer class has 58 attributes and it's related to address, marsaccounts and etc.
That's too many attributes for a single class. You should try to refactor the class and break it up into many classes. You mention addresses, if you have many attributes related to different fields of an address, you should refactor those out into an Address object. Finding other similar related fields and pulling them out into their own objects will really help reduce the number of inputs and make it easier to mock in tests.
Preparing the test inputs are painful and annoying. Imagine, 2 columns added to Customer object and they store critical values. it seems easy, but refactoring the test inputs are soul-destroying.
I would suggest making a Customer TestDouble for tests. By default, this TestDouble will be initialized to have all 58+ attributes set to valid values. The TestDouble will also have setters for all these values so they can be changed for each tests.
This way your test code doesn't duplicate setting all the irrelevant fields and makes the test more intent revealing.
For example:
#Test
public void invalidAccountIdShouldThrowBusinessRuleException(){
CustomerTestDouble cust = new CustomerTestDouble();
//only set what matters for this test
cust.setMarsaccount( -1);
BusinessRuleValidator validator = ...
//wrap with whatever you do to check that validator throws Exception
validator.validate(cust);
}
This is very clear that the exception will be thrown because the account id is -1 there's no other clutter.
And if you have to add new fields to Customer later, (or refactor it to use new objects) most of your test code is unchanged. Only the CustomerTestDouble and any directly affected tests need to be changed.
I'm starting to practice TDD in my project, as a background it also contains legacy code. We use Mockito as a mocking framework and follow a Spring MVC approach.
There are times when there's a Service class implemented with many different DAO objects as #Autowired properties. There are simple methods within these services, like for example completeTransaction.
completeTransaction will use many of the DAO objects to complete its responsibilities
Update and save the transaction
Advance a business process
Closing other pending operations
However, in performing those operations, the method requires calls to different DAO to fetch and update the transaction, fetch a business process ID, fetch pending transactions (and save their updates). This means that unit testing this method makes me add many #Mock properties. And I need to set up the mock objects before the test will actually finish for me to test a certain condition.
This seems like a code smell, and to me it almost feels like the test is ensuring the implementation of the code instead of just its contract. Again, without mocking the dependencies, the test case will not run (due to NPE and others).
What is a strategy that I can follow to clean up code like this? (I can't really provide the actual source code on the question though). I'm thinking that one possibility would be to set up a facade class with methods like ("getPendingOperations" and "advanceBusinessProcess"). Then I can mock a single dependency. But then I figure that in all other classes that have situations like this I would need to do the same, and then I'm afraid to end up with a lot of "helper" classes just for the sake of cleaner tests.
Thank you in advanced.
I think you'll want to do two things in general when you find yourself with too many mocks. These are not necessary easy, but you may find them helpful.
1) Try and make your methods and classes smaller. I think Clean Code says there are two rules, that classes should small. And that classes should be smaller then that. This makes some sense because as the units you are testing (methods and classes) get smaller, so will the dependencies. You will of course end up with more tests, but they will have less setup in each test.
2) Look at the Law of Demeter (https://en.wikipedia.org/wiki/Law_of_Demeter). There are a bunch of rules, but basically, you want to avoid long string of property/method calls. objA = objB.propertyA.SomeMethod().propertyC; If you need to mock out all of these objects just to get objA, the you will have a lot of setup. But if you can replace this with objA = objB.newProperty; then you only need to mock objB and it's one property.
Neither of these are silver bullets, but hopefully you can use some of these ideas with your project.
If the unit test is testing the completeTransaction method, then you must mock everything on which it depends. Since you are using Mockito, you can use verify to confirm that the correct mocked methods are called and that they are called in the correct order.
If the unit test is testing something that calls the completeTransaction method, then just mock the completeTransaction method.
If this is your class hierarchy:
class A -> class B -> class C
(where -> is "depends on")
In unit tests for class A, mock only class B.
In unit tests for class B, mock only class C.
I'm writing a little library for movies for myself. It's partly for learning TDD. Now I have a problem I can't solve.
The code is in here https://github.com/hasanen/MovieLibrary/blob/master/movielibrary-core/src/test/java/net/pieceofcode/movielibrary/service/MovieLibraryServiceITC.java
The problem is that when I run the whole class (right click above class name in eclipse), the second test fails because removing doesn't succeed. But when right clicking the method (getMovieGenres_getAllGenresAndRemoveOne_returnsTwoGenreAndIdsAreDifferent) and choosing Run as Junit Test, it works.
I don't necessarily need the fix, but at least some advice on how to find why junit is acting like this.
From the way you explain the problem, the problem appears to be in the setUp class. The setUp class runs before every test case invocation. This is the general sequence.
1- Add three movies.
2- Test if three movies exists.
3- Add three movies
4- remove movie item # 1.
Since sequence 1-4 works, the problem is sequence 3. Either sequence 3 swallows some exception or mutates the underlying object. (may be changes the sequence.) Without knowing how addMovie changes the underlying object, its hard to tell.
Something outside your test class (likely a superclass) is creating movieLibraryService, and it's not being recreated as often as it needs to be for independent testing.
If you add the line
movieLibraryService = new MovieLibraryService();
at the top of your testSetUp() method, this service will be properly reset before the running of each test method, and they will likely work properly.
As it is, I suspect you're getting a failure on the assertions about size, as the size is becoming 6 instead of 3.
Alternatively, you could add a teardown method (annotated with #After) which removes the contents of the movie library so that it always starts empty.
IMHO the problem is your test isn't real unit test but integration one. So while testing your service you're testing all the layers it uses. I recommend yo to use mocks for lower layers dependencies (EasyMock or something) and use integration tests only for your repository layer. This way you can avoid persistence layer influences while testing service layer.