It's a Java (using JUnit) enterprise Web application with no mock objects pre-built, and it would require a vast amount of time not estimated to create them. Is there a testing paradigm that would give me "some" test coverage, but not total coverage?
Have you tried a dynamic mocking framework such as EasyMock? It does not require you to "create" a Mock object in that you would have to write the entire class - you specify the behavior you want within the test itself.
An example of a class that uses a UserService to find details about a User in order to log someone in:
//Tests what happens when a username is found in the backend
public void testLoginSuccessful() {
UserService mockUserService = EasyMock.createMock(UserService.class);
EasyMock.expect(mockUserService.getUser("aUsername")).andReturn(new User(...));
EasyMock.replay(mockUserService);
classUnderTest.setUserService(mockUserService);
boolean isLoggedIn = classUnderTest.login("username");
assertTrue(isLoggedIn);
}
//Tests what happens when the user does not exist
public void testLoginFailure() {
UserService mockUserService = EasyMock.createMock(UserService.class);
EasyMock.expect(mockUserService.getUser("aUsername")).andThrow(new UserNotFoundException());
EasyMock.replay(mockUserService);
classUnderTest.setUserService(mockUserService);
boolean isLoggedIn = classUnderTest.login("username");
assertFalse(isLoggedIn);
}
(1) Alternatives to unit-testing (and mocks) include integration testing (with dbUnit) and FIT testing. For more, see my answer here.
(2) The mocking framework Mockito is outstanding. You wouldn't have to "pre-build" any mocks. It is relatively easy to introduce into a project.
I would echo what others are saying about EasyMock. However, if you have a codebase where you need to mock things like static method calls, final classes or methods, etc., then give JMockit a look.
Well, one easy, if not the easiest, way to get an high level of code coverage is to write the code test-first, following Test-Driven Development (TDD). Now that the code exists, without unit tests, it can be deemed as legacy code.
You could either write end-to-end test, external to your application, those won't be unit tests, but they can be written without resorting to any kind of mock. Or you could write unit tests that span over multiple classes, and only mock the classes that gets in the way of your unit tests.
Do you have real world data you can import into your testbed to use as your 'mock objects' that would be quick
I think the opposite is hard - to find a testing methodology that gives you total coverage, if at all possible in most cases.
You should give EasyMock a try.
Related
I've been struggling with understanding the minefield that is classes supported by the jmockit Mocking API. I found what I thought was a bug with File, but the issue was simply closed saying "Don't mock a File or Path" which no explanation. Can someone on here help me understand why certain classes should not be mocked and how I'm supposed to work around that. I'm trying to contribute to the library with bug reports, but each one I file just leaves me more confused. Pardon my ignorance, but if anyone can point me to the rationale for forbidden mocks etc. I'd greatly appreciate it.
I'm not certain that there's a fool proof list of rules. There's always a certain degree of knowledge and taste in these matters. Dogma is usually not a good idea.
I voted to close this question because I think a lot of it is opinion.
But with that said, I'll make an attempt.
Unit tests should be about testing individual classes, not interactions between classes. Those are called integration tests.
If your object is calling out to other remote objects, like services, you should mock those to return the data needed for your test. The idea is that services and their clients should also be tested individually in their own unit tests. You need not repeat those when testing a class that depends on them.
One exception to this rule, in my opinion, are data access objects. There is no sense in testing one of those without connecting to a remote database. Your test needs to prove the proper operation of your code. That requires a database connection in the case of data access objects. These should be written to be transactional: seed the database, perform the test, and reverse the actions of the test. The database should be in the same state when you're done.
Once your data access objects are certified as working correctly, all clients that use them should mock them. No need to re-test.
You should not be mocking classes in the JVM.
You asked for a why about File or Stream in particular - here's one. Take it or ignore it.
You don't have to test JVM classes because Sun/Oracle have already done that. You know they work. You want your class to use those classes because a failing test will expose the fact that the necessary file isn't available. A mock won't tell me that I've neglected to put a required file in my CLASSPATH. I want to find out during testing, not in production.
Another reason is that unit tests are also documentation. It's a live demonstration for others to show how to properly use your class.
In unit test you have to test that your code is doing the right thing. You can mock out any external pieces of code that is not the direct code being tested. This is a strict definition of unit test and it assumes there is another form of testing called integration testing that will be done later on. In integration testing you test how your code interacts with external elements, like a DB or another web service, or the network, or the hard drive.
If I have a piece of code that interacts with an object, like a File, and my code does 3 things to that file, then in my unit test I am going to test that my code has done those three things.
For example:
public void processFile(File f) {
if (f.exists()) {
//perform some tasks
} else {
//perform some other tasks
}
}
To properly unit-test the code above I would run at least two unit tests. One to test if the file exists, and the other to test that my code does the correct thing when the file does not exist. Because unit testing, IMHO, is only testing my code and not doing integration tests, then it is perfectly fine to mock File so that you can test both branches of this method.
During integration testing you can then test with a real File as your application will be interacting with its surroundings.
Try Mockito:
I do not know why jmockit does not allow you to mock the File class. In Mockito it can be done. Example below.
import java.io.File;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class NewMain {
public static void main(String[] args) {
File f = mock(File.class);
when(f.exists()).thenReturn(true);
System.out.println("f.exists = " + f.exists());
}
}
I'm starting to practice TDD in my project, as a background it also contains legacy code. We use Mockito as a mocking framework and follow a Spring MVC approach.
There are times when there's a Service class implemented with many different DAO objects as #Autowired properties. There are simple methods within these services, like for example completeTransaction.
completeTransaction will use many of the DAO objects to complete its responsibilities
Update and save the transaction
Advance a business process
Closing other pending operations
However, in performing those operations, the method requires calls to different DAO to fetch and update the transaction, fetch a business process ID, fetch pending transactions (and save their updates). This means that unit testing this method makes me add many #Mock properties. And I need to set up the mock objects before the test will actually finish for me to test a certain condition.
This seems like a code smell, and to me it almost feels like the test is ensuring the implementation of the code instead of just its contract. Again, without mocking the dependencies, the test case will not run (due to NPE and others).
What is a strategy that I can follow to clean up code like this? (I can't really provide the actual source code on the question though). I'm thinking that one possibility would be to set up a facade class with methods like ("getPendingOperations" and "advanceBusinessProcess"). Then I can mock a single dependency. But then I figure that in all other classes that have situations like this I would need to do the same, and then I'm afraid to end up with a lot of "helper" classes just for the sake of cleaner tests.
Thank you in advanced.
I think you'll want to do two things in general when you find yourself with too many mocks. These are not necessary easy, but you may find them helpful.
1) Try and make your methods and classes smaller. I think Clean Code says there are two rules, that classes should small. And that classes should be smaller then that. This makes some sense because as the units you are testing (methods and classes) get smaller, so will the dependencies. You will of course end up with more tests, but they will have less setup in each test.
2) Look at the Law of Demeter (https://en.wikipedia.org/wiki/Law_of_Demeter). There are a bunch of rules, but basically, you want to avoid long string of property/method calls. objA = objB.propertyA.SomeMethod().propertyC; If you need to mock out all of these objects just to get objA, the you will have a lot of setup. But if you can replace this with objA = objB.newProperty; then you only need to mock objB and it's one property.
Neither of these are silver bullets, but hopefully you can use some of these ideas with your project.
If the unit test is testing the completeTransaction method, then you must mock everything on which it depends. Since you are using Mockito, you can use verify to confirm that the correct mocked methods are called and that they are called in the correct order.
If the unit test is testing something that calls the completeTransaction method, then just mock the completeTransaction method.
If this is your class hierarchy:
class A -> class B -> class C
(where -> is "depends on")
In unit tests for class A, mock only class B.
In unit tests for class B, mock only class C.
I am working on legacy code and writing some junit tests (I know, wrong order, never mind) using jmock (also wasn't my choice, nothing I can change about that) and I have class which does some elaborate logging, messing with Strings in general. We are using log4j for logging and I want to test those logged messages. I thought about mocking Logger class, but I don't know how to do it.
As usually we have Logger done like this:
private static final Logger LOG = Logger.getLogger(SomeClass.class);
Does anyone have any idea how to mock method .getLogger(class) or any other idea how to check what exactly has been logged?
You can write own appender and redirect all output to it.
If you really think you need to do this, then you need to take a look at PowerMock. More specifically, it's ability to mock static methods. PowerMock integrates with EasyMock and Mockito, but some hunting about might result in you finding a JMock integration too if you have to stick with that.
Having said that, I think that setting up your test framework so that it logs nicely without affecting your tests, and ensuring your tests do not depend upon what gets logged is a better approach. I once had to maintain some unit tests that checked what had been logged, and they were the most brittle and useless unit tests I have ever seen. I rewrote them as soon as I had the time available to do it.
Check this similar question:
How to mock with static methods?
And by the way, it is easier to search for an existing qusetion to your problem than to post a question and wait for answers.
The easiest way I have found is using the mock log4j objects in JMockit.
You just need to the add the annotation
#UsingMocksAndStubs(Log4jMocks.class)
to your test class and all code touched by the tester class will be using a mock Logger object.
See this
But this wont log the messages. You can get away from the hassle of dealing with static objects with this.
In following code the issue is, that I cannot test dao.add() without using dao.list().size() and vice versa.
Is this approach normal or incorrect? If incorrect, how can it be improved?
public class ItemDaoTest {
// dao to test
#Autowired private ItemDao dao;
#Test
public void testAdd() {
// issue -> testing ADD but using LIST
int oldSize = dao.list().size();
dao.add(new Item("stuff"));
assertTrue (oldSize < dao.list().size());
}
#Test
public void testFind() {
// issue -> testing FIND but using ADD
Item item = new Item("stuff")
dao.add(item);
assertEquals(item, dao.find(item.getId()));
}
}
I think your test are valid integration tests as stated above, but I would use Add to aid in the testing of of Find and vice verse..
At some level you have to allow yourself to place trust in your lowest level of integration to your external dependency. I realize there is a dependency to other methods in your tests, but I find that Add and Find methods are "low level" methods that are very easy to verify.
They essentially test each other as they are basically inverse methods.
Add can easily build preconditions for find
Find can verify that an add was successful.
I can't think of a scenario where a failure in either wouldn't be caught by your test
Your testAdd method has a problem: it depends on the assumption that ItemDao.list functions properly, and yet ItemDao is the Class that you're testing. Unit tests are meant to be independent, so a better approach is use plain JDBC -as #Amir said- to verify if the record was introduced in the database.
If you're using Spring, you can relay on AbstractTransactionalDataSourceSpringContextTests to access JDBCTemplate facilities and assure a rollback after the test was executed.
I use direct JDBC (using Spring's JdbcTemplate) to test the DAO methods. I mean I call the DAO methods (which are Hibernate base), and then confirm them using JDBC direct SQL calls.
The smallest unit under test for class-based unit testing is a class.
To see why, consider that you could, in theory, test each method of the class in isolation from all other methods by bypassing, stubbing or mocking them. Some tools may not support that; this is theory not practice, assume they do.
Even so, doing things that way would be a bad idea. The specification of an individual function by itself will vary between vaguely meaningless and verbosely incomprehensible. Only in the pattern of interaction between different functions will there exist a specification simpler than the code that you can profitably use to test it.
If you add an item and the number of items reported increases, things are working. If there is some way things could not be working, but nevertheless all the patterns of interaction hold, then you are missing some needed test.
Assume the following setup:
interface Entity {}
interface Context {
Result add(Entity entity);
}
interface Result {
Context newContext();
SpecificResult specificResult();
}
class Runner {
SpecificResult actOn(Entity entity, Context context) {
return context.add(entity).specificResult();
}
}
I want to see that the actOn method simply adds the entity to the context and returns the specificResult. The way I'm testing this right now is the following (using Mockito)
#Test
public void testActOn() {
Entity entity = mock(Entity.class);
Context context = mock(Context.class);
Result result = mock(Result.class);
SpecificResult specificResult = mock(SpecificResult.class);
when(context.add(entity)).thenReturn(result);
when(result.specificResult()).thenReturn(specificResult);
Assert.assertTrue(new Runner().actOn(entity,context) == specificResult);
}
However this seems horribly white box, with mocks returning mocks. What am I doing wrong, and does anybody have a good "best practices" text they can point me to?
Since people requested more context, the original problem is an abstraction of a DFS, in which the Context collects the graph elements and calculates results, which are collated and returned. The actOn is actually the action at the leaves.
It depends of what and how much you want your code to be tested. As you mentionned the tdd tag, I suppose you wrote your test contracts before any actual production code.
So in your contract what do you want to test on the actOn method:
That it returns a SpecificResult given both a Context and an Entity
That add(), specificResult() interactions happen on respectively the Context and the Entity
That the SpecificResult is the same instance returned by the Result
etc.
Depending on what you want to be tested you will write the corresponding tests. You might want to consider relaxing your testing approach if this section of code is not critical. And the opposite if this section can trigger the end of the world as we know it.
Generally speaking whitebox tests are brittle, usually verbose and not expressive, and difficult to refactor. But they are well suited for critical sections that are not supposed to change a lot and by neophytes.
In your case having a mock that returns a mock does look like a whitebox test. But then again if you want to ensure this behavior in the production code this is ok.
Mockito can help you with deep stubs.
Context context = mock(Context.class, RETURNS_DEEP_STUBS);
given(context.add(any(Entity.class)).specificResult()).willReturn(someSpecificResult);
But don't get used to it as it is usually considered bad practice and a test smell.
Other remarks :
Your test method name is not precise enough testActOn does tell the reader what behavior your are testing. Usually tdd practitioners replace the name of the method by a contract sentence like returns_a_SpecificResult_given_both_a_Context_and_an_Entity which is clearly more readable and give the practitioner the scope of what is being tested.
You are creating mock instances in the test with Mockito.mock() syntax, if you have several tests like that I would recommend you to use a MockitoJUnitRunner with the #Mock annotations, this will unclutter a bit your code, and allow the reader to better see what's going on in this particular test.
Use the BDD (Behavior Driven Dev) or the AAA (Arrange Act Assert) approach.
For example:
#Test public void invoke_add_then_specificResult_on_call_actOn() {
// given
... prepare the stubs, the object values here
// when
... call your production code
// then
... assertions and verifications there
}
All in all, as Eric Evans told me Context is king, you shall take decisions with this context in mind. But you really should stick to best practice as much as possible.
There's many reading on test here and there, Martin Fowler has very good articles on this matter, James Carr compiled a list of test anti-patterns, there's also many reading on using well the mocks (for example the don't mock types you don't own mojo), Nat Pryce is the co-author of Growing Object Oriented Software Guided by Tests which is in my opinion a must read, plus you have google ;)
Consider using fakes instead of mocks. It's not really clear what the classes in question are meant to to, but if you can build a simple in-memory (not thread-safe, not persistent etc) implementation of both interfaces, you can use that for flexible testing without the brittleness that sometimes comes from mocking.
I like to use names beginning mock for all my mock objects. Also, I would replace
when(result.specificResult()).thenReturn(specificResult);
Assert.assertTrue(new Runner().actOn(entity,context) == specificResult);
with
Runner toTest = new Runner();
toTest.actOn( mockEntity, mockContext );
verify( mockResult ).specificResult();
because all you're trying to assert is that specificResult() gets run on the right mock object. Whereas your original assert doesn't make it quite so clear what is being asserted. So you don't actually need a mock for SpecificResult. That cuts you down to just one when call, which seems to me to be about right for this kind of test.
But yes, this does seem frightfully white box. Is Runner a public class, or some hidden implementation detail of a higher level process? If it's the latter, then you probably want to write tests around the behaviour at the higher level; rather than probing implementation details.
Not knowing much about the context of the code, I would suggest that Context and Result are likely simple data objects with very little behavior. You could use a Fake as suggested in another answer or, if you have access to the implementations of those interfaces and construction is simple, I'd just use the real objects in lieu of Fakes or Mocks.
Although the context would provide more information, I don't see any problems with your testing methodology myself. The whole point of mock objects is to verify calling behavior without having to instantiate the implementations. Creating stub objects or using actual implementing classes just seems unnecessary to me.
However this seems horribly white box, with mocks returning mocks.
This may be more about the class design than the testing. If that is the way the Runner class works with the external interfaces then I don't see any problem with having the test simulate that behavior.
First off, since nobody's mentioned it, Mockito supports chaining so you can just do:
when(context.add(entity).specificResult()).thenReturn(specificResult);
(and see Brice's comment for how to do enable this; sorry I missed it out!)
Secondly, it comes with a warning saying "Don't do this except for legacy code." You're right about the mock-returning-mock being a bit strange. It's OK to do white-box mocking generally because you're really saying, "My class ought to collaborate with a helper like <this>", but in this case it's collaborating across two different classes, coupling them together.
It's not clear why the Runner needs to get the SpecificResult, as opposed to whatever other result comes out of context.add(entity), so I'm going to make a guess: the Result contains a result with some messages or other information and you just want to know whether it's a success or failure.
That's like me saying, "Don't tell me all about my shopping order, just tell me that I made it successfully!" The Runner shouldn't know that you only want that specific result; it should just return everything that came out, the same way that Amazon shows you your total, postage and all the things you bought, even if you've shopped there lots and are perfectly aware of what you're getting.
If some classes regularly use your Runner just to get a specific result while others require more feedback then I'd make two methods to do it, maybe called something like add and addWithFeedback, the same way that Amazon let you do one-click shopping by a different route.
However, be pragmatic. If it's readable the way you've done it and everyone understands it, use Mockito to chain them and call it a day. You can change it later if you have need.