Interdependence of unit tests

Interdependence of unit tests - java

Since recently, I am trying out unit testing to get acquainted with the practice, and to ultimately write better code. I have started on one of my big projects, with a rather large untested code base, but am only unit testing utility classes, that don't have many dependencies and are quite easy to test.
I have read quite a bit of introductory material to unit testing in general, and what often comes up is that unit tests should always test the smallest possible unit of behaviour. That is to say, that whether a test passes or fails should only depend on a very specific piece of code, such as a small method.
However, I am already finding problems to put this idea into practice. Consider for example :
#Test
public void testSomethingWithFoo() {
Foo foo = new Foo(5);
Bar result = foo.bar(); //Suppose this returns new Bar(42) for some reason.
Bar expected = new Bar(42);
Assert.assertEquals(expected, result); // Implicitly calls Bar.equals, which returns true if both Bars' constructor parameters are equal.
}
Clearly, this test depends on two elements : the foo.bar() method, but also the bar.equals(otherBar) method, which is implicitly called by the assertion.
Now, I could write an other test, which asserts that this bar.equals() method works correctly. However, say it fails. Now, my first test also should fail, but for a reason beyond its scope.
My point is that, for this particular example, nothing really is problematic; I could maybe check for equality without using equals at all. However, I feel like this sort of issue will become a real problem with more complex behaviours, where avoiding existing methods because they might not work would involve rewriting large amounts of code just for tests.
How to translate these requirements into code, without making unit tests interdependent ?
Is it even possible ? If not, should I stop bothering about this issue, and assume all methods not under the current test work ?

However, say it fails. Now, my first test also should fail, but for a
reason beyond its scope.
Test isolation is a really important thing, you are right but note also that this has a limitation.
You will necessary fall into some cases where in your unit tests, you will have to rely on other general methods of other objects such as equals/hashCode or still constructors.
This is unavoidable. Sometimes, the coupling is not an issue.
For example, using a constructor to create the object passed to the method under test or a mock value is natural and should really not be avoided !
But in other cases, the coupling with the API of another class is not desirable.
It is for example the case in your sample code as your test relies on an equals() method which the fields tested may change through the time and also be functionally different from fields to watch/assert in the tested method.
In addition, this doesn't make the fields asserted explicit.
In this kind of case, to assert fields values of a returned object by the method under test, you should assert field by field via getter methods.
To avoid writing this repetitive and error prone code, I use a matcher testing library such as AssertJ or Hamcrest (included in JUnit at a time) that provides fluent and flexible way to assert the content of an instance.
For example with AssertJ :
Foo foo = new Foo(5);
Bar actualBar = foo.bar();
Assertions.assertThat(actualBar)
.extracting(Bar::getId) // supposing 42 is stored in id field
.containsExactly(42);
With such a simple example, matcher testing library has no real value. You could directly do :
Foo foo = new Foo(5);
Bar actualBar = foo.bar();
Assert.assertEquals(42, actualBar.getId())
But for cases where you want to check multiple fields (very common case), it is very helpful :
Foo foo = new Foo(5);
Bar actualBar = foo.bar();
Assertions.assertThat(actualBar)
.extracting(Bar::getId, Bar::getName, Bar::getType)
.containsExactly(42, "foo", "foo-type);

I've found it helpful to divide my classes into two main types:
Types which hold data. They represent your data's structure, and contain little to no logic.
Types which hold logic, and hold no state.
The reason for this is that many classes at many layers will inevitably be tied to your data model, so you want the representation of your data model to be rock solid and unlikely to yield any surprises.
If Bar is a data structure, then relying on the behavior of its .equals() method isn't much different than relying on the behavior of int or String's .equals() behavior. You should stop worrying about it.
On the other hand, if Bar is a logic class, you probably shouldn't be relying on its behavior at all when you're trying to test Foo. In fact, Foo shouldn't be creating a Bar at all (unless Foo is a BarFactory, e.g.). Instead, it should be relying on interfaces which can be mocked.
When it comes to testing your Data types (especially in the exceptional cases where the data type really needs to have some logic, like a Uri class, you will of a necessity be testing multiple methods within that class (like Bar's constructor and equals method). But the tests should all be testing that class specifically. Make sure you've got plenty of tests to keep these rock-solid.
When testing a service class (one with logic), you will effectively be assuming that any data types you're dealing with have been tested sufficiently so you're really testing the behavior of the type you're worried about (Foo) and not the other types you happen to be interacting with (Bar).

Related

good practice for testing mocked object

It is good practise to match mock objects widely but verify them precisely.
for example:
Using this:
when(myMock.has(any())).thenReturn(myValue);
Rather than:
when(myMock.has(eq("blah")).thenReturn(myValue);
Along with:
var result = myMethod();
assertThat(result, is(myValue));
Because it is making sure that it is always returned myValue regardless of the has method input.
There was a good explanation for this rule but I can not find it.
something along the lines: match widely and verify precisely.
It would be great if you can advise me about the name of the rule or some reference to it?

The explanation is quite simple: It will make your live easier.
Imagine the case that a caller will not call your method with "blah". In this case you rely on the mocking framework what will be returned, most likely null, zero or false. Your test will then run into a different direction or even fail with a NullpointerException. For other developers it will be hard to understand what went wrong here.
If you match widely, your test will continue as expected, but you should place a verification afterwards that makes the test fail with a clean reason. Developers tend to omit the verification step, wich renders the test useless quite often.
Usually there is no reason to match on a precise parameter value, except for the case when you want your mock to act differently on two values.
Most frameworks provide methods for the method call verification, e.g. Mockito:
#Mock
private Repository repository;
#Test
private void testReadData() {
Mockito.when(repository.findById(any())).thenReturn(yourEntity);
// run your test
Mockito.verify(repository).findById("foo");
}

Verify a temporary objects are created

I'm in a java context and am using Mockito (but I'm not bound to it) for basic mocking needs.
I have code like this
public class AuditInfoSerializer {
[..]
public Map<String, Object> doStuff(Object a) {
doOtherStuff("hello", new TempClass(someField, <someParams>));
doOtherStuff("world", new TempClass(someField, <otherParams>));
return getResult();
}
}
and in a test I want to verify that there are two instances of TempClass created with the correct set of parameters when I call the doStuff method.
Is this possible somehow?

You don't want to verify temporary data on the object under test. You want to mock dependencies and assert the object under test behavior : that is with this input you have this output.
Mock verifying is a trade off for methods to mock that return nothing but only produces side effect.
So use it only as you don't have the choice.
In your unit test, what you want is asserting what the method to test returns that is getResult().
Do that with Assert.assertEquals(...) not with Mockito.verify(...).

For the most part I agree with #davidxxx's point about the mock verifying tradeoff. If you have a setup that allows you to make assertions about an outcome like a map that is created as a result, go for it!
From an API perspective doStuff is a simple straight-forward method: Throw something at it, get something back. The information you are interested in will be contained in the map (this would be your assertion).
There is a lot going on under the hood before doStuff returns something. Many people tend to want to break up encapsulation when testing stuff. They are constantly looking for ways to uncover what is going on behind the curtains. I believe that's totally natural. But of course, it's also an anti pattern. It doesn't matter what tool you (mis)use to break natural boundaries (mocking frameworks, custom reflection, "back doors" in your code base, etc). It is always wrong. As #Michael already pointed out, the call to doOtherStuff is indeed an implementation detail. Take the perspective of client code that makes a call to doStuff. Is it interested in how the map is created? I doubt it. This should also be your testing perspective.
One last thing about using verification in tests. I would like to mitigate the trade off statement. I really don't like the generalization here. Verification is not always the less attractive choice compared to real assertions:
// Valid test without any verifaction
#Test
void method_foo_returns_gibberish (#Mock SomeInput someInput) {
// Maybe this is just to prevent an NPE ...
when(someInput.readStuff()).thenReturn("bla");
assertEquals("gibberish", Foo.foo(someInput));
}
// Test made possible by verification
#Test
void method_foo_is_readonly (#Mock SomeInput someInput) {
Foo.foo(someInput);
verify(someInput.readStuff());
verifyNoMoreInteractions(mockedList);
}
This is just the most obvious example that I could think of. There is a fraction of BDD geniuses who strive to build their whole architecture around verification driven tests! Here is an excellent article by Martin Fowler
When talking about testing, most of the time, there is no black and white. Using mocks and verification means writing different tests.
As always, it's about picking the right tool.

Should I mock all objects that my test object interacts with?

I am struggling to know how to decide what objects my test object interacts with should be mocked.
class MyClass {
private Customer customer;
private Invoice invoice;
private PrintService ps;
private DBAccessService da;
private EmailService em;
..........
}
I can see that the last 3 are to be mocked because they deal with some external systems. What about Customer and Invoice? If not, why?

The general answer to the question "Should I mock all objects that my test object interacts with?" is a resounding no: that is not a rule you should internalize or follow. Mocks are one of many types of test double, and you'll have to use judgment about where to use mocks, where to use other fakes or test doubles, and where to use real collaborators. I do agree with your decision that the last three fields (services) are probably worth mocking, and if it were up to me, I would use real objects for the former two fields (Customer and Invoice).
Here, I would adhere to a guideline, don't mock data objects, which follows from a few observations:
Data objects are often extremely stateful, and mocking frameworks tend to stub state poorly. Mockito doesn't really have a great syntax for "getX returns 15 indefinitely, until you call setX(20), then getX returns 20". Thus, stubbing correctly is often hard.
For data objects that are just fields/getters/setters, there's not much value to verifying that couldn't be done by reading mutable state out of a real implementation. Who cares whether getY was called, as long as the value in the object was read? Who cares how many times setY was called, as long as the correct value ends up in the object? Thus, verifying is often unnecessary.
Data objects are usually written before the objects that consume them, so there's often a working implementation that already exists.
Data objects often have determistic behavior with few external interactions, so there's often little to be gained in improving test stability or reducing test flakiness.
As biziclop mentioned in the question comments,
always remember what the purpose of automated testing is: to find errors in the shortest possible time with the smallest possible effort. Or to turn it around: to build up confidence about the correctness of your code.
In this sense, mocking data objects has a high cost to the readability and correctness of your test, and is also unlikely to provide benefit regarding test correctness or stability. I'd avoid it.

Proper way to compare object in a Unit test

I'm writing unit tests for my grails application, and I realized I don't really know the proper way to assert whether an object is the proper object or not.
For example, given this test:
void testExampleTest() {
mockSession.person = new Person(firstName:'John', lastName:'Doe', middleInitial:'E')
def model = controller.testMethod()
...assertions...
}
and
def testMethod = {
Person currPerson = session.getAttribute("person")
render(view:'view',model:[person:currPerson]
}
how should I make sure that the person object I added to the session is properly being passed in the model? Is it sufficient to use
assertEquals( person,model['person'] )
or because I injected the object myself into the session does it make more sense to use
assertEquals( person.firstName, model['person'].firstName )
assertEquals( person.lastName, model['person'].lastName )
assertequals( person.middleName, model['person'].middleName )
It seems to me that the first way should suffice as long as the object has a properly defined equals method, but I just wanted to see what the conventional way is.
Thanks

Property-by-property comparison needs to be repeated in every test - so it's a good old code duplication, a test smell described in XUnitPatterns. Better have a proper equals().
Of course, you can add an utility method personEquals() or even override Person.equals() in runtime. For mocked class, you will probably have to. I personally stick to shorter code which is just one assertEquals() when possible.

Funny, I and a colleague had a similar discussion today. Our conclusion was that
An advantage of the more laborious attribute-by-attribute comparison is that it reports a specific difference rather than just a "no, they are not equals", and this may be convenient.
Also we did not have control over certain classes, and some of those lacked an equals method.
We intend to investigate whether it's possible to use reflection to implement a comparator, hence removing some of the tedium.

I have found that doing property by property is a little more reliable and gives you a little bit more fine grain control over how something is compared, the down side is it's a little more work to set up and maintain

If equals is defined properly you are right. The problem is, that you might have to first unit test if equals is defined properly (meaning it behaves the way you expect it to).
This might get a little more difficult if you create a mockup for the Person class. In that case you don't care if equals works properly because you only want to check if some attributes are being set/accessed properly. This is why I prefer checking for primitive values if possible and necessary. I find that it makes the tests also more descriptive (although it can become pretty verbose).

In this particular instance, testing the individual properties is only a way for you to identify a specific instance of an object, and it clouds the meaning of the test. What you specifically care about and should assert is that model['person'] is the exact same object as what you initially put in as person:
assertSame(person, model['person'])
Or with Hamcrest, which allows much more expressive assertions overall:
assertThat(model['person'], sameInstance(person))

As you wrote, if the test data has a proper equals method, you can use it. "Proper" here means that it tests the attributes you want to be tested.
I often work with database entities which only compare their ID attribute. With these objects, I need to test each attribute separately to see if they are equal. I wrote a little helper that allows me to write a single assert for many properties, like this:
assertEqualProperties(person, model['person'], "firstName", "lastName", "middleName");
This helper method uses reflection to access the attributes (not directly, I invoke the commons-beans library). In Groovy, there surely is a syntax that does not need explicit reflection. The method reports the first non-equal attribute as a test failure.

In Grails every object is serializable, so you could compare the two using their XML Serializations:
public void compareXML(Object a, Object b)
ByteArrayOutputStream aBaos = new ByteArrayOutputStream();
XMLEncoder aEncoder = new XMLEncoder(aBaos);
aEncoder.writeObject(a);
aEncoder.close();
String xmlA = baos.toString();
ByteArrayOutputStream bBaos = new ByteArrayOutputStream();
XMLEncoder bEncoder = new XMLEncoder(bBaos);
bEncoder.writeObject(b);
bEncoder.close();
String xmlB = bBaos.toString();
assertEquals(xmlA, xmlB);
}
If you're working in eclipse, you'll get a great textual comparison of the two XML strings showing all of the differences.

I use assertSame(). Comparing field by field is way more work than necessary - you mocked the data, so just assert that the mocked values are properly returned.

How to test for equality of complex object graphs?

Say I have a unit test that wants to compare two complex for objects for equality. The objects contains many other deeply nested objects. All of the objects' classes have correctly defined equals() methods.
This isn't difficult:
#Test
public void objectEquality() {
Object o1 = ...
Object o2 = ...
assertEquals(o1, o2);
}
Trouble is, if the objects are not equal, all you get is a fail, with no indication of which part of the object graph didn't match. Debugging this can be painful and frustrating.
My current approach is to make sure everything implements toString(), and then compare for equality like this:
assertEquals(o1.toString(), o2.toString());
This makes it easier to track down test failures, since IDEs like Eclipse have a special visual comparator for displaying string differences in failed tests. Essentially, the object graphs are represented textually, so you can see where the difference is. As long as toString() is well written, it works great.
It's all a bit clumsy, though. Sometimes you want to design toString() for other purposes, like logging, maybe you only want to render some of the objects fields rather than all of them, or maybe toString() isn't defined at all, and so on.
I'm looking for ideas for a better way of comparing complex object graphs. Any thoughts?

The Atlassian Developer Blog had a few articles on this very same subject, and how the Hamcrest library can make debugging this kind of test failure very very simple:
How Hamcrest Can Save Your Soul (part 1)
Hamcrest saves your soul - Now with less suffering! (part 2)
Basically, for an assertion like this:
assertThat(lukesFirstLightsaber, is(equalTo(maceWindusLightsaber)));
Hamcrest will give you back the output like this (in which only the fields that are different are shown):
Expected: is {singleBladed is true, color is PURPLE, hilt is {...}}
but: is {color is GREEN}

What you could do is render each object to XML using XStream, and then use XMLUnit to perform a comparison on the XML. If they differ, then you'll get the contextual information (in the form of an XPath, IIRC) telling you where the objects differ.
e.g. from the XMLUnit doc:
Comparing test xml to control xml [different]
Expected element tag name 'uuid' but was 'localId' -
comparing <uuid...> at /msg[1]/uuid[1] to <localId...> at /msg[1]/localId[1]
Note the XPath indicating the location of the differing elements.
Probably not fast, but that may not be an issue for unit tests.

Because of the way I tend to design complex objects, I have a very easy solution here.
When designing a complex object for which I need to write an equals method (and therefore a hashCode method), I tend to write a string renderer, and use the String class equals and hashCode methods.
The renderer, of course, is not toString: it doesn't really have to be easy for humans to read, and includes all and only the values I need to compare, and by habit I put them in the order which controls the way I'd want them to sort; none of which is necessarily true of the toString method.
Naturally, I cache this rendered string (and the hashCode value as well). It's normally private, but leaving the cached string package-private would let you see it from your unit tests.
Incidentally, this isn't always what I end up with in delivered systems, of course - if performance testing shows that this method is too slow, I'm prepared to replace it, but that's a rare case. So far, it's only happened once, in a system in which mutable objects were being rapidly changed and frequently compared.
The reason I do this is that writing a good hashCode isn't trivial, and requires testing(*), while making use of the one in String avoids the testing.
(* Consider that step 3 in Josh Bloch's recipe for writing a good hashCode method is to test it to make sure that "equal" objects have equal hashCode values, and making sure that you've covered all possible variations are covered isn't trivial in itself. More subtle and even harder to test well is distribution)

The code for this problem exists at http://code.google.com/p/deep-equals/
Use DeepEquals.deepEquals(a, b) to compare two Java objects for semantic equality. This will compare the objects using any custom equals() methods they may have (if they have an equals() method implemented other than Object.equals()). If not, this method will then proceed to compare the objects field by field, recursively. As each field is encountered, it will attempt to use the derived equals() if it exists, otherwise it will continue to recurse further.
This method will work on a cyclic Object graph like this: A->B->C->A. It has cycle detection so ANY two objects can be compared, and it will never enter into an endless loop.
Use DeepEquals.hashCode(obj) to compute a hashCode() for any object. Like deepEquals(), it will attempt to call the hashCode() method if a custom hashCode() method (below Object.hashCode()) is implemented, otherwise it will compute the hashCode field by field, recursively (Deep). Also like deepEquals(), this method will handle Object graphs with cycles. For example, A->B->C->A. In this case, hashCode(A) == hashCode(B) == hashCode(C). DeepEquals.deepHashCode() has cycle detection and therefore will work on ANY object graph.

Unit tests should have well-defined, single thing they test. This means that in the end you should have well-defined, single thing that can be different about those two object. If there are too many things that can differ, I would suggest splitting this test into several smaller tests.

I followed the same track you are on. I also had additionnal troubles:
we can't modify classes (for equals or toString) that we don't own (JDK), arrays etc.
equality is sometimes different in various contexts
For example, tracking entities equality might rely on database ids when available ("same row" concept), rely the equality of some fields (the business key) (for unsaved objects). For Junit assertion, you might want all fields equality.
So I ended up creating objects that run through a graph, doing their job as they go.
There is typically a superclass Crawling object:
crawl through all properties of the objects ; stop at:
enums,
framework classes (if applicable),
at unloaded proxies or distant connections,
at objects already visited (to avoid looping)
at Many-To-One relationship, if they indicate a parent (usually not included in the equals semantic)
...
configurable so that it can stop at some point (stop completely, or stop crawling inside the current property):
when mustStopCurrent() or mustStopCompletely() methods return true,
when encountering some annotations on a getter or a class,
when the current (class, getter) belong to a list of exceptions
...
From that Crawling superclass, subclasses are made for many needs:
For creating a debug string (calling toString as needed, with special cases for Collections and arrays that don't have a nice toString ; handling a size limit, and much more).
For creating several Equalizers (as said before, for Entities using ids, for all fields, or solely based on equals ;). These equalizers often need special cases also (for example for classes outside your control).
Back to the question : These Equalizers could remember the path to the differing values, that would be very useful your JUnit case to understand the difference.
For creating Orderers. For example, saving entities need to be done is a specific order, and efficiency will dictate that saving the same classes together will give a huge boost.
For collecting a set of objects that can be found at various levels in the graph. Looping on the result of the Collector is then very easy.
As a complement, I must say that, except for entities where performance is a real concern, I did choose that technology to implements toString(), hashCode(), equals() and compareTo() on my entities.
For example, if a business key on one or more fields is defined in Hibernate via a #UniqueConstraint on the class, let's pretend that all my entities have a getIdent() property implemented in a common superclass.
My entities superclass has a default implementation of these 4 methods that relies on this knowledge, for example (nulls need to be taken care of):
toString() prints "myClass(key1=value1, key2=value2)"
hashCode() is "value1.hashCode() ^ value2.hashCode()"
equals() is "value1.equals(other.value1) && value2.equals(other.value2)"
compareTo() is combine the comparison of the class, value1 and value2.
For entities where performance is of concern, I simply override these methods to not use reflexion. I can test in regression JUnit tests that the two implementations behave identically.

We use a library called junitx to test the equals contract on all of our "common" objects:
http://www.extreme-java.de/junitx/
The only way I can think of to test the different parts of your equals() method is to break down the information into something more granular. If you are testing a deeply-nested tree of objects, what you are doing is not truly a unit test. You need to test the equals() contract on each individual object in the graph with a separate test case for that type of object. You can use stub objects with a simplistic equals() implementation for the class-typed fields on the object under test.
HTH

I would not use the toString() because as you say, it is usually more useful for creating a nice representation of the object for display or logging purposes.
It sounds to me that your "unit" test is not isolating the unit under test. If, for example, your object graph is A-->B-->C and you are testing A, your unit test for A should not care that the equals() method in C is working. Your unit test for C would make sure it works.
So I would test the following in the test for A's equals() method:
- compare two A objects that have identical B's, in both directions, e.g. a1.equals(a2) and a2.equals(a1).
- compare two A objects that have different B's, in both directions
By doing it this way, with a JUnit assert for each comparison, you will know where the failure is.
Obviously if your class has more children that are part of determining equality, you would need to test many more combinations. What I'm trying to get at though is that your unit test should not care about the behavior of anything beyond the classes it has direct contact with. In my example, that means, you would assume C.equals() works correctly.
One wrinkle may be if you are comparing collections. In that case I would use a utility for comparing collections, such as commons-collections CollectionUtils.isEqualCollection(). Of course, only for collections in your unit under test.

If you're willing to have your tests written in scala you could use matchete. It is a collection of matchers that can be used with JUnit and provide amongst other things the ability to compare objects graphs:
case class Person(name: String, age: Int, address: Address)
case class Address(street: String)
Person("john",12, Address("rue de la paix")) must_== Person("john",12,Address("rue du bourg"))
Will produce the following error message
org.junit.ComparisonFailure: Person(john,12,Address(street)) is not equal to Person(john,12,Address(different street))
Got : address.street = 'rue de la paix'
Expected : address.street = 'rue du bourg'
As you can see here I've been using case classes, which are recognized by matchete in order to dive into the object graph.
This is done through a type-class called Diffable. I'm not going to discuss type-classes here, so let's say that it is the corner stone for this mechanism, which compare 2 instances of a given type. Types that are not case-classes (so basically all types in Java) get a default Diffable that uses equals. This isn't very useful, unless you provide a Diffable for your particular type:
// your java object
public class Person {
public String name;
public Address address;
}
// you scala test code
implicit val personDiffable : Diffable[Person] = Diffable.forFields(_.name,_.address)
// there you go you can now compare two person exactly the way you did it
// with the case classes
So we've seen that matchete works well with a java code base. As a matter of fact I've been using matchete at my last job on a large Java project.
Disclaimer : i'm the matchete author :)

Develop Reference

Java is a programming language and computing platform first released by Sun Microsystems in 1995.