Related
I'm working on testing a method that is of the data type int. It accepts 2 ints as its parameters, and return which is the larger number. How do I properly call this from my test package? Here is my code so far:
public void testFindMaxNum()
{
assertEquals("findMaxNum(FAILS)", ProgrammingProject4.findMaxNum(1,2),1);
assertEquals("findMaxNum(PASS)", ProgrammingProject4.findMaxNum(1,2),;
}
Without the code that this is calling (findMaxNum) it is hard to be completely accurate in my answer. However, it looks like you are missing some annotations from your Test class.
I am going to assume that the signature of your method findMaxNum looks like the following:
public int findMaxNum(int i, int j)
Based on this your test code should look something like the following:
import static org.junit.Assert.assertEquals;
import org.junit.Test;
public class ProgrammingProject4Test {
#Test
public void testFindMaxNumFirst() {
// Arrange
ProgrammingProject4 programmingProject4 = new ProgrammingProject4();
// Act and Assert
assertEquals("First number should be bigger than second",2, programmingProject4.findMaxNum(1,2));
}
#Test
public void testFindMaxNumSecond() {
// Arrange
ProgrammingProject4 programmingProject4 = new ProgrammingProject4();
// Act and Assert
assertEquals("Second number should be bigger,2, programmingProject4.findMaxNum(2,1));
}
}
Couple of things to note from your code:
You are missing the annotations for the testing framework e.g. #Test. This means your build tool won't be able to find and run the tests.
You have the expected result after the result from the method. In JUnit asserts they should be the other way (as per my example).
Your code runs two tests in one method. This is bad practice because it makes debugging the tests harder. Also if the first assert fails the second one won't be run. Therefore splitting the two tests into two different methods is better.
You can take a look at Diffblue's playground (full disclosure I work for Diffblue), this is a tool that will write unit tests for a small sample piece of Java code. https://playground.diffblue.com
Hope this all makes sense. There are quite a few points in the answer. Feel free to comment if anything needs clarifying.
I have 2 questions:
1) In JUnit you shouldn't test or mock private methods. But how do I deal with, when they are getting called inside a public method. Let's assume I have following setup:
public void method(String value){
if(value.contains("something")){
doSomethingToString(value);
}
else{
//do something else
}
}
private void doSomethingToString(String value){
Object obj = service.getObject(); //service is mocked in my test-class
//do something with the obj and value
}
I am doing a whitebox test, so I know the methods and what's going on. Now I want to test the public method method(String value). When I now only consider what happens in there, I would get into trouble, since I need to influence what service.getObject() in my private method returns. Is it OK, when I just go on, like I would, meaning using doReturn(objectICreatedInMyTestClass).when(service.getObject()) or do I need to find another way?
2) Methods which have more than one conditions. For example:
public void method(String value){
if(value.contains("something")){
Object obj = service.getObj(value);
}
else{
//do something else
}
if(obj.getAddress == null){
//do something
}
else{
//do something else
}
if (obj.getName == "Name") {
// do something
}
else
{
// do something else
}
}
How many times do I need to test this method? Only twice, where once all conditions return true, and second, where they all return false? Or is it advised to test every possible scenario? This would mean test with condition 1 = true, condition 2 = false, condition 3=false, and then condition 1 = true, condition 2 = true, condition 3 = false and so on (= 8 possibilites).
1) Should I test private methods on their own, outside of the public methods that call them?
Typically I've always followed the assumption that if the only way your code will access that method is by going through another, then that's how you should test it. My experience with testing systems has lead me to believe that this is generally the way it's done in 'the real world' as well.
If we take your example, we can assume that our first step is to write tests to thoroughly test our main method. As part of this we should be testing scenarios that include properly exercising all the conditions we could expect it to face. This will include at least a subset of the scenarios that your private method will face.
Your private method may be used by multiple (possibly wildly different) methods, so its space of possible inputs and outputs may be greater than any single public method that uses it. If you thoroughly test the public methods that use them however, you should be in the situation where you are testing all of the possible scenarios that that private method will encounter.
Because of this you shouldn't need to write tests specifically for the private method. There may be other scenarios where it is unavoidable to try and test private methods, or private classes. Usually I would argue this is because the code is simply written in a way that makes it hard/impossible and could be rewritten to make it friendlier to tests (And therefore friendlier to being updated/refactored at a later date).
2) Should all of those combinations be tested?
This depends on what is happening in the example. There are two different scenarios to consider
a) Neither of these branches have anything to do with each other. That is, what ever happens in the first set of branches will have no way of impacting the logic of what happens in the second branch.
b) Some possible implications of running any logic in either of the first set of branches would result in a different result in the logic of the code in one or more of the second branches.
This will be down to your reading and understanding what is happening in the code, so your example isn't enough to point to one or the other way of doing things.
While unit testing some methods, there can be some scenarios where value of some parameters do not matter and can be any value.
For example in this piece of code:
public void method(String arg1, String arg2, int arg3){
if(arg1 == null) throw new NullPointerException("arg1 is null");
//some other code
}
unit testing the behavior that when arg1 is null then NPE must be thrown, the values of other arguments do not matter, they can be any value or be null.
So I wanted to document the fact that the values do not matter for the method under test.
I thought of following options:
Option 1: Define constants of ANY_XXX
I thought of explicitly creating constants ANY_STRING and ANY_INT, which contain a fixed value which documents that it can be any value and the method under test does not care about the actual value.
I can put all these constants in a single class called Any and reuse them across all test classes.
Option 2: Random values for ANY_XXX
This option seems a bit hacky to me as I have read somewhere that randomness should not be brought into test cases. But in this scenario this randomness will not be visible as the parameters will not create any side effect.
Which approach would be more suitable for better, readable tests?
UPDATE:
While I can use ANY_XXX approach by defining constants in Any class, but I am also thinking of generating ANY_XXX values with some constraints such as
Any.anyInteger().nonnegative();
Any.anyInteger().negative();
Any.anyString().thatStartsWith("ab");
I am thinking that maybe Hamcrest Matchers can be used for creating this chaining. But I am not sure if this approach is a good one. Similar methods for anyObject() are already provided by Mockito but those only work on Mocks and spies and not on normal objects. I want to achieve the same for normal objects for more readable tests.
Why I want to do this?
Suppose I have a class
class MyObject{
public MyObject(int param1, Object param2){
if(param1 < 0) throw new IllegalArgumentException();
if(param2 == null) throw new NullPointerException();
}
}
And now while writing tests for constructor
class MyObjectTest{
#Test(expected=NullPointerException.class)
public void testConstructor_ShouldThrowNullpointer_IfSecondParamIsNull(){
//emphasizing the fact that value of first parameter has no relationship with result, for better test readability
new MyObject(Any.anyInteger().nonnegative(), null);
}
}
I see both og them quite a lot
Personally I disagree that randomness should not be brought into tests. Using randomness to some degree should make your tests more robust, but not necessarily easier to read
If you go for the first approach I would not create a constants class, but rather pass the values (or nulls) directly, since then you see what you pass in without the need to have a look in another class - which should make your tests more readable. You can also easily modify your tests later if you need the other parameters later on
My preference is to build up a utility class of constants along with methods to help with the creation of the constant values for tests, e.g.:
public final class Values {
public static final int ANY_INT = randomInt(Integer.MIN_VALUE, Integer.MAX_VALUE);
public static final int ANY_POSITIVE_INT = randomInt(0, Integer.MAX_VALUE);
public static final String ANY_ISBN = randomIsbn();
// etc...
public static int randomInt(int min, int max) { /* omitted */ }
public static String randomIsbn() { /* omitted */ }
// etc...
}
Then I would use static imports to pull the constants and methods I needed for a particular test class.
I use the ANY_ constants only in situations where I do not care about the value, I find that they can make the intent of the test clearer, for example:
// when
service.fooBar(ANY_INT, ANY_INT, ANY_INT, ANY_INT, 5);
It's clear that the value 5 is of some significance - although it would be better as a local variable.
The utility methods can be used for adhoc generation of values when setting up tests, e.g.:
// given
final String isbn1 = randomIsbn();
final String isbn2 = randomIsbn();
final Book[] books = { new Book(isbn1), new Book(isbn2) };
// when
bookRepository.store(books);
Again, this can help to keep the test classes concerned about the tests themselves and less about data set up.
In addition to this I have also used a similar approach from domain objects. When you combine the two approaches it can be quite powerful. e.g.:
public final class Domain {
public static Book book() {
return new Book(randomIsbn());
}
// etc...
}
I've faced the same problem when i've started to write unit tests for my project and had to deal with numerous of arrays, lists, integer inputs, strings etc.
So I decided to use QuickCheck and create a generator util class.
Using Generators in this library, you can generate primitive data types and String easily.
For example, when you want to Generate an integer; simply use IntegerGenerator class.You can define maximum and minimum values in the constructor of the generator.You can also use CombinedGeneratorSamples class to generate data structures like lists, maps and arrays.
Another feature of this library is implementing Generator interface for custom class generators.
You're overthinking and creating unnecessary barriers for your project :
if you want to document your method, do it with words! that's why the Javadoc is here for
if you want to test your method with "any positive int" then just call it with a couple different positive ints. In your case, ANY does not mean testing every possible integer value
if you want to test your method with "a string that starts with ab", call it with "abcd", then "abefgh" and just add a comment on the test method !
Sometimes we are so caught with frameworks and good practices that it takes common sense away.
In the end : most readable = simplest
How about using a caller method for the actual method.
//This is the actual method that needs to be tested
public void theMethod(String arg1, String arg2, int arg3, float arg4 ){
}
Create a caller method that calls the method with the required parameters and default(or null) values for the rest of the params and run your test case on this caller method
//The caller method
#Test
public void invokeTheMethod(String param1){
theMethod(param1, "", 0, 0.0F); //Pass in some default values or even null
}
Although you will have to be pretty sure that passing default values on theMethod(...) for the other parameters wont cause any NPE.
i see 3 options:
never pass nulls, forbid your team passing nulls. nulls are evil. passing null should be an exception, not a rule
simply use annotation in production code: #NotNull or sth like that. if u use lombok, this annotation will also do the actual validation
and if u really have to do it in tests then simply create a test with proper name:
static final String ANY_STRING = "whatever";
#Test
public void should_throw_NPE_when_first_parameter_is_null() {
object.method(null, ANY_STRING, ANY_STRING); //use catch-exception or junit's expected
}
If you're willing to give JUnitParams' framework a go, you could parametrize your tests specifying meaningful names to your parameters:
#Test
#Parameters({
"17, M",
"2212312, M" })
public void shouldCreateMalePerson(int ageIsNotRelevant, String sex) throws Exception {
assertTrue(new Person(ageIsNotRelevant, sex).isMale());
}
I'm always in favor of the constants approach. The reason is that I believe it gets more readable than chaining several matchers.
Instead of your example:
class MyObjectTest{
#Test(expected=NullPointerException.class)
public void testConstructor_ShouldThrowNullpointer_IfSecondParamIsNull(){
new MyObject(Any.anyInteger().nonnegative(), null);
}
}
I would d:
class MyObjectTest{
private static final int SOME_NON_NEGATIVE_INTEGER = 5;
#Test(expected=NullPointerException.class)
public void testConstructor_ShouldThrowNullpointer_IfSecondParamIsNull(){
new MyObject(SOME_NON_NEGATIVE_INTEGER, null);
}
}
Also, I prefer the use of 'SOME' over 'ANY', but that's also a matter of personal taste.
If you're considering testing the constructor with a number of different variants as you mentioned (nonNegative(), negative(), thatStartsWith(), etc.), I would that instead you write parameterized tests. I recommend JUnitParams for that, here's how I'd do it:
#RunWith(JUnitParamRunner.class)
class MyObjectTest {
#Test(expected = NullPointerException.class)
#Parameters({"-4000", "-1", "0", "1", "5", "10000"})
public void testConstructor_ShouldThrowNullpointer_IfSecondParamIsNull(int i){
new MyObject(i, null);
}
...
}
I suggest you go with constant values for those parameters which may be arbitrary. Adding randomness makes your test runs not repeatable. Even if parameter values "don't matter" here, actually the only "interesting" case is when a test fails, and with random behavior added in, you might not be able to reproduce the error easily. Also, simpler solutions are often better, and easier to maintain: using a constant is certainly simpler than using random numbers.
Of course if you go with constant values, you could put these values in static final fields, but you could also put them in methods, with names such as arbitraryInt() (returning e.g. 0) and so on. I find the syntax with methods cleaner than with constants, as it resembles Mockito's any() matchers. It also allows you to replace the behavior more easily in case you need to add more complexity later on.
In case you want to indicate that a parameter doesn't matter and the parameter is an object (not primitive type), you can also pass empty mocks, like so: someMethod(null, mock(MyClass.class)). This conveys to a person reading the code that the second parameter can be "anything", since a newly created mock has only very basic behavior. It also doesn't force you to create your own methods for returning "arbitrary" values. The downside is it doesn't work for primitive types or for classes which can't be mocked, e.g. final classes like String.
Ok.... I see a big Problem with you approach!
The other value doesn't matter? Who guarantees this? The Writer of the Test, the writer of the Code? What if you have a Method, which throws some unrelated Exception if the first Parameter is exactly 1000000 even if the second parameter is NULL ?
You have to formulate your Test-Cases: What is the Test-Specification... What do you want to proof? Is it:
In some cases if the first parameter is some arbitrary value and the second is null, this method should throw a NullPointerException
For any possible first Input value, if the second value is NULL the method should always throw a NullPointerException
If you want to test the first case - your approach is ok. Use a constant, a random value, a Builder... whatever you like.
But if your specification actually requires the 2nd condition all of your presented solutions are not up for the task, since they only test some arbitrary value. A good test should still be valid if the programmer changes some code in the method. This means the right way to test this method would be a whole series of Testcases, testing all corner-cases as with all other methods. So each critical value which can lead to a different execution-path should be checked - or you need a testsuite which checks for code-path completeness...
Otherwise your test is just bogus and there to look pretty...
I have a Parametrized test class with bunch of unit tests that generally control the creation of custom email messages. Right now class has a lot of test which depend on factor(s) used in parametrized class, the flow of the tests is the same for every test. The example of a test:
#Test
public void testRecipientsCount() {
assertEquals(3, recipientsCount);
}
I had to add extra funcionality to my email class that adds some extra internal emails to the list of recipients, and that only happens for some of the cases and that leads to my problem.
Lets say I want to assert the amount of messages created. For the old test it was the same for each case, but now its different depending on cases. The most intuitive way for me was to add if statements:
#Test
public void testRecipientsCount(){
if(something) {
assertEquals(3, recipientsCount);
}
else {
assertEquals(4, recipientsCount);
}
}
...
but my more experienced co-worker says we should avoid ifs in test classes (and I kinda agree on that).
I thought that splitting test on two test classess may work, but that would lead to redundant code in both classes (I still have to check if non-iternal messages were created, their size, content etc.), and a few lines added for one of them.
My question is: how do I do this so I don't use if's or loads of redundant code (not using parametrized class would produce even more redundant code)?
In my opinion a Junit should be read like a protocol.
That means you can write redundant code to make the test case better readable.
Write a testcase for each possible if-statement in your business logic as well as the negative cases. Thats the only way to get a 100% test coverage.
I use the structure:
- testdata preparation
- executing logic
- check results
- clear data
Furthermore you should write complex asserts of big objects in own abstract classes:
abstract class YourBusinessObjectAssert{
public static void assertYourBussinessObjectIsValid(YourBusinessObject pYourBusinessObject, Collection<YourBusinessObject> pAllYourBusinessObject) {
for (YourBusinessObject lYourBusinessObject : pAllYourBusinessObject) {
if (lYourBusinessObject.isTechnicalEqual(pYourBusinessObject)) {
return;
}
}
assertFalse("Could not find requested YourBusinessObject in List<YourBusinessObject>!", true);
}
}
it will reduce the complexity of your code and you're making it available to other developers.
A unit test should, in my opinion, test only one thing if possible. As such I'd say that if you need an if statement then you probably need more than one unit test - one for each block in the if/else code.
If possible I'd say a test should read like a story - my preferred layout (and its not my idea :-) - its fairly widely used) is:
- given: do setup etc
- when: the place you actually execute/call the thing under test
- expect: verify the result
Another advantage of a unit test testing only one thing is that when a failure occurs its unambiguous what the cause was - if you have a long test with many possible outcomes it becomes much harder to reason why a test has failed.
I'm not sure if it's possible to cleanly do what you're after in a parametrized test. If you need different test case behavior based on which parameter for some features, you might just be better off testing those features separately - in different test classes that are not parametrized.
If you really do want to keep everything in the parametrized test classes, I would be inclined to make a helper function so that your example test at least reads as a simple assertion:
#Test
public void testRecipientsCount(){
assertEquals(expectedCount(something), recipientsCount)
}
private int expectedCount(boolean which) {
if (something){
return 3;
}
else {
return 4;
}
}
Why not have a private method that tests the things that are common for each method? Something like (but probably with some input parameter for the testCommonStuff() method):
#Test
public void testRecipientsCountA(){
testCommonStuff();
// Assert stuff for test A
}
#Test
public void testRecipientsCountB(){
testCommonStuff();
// Assert stuff for test B
}
private void testCommonStuff() {
// Assert common stuff
}
This way you don't get redundant code and you can split your test into smaller tests. Also you make your tests less error prone IF they should actually test the same things. You will still know which test that failed, so traceability should be no problem.
Functions (side-effect free ones) are such a fundamental building block, but I don't know of a satisfying way of testing them in Java.
I'm looking for pointers to tricks that make testing them easier. Here's an example of what I want:
public void setUp() {
myObj = new MyObject(...);
}
// This is sooo 2009 and not what I want to write:
public void testThatSomeInputGivesExpectedOutput () {
assertEquals(expectedOutput, myObj.myFunction(someInput);
assertEquals(expectedOtherOutput, myObj.myFunction(someOtherInput);
// I don't want to repeat/write the following checks to see
// that myFunction is behaving functionally.
assertEquals(expectedOutput, myObj.myFunction(someInput);
assertEquals(expectedOtherOutput, myObj.myFunction(someOtherInput);
}
// The following two tests are more in spirit of what I'd like
// to write, but they don't test that myFunction is functional:
public void testThatSomeInputGivesExpectedOutput () {
assertEquals(expectedOutput, myObj.myFunction(someInput);
}
public void testThatSomeOtherInputGivesExpectedOutput () {
assertEquals(expectedOtherOutput, myObj.myFunction(someOtherInput);
}
I'm looking for some annotation I can put on the test(s), MyObject or myFunction to make the test framework automatically repeat invocations to myFunction in all possible permutations for the given input/output combinations I've given, or some subset of the possible permutations in order to prove that the function is functional.
For example, above the (only) two possible permutations are:
myObj = new MyObject();
myObj.myFunction(someInput);
myObj.myFunction(someOtherInput);
and:
myObj = new MyObject();
myObj.myFunction(someOtherInput);
myObj.myFunction(someInput);
I should be able to only provide the input/output pairs (someInput, expectedOutput), and (someOtherInput, someOtherOutput), and the framework should do the rest.
I haven't used QuickCheck, but it seems like a non-solution. It is documented as a generator. I'm not looking for a way to generate inputs to my function, but rather a framework that lets me declaratively specify what part of my object is side-effect free and invoke my input/output specification using some permutation based on that declaration.
Update: I'm not looking to verify that nothing changes in the object, a memoizing function is a typical use-case for this kind of testing, and a memoizer actually changes its internal state. However, the output given some input always stays the same.
If you are trying to test that the functions are side-effect free, then calling with random arguments isn't really going to cut it. The same applies for a random sequence of calls with known arguments. Or pseudo-random, with random or fixed seeds. There's a good chance are that a (harmful) side-effect will only occur with any of the sequence of calls that your randomizer selects.
There is also a chance that the side-effects won't actually be visible in the outputs of any of the calls that you are making ... no matter what the inputs are. They side-effects could be on some other related objects that you didn't think to examine.
If you want to test this kind of thing, you really need to implement a "white-box" test where you look at the code and try and figure out what might cause (unwanted) side-effects and create test cases based on that knowledge. But I think that a better approach is careful manual code inspection, or using an automated static code analyser ... if you can find one that would do the job for you.
OTOH, if you already know that the functions are side-effect free, implementing randomized tests "just in case" is a bit of a waste of time, IMO.
I'm not quite sure I understand what you are asking, but it seems like Junit Theories (http://junit.sourceforge.net/doc/ReleaseNotes4.4.html#theories) could be an answer.
In this example, you could create a Map of key/value pairs (input/output) and call the method under test several times with values picked from the map. This will not prove, that the method is functional, but will increase the probability - which might be sufficient.
Here's a quick example of such an additional probably-functional test:
#Test public probablyFunctionalTestForMethodX() {
Map<Object, Object> inputOutputMap = initMap(); // this loads the input/output values
for (int i = 0; i < maxIterations; i++) {
Map.Entry test = pickAtRandom(inputOutputMap); // this picks a map enty randomly
assertEquals(test.getValue(), myObj.myFunction(test.getKey());
}
}
Problems with a higher complexity could be solved based on the Command pattern: You could wrap the test methods in command objects, add the command object to a list, shuffle the list and execute the commands (= the embedded tests) according to that list.
It sounds like you're attempting to test that invoking a particular method on a class doesn't modify any of its fields. This is a somewhat odd test case, but it's entirely possible to write a clear test for it. For other "side effects", like invoking other external methods, it's a bit harder. You could replace local references with test stubs and verify that they weren't invoked, but you still won't catch static method calls this way. Still, it's trivial to verify by inspection that you're not doing anything like that in your code, and sometimes that has to be good enough.
Here's one way to test that there are no side effects in a call:
public void test_MyFunction_hasNoSideEffects() {
MyClass systemUnderTest = makeMyClass();
MyClass copyOfOriginalState = systemUnderTest.clone();
systemUnderTest.myFunction();
assertEquals(systemUnderTest, copyOfOriginalState); //Test equals() method elsewhere
}
It's somewhat unusual to try to prove that a method is truly side effect free. Unit tests generally attempt to prove that a method behaves correctly and according to contract, but they're not meant to replace examining the code. It's generally a pretty easy exercise to check whether a method has any possible side effects. If your method never sets a field's value and never calls any non-functional methods, then it's functional.
Testing this at runtime is tricky. What might be more useful would be some sort of static analysis. Perhaps you could create a #Functional annotation, then write a program that would examine the classes of your program for such methods and check that they only invoke other #Functional methods and never assign to fields.
Randomly googling around, I found somebody's master's thesis on exactly this topic. Perhaps he has working code available.
Still, I will repeat that it is my advice that you focus your attention elsewhere. While you CAN mostly prove that a method has no side effects at all, it may be better in many cases to quickly verify this by visual inspection and focus the remainder of your time on other, more basic tests.
have a look at http://fitnesse.org/: it is used often for Acceptance Test but I found it is a easy way to run the same tests against huge amount of data
In junit you can write your own test runner. This code is not tested (I'm not sure if methods which get arguments will be recognized as test methods, maybe some more runner setup is needed?):
public class MyRunner extends BlockJUnit4ClassRunner {
#Override
protected Statement methodInvoker(final FrameworkMethod method, final Object test) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
Iterable<Object[]> permutations = getPermutations();
for (Object[] permutation : permutations) {
method.invokeExplosively(test, permutation[0], permutation[1]);
}
}
};
}
}
It should be only a matter of providing getPermutations() implementation. For example it can take data from some List<Object[]> field annotated with some custom annotation and produce all the permutations.
I think the term you're missing is "Parametrized Tests". However it seems to be more tedious in jUnit that in the .Net flavor. In NUnit, the following test executes 6 times with all combinations.
[Test]
public void MyTest(
[Values(1,2,3)] int x,
[Values("A","B")] string s)
{
...
}
For Java, your options seem to be:
JUnit supports this with version 4. However it's a lot of code (it seems, jUnit is adamant about test methods not taking parameters). This is the least invasive.
DDSteps, a jUnit plugin. See this video that takes values from appropriately named excel spreadsheet. You also need to write a mapper/fixture class that maps values from the spreadsheet into members of the fixture class, that are then used to invoke the SUT.
Finally, you have Fit/Fitnesse. It's as good as DDSteps, except for the fact that the input data is in HTML/Wiki form. You can paste from an excel sheet into Fitnesse and it formats it correctly at the push of a button. You need to write a fixture class here too.
Im afraid that I dont find the link anymore, but Junit 4 has some help functions to generate testdata. Its like:
public void testData() {
data = {2, 3, 4};
data = {3,4,5 };
...
return data;
}
Junit will then thest your methods will this data. But as I said, I cant' find the link anymore (forgot the keywords) for a detailed (and correct) example.