Sonar rule S2699: Not all asserts are recognized as valid assertions - java

We are running Sonarqube 5.6.1 with the Java Plugin 4.1 and having some troubles using the Sonar rule S2699 (Test should include assertions).
Using this example test class
import mypackage.Citit1543Dummy;
import mypackage.Citit1543OtherDummy;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.mockito.MockitoAnnotations;
import java.util.Arrays;
import static org.hamcrest.Matchers.is;
import static org.hamcrest.Matchers.isIn;
import static org.hamcrest.Matchers.lessThan;
import static org.hamcrest.core.IsNot.not;
import static org.mockito.Matchers.notNull;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.junit.Assert.assertThat;
public class Citit1543Test {
#Before
public void setUp() {
MockitoAnnotations.initMocks(this);
}
#Test
public void test1() {
assert true;
}
#Test
public void test2() {
Assert.assertTrue(1 > (2-3));
}
#Test
public void test3() {
Assert.assertFalse(1 > (100-1));
}
#Test
public void test4() {
Assert.assertThat("test", 1, is(1));
}
#Test
public void test5() {
Assert.assertArrayEquals(new String[0], new String[0]);
}
#Test
public void test6() {
Assert.assertEquals(1 > 0, true);
}
#Test
public void test7() { // asserts in another method
test7asserts(1, 1);
}
private void test7asserts(int a, int b) {
Assert.assertTrue(a == b);
}
#Test
public void test8() {
test8asserts(1, 2);
}
private void test8asserts(int a, int b) {
Assert.assertNotSame(a, b);
}
#Test
public void test9() {
Citit1543Dummy dummy = new Citit1543Dummy();
dummy.otherDummy = mock(Citit1543OtherDummy.class);
dummy.doSomething();
verify(dummy.otherDummy, times(1)).doSomething();
}
#Test
public void test10() {
Citit1543Dummy dummy = new Citit1543Dummy();
dummy.otherDummy = mock(Citit1543OtherDummy.class);
dummy.doSomething();
test10verifies(dummy.otherDummy);
}
private void test10verifies(Citit1543OtherDummy otherDummy) {
verify(otherDummy, times(1)).doSomething();
}
#Test
public void test11() {
Assert.assertThat("test", "", not(1));
}
#Test
public void test12() {
Assert.assertThat("test", 1, lessThan(2));
}
#Test
public void test13() {
Long[] arr = new Long[] { 1L, 2L, 3L, 4L };
assertThat("Just testing", arr, is(new Long[] {
1L, 2L, 3L, 4L
}));
}
}
our Sonarqube instance flags the test cases test1 (assert statement not recognized), test7 (assert statements in another method), test8 (same) , test10 (Mockitos verify in another method), test11 and test13 as methods without assertions. I'm pretty sure that there are a lot more methods which aren't recognized (yes, unfortunately we use a bunch of different mocking/testing framework across our projects).
For now, we started to //NOSONAR whenever one of the asserts/verifies aren't recognized.
Is there an easy way to include these methods to be recognized as valid asserts?

Many of your stated issues are known and indeed (in some form of another) marked as FP:
test1: The current flow analysis ignores assert statements. See this post over at the groups.
The cases test7, test8 and test10 are related to the lack of not having cross-procedural analysis: They are valid cases but the current flow doesn't know that (ex.) test7assert is a valid assert statement for another method. See this post over at the groups.
Your other cases also produce false positives in the tests of S2699. I'd expect that once a SonarSource dev reads this topic that they'll create a ticket to resolve the cases in test11/13. But as I'm not a dev of them I can't guarantee that of course.
As to :
Is there an easy way to include these methods to be recognized as valid asserts?
No, the valid assertions are defined within the code of S2699 and are not a parameter. Some of your cases will require a more complex flow analysis whilst the last couple just seem to boil down to some missing definitions or too strict definitions, but I didn't deep-dive into the reasons why they produce FPs.

Related

Is it possible to access instance of exception "catched" with JUnit Rule ExpectedException?

Suppose that in a test is:
#Rule
public ExpectedException thrown = ExpectedException.none();
and some test contains:
thrown.expect(SomeNastyException.class);
Now if there is a need to inspect this SomeNastyException more detailed what would be the way?
Of course it would be possible just not to use ExcpectedException and wrap call with try/catch and then inspect the exception in any way it is needed but now question is about can it be done with ExpectedException, something like:
assertEquals("42", thrown.getException().getUniversalAnswerToEverything());
There is a less specialized version of expect method in that rule, that accepts a Hamcrest's matcher:
expect(Matcher)
This method would allow you to assert almost anything about the thrown exception, provided there is a Matcher implementation for that.
For instance, for your case it'd look something like this:
import static org.hamcrest.Matchers.hasProperty;
import static org.hamcrest.Matchers.is;
....
exceptions.expect(hasProperty("UniversalAnswerToEverything", is(42)));
Hamcrest has a pretty flexible matcher model, so you can also easily write your own if you aren't satisfied with what's inlucded in the library.
You can use hasProperty hamcrest Matcher. And if your SomeNastyException does not conform to java bean protocol, you can create a custom hamcrest matcher for your exception:
package stackoverflow.q59946794;
import org.hamcrest.Description;
import org.hamcrest.FeatureMatcher;
import org.hamcrest.Matcher;
import org.hamcrest.TypeSafeMatcher;
import org.hamcrest.core.IsEqual;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;
import java.util.logging.Logger;
public class SomeNastyExceptionSpec {
private final static Logger logger = Logger.getLogger(SomeNastyExceptionSpec.class.getName());
public static class SomeNastyException extends Exception implements java.io.Serializable {
SomeNastyException(String message) { super(message);}
public Integer getUniversalAnswerToEverything() {return 42;}
}
public static Matcher<SomeNastyException> hasUniversalAnswerToEverythingFeature(final int expectedAnswer) {
return new FeatureMatcher<SomeNastyException, Integer>(
new IsEqual<Integer>(expectedAnswer),
"SomeNastyException actual answer",
"SomeNastyException expected answer"
) {
#Override
protected Integer featureValueOf(SomeNastyException actual) {
return actual.getUniversalAnswerToEverything();
}
};
}
public static Matcher<SomeNastyException> hasUniversalAnswerToEverything(final int expectedAnswer) {
return new TypeSafeMatcher<SomeNastyException>() {
#Override
protected void describeMismatchSafely(SomeNastyException e, Description description) {
description.appendText("was ").appendValue(e.getUniversalAnswerToEverything());
}
#Override
public void describeTo(Description description) {
description.appendText("SomeNastyException with answer ").appendValue(expectedAnswer);
}
#Override
protected boolean matchesSafely(SomeNastyException e) {
return e.getUniversalAnswerToEverything() == expectedAnswer;
}
};
}
#Rule(order = Integer.MAX_VALUE)
public ExpectedException thrown = ExpectedException.none();
#Test
public void shouldTestExceptionMessage() throws Exception {
thrown.expect(SomeNastyException.class);
thrown.expect(hasUniversalAnswerToEverything(42));
thrown.expect(hasUniversalAnswerToEverythingFeature(42));
thrown.expectMessage("always 42");
throw new SomeNastyException("always 42");
}
}
My question was asked while still using JUnit4. Recently migrated to JUnit5 and it has exactly what I was looking for, namely assertThrows that returns the exception thrown. As a dummy example:
#org.junit.jupiter.api.Test
void testThrows() {
Exception exception = assertThrows(NotAnswerableException.class, () -> {
throw new NotAnswerableException("please rephrase your question", param2, param3);
});
assertTrue(exception.getMessage().startsWith("please"));
assertEquals(param2, exception.getParam2());
}

TestNG dependsOnMethods

package test;
import org.testng.annotations.Test;
public class Day3 {
#Test
public void webLoginCarLoan() {
System.out.println("WebLoginCarLoan");
}
#Test
public void mobileLoginCarLoan() {
System.out.println("MobileLoginCarLoan");
}
#Test
public void mobileSignoutCarLoan() {
System.out.println("MobileSignoutCarLoan");
}
#Test(dependsOnMethods = { "webLoginCarLoan" })
public void apiCarLoan() {
System.out.println("LoginAPICarLoan");
}
}
Output:
MobileLoginCarLoan
WebLoginCarLoan
MobileSignoutCarLoan
LoginAPICarLoan
Why WebLoginCarLoan comes before MobileSignoutCarLoan in Output?
TestNG does not execute tests in same order as noted in class. If you think, need to execute tests/#Test methods in specified order then use priority
#Test(priority=1)
Also as pointed by Mark "I think that dependsOnMethods will make sure webLoginCarLoan is executed before apiCarLoan however not immediately after it perse"
see in Java Doc https://jitpack.io/com/github/cbeust/testng/master/javadoc/org/testng/annotations/Test.html#dependsOnMethods--

ND4J JUnit Testing Exceptions

I have written a simple class and test it manually in main() and works as expected:
import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.factory.Nd4j;
public class ND4J {
public static void main(String[] args) {
ND4J actualObject = new ND4J(Nd4j.zeros(2,2).add(4.0));
INDArray testObject = Nd4j.create(new double[][]{{4.0,4.0}, {4.0,4.0}});
if(testObject.equals(actualObject.getMatrix())){
System.out.println("OK"); // prints “OK”
}
}
private INDArray matrix;
public ND4J (INDArray matrix){
this.matrix = matrix;
}
public INDArray getMatrix(){
return this.matrix;
}
public String toString(){
return this.getMatrix().toString();
}
}
But trying to unit test this class with JUnit 4 is throwing java.lang.AbstractMethodError:
import org.junit.Test;
import org.junit.runner.RunWith;
import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.factory.Nd4j;
import static org.junit.Assert.*;
public class ND4JTest {
#Test
public void print() {
ND4J actualObject = new ND4J(Nd4j.zeros(2,2).add(4.0));
//above statement throws this error
//java.lang.AbstractMethodError: org.nd4j.linalg.factory.Nd4jBackend.getConfigurationResource()Lorg/springframework/core/io/Resource;
INDArray testObject = Nd4j.create(new double[][]{{4.0,4.0}, {4.0,4.0}});
assertEquals(testObject, actualObject.getMatrix());
}
}
In fact more complex classes classes using ND4J that run fine from main() are having similar problems in testing. My pom file has the following ND4J dependencies: javacpp, nd4j-jblas, nd4j-native-platform, nd4j-native.
Thanks
Rather than the prerequisites mentioned on the ND4J get started page I got it working by following instructions from here: https://github.com/deeplearning4j/dl4j-examples/blob/master/standalone-sample-project/pom.xml

Test that RxJava BehaviorProcessor emits a value

I'm having trouble understanding why all those processors pass the test but Behavior does not:
package com.example;
import org.junit.Test;
import io.reactivex.Flowable;
import io.reactivex.processors.*;
public class ProcessorTest {
private static Flowable<String> justFoo() {
return Flowable.just("foo");
}
private static FlowableProcessor<String> subscribeToFoo(
FlowableProcessor<String> processor) {
justFoo().subscribe(processor);
return processor;
}
#Test public void flowable() { // pass
justFoo().test().assertValue("foo");
}
#Test public void async() { // pass
subscribeToFoo(AsyncProcessor.create()).test().assertValue("foo");
}
#Test public void replay() { // pass
subscribeToFoo(ReplayProcessor.create()).test().assertValue("foo");
}
#Test public void unicast() { // pass
subscribeToFoo(UnicastProcessor.create()).test().assertValue("foo");
}
#Test public void behaviorFail() { // fail
subscribeToFoo(BehaviorProcessor.create()).test().assertValue("foo");
}
#Test public void behaviorPassing() { // pass
subscribeToFoo(BehaviorProcessor.create())
.test()
.assertNoValues()
.assertSubscribed()
.assertComplete()
.assertNoErrors()
.assertNoTimeout()
.assertTerminated();
}
}
The docs say that BehaviorProcessor is a:
Processor that emits the most recent item it has observed and all subsequent observed items to each subscribed Subscriber.
So in my understanding it should pass the behaviorFail test, not the behaviorPassing. How come is that?
How would I write a valid test, to know that a BehaviorProcessor emitted a certain value?
Getting rid of the terminal event passed to the processor would help:
#Test public void behavior() {
final BehaviorProcessor<String> processor = BehaviorProcessor.create();
justFoo().concatWith(Flowable.never()).subscribe(processor);
processor.test().assertValue("foo");
}

Eclipse Junit run configuration that takes the current selected test class as an arg?

Instead of constantly creating identical debug configuraitons for my test cases, I would like to be able to simply save a few arguments common across all my Junit tests, right click on a specific test, then run that single run config. IE I would like a single debug configuration that can take as an argument the current selected test case instead of requiring me to manually specify it every time in the JUnit run configuration. My only options in the dialog appear to be either to specify a single test class or run all the tests in the project. As a result, Eclipse is littered with dozens of run configurations for all my test cases.
Instead of specifying a specific test class, I'd like it to specify a variable like ${container_loc} or ${resource_loc} for the class to run as in this question. Is there a variable in Eclipse that specifies the current selected Java class that I could place in the test class field in the dialog?
A specific example where this is useful is when running the Lucene unit tests. There's lots of arguments you can specify to customize the tests, some of which like -ea are required. Everytime I want to test a specific test case in Lucene in Eclipse, I have to manually setup these variables in the Eclipse debug config dialog :-/.
Have you looked at Parameterized Tests in JUnit? Here is an example:
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
#RunWith(Parameterized.class)
public class ParamTest {
#Parameters(name = "{index}: fib({0})={1}")
public static Iterable<Object[]> data() {
return Arrays.asList(new Object[][] {
{ 0, 0 }, { 1, 1 }, { 2, 1 }, { 3, 2 }, { 4, 3 }, { 5, 5 }, { 6, 8 }
});
}
private int input;
private int expected;
public ParamTest(int input, int expected) {
this.input = input;
this.expected = expected;
}
#Test
public void test() {
Assert.assertEquals(expected, input);
}
}
If you just want to run one test at a time you can use private variables as in:
public class MultipleTest {
private int x;
private int y;
public void test1(){
Assert.assertEquals(x, y);
}
public void test2(){
Assert.assertTrue(x >y);
}
public void args1(){
x=10; y=1;
}
public void args2(){
x=1;y=1;
}
public void args3(){
x=1;y=10;
}
#Test
public void testArgs11(){
args1();
test1();
}
#Test
public void testArgs21(){
args2();
test1();
}
#Test
public void testArgs31(){
args3();
test1();
}
#Test
public void testArgs12(){
args1();
test2();
}
#Test
public void testArgs22(){
args2();
test2();
}
#Test
public void testArgs32(){
args3();
test2();
}
}

Categories

Resources