This question already has answers here:
Conditionally ignoring tests in JUnit 4
(5 answers)
Closed 5 years ago.
I have a junit integration test that runs on our build machine, and calls an external server. We use a gating build - so code doesn't make it in to the main branch unless we get 100% of tests passing
Occasionally, the external server goes down for a while, and the test fails, but with a definitive exception that I can catch. I really don't want the test to fail the build, therefore blocking code getting in - but I also would prefer it's not marked as "passed". So I want to sort of mark it as a warning, ignored, or indeterminate result. This would be ideal:
#Test
public void someTest()
{
try
{
do whatever
}
catch (ServerDownException theException)
{
junit.markThisTestAsIgnored(); <---- something like this
}
}
found it -
throw new AssumptionViolationException( "skipping test because xxx is down");
One option is to set a category(stress category for example).
Check out this link:
https://github.com/junit-team/junit4/wiki/categories
If you want, you can put the #Ignore rule:
https://dzone.com/articles/allowing-junit-tests-pass-test
Regards
Related
I am trying to understand which call causes Error and Which causes failures in Junit4. Until Junit3,
Failure can be created using
junit.framework.AssertionFailedError
And Error with
junit.framework.Assert.assertEquals
But with the deprecation of junit.framework.Assert, which is not moved to org.junit.Assert, I am not able to find a way in junit4 to throw a failure. Anything I try with org.junit.Assert (even Assert.fail() ) , JUnit considers it as Error.
Any idea on how to properly generate failures in Junit4 style tests?
Update
I figured out that there is a std.err at the end of XML generated by JUnit ant target.
<system-err>TEXT here</system-err>
and I suspected this is the cause that making it ERROR instead of Failure. But when I cleared all sys.err, it still marking it ERROR.
You can still use Assert.assertThat for getting assertion failure
assertThat(0, is(1)); // fails:
assertThat(0, is(not(1))) // passes
It may not what you need, but also JUnit 4 has ComparisonFailure
Thrown when an assertEquals(String, String) fails. Create and throw a ComparisonFailure manually if you want to show users the difference between two complex strings.
I am having the same issue. The only solution I have found so far is to use a try block followed by
catch (AssertionError ae) {
fail(ae.toString());
}
But I can see downsides to this and I have seen many people say this is bad practice. Unfortunately I don't see another way around it when using ant to make a report.
I have a method that I am trying to unit-test. I cannot post the actual code, but it looks like this:
public int getTotal() throws MyException {
int total = 0;
try (ExternalResource externalResource = ExternalService.getResource()) {
try (OtherExternal otherResource = externalResource.getOtherResource()) {
if (someCondition) {
total = otherResource.getTotal();
}
}
}
}
JaCoCo is telling me that I am missing 4/8 branches on each of the try-with-resource blocks. I am testing that someCondition is true and someCondition is false, and JaCoCo shows that block completely covered.
I read this question, and I understand from the accepted answer that the issue is in how the byte code is generated.
I would like to be able to better understand how to identify the various branches that are generated, and then I can make a better judgement on wether to test them or not (are they unreachable, etc).
Per the change history in version 0.8.2:
Branches and instructions generated by javac 11 for try-with-resources statement are filtered out
I've tested this out locally using openjdk java8, and my try-with-resources now reports 100% branch coverage (even though the IOException is never thrown in my tests).
While it is good to test this behavior out, there are times when you can't easily reproduce such exceptions. For instance, in a method that just returns an open port:
public int getOpenPort() {
try (ServerSocket boundSocket = new ServerSocket(0)) {
return boundSocket.getLocalPort();
}
}
I know of no simple way to force this code to throw IOException without adding a bunch of confusing and unnecessarily complicated code, just to pass a branch coverage check. Luckily, the new (v0.8.2) jacoco library gives this method 100% coverage with a single test just calling Assert.assertNotEquals(0, portChecker.getOpenPort());.
You have to test every Exception and every condition. But JaCoCo sometimes failed to identify correctly what is covered or not.
This question already has answers here:
How to continue execution when Assertion is failed
(5 answers)
Closed 7 years ago.
I'm using Selenium WebDriver + Java + TestNG for automation. In my test methods, sometimes there are more than one assertions.
Suppose, there are four assertions and second assertion fails, then rest of the execution is terminated.
What I want is - even after second assertion fails, code after it should be executed. And at the end(after test method is executed), it should return what assertions are failed out of four and test should be marked as "Fail".
Is there any way using Java + TestNG, I can achieve this?(And I would like to put this code at some central place, so that I won't have to add it in every test method)
If no assertion fails, then no worries. It'll execute as usual.
Here's something that you could be looking for:
https://rameshbaskar.wordpress.com/2013/09/11/soft-assertions-using-testng/
import org.testng.asserts.Assertion;
import org.testng.asserts.SoftAssert;
public class MyTest {
private Assertion hardAssert = new Assertion();
private SoftAssert softAssert = new SoftAssert();
softAssert.assertTrue(false);
}
Can anyone help me with my problem?
I test my program with Robotium in Junit.
My problem is:
When I detect there is a failure in junit, how can I use code to detect there is failure in program? So, I can continue run if no error occur? e.g. if no error, continue testing, else exit.
I suggest using Java's built-in assertions for your test. To create an assertion:
assert someBoolean : message;
For example:
assert (myValue == 3) : "myValue was " + myValue + ", should have been 3";
Assertions are disabled by default when running your program. To run your program with assertions, run it like this:
java -enableassertions MyClass
Then, if your program is running with this runtime option, whenever an assert is reached, the following happens:
If the boolean is true, the program will continue.
If it is false, an AssertionError is thrown with the specified message.
For example:
int myVar = 5;
assert (myVar == 3) : "myVar is " + myVar + " not 3";
results in
Exception in thread "main" java.lang.AssertionError: myVar is 5 not 3
IF assertions are enabled. Remember: all of that only happens when you enable asserts using -enableassertions or -ea. If you don't, the asserts are skipped.
When I detect there is a failure in junit, how can I use code to detect there is failure in program? So, I can continue run if no error occur? e.g. if no error, continue testing, else exit.
This doesn't make much sense. If you've got a failure in a JUnit test, that means there is a failure in your program. If no failure occurs, the unit testing will proceed to the next test automatically.
But maybe you are asking if you can do this:
// in some unit test
assert(....); // <<--- this test fails:
// Do something so that the unit test will continue to the next assertion ...
assert(....)
The answer is that you can't do that in any useful way:
The unit test framework can only report unit test failures that indicate that they have failed by terminating with an exception.
You could write a unit test to catch the exception that an assert(...) or fail(...) call throws and continue to the next assertion. But that would destroy all evidence of the previous unit test failure.
So if you want to be able to do the second assertion despite the first one failing, you need to make them separate testcases.
You might also be asking if there is a way to get the JUnit test runner to abort on the first failed unit test. The answer is yes that it is possible, but how you would do it would depend on the test runner you are using.
You can make assertions for a condition to be true or false-
assertTrue(soloObject.waitForActivity("Activity Name"));
Instead of wating for an activity you can use all the methods provided by robotium to make assertions example isTextFound("text"), isTextFound("text"), isCheckBoxChecked(index), etc.
Im am currently developing an automated "test" class (running several individual tests on other classes in the same package). The aim of the test file is to show whether each test either passed or failed. Some of the files being tested are not written correctly creating an ArrayOutOfBoundsException when running the test, that produces my test file to crash and not carry on until the end performing other tests. I am not in the position to modify code to fix the errors on the project being tested.
-> how to stop an exception from halting program execution in Java without creating any new classes
Thank for all your help, advice and sharing.
Best way to stop it happening: fix the code to perform appropriate checking.
If you can't fix the code which is actually failing, you could catch the exception explicitly at an "outer" level, log a warning and continue with the next file:
try
{
operationWhichMightThrow();
}
catch (ArrayIndexOutOfBoundsException e)
{
log.warning("Failed file " + filename, e);
// Do whatever you need to continue to the next file.
}
Catch the exception and log it as a test failure.