Can anyone help me with my problem?
I test my program with Robotium in Junit.
My problem is:
When I detect there is a failure in junit, how can I use code to detect there is failure in program? So, I can continue run if no error occur? e.g. if no error, continue testing, else exit.
I suggest using Java's built-in assertions for your test. To create an assertion:
assert someBoolean : message;
For example:
assert (myValue == 3) : "myValue was " + myValue + ", should have been 3";
Assertions are disabled by default when running your program. To run your program with assertions, run it like this:
java -enableassertions MyClass
Then, if your program is running with this runtime option, whenever an assert is reached, the following happens:
If the boolean is true, the program will continue.
If it is false, an AssertionError is thrown with the specified message.
For example:
int myVar = 5;
assert (myVar == 3) : "myVar is " + myVar + " not 3";
results in
Exception in thread "main" java.lang.AssertionError: myVar is 5 not 3
IF assertions are enabled. Remember: all of that only happens when you enable asserts using -enableassertions or -ea. If you don't, the asserts are skipped.
When I detect there is a failure in junit, how can I use code to detect there is failure in program? So, I can continue run if no error occur? e.g. if no error, continue testing, else exit.
This doesn't make much sense. If you've got a failure in a JUnit test, that means there is a failure in your program. If no failure occurs, the unit testing will proceed to the next test automatically.
But maybe you are asking if you can do this:
// in some unit test
assert(....); // <<--- this test fails:
// Do something so that the unit test will continue to the next assertion ...
assert(....)
The answer is that you can't do that in any useful way:
The unit test framework can only report unit test failures that indicate that they have failed by terminating with an exception.
You could write a unit test to catch the exception that an assert(...) or fail(...) call throws and continue to the next assertion. But that would destroy all evidence of the previous unit test failure.
So if you want to be able to do the second assertion despite the first one failing, you need to make them separate testcases.
You might also be asking if there is a way to get the JUnit test runner to abort on the first failed unit test. The answer is yes that it is possible, but how you would do it would depend on the test runner you are using.
You can make assertions for a condition to be true or false-
assertTrue(soloObject.waitForActivity("Activity Name"));
Instead of wating for an activity you can use all the methods provided by robotium to make assertions example isTextFound("text"), isTextFound("text"), isCheckBoxChecked(index), etc.
Related
I am trying to understand which call causes Error and Which causes failures in Junit4. Until Junit3,
Failure can be created using
junit.framework.AssertionFailedError
And Error with
junit.framework.Assert.assertEquals
But with the deprecation of junit.framework.Assert, which is not moved to org.junit.Assert, I am not able to find a way in junit4 to throw a failure. Anything I try with org.junit.Assert (even Assert.fail() ) , JUnit considers it as Error.
Any idea on how to properly generate failures in Junit4 style tests?
Update
I figured out that there is a std.err at the end of XML generated by JUnit ant target.
<system-err>TEXT here</system-err>
and I suspected this is the cause that making it ERROR instead of Failure. But when I cleared all sys.err, it still marking it ERROR.
You can still use Assert.assertThat for getting assertion failure
assertThat(0, is(1)); // fails:
assertThat(0, is(not(1))) // passes
It may not what you need, but also JUnit 4 has ComparisonFailure
Thrown when an assertEquals(String, String) fails. Create and throw a ComparisonFailure manually if you want to show users the difference between two complex strings.
I am having the same issue. The only solution I have found so far is to use a try block followed by
catch (AssertionError ae) {
fail(ae.toString());
}
But I can see downsides to this and I have seen many people say this is bad practice. Unfortunately I don't see another way around it when using ant to make a report.
This question already has answers here:
Conditionally ignoring tests in JUnit 4
(5 answers)
Closed 5 years ago.
I have a junit integration test that runs on our build machine, and calls an external server. We use a gating build - so code doesn't make it in to the main branch unless we get 100% of tests passing
Occasionally, the external server goes down for a while, and the test fails, but with a definitive exception that I can catch. I really don't want the test to fail the build, therefore blocking code getting in - but I also would prefer it's not marked as "passed". So I want to sort of mark it as a warning, ignored, or indeterminate result. This would be ideal:
#Test
public void someTest()
{
try
{
do whatever
}
catch (ServerDownException theException)
{
junit.markThisTestAsIgnored(); <---- something like this
}
}
found it -
throw new AssumptionViolationException( "skipping test because xxx is down");
One option is to set a category(stress category for example).
Check out this link:
https://github.com/junit-team/junit4/wiki/categories
If you want, you can put the #Ignore rule:
https://dzone.com/articles/allowing-junit-tests-pass-test
Regards
I am trying to learn python to work on a test project.
Is there a way to implement TestNG Listeners like functionality in python test framework.
Listeners have method like OnTestFailure(), OnTestSuccess, OnStart(), and many more which are really helpful when you want to do certain things.
Let's say, a test case failed and you want to perform some actions like taking a screenshot. Then you can just write that in one place and not have write that in every afterTest method also.
This class will be called from the test case like this
TestStatus.mark('testName', result, 'message you want to log')
result is a boolean
class TestStatus(unittest.TestCase):
def __init__(self):
super(TestStatus, self).__init__()
def mark(self, testName, result, resultMessage):
testName = testName.lower()
try:
if result:
self.log.info("Verification successful :: " + resultMessage)
else:
# If the test fails,
# this calls screenshot method from util class
self.util.screenShot("FAIL" + mapKey)
self.log.info("Verification failed :: " + resultMessage)
except:
self.log.info("### Exception Occurred !!!")
traceback.print_stack()
This is a sample test case in the test case class:
def test_accountSignup(self):
# Generate a username and password to use in the test case
userName = self.util.getUniqueName()
password = self.util.getUniqueName()
# You can ignore this, this is calling a method
# signup from the account page (page object model)
self.accounts.signup(userName, password)
# This call is also from the page object,
# it checks if the sign up was successful
# it returns a boolean
result = isSignUpSuccessful()
# test_status object was created in the setUp method
#
self.test_status.mark("test_accountSignup", result,
"Signup was successful")
#Sunshine, Thanks for the reply. One question.
I see that, inside the test case function "def test_accountSignup(self)", towards the end of the test case, you added the line
self.test_status.mark("test_accountSignup", result,
"Signup was successful")
What if the test cases fails some where before reaching test_status.mark call in the test case. For eg: What will happen if the test case fails on the line
self.accounts.signup(userName, password)
I guess, in this case, the test result won't be captured for this particular test case. Correct?
Do you think, we should instead write the test cases as shown below?
def test_accountSignup(self):
try:
# Write all the test case specific code here
self.test_status.mark("test_accountSignup", "Passed",
"test_accountSignup passed")
except Exception as e:
self.test_status.mark("test_accountSignup", "Failed",
"test_accountSignup failed. Exception is: "+e.message)
Please let me know what you think. Thanks again!
I have a method that during a specific point of processing it expects that certain invariants are kept.
To keep this trivial let's say that at point X during the code processing the variable high and the variable low MUST be divisible.
So in the code I do:
if(high % low != 0){
throw new AssertionError("Invalid state of low and high [low = "+ low+ ", high=" + high+"]");
}
During unit testing with JUnits I had a test case to test for this.
So I did:
try {
//At this point I know for sure that the low and high are not divible so process() should throw an Assertion Error
process();
fail();
}
catch(AssertionError e){
}
But the test case was green!
But I realised that junit raises an assert error due to fail but I catch it and as a result the test case is pass instead of failed.
From my point of view the proper error to raise in my code is also AssertionError and not some generic e.g. IllegalArgumentsException
So is there a way to fix this so that the test case works or should I not be using the AssertionError in my code in the first place? If this is the case, what exception should I be raising?
You should not have a try-catch block in a unit test. If you expect an exception use an annotation:
#Test(expected=Exception.class)
public void youTestMethod() {
...
}
You will have to use a different exception such as IllegalArgumentsException because JUnit uses AssertionError internally. I find IllegalArgumentsException more descriptive of what actually went wrong.
boolean shouldFail = false;
try {
callTheMethod();
shouldFail = true;
}
catch (AssertionError e) {
// expected
}
if (shouldFail) {
fail();
}
But if the condition is an invariant, then it should always be true, and the AssertionError should thus never be thrown, and you should thus not even be able to unit test it. If you're able to test it, it means that the invariant is not really an invariant, and that the condition might be true depending on the sequence of calls or on the arguments provided. You should thus prefer an IllegalStateExcetion over the AssertionError.
> should I not be using the AssertionError in my code in the first place?
No JUnit is using AssertionError and its decendants to tell the junit-runner that a test has failed. (Simmilar applies to .net NUnit-runner)
> If this is the case, what exception should I be raising?
I would use one of the generic Exceptoin like IllegalArgumentsException or create my own Exception
Yes, there is a conflict between your code and JUnit, but it's easy to get around.
When you write a JUnit test case, as you've already deduced, an AssertionError is thrown when the test case has failed.
To let JUnit know that your test case has passed, the test code should NOT throw an AssertionError on any other Exception/Error for that matter.
There will be two test cases (at least) -
A. high and low are exactly divisible - the test case code doesn't throw an AssertionError.
//test case setup
yourClass.callYourMethod(4,2);
//verification
Here, if the test case behaves correctly, there is no AssertionError and JUnit knows it has passed.
B. high and low are not exactly divisible - the code should throw AssertionError but the test case should not.
boolean failed;
try {
//test case setup
yourClass.callYourMethod(4,2);
failed = true;
} catch (AssertionError e) {
failed = false;
}
if (failed) {
fail();
}
Im am currently developing an automated "test" class (running several individual tests on other classes in the same package). The aim of the test file is to show whether each test either passed or failed. Some of the files being tested are not written correctly creating an ArrayOutOfBoundsException when running the test, that produces my test file to crash and not carry on until the end performing other tests. I am not in the position to modify code to fix the errors on the project being tested.
-> how to stop an exception from halting program execution in Java without creating any new classes
Thank for all your help, advice and sharing.
Best way to stop it happening: fix the code to perform appropriate checking.
If you can't fix the code which is actually failing, you could catch the exception explicitly at an "outer" level, log a warning and continue with the next file:
try
{
operationWhichMightThrow();
}
catch (ArrayIndexOutOfBoundsException e)
{
log.warning("Failed file " + filename, e);
// Do whatever you need to continue to the next file.
}
Catch the exception and log it as a test failure.