I am doing some queries in a data provider. If the data queries do not return data I can use, rather than fail the test I would like to skip it. So I
throw new SkipException( "Could not find adequate data" ) but this is failing the test rather than skipping it.
Some research shows that SkipException has a method isSkip() which will skip if true and fail if false. I dumped it before throwing the exception and it showed true, but the test is still failing.
Am I doing something wrong or is there a better way to skip? (yes I know you can put it in the #test() but I don't know how to do that after the test is running.
If you throw a SkipException yes the test will fail. It's just like any other exception. If you want to skip a test you can do the following
Do #Test(enabled=false)
Implement iAnnotationTransformer and then do a annotation.setEnabled(false);
This will be a listener and you have to include that in your test or testng.xml or maven
You are not supposed to use SkipException in a data provider. Just send the result of the next query.
Related
I am writing a test in Scala where I do something like this --
val svc = mock[Service]
Initializer.fun1(svc)
fun1 internally calls a function svc.fun2 which returns Unit.
My test is failing when it tries to execute svc.fun2 with this error -
java.lang.NoSuchMethodError: org.mockito.internal.invocation.ArgumentsProcessor.expandArgs
As svc has already been mocked, I shouldn't have to mock fun2 ,right ?
Unless I want fun2 to return a specific value
Same thing worked for another codebase of mine, but that's using a different library for Mockito
Update :
I also tried adding a when/thenReturn for fun2 but still getting the same error.
I just did this
when(svc.fun2()) thenReturn()
since it's supposed to return UNIT
I have a test case I'm trying to finish. It should try to find location ABC, but that doesn't actually exist in the DB. Essentially, it should not load the data I'm trying to find. I've tried a bunch of things, and haven't figured it out yet. Here is my code:
#Test
public void testFindByInvalidLocABC() {
System.out.println("findByInvalidLocABC");
Storage result = StorageFacadeTest.facade.findByLoc("ABC");
assertNotNull(result);
assertEquals("NOK-0000001402", result.getId());
assertEquals("ABC", result.getLoc());
}
Any suggestions is greatly appreciated!
I have a test case I'm trying to finish. It should try to find
location ABC, but that doesn't actually exist in the DB
To ensure that data be present or not present during test executions, you cannot rely on a applicative or shared database.
Automated tests have to be repeatable. Otherwise, these will be reliable today but useless and error prone tomorrow.
So I encourage you to clean/populate a test database( or schema) before the tests be executed.
Besides, as others commented, your test doesn't look like a "not found" scenario. You assert the retrieved Storage content. It makes no sense.
It should rather look like :
#Test
public void findByLoc_with_invalidLoc_returns_null() {
Storage result = StorageFacadeTest.facade.findByLoc("ABC");
assertNull(result);
}
Some improvements for your unit test:
1) Returning Optional instead of null in the method under test is probably better but you don't use it in your actual implementation. So I follow it in my example.
2) System.out is really not advised in the test code.
3) The test prefix in the test method is not advised either. It is a legacy way as Java annotations didn't exist.
Usually when we run testcases using TestNG, assertion error stops further execution after that point. But, sometimes it would be better if we could run the whole script. Manually blocking/disabling those assertions become tedious process. Hence, if there were some ways to programmatically enable/disable assertions other than manual it would be of great help indeed.
Does TestNG support this? If not, can anyone help me please?
As Julien mentioned above you are better off making a custom softAssert of your own. I don't know, I could be horribly wrong but the standard softAssert that comes with testNG didn't give me the behaviour that I was after.
I suppose the most common reason that your tests are failing is an ElementNotFound or TimeOutException. So in your waitForElement method you can capture these exceptions (or any exception for that matter) and print a warning msg on the console (or don't print anything or even take a screenshot if you might, like a warning error but not a show-stopper error). Something like the below:
public boolean waitForElement(String elementName, int timeOut) {
try{
elementPresent=wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath(findXpath(elementName)))).isDisplayed());
}
catch(org.openqa.selenium.TimeOutException e1){e1.printStackTrace();elementPresent=false;takeScreenshot();}
}
return elementPresent;
}
Hope that helps!
Using QAF Validation you can fulfill your requirements, for selenium it provides inbuilt verification methods, if it fails then testcase also continues to execution.
As suggested , SoftAssert can be used which does not halt execution even if assertion fails.
Also enable/disable assertions is possible by flagging tests as enabled=false or enabled=true. This is in turn runs all tests [and thereby assertions] except those are marked as enabled=false
Example. Assertion in example won't be executed as test is disabled.
#Test (enabled=false)
public void verifySearchReport {
soft.assertEquals("About*", "About123Soft","FAILED ASSERT");
soft.assertAll();
}
Assertion in example will be executed as test is enabled. Further execution of tests won't be halted [Even if assertion is failing] as SoftAssert is used.
#Test (enabled=true)
public void verifySearchReport {
soft.assertEquals("About*", "About123Soft","FAILED ASSERT");
soft.assertAll();
}
//Further #Test here
testNg binds data providers by name (a string). when i misspelled it, it returned success. just no tests were run. is there any configuration option to fail-fast in case of such error?
Actually, when a test case does not find its data provider (which happens in your case of misspelled data provider name), that test case is skipped. And therefore, no test is run. What you need is a way to see skipped test cases.
You can print messages or throw exceptions whenever a test is skipped. However, throwing exceptions may not be recommended because it may stop build after the first skipped tests and your entire test suite may remain untested.
Approach 1
You need to implement ITestListener which provides method
onTestSkipped(ITestResult testResult)
Approach 2 TestNG also enables you to generate reports at the end of test run. You need to implement
IReporter interface
You need to code for generateReport() method.
In addition for both the above approaches, you need to configure your implementation class as a listener in testng.xml like following.
<listeners>
<listener class-name="com...test.reporter.TestStatusReporter" />
</listeners>
Next
Once you run
mvn install
and is successful, you can view test results including skipped tests details in the following location of your project.
../target/surefire-reports/index.html
Hope this helps.
I want to check some external log files after each test, if there were ary errors during execution. Throwing an exception in an AfterMethod doesn't work, because it is handled differently by TestNG: it will just fail the configuration method and not the preceding test.
My approach would be like this:
#AfterMethod(alwaysRun = true)
protected void tearDown(ITestResult result) {
if (thereWasAProblemDuringTestExecution()) {
result.setStatus(ITestResult.FAILURE);
result.setThrowable(getSomeThrowableSomebodyStoredAnywhere());
}
// doing other cleanUp-tasks
}
But still, my Eclipse TestNG plugin says the test passed.
Is it possible (and how) to fail a test (and not only a configuration method) in a configuration method?
Try setCurrentTestResult() method from class org.testng.Reporter:
result.setStatus(ITestResult.FAILURE);
Reporter.setCurrentTestResult(result);
This might be too late but I can help anyone else who is looking for this.
You can do this using the ITestContext
For some reason you cant really update a result which is already generated. The workaround is to add the same result, remove it, update it then add it again. The only way I have found to remove a result is to do the add and remove sequence.
def testContext = Reporter.currentTestResult.testContext
testContext.passedTests.addResult(result,Reporter.currentTestResult.method)
testContext.passedTests.getAllMethods().remove(Reporter.currentTestResult.method)
result.setStatus(ITestResult.FAILURE)
testContext.failedTests.addResult(result,Reporter.currentTestResult.method)
I am using Groovy.