I am using Selenium RC using Java with eclipse and TestNG framework. I have the following code snippet:
assertTrue(selenium.isTextPresent("Please enter Email ID"));
assertTrue(selenium.isTextPresent("Please enter Password"));
First assertion was failed and execution was stopped. But I want to continue the further snippet of code.
I suggest you to use soft assertions, which are provided in TestNg natively
package automation.tests;
import org.testng.asserts.Assertion;
import org.testng.asserts.SoftAssert;
public class MyTest {
private Assertion hardAssert = new Assertion();
private SoftAssert softAssert = new SoftAssert();
}
#Test
public void testForSoftAssertionFailure() {
softAssert.assertTrue(false);
softAssert.assertEquals(1, 2);
softAssert.assertAll();
}
Source: http://rameshbaskar.wordpress.com/2013/09/11/soft-assertions-using-testng/
Selenium IDE uses verify to perform a soft assertion, meaning that the test will continue even if the check fails and either report the failures at the end of the test or on the event of a hard assertion.
With TestNG it is possible to have these soft assertions by using custom test listeners. I have documented how to do this on my blog: http://davehunt.co.uk/2009/10/08/using-soft-assertions-in-testng.html
Basically, you need to create your own verify* methods, in these you can catch assertion failures and add them to a map. Then in a custom afterInvocation listener you can set the test to failed if the map is not empty.
I am adding again one of the easiest ways to continue on assertion failure. This was asked here.
try{
Assert.assertEquals(true, false);
}catch(AssertionError e)
{
System.out.println("Assertion error. ");
}
System.out.println("Test Completed.");
Change your assertions to verifications:
verifyTrue(selenium.isTextPresent("Please enter Email ID"));
verifyTrue(selenium.isTextPresent("Please enter Password"));
Once an assertion fails, execution should stop, that's the point of using them.
You can declare an assertion that tests both things, but then you're testing two things at once. Better to fix the cause of the first failure, then move on to the second assertion.
Related
Background
I am using Extent Report Cucumber Adapter for my Cucumber based test automation framework built using Java which is running on Junit. I am using AssertJ assertions for the test conditions.
Scenario
One of the test scenarios require to test all the links on a web page. I have written the code for the same and it's working fine. I am using AssertJ assertion for the test condition under a try block and catching the SoftAssertionError so that my test execution doesn't halt because of the exception and continue validating all the remaining links even if it finds any broken link.
The report mentions the links which are found broken. However, ideally this step should get failed as the script found some broken links. But the report marks the overall step as passed and as a result the scenario is also marked as passed. Now I am not able to figure out how can I mark the step as failed in my Extent Report provided there are some links found which are broken. Kindly suggest a way to do this. I am providing a small snippet of my code for a better understanding.
public void ValidateAllLinks(String linkURL) {
try
{
URL url = new URL(linkURL);
//Creating url connection and getting the response code
HttpURLConnection httpURLConnect=(HttpURLConnection)url.openConnection();
httpURLConnect.setConnectTimeout(5000);
httpURLConnect.connect();
try {
SoftAssertions softly = new SoftAssertions();
softly.assertThat(httpURLConnect.getResponseCode()).as("This is a broken link: " + linkURL).isGreaterThanOrEqualTo(400);
softly.assertAll();
}catch (SoftAssertionError e)
{
e.printStackTrace();
}
if(httpURLConnect.getResponseCode()>=400)
{
System.out.println(linkURL+" - "+httpURLConnect.getResponseMessage()+" is a broken link.");
ExtentCucumberAdapter.addTestStepLog("<b>" + "<font color=" + "red>" + linkURL+" - "+httpURLConnect.getResponseMessage()+" is a broken link." + "</font>" + "</b>");
}
//Fetching and Printing the response code obtained
else{
System.out.println(linkURL+" - "+httpURLConnect.getResponseMessage()+" is working as expected.");
}
}catch (Exception e) {
}
}
Your example is not a good fit for soft assertions as you are testing only one thing. Soft assertions are meant to assert a bunch of things and once you have asserted all the things you wanted to, you call assertAll().
I don't understand why you test twice httpURLConnect.getResponseCode() you could do it once, add the test step log and then fail the test with a fail() method call (either from AssertJ or JUnit)
Note: I am unable to paste the exact framework and code as the server I work on is not accessible from outside. Hence, I will try to explain my problem in simple words and examples.
Overview - I have created a Selenium automation framework which includes TestNG, Maven (POM.XML), Test data files, Scripts and few reusable functions.
Issue I am facing - I use Jenkins to execute my scripts. Jenkins calls POM.XML file which inturn calls testng.xml file (In testng.xml file, I have mentioned the scripts to be executed)
Let's say, I have to perform login action
Main Script
#Test
Public void maintest ()
{
//I use Extent reports for reporting purpose and hence I have created extent
//reporting reusable function which is called in the below fashion.
//If Login method returns pass, ExtentReportingFunc passes Pass to its
//function and displays Pass for that particular Login step in the report.
ExtentReportingFunc (Login(UserName, Password));
}
Reusable functions
Public String Login (String UN, String Pass)
{
//Sendkeys and set UN
driver.findelement (By.id("username")).sendkeys(UN);
//Sendkeys and set Password
driver.findelement (By.id("password")).sendkeys(pass);
//Click Login
driver.findelement (By.id("login")).click ();
//Verifying the message "Welcome User" is displayed after login
try
{
if (driver.findlement (By.id("welcomemessage")).isdisplayed ();
{
return pass;
}
} catch (Exception e)
{
//If welcome message is not found then return fail to the Reporting function
return "fail";
//Below code will be unreachable which is correct as per Java but that is the challenge.
// I tried several ways to find a work around to return fail as above as
// well throw exception but no luck till now.
// Upon throwing exception in the below way, Jenkins displays build as
//failed. If it is not done, though the script has failed,
//Jenkins displays "BUILD PASSED"
throw e;
}
}
//Extent Reporting function
ExtentReportingFunc (String status)
{
log.report (status);
}
Here, the challenge is - In the catch block, If I DO NOT mention "throw e", Jenkins won't understand that the failure has occurred and display "BUILD PASSED" in it's console output. I want it to display "BUILD FAILURE" in Jenkins console. Reason why I want it to display "BUILD FAILED" is - I have integrated JIRA with Jenkins. Only when jenkins show BUILD FAILED, it logs automatically bugs to JIRA. If it is "BUILD PASSED" and though finished status is UNSTABLE, no failures will be displayed in the test result section of Jenkins NOR it logs any bugs in JIRA.
However, that time I won't be able to pass return "fail" to the main reporting function so that it can display login step as failure in the report.
I understand, as per JAVA we can either throw or return in the catch block but not both. Is there any other way we can make this work?
I already had created end to end framework but later realised this problem when I started integrating with Jenkins (or else everything was fine till then).
Why Don't you add assert fail inside you catch statement, that way you are forcing testng test to fail when its inside the catch statement
org.testng.Assert.fail("Im failing here due to ... you can add your e message here");
Just add the line of code before
return "fail";
and keep the rest of the function as is
You can solve this problem by using asserts, so whenever your condition doesn't met the assert will fail and so will your test case and jenkins will show the status of the build as "UNSTABLE" instead of "PASS".
For example, in the above example, instead of using try catch and if condition in the try, it can be solved by a single line of assert which will also give you your desired build status as well.
You can replace the above code with:
Assert.assertTrue(driver.findElement(By.id("welcomemessage")).isDisplayed(), "Element is not present on the page");
So in this, if the element is not displayed on the page, assert will be failed as it is expecting true value but will get false and your jenkins build status will be shown as unstable.
Is there any reason to group multiple assertions:
public void shouldTellIfPrime(){
Assertions.assertAll(
() -> assertTrue(isPrime(2)),
() -> assertFalse(isPrime(4))
);
}
instead of doing this:
public void shouldTellIfPrime(){
Assertions.assertTrue(isPrime(2));
Assertions.assertFalse(isPrime(4));
}
The interesting thing about assertAll is that it always checks all of the assertions that are passed to it, no matter how many fail. If all pass, all is fine - if at least one fails you get a detailed result of all that went wrong (and right for that matter).
It is best used for asserting a set of properties that belong together conceptionally. Something where your first instinct would be, "I want to assert this as one".
Example
Your specific example is not an optimal use case for assertAll because checking isPrime with a prime and a non-prime is independent of each other - so much so that I would recommend writing two test methods for that.
But assume you have a simple class like an address with fields city, street, number and would like to assert that those are what you expect them to be:
Address address = unitUnderTest.methodUnderTest();
assertEquals("Redwood Shores", address.getCity());
assertEquals("Oracle Parkway", address.getStreet());
assertEquals("500", address.getNumber());
Now, as soon as the first assertion fails, you will never see the results of the second, which can be quite annoying. There are many ways around this and JUnit Jupiter's assertAll is one of them:
Address address = unitUnderTest.methodUnderTest();
assertAll("Should return address of Oracle's headquarter",
() -> assertEquals("Redwood Shores", address.getCity()),
() -> assertEquals("Oracle Parkway", address.getStreet()),
() -> assertEquals("500", address.getNumber())
);
If the method under test returns the wrong address, this is the error you get:
org.opentest4j.MultipleFailuresError:
Should return address of Oracle's headquarter (3 failures)
expected: <Redwood Shores> but was: <Walldorf>
expected: <Oracle Parkway> but was: <Dietmar-Hopp-Allee>
expected: <500> but was: <16>
According to documentation here
Asserts that all supplied executables do not throw an AssertionError.
If any supplied Executable throws an AssertionError, all remaining
executables will still be executed, and all failures will be
aggregated and reported in a MultipleFailuresError. However, if an
executable throws an exception that is not an AssertionError,
execution will halt immediately, and the exception will be rethrown as
is but masked as an unchecked exception.
So main difference is that the assertAll will allow all the asserts to execute without breaking the flow while the others like assertTrue and the lot will stop the test with the AssertionError
So in your first example both assertions will execute regardless of pass to fail, while in the second example test will stop if first assertion fails.
Is there any reason to group multiple assertions
If you want all assertions exercised in the unit test.
assert and assertAll, both methods are designed to validate expected output vs actual output.
In simple assert, if the first assertion fails, it fails the entire test case and doesn't validate the rest of asserts. assertAll validates all test cases.
If some assertions fail, then also it will continue the rest of the assertions and return the validation result for all failed assertion.
For example:
public Apple addApple(int appleId, String appleName) {
Apple apple = new Apple(appleId, appleName);
return apple;
}
Test case:
#Test
void addAppleAssertTest() {
System.out.println("AppleCalculatorTest.addAppleTest");
AppleCalculator appleCalculator = new AppleCalculator();
Apple apple = appleCalculator.addApple(1, "apple");
assertNotNull(apple, "apple object should not be null");
assertEquals(11, apple.getAppleId(), "appleId should be 1");
assertEquals("apple1", apple.getAppleName(), "appleName should be apple");
}
#Test
void addAppleAssertAllTest() {
System.out.println("AppleCalculatorTest.addAppleTest");
AppleCalculator appleCalculator = new AppleCalculator();
Apple apple = appleCalculator.addApple(1, "apple");
assertAll(() -> assertNotNull(apple, "apple object should not be null"),
() -> assertEquals(11, apple.getAppleId(), "appleId should be 1"),
() -> assertEquals("apple1", apple.getAppleName(), "appleName should be apple"));
}
I'm currently writing a test and I was wondering if there was a way to assert that a text box is empty.
The test is for a logout command that "clears data" instead of remembering your email or username. The way I modeled this test has the test log in, log out, and the part that I'm stuck at - asserting that the email text box on the login screen is empty after logging out.
I've tried stuff like this:
if (driver.findElement(By.cssSelector("input[type=\"text\"]").equals(""))) {
// This will throw an exception
}
But that doesn't work because the arguments aren't accepted.
Any ideas?
I think you need to get the value attribute:
WebElement myInput = driver.findElement(By.cssSelector("input[type=\"text\"]"));
if (!myInput.getAttribute("value").equals("")) {
fail()
}
The previous answer works, but the assertion can be a lot cleaner. Assertions should always give some reasonable message. Here is an example using the JUnit assertThat and the hamcrest matchers.
import org.junit.Assert;
import static org.hamcrest.Matchers.isEmptyString;
...
WebElement myInput = driver.findElement(By.cssSelector("input[type=\"text\"]"));
Assert.assertThat(myInput.getAttribute("value"), isEmptyString());
Or better yet, give a reason message:
Assert.assertThat("Field should be empty", myInput.getAttribute("value"), isEmptyString());
testLogicalDoc = new LogicalDocumentImpl(-4);
assertTrue(testLogicalDoc==null);
In my code above, I have an assert condition with which I want to make sure I don't create my object with negative size. It is a stringBuilder beneath the covers which throws NegativeArrayBoundsException for a size less than zero. But my junit test fails here. I don't know any other way of making sure an object is not created with a negative size.
Any thoughts on how this could be tested ? or should it be a Junit test at all ??
Many thanks,
-Pan
EDIT:
#Test(expected=NegativeArraySizeException.class)
public void testCreate4b()
{
LogicalDocumentImpl testLogicalDoc = new LogicalDocumentImpl(-4);
}
I'm catching the exception in the LogicalDocumentImpl class but still this test fails with an assertion error but only succeeds when I do a try catch on assertion error ..why is that so ??
if you are throwing NegativeArrayBoundsException your test case could check like this
#Test(expected= NegativeArrayBoundsException.class)
That means your test should throw the exception NegativeArrayBoundsException.
Alternatively you can use fail('should never come here for negative values..')
testLogicalDoc = new LogicalDocumentImpl(-4);
fail('should never come here for negative values..');
Catch AssertionError and fail otherwise:
try {
LogicalDocumentImpl testLogicalDoc = new LogicalDocumentImpl(-4);
fail("should throw");
}
catch (AssertionError e) {
}
Usually Junit test cases are meant to test that the behavior of your code in certain cases is what you expect. Therefore, for this case you expect that an exception be thrown.
Looking at the JUnit faq (http://junit.sourceforge.net/doc/faq/faq.htm#tests_7) you want to use something like the following:
#Test(expected=NegativeArrayBoundsException.class)