Develop Reference

Cucumber Scenarios to be run in Sequential Order

I have few concerns regarding cucumber framework:-
1. I have single Feature file(steps are dependent on each other)and i want to run all the scenarios in order, by default they are running in random order.
2. How to run a single feature file multiple times?
I put some tags and tried to run but no luck.
#Given("Get abc Token")
public void get_abc_Token(io.cucumber.datatable.DataTable dataTable) throws URISyntaxException {
DataTable data=dataTable.transpose();
String tkn= given()
.formParam("parm1",data.column(0).get(1))
.formParam("parm2", data.column(1).get(1))
.formParam("parm3", data.column(2).get(1))
.when()
.post(new URI(testurl)+"/abcapi")
.asString();
jp=new JsonPath(tkn);
Token=jp.getString("access_token");
if (Token==null) {
Assert.assertTrue(false,"Token is NULL");
}else {
}
}
#Given("Get above token")
public void get_abovetoken(io.cucumber.datatable.DataTable dataTable) throws URISyntaxException {
System.out.println("Token is " +Token);
}
}
So in the above steps i am getting token from one step and trying to print token in other step but i got null and not the actual value, because my steps are nunning randommmally
Please note i am running TestRunner via testng.xml file.

Cucumber and testing tools in general are designed to run each test/scenario as a completely independent thing. Linking scenarios together is a terrible anti-pattern don't do it.
Instead learn to write scenarios properly. Scenarios and feature files should have no programming in them at all. Programming needs to be pushed down into the step definitions.
Any scenario, no matter how complicated can be written in 3 steps if you really want to. Your Given can set up any amount of state. Your When deals with what you are doing, and your Then can check up any number of conditions.
You do this by pushing all the detail down out of the scenario and into the step definitions. You improve this further by having the step definitions call helper methods that do all the work.

Jenkins displays "Build Success" even when tests are failed due to error handling in scripts (try and catch block)

Note: I am unable to paste the exact framework and code as the server I work on is not accessible from outside. Hence, I will try to explain my problem in simple words and examples.
Overview - I have created a Selenium automation framework which includes TestNG, Maven (POM.XML), Test data files, Scripts and few reusable functions.
Issue I am facing - I use Jenkins to execute my scripts. Jenkins calls POM.XML file which inturn calls testng.xml file (In testng.xml file, I have mentioned the scripts to be executed)
Let's say, I have to perform login action
Main Script
#Test
Public void maintest ()
{
//I use Extent reports for reporting purpose and hence I have created extent
//reporting reusable function which is called in the below fashion.
//If Login method returns pass, ExtentReportingFunc passes Pass to its
//function and displays Pass for that particular Login step in the report.
ExtentReportingFunc (Login(UserName, Password));
}
Reusable functions
Public String Login (String UN, String Pass)
{
//Sendkeys and set UN
driver.findelement (By.id("username")).sendkeys(UN);
//Sendkeys and set Password
driver.findelement (By.id("password")).sendkeys(pass);
//Click Login
driver.findelement (By.id("login")).click ();
//Verifying the message "Welcome User" is displayed after login
try
{
if (driver.findlement (By.id("welcomemessage")).isdisplayed ();
{
return pass;
}
} catch (Exception e)
{
//If welcome message is not found then return fail to the Reporting function
return "fail";
//Below code will be unreachable which is correct as per Java but that is the challenge.
// I tried several ways to find a work around to return fail as above as
// well throw exception but no luck till now.
// Upon throwing exception in the below way, Jenkins displays build as
//failed. If it is not done, though the script has failed,
//Jenkins displays "BUILD PASSED"
throw e;
}
}
//Extent Reporting function
ExtentReportingFunc (String status)
{
log.report (status);
}
Here, the challenge is - In the catch block, If I DO NOT mention "throw e", Jenkins won't understand that the failure has occurred and display "BUILD PASSED" in it's console output. I want it to display "BUILD FAILURE" in Jenkins console. Reason why I want it to display "BUILD FAILED" is - I have integrated JIRA with Jenkins. Only when jenkins show BUILD FAILED, it logs automatically bugs to JIRA. If it is "BUILD PASSED" and though finished status is UNSTABLE, no failures will be displayed in the test result section of Jenkins NOR it logs any bugs in JIRA.
However, that time I won't be able to pass return "fail" to the main reporting function so that it can display login step as failure in the report.
I understand, as per JAVA we can either throw or return in the catch block but not both. Is there any other way we can make this work?
I already had created end to end framework but later realised this problem when I started integrating with Jenkins (or else everything was fine till then).

Why Don't you add assert fail inside you catch statement, that way you are forcing testng test to fail when its inside the catch statement
org.testng.Assert.fail("Im failing here due to ... you can add your e message here");
Just add the line of code before
return "fail";
and keep the rest of the function as is

You can solve this problem by using asserts, so whenever your condition doesn't met the assert will fail and so will your test case and jenkins will show the status of the build as "UNSTABLE" instead of "PASS".
For example, in the above example, instead of using try catch and if condition in the try, it can be solved by a single line of assert which will also give you your desired build status as well.
You can replace the above code with:
Assert.assertTrue(driver.findElement(By.id("welcomemessage")).isDisplayed(), "Element is not present on the page");
So in this, if the element is not displayed on the page, assert will be failed as it is expecting true value but will get false and your jenkins build status will be shown as unstable.

Trying to suppress errors while attaching browser

EDIT: I'm using the LeanFT Java SDK 14.50
EDIT2: for text clarification
I'm writing test scripts for a web application that sometimes opens popup browsers for specific actions. So natually when that happens, I will attach the new browser using BrowserFactory.attach(...). The problem is that leanFT does not seem to have a way to validate that the browser exists before attaching it, and if I try to attach it too early, it will fail. And I don't like to use an arbitrairy wait/sleep time as I can never really know how much time it's going to take for the browser to get be ready. So my solution to this is below
private Browser attachPopUpBrowser(BrowserType bt, RegExpProperty url){
Browser browser = null;
int iteration = 0;
//TimeoutLimit.SHORT = 15000
while (browser == null && iteration < TimeoutLimit.SHORT.getLimit()) {
try {
Reporter.setReportLevel(ReportLevel.Off);
browser = BrowserFactory.attach(
new BrowserDescription.Builder()
.type(bt)
.url(url)
.build()
);
Reporter.setReportLevel(ReportLevel.All);
} catch (GeneralLeanFtException e) {
try {
Thread.sleep(1000);
iteration += 1000;
} catch (InterruptedException e1) {
}
}
}
return browser;
}
Now, this works wonderfully with one exception. It generates errors in the leanft test result. Errors that I want to ignore because I know that it will fail a few times before it will succeed. As you can see, I've tried changing the ReportLevel while doing this in order to suppress the error logging, but it doesn't work. I've tried using
Browser[] browsers = BrowserFactory.getallOpenBrowsers(BrowserDescription);
thinking that it will return an empty Array if it finds nothing, but I still get errors while the browser is not ready. Does anyone have suggestions as to how I could work around this?
TL;DR
I'm looking for a way to either suppress the errors generated within my While..Loop or to validate that the browser is ready before attaching it. All of that, so that I can have a nice and clean Run Result at the end of my test (because these errors will present false negatives in all nearly all of my tests)
Addendum
Also, when the attach fails for the first time, I get a an exception
com.hp.lft.sdk.ReplayObjectNotFoundException: attachApplication
as expected, but all subsequent failures are throwing
com.hp.lft.sdk.GeneralLeanFtException: Cannot read property 'match' of null
I've compared both stack traces and they are identical except for the last 2 lines which happen within the ReplayExceptionFactory.CreateDefault() so I think that there is something that gets corrupted during the exception generation, but that is within the leanft.sdk.internal package so there might not be a lot we can do about it right now.I'm guessing that if I did not get that second "cannot read property" exception, I would correctly get the ReplayObjectNotFoundException until the browser is correctly attached.

I'd rather not force an attach endlessly until it works. Even if we'd solve the false negatives, we'd still have a not so good approach to the problem.
The cleanest solution would be to see if there is anything to attach to in the first place.
And you can do just that by getting all the browser instances that meets your description.
Browser[] browsers = BrowserFactory.getAllOpenBrowsers(new BrowserDescription.Builder().build());
Any element in this collection is an already "attached" browser - you can start using it.
If the list doesn't contain your browser instance, rerun the query.

How to continue execution when Assertion is failed

I am using Selenium RC using Java with eclipse and TestNG framework. I have the following code snippet:
assertTrue(selenium.isTextPresent("Please enter Email ID"));
assertTrue(selenium.isTextPresent("Please enter Password"));
First assertion was failed and execution was stopped. But I want to continue the further snippet of code.

I suggest you to use soft assertions, which are provided in TestNg natively
package automation.tests;
import org.testng.asserts.Assertion;
import org.testng.asserts.SoftAssert;
public class MyTest {
private Assertion hardAssert = new Assertion();
private SoftAssert softAssert = new SoftAssert();
}
#Test
public void testForSoftAssertionFailure() {
softAssert.assertTrue(false);
softAssert.assertEquals(1, 2);
softAssert.assertAll();
}
Source: http://rameshbaskar.wordpress.com/2013/09/11/soft-assertions-using-testng/

Selenium IDE uses verify to perform a soft assertion, meaning that the test will continue even if the check fails and either report the failures at the end of the test or on the event of a hard assertion.
With TestNG it is possible to have these soft assertions by using custom test listeners. I have documented how to do this on my blog: http://davehunt.co.uk/2009/10/08/using-soft-assertions-in-testng.html
Basically, you need to create your own verify* methods, in these you can catch assertion failures and add them to a map. Then in a custom afterInvocation listener you can set the test to failed if the map is not empty.

I am adding again one of the easiest ways to continue on assertion failure. This was asked here.
try{
Assert.assertEquals(true, false);
}catch(AssertionError e)
{
System.out.println("Assertion error. ");
}
System.out.println("Test Completed.");

Change your assertions to verifications:
verifyTrue(selenium.isTextPresent("Please enter Email ID"));
verifyTrue(selenium.isTextPresent("Please enter Password"));

Once an assertion fails, execution should stop, that's the point of using them.
You can declare an assertion that tests both things, but then you're testing two things at once. Better to fix the cause of the first failure, then move on to the second assertion.

How to run one test against multiple sites using Selenium and TestNG

Given 3 web applications under test with given URLs:
www.A.com
www.B.com
www.C.com
How do I proceed to design a way using Selenium to run a single TestNG test against these three browsers and print out the results.
Current Strategy:
I have a java class with a main method, a properties file containing the the 3 urls listed above.
In this class i have a while loop that parses these properties file like below snippet, and for each url, programmatically calls an ant task that automates the build from compilation to test-run to result archiving. The problem is that after the first run completes, it doesn't return to the while loop to do it again. You might ask why i want to run it three times. The idea as already explained is to be able to run a suite of tests against multiple websites automatically and printout results without intervention. Code Snippet
try {
reader = new BufferedReader(new FileReader(new File(filename)));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
try {
while((line=reader.readLine()) != null){
//call ant target to archive result
userprops.setProperty("url", line);
org.apache.tools.ant.Main.start(target, userprops, loader);
}
}catch (IOException e) {
e.printStackTrace();
}
I hope somebody understands what am trying to do and can help me understand why the while loop terminates after the first test run. Also maybe can offer another easier strategy with TestNG.
thanks Guys. Y.ou guys Rock!!

It seems to me that if you are using ANT you shouldn't need your class. I would just use three targets and assign the different properties within those targets.