I received a table and was able to get and validate the data (whether the email is ACTUALLY an email and so forth). We want to validate the data that is displayed on the front end with the backend. There was a table (I had seen the table- the first column was name and then email , phone number, company , country and the date).
Now, the person on the front-end, switched up the columns. I had seen the table before and therefore I knew the order I would receive my information. I will have to change the code everytime a small change is made on the front end. FYI, the table headers are defined with "data-name" so I will be able to use it if your solution involves something w/ table headers. My code is posted below:
public static void getTableContents(){
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
//TODO Replace this w. until it find command
e.printStackTrace();
}
log("TABLE CONTENTS VERIFICATION");
//Get the table contents
WebElement table_element = driver.findElement(By.className(TABLE_RESPONSE));
//Click on a button to switch into the desired column
WebElement switchColumn = driver.findElement(By.cssSelector(".switchColumn img.pull-right"));
switchColumn.click();
// We do not want the first two contents as they represent default or error case.
List<WebElement> tr_collection=table_element.findElements(By.xpath("//tbody//tr[position()>2]"));
int i=1;
for(WebElement trElement : tr_collection)
{
List<WebElement> td_collection=trElement.findElements(By.xpath("td"));
int j=1;
for(WebElement tdElement : td_collection)
{
String check = tdElement.getText();
if(j==1) {
if(isValidName(check))
passed("Name Test : " );
else{
log(check + " is not a valid name");
}
}
if(j==2) {
if(isValidEmail(check))
{ passed("Email address Test : ");
}
else
log(check + " is not a valid email");
}
if(j==3) {
if(isValidNumber(check))
passed("Valid Number test : ");
else
log(check + " is not a valid number");
}
if(j==6){
if(isValidIccid(check))
passed("Valid Iccid test : ");
else
log(check+ " is not a valid Iccid");
}
if(j==4){
//Blank (for a while)
}
if(j==5){
if(isValidCountry(check))
passed("Valid country test : ");
else
log(check+ " is not a valid country");
}
j++;
}
System.out.println();
i++;
}
}
Is there any quick way of changing this code to my requirements? I can't keep changing the numbers (1,2,3,4,5,6) all the times. I am just looking for clever ways to UPDATE my code rather than CHANGE my code completely.
Any help/tip is greatly appreciated.EDIT: Also, I will be changing my if/else statements to switch cases so it is easier to understand. But as of now, I got a great problem cause I might have to change my entire code so please do not mind.
If your table has the table headers with classes like:
data-Name
data-Email
then you can use this in the code as follows:
// We do not want the first two contents as they represent default or error case.
List<WebElement> tr_collection=table_element.findElements(By.xpath("//tbody//tr[position()>2]"));
int i=1;
for(WebElement trElement : tr_collection)
{
List<WebElement> td_collection=trElement.findElements(By.xpath("td"));
int j=1;
for(WebElement tdElement : td_collection)
{
String check = tdElement.getText();
String header = driver.findElement(By.xpath("//tbody//tr/th[" + j + "]")).getAttribute("class");
if(header.equals("data-Name")) {
if(isValidName(check))
passed("Name Test : " );
else{
log(check + " is not a valid name");
}
}
if(header.equals("data-Email")) {
if(isValidEmail(check))
{ passed("Email address Test : ");
}
else
log(check + " is not a valid email");
}
//... and so on
}
}
You might need to change this line to reflect the correct way for you to identify the table headers:
String header = driver.findElement(By.xpath("//tbody//tr/th[" + j + "]")).getAttribute("class");
but the point is to use the "j" counter to track the column that you are on and check the class name of the header for that column. The order of the columns should no longer matter if you identify them this way.
Related
My object is to scrape data by using Java Selenium. I am able to load selenium driver, connect to the website and fetch the first column then go to the next pagination button until its become disable and write it to the console. Here is what I did so far:
public static WebDriver driver;
public static void main(String[] args) throws Exception {
System.setProperty("webdriver.chrome.driver", "E:\\eclipse-workspace\\package-name\\src\\working\\selenium\\driver\\chromedriver.exe");
System.setProperty("webdriver.chrome.silentOutput", "true");
driver = new ChromeDriver();
driver.get("https://datatables.net/examples/basic_init/zero_configuration.html");
driver.manage().window().maximize();
compareDispalyedRowCountToActualRowCount();
}
public static void compareDispalyedRowCountToActualRowCount() throws Exception {
try {
Thread.sleep(5000);
List<WebElement> namesElements = driver.findElements(By.cssSelector("#example>tbody>tr>td:nth-child(1)"));
System.out.println("size of names elements : " + namesElements.size());
List<String> names = new ArrayList<String>();
//Adding column1 elements to the list
for (WebElement nameEle : namesElements) {
names.add(nameEle.getText());
}
//Displaying the list elements on console
for (WebElement s : namesElements) {
System.out.println(s.getText());
}
//locating next button
String nextButtonClass = driver.findElement(By.id("example_next")).getAttribute("class");
//traversing through the table until the last button and adding names to the list defined about
while (!nextButtonClass.contains("disabled")) {
driver.findElement(By.id("example_next")).click();
Thread.sleep(1000);
namesElements = driver.findElements(By.cssSelector("#example>tbody>tr>td:nth-child(1)"));
for (WebElement nameEle : namesElements) {
names.add(nameEle.getText());
}
nextButtonClass = driver.findElement(By.id("example_next")).getAttribute("class");
}
//printing the whole list elements
for (String name : names) {
System.out.println(name);
}
//counting the size of the list
int actualCount = names.size();
System.out.println("Total number of names :" + actualCount);
//locating displayed count
String displayedCountString = driver.findElement(By.id("example_info")).getText().split(" ")[5];
int displayedCount = Integer.parseInt(displayedCountString);
System.out.println("Total Number of Displayed Names count:" + displayedCount);
Thread.sleep(1000);
// Actual count calculated Vs Dispalyed Count
if (actualCount == displayedCount) {
System.out.println("Actual row count = Displayed row Count");
} else {
System.out.println("Actual row count != Displayed row Count");
throw new Exception("Actual row count != Displayed row Count");
}
} catch (Exception e) {
e.printStackTrace();
}
}
I want to:
scrape more than one column or may be selected columns for example on this LINK name, office and age column
Then want to write these columns data in CSV file
Update
I tried like this but not running:
for(WebElement trElement : tr_collection){
int col_num=1;
List<WebElement> td_collection = trElement.findElements(
By.xpath("//*[#id=\"example\"]/tbody/tr[rown_num]/td[col_num]")
);
for(WebElement tdElement : td_collection){
rows += tdElement.getText()+"\t";
col_num++;
}
rows = rows + "\n";
row_num++;
}
Scraping:
Usually when I want to gather list elements I will select by Xpath instead of CssSelector. The structure of how to access elements through the Xpath is usually more clear, and depends on one or two integer values specifying the element.
So for your example where you want to find the names, you would find an element by the Xpath, the next element in the list's Xpath, and find the differing value:
The first name, 'Airi Satou' is found at the following Xpath:
//*[#id="example"]/tbody/tr[1]/td[1]
Airi's position has the following Xpath:
//*[#id="example"]/tbody/tr[1]/td[2]
You can see that across rows the Xpath for each piece of information differs on the 'td' markup.
The next name in the list, 'Angela Ramos' is found:
//*[#id="example"]/tbody/tr[2]/td[1]
And Angela's position is found:
//*[#id="example"]/tbody/tr[2]/td[2]
You can see that the difference in the column is controlled by the 'tr' markup.
By iterating over values of 'tr' and 'td' you can get the whole table.
As for writing to a CSV, there are a some solid Java libraries for writing to CSVs. I think a straightforward example to follow is here:
Java - Writing strings to a CSV file
UPDATE:
#User169 It looks like you're gathering a list of elements for each row in the table. You want to gather the Xpaths one by one, iterating over the list of webElements that you found originally. Try this, then add to it so it will get text and save it to an array.
for (int num_row = 1; num_row < total_rows; num_row++){
for (int num_col = 1; num_col < total_col; num_col++){
webElement info = driver.findElement(By.xpath("//*[#id=\"example\"]/tbody/tr[" + row_num + ']/td[' + col_num + "]");
}
}
I haven't tested it so it may need a few small changes.
I have List which has names. I am providing name by using Scanner by using advance for loop and checking if the name is in the list it will update the client table else it will update . Category and client table both are in database .The problem is if my list has 4 names it should check if condition only and not go to the else statement till it check the 4 names , if name is not there than(which i give from scanner) should go to else condition needed a logic
String n = scann.nextLine();
List<String> li = new ArrayList<>();
li.add("abc");
li.add("def");
li.add("ghi");
li.add("jkl");
for (int i = 0; i < li.size(); i++) {
if (n.equals(li.get(i))) {
System.out.println("client table update");
} else {
System.out.println("category and client table update");
}
}
}
I kinda get the gist of what you are asking, but please be more precise.
From what i understand from your question is, you want to print either the first or the second thing, depending on wether your name is in the list or not. And in your case, both get printed.
That is because your prints are inside the for loop, thus printing 4 times and printing both options regardless, since your name cant be all 4 names at the same time.
Here is how i would solve this:
List<String> li = new ArrayList<>();
li.add("abc");
li.add("def");
li.add("ghi");
li.add("jkl");
boolean inList = false;
for (int i = 0; i < li.size(); i++) {
if (n.equals(li.get(i))) {
inList = true;
}
}
if (inList) {
System.out.println("client table update");
} else {
System.out.println("category and client table update");
}
You can simply use contains() to check whether the name exists in the list i.e.
if(li.contains(n)){
System.out.println("client table update");
}
else{
System.out.println("category and client table update");
}
String n = scann.nextLine();
List<String> li = new ArrayList<>();
li.add("abc");
li.add("def");
li.add("ghi");
li.add("jkl");
for (String data : li) {
if (li.contains(n)) {
System.out.println("client table update");
} else {
System.out.println("category and client table update");
}
}
System.setProperty("webdriver.chrome.driver", "C:\\Users\\Testing\\Downloads\\chromedriver_win32\\chromedriver.exe");
driver = new ChromeDriver();
driver.navigate().to("https://jpetstore.cfapps.io/catalog");
driver.findElement(By.xpath("//a[contains(text(),'Sign In')]")).click();
driver.findElement(By.name("username")).sendKeys("Testing6738788");
driver.findElement(By.name("password")).sendKeys("test#123");
driver.findElement(By.id("login")).click();
driver.findElement(By.xpath("//div[#id='SidebarContent']/a[contains(#href,'FISH')]/img")).click();
driver.findElement(By.xpath("//td[contains(text(),'Angelfish')]//preceding-sibling::td//a")).click();
List<WebElement> tablelist = driver.findElements(By.xpath("//div[#id='Catalog']//tr"));
for(int i = 0; i < tablelist.size(); i++)
{
String gotvalues = tablelist.get(i).getText();
System.out.println("Values got from the table " +gotvalues);
// Here im using split function but no luck
String[] splitword = gotvalues.split(" ");
for(String words : splitword)
{
System.out.println("Got single words from the split " + words);
// I want to compare the Large Angelfish value from the output
if(words.equalsIgnoreCase("Large Angelfish"))
{
System.out.println("Element present " + words);
}
}
}
Words should be split as "Item ID" -EST-1. I'm facing an issue with the description. The complete word is not getting displayed. How to write code to get item ID, product ID, and description?
Without String Array also you can verify.Try this code see if this help.Take input from keyboard.You have to type both values in console Like EST-1 then Enter and Then Large Angelfish and its compared later.Try Now.
Scanner scan= new Scanner(System.in);
String textID= scan.nextLine(); //Enter ID Here
String textDesc= scan.nextLine();//Enter Desc Here
System.setProperty("webdriver.chrome.driver", "C:\\Users\\Testing\\Downloads\\chromedriver_win32\\chromedriver.exe");
driver = new ChromeDriver();
driver.navigate().to("https://jpetstore.cfapps.io/catalog");
driver.findElement(By.xpath("//a[contains(text(),'Sign In')]")).click();
driver.findElement(By.name("username")).sendKeys("Testing6738788");
driver.findElement(By.name("password")).sendKeys("test#123");
driver.findElement(By.id("login")).click();
driver.findElement(By.xpath("//div[#id='SidebarContent']/a[contains(#href,'FISH')]/img")).click();
driver.findElement(By.xpath("//td[contains(text(),'Angelfish')]//preceding-sibling::td//a")).click();
List<WebElement> tablelist = driver.findElements(By.xpath("//div[#id='Catalog']//tr/td"));
System.out.println(tablelist.size());
for(int i=0;i<tablelist.size();i++)
{
String gotvalues = tablelist.get(0).getText();
String gotvaluesdesc = tablelist.get(2).getText();
// System.out.println("Values got from the table " +gotvalues );
if(gotvalues.trim().equalsIgnoreCase(textID) && gotvaluesdesc.trim().equalsIgnoreCase(textDesc))
{
System.out.println("Element present ID: " + gotvalues + " Desc :" + gotvaluesdesc);
break;
}
As you want to compare the table data, you need to modify your xPath little to fetch the data effectively so that you can avoid the splitting part and you can compare the data easily.
Try the below code :
String xPath = "//div[#id='Catalog']//tr";
List<WebElement> tableList = driver.findElements(By.xpath(xPath));
System.out.println("Item ID\t\tProduct ID\t\tDescription\t\tList Price\t\tOther");
System.out.println("--------------------------------------------------------------------------------------------------");
for(int i=1;i<tableList.size();i++) {
// Below line fetches/stores each table data as column wise
List<WebElement> listData = driver.findElements(By.xpath(xPath+"["+(i+1)+"]/td"));
for(int j=0;j<listData.size();j++) {
// Printing the column data
System.out.print(listData.get(j).getText()+"\t\t");
}
System.out.println();
}
// As per the above output, description is in the 3rd(2nd index) column so you can fetch that with the index number 2.
for(int i=1;i<tableList.size();i++) {
List<WebElement> listData = driver.findElements(By.xpath(xPath+"["+(i+1)+"]/td"));
if(listData.get(2).getText().trim().equals("Large Angelfish")) {
System.out.println("=> 'Large Angelfish' is Matching...");
}
if(listData.get(2).getText().trim().equals("Large Angelfish")) {
System.out.println("=> 'Small Angelfish' is Matching...");
}
}
If you execute the above code, it will print output as below :
Item ID Product ID Description List Price Other
----------------------------------------------------------------------------
EST-1 FI-SW-01 Large Angelfish $16.50 Add to Cart
EST-2 FI-SW-01 Small Angelfish $16.50 Add to Cart
In the above output, Description column number is 3 so you can substitute an index number in the below line for the corresponding column :
listData.get(2).getText().trim().equals("Large Angelfish")
I hope it helps...
I need to get all items in the list, but my script shows only the first item. Please refer to the verifyDisplay assertion in for loop, where I want to show all items in the list. Thanks for the help.
My script:
List<WebElement> groups = driver.findElements(By
.xpath(".//*[#id='competitiveCategoryTemp']/option"));
verifyDisplay("'" + competitive_categories_id_with_space + "'" + "===> The newly added Competitive Category is listed",
By.xpath(".//*[#id='competitiveCategoryTemp']/option"));
boolean sortedGroups = false;
for (int i = 0; i < groups.size() - 1; i++) {
verifyDisplay("'" + groups.get(i).getText() + "'" + "\n"+
"Other Competitive Categories are available and names are SORTED",
By.xpath(".//*[#id='competitiveCategoryTemp']/option"));
if (groups.get(i).getText()
.compareToIgnoreCase(groups.get(i + 1).getText()) > 0) {
sortedGroups = true;
break;
}
sortedGroups = true;
}
if (sortedGroups) {
System.out.println("The Competitive Category names are SORTED");
} else
System.out.println("The Competitive Category names are UN-SORTED");
}
if (groups.get(i).getText()
.compareToIgnoreCase(groups.get(i + 1).getText()) > 0) {
sortedGroups = true;
break;
}
If this condition is met, it breaks out of the for loop, and thus does not proceed forward to the second element. Could that be the problem?
WebDriver has Select class to control dropdown objects. Your approach will also work. But this way it will look neat and you can reuse the existing methods.
Import this lib.
import org.openqa.selenium.support.ui.Select;
Then,
Select dropdown = new Select(driver.findElement(By.id("competitiveCategoryTemp")));
dropdown.getOptions() // will return all the options - it is a List<WebElement>
//To use
for(WebElement option: dropdown.getOptions()){
System.out.println(option.getText());
}
dropdown.getAllSelectedOptions() // will return the default selected options - it is a List<WebElement>
Hello i have to save from a page same <a> in the page and has got as class=my_img i save this elements in a List an after i try to go in firts element of list and after go back get the second element but the selenium give me this error
Exception in thread "main"
org.openqa.selenium.StaleElementReferenceException: Element not found
in the cache - perhaps the page has changed since it was looked up
and this is my code
List <WebElement> Element = drivers.findElements(By.cssSelector(".my_img"));
System.out.println("Megethos"+Element.size());
System.out.println("Pame stous epomenous \n");
for (i = 1; i < Element.size(); i++) {
drivers.manage().timeouts().implicitlyWait(35, TimeUnit.SECONDS);
System.out.println(i+" "+Element.size());
System.out.println(i+" "+Element.get(i));
action.click(Element.get(i)).perform();
Thread.sleep(2000);
System.out.println("go back");
drivers.navigate().back();
Thread.sleep(6000);
drivers.navigate().refresh();
Thread.sleep(6000);
}
Your action.click() and/or navigate() calls are resulting in a page reload, causing the WebElement's in your list to no longer be valid. Put your findElements() call inside the loop:
List <WebElement> Element = drivers.findElements(By.cssSelector(".my_img"));
for (i = 1; i < Element.size(); i++) {
Element = drivers.findElements(By.cssSelector(".my_img"));
drivers.manage().timeouts().implicitlyWait(35, TimeUnit.SECONDS);
System.out.println(i+" "+Element.size());
System.out.println(i+" "+Element.get(i));
action.click(Element.get(i)).perform();
Thread.sleep(2000);
System.out.println("go back");
drivers.navigate().back();
Thread.sleep(6000);
drivers.navigate().refresh();
Thread.sleep(6000);
}
If the primary purpose is to click on the links and get back to the previous page, its better to get "href" attributes for all the "a" elements in the page and navigate to each of them. The way you've followed will always result in StaleElementReferenceExeception, as when you navigate back to the original DOM changes.
Below is the way like I suggested:
List<WebElement> linkElements = driver.findElements(By.xpath("//a[#class='my_img']"));
System.out.println("The number of links under URL is: "+linkElements.size());
//Getting all the 'href' attributes from the 'a' tag and putting into the String array linkhrefs
String[] linkhrefs = new String[linkElements.size()];
int j = 0;
for (WebElement e : linkElements) {
linkhrefs[j] = e.getAttribute("href");
j++;
}
// test each link
int k=0;
for (String t : linkhrefs) {
try{
if (t != null && !t.isEmpty()) {
System.out.println("Navigating to link number "+(++k)+": '"+t+"'");
driver.navigate().to(t);
String title;
title = driver.getTitle();
System.out.println("title is: "+title);
//Some known errors, if and when, found in the navigated to page.
if((title.contains("You are not authorized to view this page"))||(title.contains("Page not found"))
||(title.contains("503 Service Unavailable"))
||(title.contains("Problem loading page")))
{
System.err.println(t + " the link is not working because title is: "+title);
} else {
System.out.println("\"" + t + "\"" + " is working.");
}
}else{
System.err.println("Link's href is null.");
}
}catch(Throwable e){
System.err.println("Error came while navigating to link: "+t+". Error message: "+e.getMessage());
}
System.out.println("++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++");
}