Develop Reference

Click on table row if text is found

I use this Java code with Selenium to select table row based on found text:
WebElement tableContainer = driver.findElement(By.xpath("//div[#class='ag-center-cols-container']"));
List<WebElement> list = tableContainer.findElements(By.xpath("./child::*"));
// check for list elements and print all found elements
if(!list.isEmpty())
{
for (WebElement element : list)
{
System.out.println("Found inner WebElement " + element.getText());
}
}
// iterate sub-elements
for ( WebElement element : list )
{
System.out.println("Searching for " + element.getText());
if(element.getText().equals(valueToSelect))
{
element.click();
break; // We need to put break because the loop will continue and we will get exception
}
}
Full code: https://pastebin.com/ANMqY01y
For some reason table text is not clicked. I don't have exception. Any idea why it's not working properly?

See there are 2 divs with //div[#class='ag-center-cols-container'] with this xpath.
first div does not have anything, while second div has child divs.
I would suggest you to use :
List<WebElement> list = driver.findElements(By.xpath("//div[#class='ag-center-cols-container']//div"));
Remove this line from your code :
WebElement tableContainer = driver.findElement(By.xpath("//div[#class='ag-center-cols-container']"));

How to scrape selected table columns and write them in CVS in Java Selenium

My object is to scrape data by using Java Selenium. I am able to load selenium driver, connect to the website and fetch the first column then go to the next pagination button until its become disable and write it to the console. Here is what I did so far:
public static WebDriver driver;
public static void main(String[] args) throws Exception {
System.setProperty("webdriver.chrome.driver", "E:\\eclipse-workspace\\package-name\\src\\working\\selenium\\driver\\chromedriver.exe");
System.setProperty("webdriver.chrome.silentOutput", "true");
driver = new ChromeDriver();
driver.get("https://datatables.net/examples/basic_init/zero_configuration.html");
driver.manage().window().maximize();
compareDispalyedRowCountToActualRowCount();
}
public static void compareDispalyedRowCountToActualRowCount() throws Exception {
try {
Thread.sleep(5000);
List<WebElement> namesElements = driver.findElements(By.cssSelector("#example>tbody>tr>td:nth-child(1)"));
System.out.println("size of names elements : " + namesElements.size());
List<String> names = new ArrayList<String>();
//Adding column1 elements to the list
for (WebElement nameEle : namesElements) {
names.add(nameEle.getText());
}
//Displaying the list elements on console
for (WebElement s : namesElements) {
System.out.println(s.getText());
}
//locating next button
String nextButtonClass = driver.findElement(By.id("example_next")).getAttribute("class");
//traversing through the table until the last button and adding names to the list defined about
while (!nextButtonClass.contains("disabled")) {
driver.findElement(By.id("example_next")).click();
Thread.sleep(1000);
namesElements = driver.findElements(By.cssSelector("#example>tbody>tr>td:nth-child(1)"));
for (WebElement nameEle : namesElements) {
names.add(nameEle.getText());
}
nextButtonClass = driver.findElement(By.id("example_next")).getAttribute("class");
}
//printing the whole list elements
for (String name : names) {
System.out.println(name);
}
//counting the size of the list
int actualCount = names.size();
System.out.println("Total number of names :" + actualCount);
//locating displayed count
String displayedCountString = driver.findElement(By.id("example_info")).getText().split(" ")[5];
int displayedCount = Integer.parseInt(displayedCountString);
System.out.println("Total Number of Displayed Names count:" + displayedCount);
Thread.sleep(1000);
// Actual count calculated Vs Dispalyed Count
if (actualCount == displayedCount) {
System.out.println("Actual row count = Displayed row Count");
} else {
System.out.println("Actual row count != Displayed row Count");
throw new Exception("Actual row count != Displayed row Count");
}
} catch (Exception e) {
e.printStackTrace();
}
}
I want to:
scrape more than one column or may be selected columns for example on this LINK name, office and age column
Then want to write these columns data in CSV file
Update
I tried like this but not running:
for(WebElement trElement : tr_collection){
int col_num=1;
List<WebElement> td_collection = trElement.findElements(
By.xpath("//*[#id=\"example\"]/tbody/tr[rown_num]/td[col_num]")
);
for(WebElement tdElement : td_collection){
rows += tdElement.getText()+"\t";
col_num++;
}
rows = rows + "\n";
row_num++;
}

Scraping:
Usually when I want to gather list elements I will select by Xpath instead of CssSelector. The structure of how to access elements through the Xpath is usually more clear, and depends on one or two integer values specifying the element.
So for your example where you want to find the names, you would find an element by the Xpath, the next element in the list's Xpath, and find the differing value:
The first name, 'Airi Satou' is found at the following Xpath:
//*[#id="example"]/tbody/tr[1]/td[1]
Airi's position has the following Xpath:
//*[#id="example"]/tbody/tr[1]/td[2]
You can see that across rows the Xpath for each piece of information differs on the 'td' markup.
The next name in the list, 'Angela Ramos' is found:
//*[#id="example"]/tbody/tr[2]/td[1]
And Angela's position is found:
//*[#id="example"]/tbody/tr[2]/td[2]
You can see that the difference in the column is controlled by the 'tr' markup.
By iterating over values of 'tr' and 'td' you can get the whole table.
As for writing to a CSV, there are a some solid Java libraries for writing to CSVs. I think a straightforward example to follow is here:
Java - Writing strings to a CSV file
UPDATE:
#User169 It looks like you're gathering a list of elements for each row in the table. You want to gather the Xpaths one by one, iterating over the list of webElements that you found originally. Try this, then add to it so it will get text and save it to an array.
for (int num_row = 1; num_row < total_rows; num_row++){
for (int num_col = 1; num_col < total_col; num_col++){
webElement info = driver.findElement(By.xpath("//*[#id=\"example\"]/tbody/tr[" + row_num + ']/td[' + col_num + "]");
}
}
I haven't tested it so it may need a few small changes.

Shadow Root - click in a href under several shadow roots

I have a list of links inside of several shadowRoots. Already solved this problem.
public WebElement expandRootElement(WebElement element) {
WebElement ele = (WebElement) ((JavascriptExecutor) driver).executeScript("return arguments[0].shadowRoot",element);
return ele;
}
WebElement root5_adminPanel = shadowRoot4_MduiContainerChild2.findElement(By.cssSelector("#layout > border-layout > ng-view > admin-panel"));
WebElement shadowRoot5_AdminPanel= expandRootElement(root5_adminPanel);
WebElement root6_breadCrumb = shadowRoot5_AdminPanel.findElement(By.cssSelector("#layout > border-layout > breadcrumb"));
WebElement shadowRoot6_breadCrumb = expandRootElement(root6_breadCrumb);
WebElement root6_domainPanel = shadowRoot5_AdminPanel.findElement(By.cssSelector("#layout > border-layout > ng-view > gdsr-domain-panel"));
WebElement shadowRoot6_domainPanel = expandRootElement(root6_domainPanel);
WebElement root7_selectDomain = shadowRoot6_domainPanel.findElement(By.cssSelector("#domainContainer > domain-panel-item.ng-binding.last"));
WebElement shadowRoot7_selectDomain = expandRootElement(root7_selectDomain);
When I reach this shadowRoot7, I have a list of items with the same name, which I already created a List to fix it.
List<WebElement> rows_table = shadowRoot6_domainPanel.findElements(By.cssSelector("#domainContainer > domain-panel-item:nth-child(n)"));
(They are around 45 items)
This will select all of them, in this case all the domain-panel-item rows.
My problem is that each domain-panel-item still contain another shadowRoot (the same path for all of them) an i would like to select a random item, not the first or last one, for example, the item number 43.
enter image description here
My solution was this one but it doesn't work because it doesnt access to the link that i want:
public void clickSelectedDomain(String domain) {
List<WebElement> rows_table = shadowRoot6_domainPanel.findElements(By.cssSelector("#domainContainer > gdsr-domain-panel-item:nth-child(n)"));
int rows_count = rows_table.size();
for (int row=0; row<rows_count; row++) {
if(rows_table.get(row).getAttribute("href").contains(domain)) {
rows_table.get(row).click();
}
}
}
Some have an idea how to fix this?

You solved the problem by calling recursively executeScript() in order to get the imbricated Shadow DOMs but actually you could have just called executeScript() once, and inside got the Shadow DOMs successively.
driver.executeScript( function ()
{
var root1 = document.querySelector( 'selector string 1' ).shadowRoot
var root2 = root1.querySelector( 'selector string 2' ).shadowRoot
var root3 = root2.querySelector( 'selector string 3' ).shadowRoot
...
return foundElement
}
Anyways, in the for() {} loop, you should extract the ultimate Shadow DOM one last time, and then select the <a> element to check its content.

I am trying to click on an option in the dropdown with the help of the li tag but not getting the output neither am i getting any error

public class CssSelector3 {
public static void main(String[] args) throws InterruptedException {
WebDriver driver = new FirefoxDriver();
driver.get("http://qa.letslearnindia.com");
driver.manage().window().maximize();
driver.findElement(By.linkText("Sign in")).click();
Thread.sleep(5000);
driver.findElement(By.cssSelector("input[id='inputSuccess2']")).sendKeys("tester42#gmail.com");
driver.findElement(By.cssSelector("input[id='inputSuccess3']")).sendKeys("123456");
driver.findElement(By.cssSelector("input[id='btn_login']")).click();
Thread.sleep(5000);
driver.findElement(By.xpath("//*[#id='navbar']/ul/li[2]/a")).click();
driver.findElement(By.xpath("//*[#id='horizontalTab']/div/div[1]/div[1]/div[2]/a/input")).click();
Thread.sleep(5000);
driver.findElement(By.xpath("//*[#id='full_height_base']/div/div[3]/div[3]/div[2]/div/ul[2]/li[1]/a")).click();
driver.findElement(By.xpath("//*[#id='courseTitle']")).sendKeys("Automation Test");
driver.findElement(By.xpath("//*[#id='courseSubtitle']")).sendKeys("Automating the test cases");
Thread.sleep(5000);
WebElement dropdown = driver.findElement(By.xpath("//*[#id='validate-me-plz']/div[1]/div[2]/div/p/span"));
List<WebElement> li = dropdown.findElements(By.tagName("li"));
System.out.println(li.size());
String element;
for(int i =0; i<li.size();i++){
element = li.get(i).getAttribute("data-val");
if(element.equals("English")){
li.get(i). click();

When choosing from <select> tag you should use Select class
WebElement dropdown = driver.findElement(By.id("courseLanguage")); // locate the dropdown
Select select = new Select(dropdown); // initialize select
select.selectByVisibleText("English"); // choose the option with "English" as text
// select.selectByValue("English"); // choose the option with "English" as value
Its still giving an error as "Element is not currently visible and so may not be interacted with"
To make sure the element is visible before interaction use explicit wait
// this will wait up to 10 seconds for the dropdown to be visible and will return the dropdown element
WebDriverWait wait = new WebDriverWait(driver, 10);
WebElement dropdown = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("courseLanguage")));
Select select = new Select(dropdown);
select.selectByVisibleText("English");

try by selecting all the elements with a select tag by using findElements method
and then pass the desired element to the Select class as below :
List<WebElement> AllselectTags= driver.findElements(By.tagName("select"));
WebElement selectedElement = AllselectTags.get(0);
Select s = new Select(selectedElement);
s.selectByValue("English");

How to select all list options from drop downlist loop through them selcet each option and click submit button using selenium webdriver

Here is the link to print name and meaning columns of all pages using drop down
Try to build the script for following:
1. Go to http://babynames.merschat.com/index.cgi?function=Search&origin=Sanskrit&gender=f
2. print the name and meaning columns to syso.
I was able to print page 1 as it is a default page.
Here is the code:
public class BabyNamesAndMeanings {
WebDriver driver = new FirefoxDriver();
#BeforeClass
public void setUp() {
driver.get("http://babynames.merschat.com/index.cgi?function=Search&origin=Sanskrit&gender=f");
driver.manage().window().maximize();
}
#Test
public void printBabyNamesAndMeaningsOfFirstPage() {
WebElement baby_names = driver
.findElement(By
.xpath("//tbody/tr[7]/td[3]/table[2]/tbody/tr[2]/td[2]/font/table[1]/tbody"));
List<WebElement> names = baby_names.findElements(By
.xpath("//tr/td[1]/font/a"));
List<WebElement> meanings = baby_names.findElements(By
.xpath("//tr/td[4]/font/a"));
for (int i = 0; i < names.size(); i++) {
System.out.println("Name: " + names.get(i).getText()
+ " Meaning: " + meanings.get(i).getText());
}
}
I don't know how to loop through rest of the options in the drop down list at the bottom of the page and hit submit button to print name and meaning of all the pages.
There are 100+ pages.
Thanks in advance.

The code below will do your job.
driver.get("http://babynames.merschat.com/index.cgi?function=Search&origin=Sanskrit&gender=f");
List<WebElement> pageOptions = new Select(driver.findElement(By.xpath("//select[#name='page']"))).getOptions();//Get all options in dropdown
ArrayList<String> pageDd = new ArrayList<String>();
for(WebElement eachPage:pageOptions){
pageDd.add(eachPage.getText());//Save text of each option
}
int i=1;
for(String eachVal:pageDd){
new Select(driver.findElement(By.xpath("//select[#name='page']"))).selectByVisibleText(eachVal);//Select page
driver.findElement(By.xpath("//input[#value='Go']")).click();//Click on go
List<WebElement> names = driver.findElements(By.xpath("//a[contains(#title,' meanings and popularity')]"));//Get all names on page
for(WebElement eachName:names){
String name = eachName.getText(); //Get each name's text
WebElement mean = eachName.findElement(By.xpath("./../../..//a[contains(#title,'Names for baby with meanings like ')]"));//Get meaning for that name
String meaning = mean.getText();//Get text of meaning
System.out.println(i+") Name: " +name+ " Meaning: " + meaning);//Print the data
i++;
}
}
Try and understand the way requirement is achieved. If you have any doubt ask.

Another method to iterate and select all the Dropdown values
Select dropdown= new Select(WebUIDriver.webDr.findElement(By.xpath("enter xpath")));
int noOfDropDownValues= dropdown.getOptions().size()-1;
for(int i=0;i<noOfDropDownValues;i++){
new Select(WebUIDriver.webDr.findElement(By.xpath("Enter Xpath']"))).selectByValue(String.valueOf(i));
}