Develop Reference

Limiting WebDriver scope is not working as expected

Can anyone please explain why data.size() is coming up as 13 and why data1.size() is coming up as 364?
As per my understanding, data.size() should be 0 because <td> is not a valid xpath expression and data1.size() should be 13 as there are 13 <td> tags inside/under the precipitation element. 364 is actually the total number of "td" tags in the particular webpage.
System.setProperty("webdriver.chrome.driver", "C:\\Program Files\\Java\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.manage().window().maximize();
driver.get("https://en.wikipedia.org/wiki/York");
Actions a = new Actions(driver);
WebElement precipitation = driver.findElement(By.xpath("//a[#title='Precipitation']//ancestor::tr"));
a.moveToElement(precipitation).build().perform();
List<WebElement> data = precipitation.findElements(By.xpath("td"));
List<WebElement> data1 = precipitation.findElements(By.xpath("//td"));
List<WebElement> data2 = precipitation.findElements(By.tagName("td"));
List<WebElement> data3 = precipitation.findElements(By.cssSelector("td"));
List<WebElement> data4 = driver.findElements(By.cssSelector("td"));
List<WebElement> data5 = driver.findElements(By.xpath("td"));
List<WebElement> data6 = driver.findElements(By.xpath("abcxyz"));
System.out.println("data = " +data.size());
System.out.println("data1 = " +data1.size());
System.out.println("data2 = " +data2.size());
System.out.println("data3 = " +data3.size());
System.out.println("data4 = " +data4.size());
System.out.println("data5 = " +data5.size());
System.out.println("data6 = " +data6.size());
driver.close();

Actually "td" is a valid expression, it selects the nodes with node name td, and since you are not using / or // the search doesn't start in a higher element in the DOM hierarchy. This means that
precipitation.findElements(By.xpath("td"));
is the equivalent of
driver.findElement(By.xpath("//a[#title='Precipitation']//ancestor::tr/td"));
// will locate all the matching elements in the DOM, i.e. all the <td> nodes. If you want to use precipitation as the root node and start the search from there you need to add . for current context
precipitation.findElements(By.xpath(".//td"));
for all <td>s under the <tr>, or
precipitation.findElements(By.xpath("./td"));
for direct children of the <tr>
See XPath Syntax for reference.

Selecting Nodes
XPath uses path expressions to select nodes in an XML document. The node is selected by the help of the following semantics:
So as per the discussion above once you identify the webElement precipitation
WebElement precipitation = driver.findElement(By.xpath("//a[#title='Precipitation']//ancestor::tr"));
next to get the descendent <td> tags instead of:
List<WebElement> data = precipitation.findElements(By.xpath("td"));
As per the discussion above to identify the desired descendant <td> elements in the most effective way, you need to append the / character to denote from the from which node you would initiate the search.
So effectively, your line of code will be:
List<WebElement> data = precipitation.findElements(By.xpath(".//td"));
In a single line:
List<WebElement> data = precipitation.findElements(By.xpath("//a[#title='Precipitation']//ancestor::tr//td"));
which would select for you the correct 13 nos of desired <td> nodes.

UnexpectedTagNameException: Element should have been "select" but was "a" while trying to get the texts of dropdown menu using Selenium and Java

While trying to get the menu list, I'm getting this error message:
Exception in thread "main" org.openqa.selenium.support.ui.UnexpectedTagNameException: Element should have been "select" but was "a".
Here below is the code:
public static void main(String[] args) {
// TODO Auto-generated method stub
System.setProperty("webdriver.chrome.driver", "D:\\selenium files\\chromedriver_win32_new\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.manage().window().maximize();
driver.get("https://www.tutorialspoint.com/tutor_connect/index.php");
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
WebElement ele = driver.findElement(By.xpath("//*[#id=\"logo-menu\"]/div/div[1]/div/a"));
Select s = new Select(ele);
//getting list of menu
List <WebElement> op = s.getOptions();
int size = op.size();
for(int i =0; i<size ; i++){
String options = op.get(i).getText();
System.out.println(options);
}
}
}

That is because the element you are trying to cast is a link tag and not a select tag.
You need to give the Xpath or CSS of the correct Select element and then cast it from WebElement into a Select ojbect.
In the example you are using there is not real selector, you first need to click on the buttons that says "Categories" and later take the options that appear:
WebElement button = driver.findElementByCSS("div[class='mui-dropdown']");
button.click();
WebElement SelectObj = driver.findElementByCSS("ul[class*='mui--is-open']");
Select s = new Select(SelectObj);

The desired element is not a <select> element but a <ul> element. Once you click on the <a> element then only the classname mui--is-open is appended to the desired <ul> element.
Solution
So to get the contents of the dropdown menu you need to induce WebDriverWait for the visibilityOfAllElementsLocatedBy() and you can use Java8 stream() and map() and you can use either of the following Locator Strategies:
Using cssSelector:
new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.cssSelector("a.mui-btn.mui-btn--primary.categories"))).click();
System.out.println(new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.cssSelector("ul.mui-dropdown__menu.cat-menu.mui--is-open a"))).stream().map(element->element.getText()).collect(Collectors.toList()));
Using xpath:
new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.xpath("//a[#class='mui-btn mui-btn--primary categories']"))).click();
System.out.println(new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.xpath("//ul[#class='mui-dropdown__menu cat-menu mui--is-open']//a"))).stream().map(element->element.getText()).collect(Collectors.toList()));
References
You can find a couple of relevant detailed discussions in:
How to extract the text iterating specific rows within a table using XPath with Selenium and Java
How to extract the dynamic values of the id attributes of the table elements using Selenium and Java
How to print runs scored by a batsmen in a scoreboard format webelements through CSS selector using Selenium and Java

how to get the id from the Xpath as a text

I had taken the xPath as below but I am not able to get the value 145666 when I try to print.
WebDriverWait wait = new WebDriverWait(driver, 20);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//span[contains(#class,'mat-content')]//div[#id='demandTrackingID']")));
Now iam getting the below Expception :
org.openqa.selenium.TimeoutException: Expected condition failed: waiting for visibility of element located by By.xpath: //span[contains(#class,'mat-content')]//div[#id='TrackID']
Note: This Html Element is a Non Visible Element I need to inspect the panel and get this element as there is no field for this Element
HTML:
<div _ngcontent-wbh-c179="" id="TrackID" class="disp-none">14566 </div>

Please try this:
WebElement trackid= driver.findElement(By.xpath("//span[contains(#class,'mat-content')]//div[#id='TrackID']"));
System.out.println("getid:"+ trackid.getText());
You will possible need to add an explicit wait before that to make element fully loaded before retrieving it's content
WebDriverWait wait = new WebDriverWait(driver, 30);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//span[contains(#class,'mat-content')]//div[#id='TrackID']")));
WebElement trackid= driver.findElement(By.xpath("//span[contains(#class,'mat-content')]//div[#id='TrackID']"));
System.out.println("getid:"+ trackid.getText());

id is ID not id
so use this :
WebElement trackid= driver.findElement(By.id("TrackID"));
System.out.println("getid:"+ trackid.getText());

The element that contains the text you want has an ID, TrackID, not to be confused with the "ID" text on the page. You can use that to locate the element more easily than the XPath you are currently using. Once you have the element, you can use WebDriverWait to wait for the element to contain text. Code is below.
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until((ExpectedCondition<Boolean>) d -> d.findElement(By.id("TrackID")).getText().length() != 0);
Given the class class="disp-none" on the element, I'm guessing that the element might not be visible. If this is the case, you'll have to use JavaScript to get the invisible text. Selenium was designed to only interact with visible elements and will throw an exception if you try to interact with elements that are not visible.
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until((ExpectedCondition<Boolean>) d -> d.findElement(By.id("TrackID")).getText().length() != 0);
WebElement element = driver.findElement(By.id("TrackID"));
String trackID = (JavascriptExecutor)driver.executeScript("return arguments[0].text", element);

Unable to select the drop down value inside shadow element(webpage built by Polymar JS)

My web page built using the PolymerJS framework,I am trying to select the drop down value inside my shadow element using below code, I went to last point but the selection has not happened, I want to select any dropdown item from the screen based on text (trip.com, priceline.com etc), please help.
WebElement root2 = shadowRoot1.findElement(By.id("hotels-search"));
WebElement shadowRoot2 = expandRootElement(root2);
WebElement root3 = shadowRoot2.findElement(By.id("detail"));
WebElement shadowRoot3 = expandRootElement(root3);
WebElement root4 = shadowRoot3.findElement(By.id("rates"));
WebElement shadowRoot4 = expandRootElement(root4);
WebElement root5 = shadowRoot4.findElement(By.tagName("hotel-rates-sort-bar"));
WebElement shadowRoot5 = expandRootElement(root5);
WebElement element = shadowRoot5.findElement(By.cssSelector("#__providersDropdown > div.dropdown > div"));
Select select= new Select(element);
select.selectByIndex(2);```
**My elements in the below screenshot**
[![enter image description here][1]][1]
[1]: https://i.stack.imgur.com/buYuq.png

Selenium cannot select dropdown on span element

To get the element I have used a nested loop.I am able to click on dropdwn.PFB the code:
List<WebElement> webElements1 = driver.findElements(By.className("selectboxit"));
for(WebElement webElement1 : webElements1) {
if( webElement1.getAttribute("name").equals("TransactionHistoryFG.OUTFORMAT"))
{
WebElement web1 = webElement1.findElement(By.className("selectboxit-text"));
web1.click();
}
}
When i am trying to use Select on webelement i am getting error :
org.openqa.selenium.support.ui.UnexpectedTagNameException: Element
should have been "select" but was "span"
How can i select dropdown i span element?

Possible solution for selecting dropdown using selenium webdriver is:
Select select = new Select(driver.findElement(By.xpath("//path_to_drop_down")));
select.deselectAll();
select.selectByVisibleText("Value1");
Instead of the approach you mentioned above, let me know if this helps :)

List<WebElement> webElements1 = driver.findElements(By.cssSelect(".selectboxit"));
for(WebElement webElement1 : webElements1) {
if( webElement1.getAttribute("name").equals("TransactionHistoryFG.OUTFORMAT"))
{
WebElement web1 = webElement1.findElement(By.className("selectboxit-text"));
JavascriptExecutor js = (JavascriptExecutor)driver;
js.executeScript("arguments[0].click();", web1);
}
}

Well, it is not the best way to do it, but in some cases it can be used:
it will open your combobox
driver.findElements(By.cssSelect(".selectboxit")).click()
now, you just need to write the specified value
driver.findElements(By.cssSelect(".selectboxit")).sendKeys("<value>");
OR
driver.findElements(By.cssSelect(".selectboxit")).sendKeys(Keys.ARROW_DOWN).
Use "ARROW_DOWN" as wanted to select your specified value.