Develop Reference

Locating WebElement using different locators(NoSuchElementException)

I am having problem with locating WebElement using different locators. In the below html tag I tried locating the "write a review" WebElement with different locators like linkText,xpath,classname but still getting NoSuchElementException
-->url https://www.tripadvisor.in/-->search for Club Mahindra-->click on Club Mahindra-->click on write a review.
<a href="/UserReview-g641714-d1156207-Club_Mahindra_Madikeri_Coorg-
Madikeri_Kodagu_Coorg_Karnataka.html" target="_blank" class="ui_button
primary">Write a review</a>
Locators used
By.xpath("//*[#id="component_12"]/div/div[2]/div/div[2]/div/div[1]/a")
By.xpath("//a[#href='/UserReview-g641714-d1156207-
Club_Mahindra_Madikeri_Coorg-Madikeri_Kodagu_Coorg_Karnataka.html']")
By.className("ui_button primary")
By.linkText("Write a review")
I am really confused. What am I doing wrong?

I have tired to analyse and implement the same. Below are my findings:
-> Application waiting time is more as there are lots of dynamic loads applicable for the page.
-> Proper waits needs to be implemented
-> Check whether all the pages are getting opened in the same tab or clicking on each link is redirecting to new tabs, if so then we have to switch to that particular window.
-> Below code works like a pro for me.
driver.get("https://www.tripadvisor.in/");
WebDriverWait wait = new WebDriverWait(driver, 120);
WebElement ele1 =
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//*[text()='Where to?']")));
ele1.click();
WebElement ele2= wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//*[#placeholder='Where to?']")));
ele2.sendKeys("club mahindra, india");
WebElement ele3= wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//span[contains(text(),'Search for ')]")));
ele3.click();
WebElement ele4= wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//span[contains(text(),'Club Mahindra Madikeri, Coorg')]")));
ele4.click(); //this click leads to a new tab
Set<String> winHandles = driver.getWindowHandles();
for(String str : winHandles) {
driver.switchTo().window(str);
}
System.out.println(driver.getTitle());
WebElement ele;
int i=1;
while(true) {
try {
ele = wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//*[text()='Write a review']")));
break;
}catch(Exception e) {
System.out.print(i++);
}
}
System.out.println();
Actions action = new Actions(driver);
action.moveToElement(ele);
ele.click();
System.out.println("Clicked on the 'Write a review button'");

you can try
//a[contains(text(),'Write a review')]

How to interact with the elements within #shadow-root (open) while Clearing Browsing Data of Chrome Browser using cssSelector

I had been following the discussion How to automate shadow DOM elements using selenium? to work with #shadow-root (open) elements.
While in the process of locating the Clear data button within the Clear browsing data popup, which appears while accessing the url chrome://settings/clearBrowserData through Selenium I am unable to locate the following element:
#shadow-root (open)
<settings-privacy-page>
Snapshot:
Using Selenium following are my code trials and the associated errors encountered:
Attempt 1:
WebElement root5 = shadow_root4.findElement(By.tagName("settings-privacy-page"));
Error:
Exception in thread "main" org.openqa.selenium.JavascriptException: javascript error: b.getElementsByTagName is not a function
Attempt 2:
WebElement root5 = shadow_root4.findElement(By.cssSelector("settings-privacy-page"));
Error:
Exception in thread "main" org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"settings-privacy-page"}
Attempt 3:
WebElement root5 = (WebElement)((JavascriptExecutor)shadow_root4).executeScript("return document.getElementsByTagName('settings-privacy-page')[0]");
Error:
Exception in thread "main" java.lang.ClassCastException: org.openqa.selenium.remote.RemoteWebElement cannot be cast to org.openqa.selenium.JavascriptExecutor
Incase if it is helpful the initial code block (till the above line) works perfect:
driver.get("chrome://settings/clearBrowserData");
WebElement root1 = driver.findElement(By.tagName("settings-ui"));
WebElement shadow_root1 = expand_shadow_element(root1);
WebElement root2 = shadow_root1.findElement(By.cssSelector("settings-main#main"));
WebElement shadow_root2 = expand_shadow_element(root2);
WebElement root3 = shadow_root2.findElement(By.cssSelector("settings-basic-page[role='main']"));
WebElement shadow_root3 = expand_shadow_element(root3);
WebElement root4 = shadow_root3.findElement(By.cssSelector("settings-section[page-title='Privacy and security']"));
WebElement shadow_root4 = expand_shadow_element(root4);
PS: expand_shadow_element() works flawless.

If you are trying to get 'Clear Data' element then you can use the below js to get the element and then perform.
return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')
Here is the sample script.
driver.get("chrome://settings/clearBrowserData");
driver.manage().window().maximize();
JavascriptExecutor js = (JavascriptExecutor) driver;
WebElement clearData = (WebElement) js.executeScript("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')");
// now you can click on clear data button
clearData.click();
Edit 2: Explanation
Problem: Selenium does not provide explicit support to work with Shadow DOM elements, as they are not in the current dom. That's the reason why we will get NoSuchElementException exception when try to access the elements in the shadow dom.
Shadow DOM:
Note: We will be referring to the terms shown in the picture. So please go through the picture for better understanding.
Solution:
In order to work with shadow element first we have to find the shadow host to which the shadow dom is attached. Here is the simple method to get the shadow root based on the shadowHost.
private static WebElement getShadowRoot(WebDriver driver,WebElement shadowHost) {
JavascriptExecutor js = (JavascriptExecutor) driver;
return (WebElement) js.executeScript("return arguments[0].shadowRoot", shadowHost);
}
And then you can access the shadow tree element using the shadowRoot Element.
// get the shadowHost in the original dom using findElement
WebElement shadowHost = driver.findElement(By.cssSelector("shadowHost_CSS"));
// get the shadow root
WebElement shadowRoot = getShadowRoot(driver,shadowHost);
// access shadow tree element
WebElement shadowTreeElement = shadowRoot.findElement(By.cssSelector("shadow_tree_element_css"));
In order to simplify all the above steps created the below method.
public static WebElement getShadowElement(WebDriver driver,WebElement shadowHost, String cssOfShadowElement) {
WebElement shardowRoot = getShadowRoot(driver, shadowHost);
return shardowRoot.findElement(By.cssSelector(cssOfShadowElement));
}
Now you can get the shadowTree Element with single method call
WebElement shadowHost = driver.findElement(By.cssSelector("shadowHost_CSS_Goes_here));
WebElement shadowTreeElement = getShadowElement(driver,shadowHost,"shadow_tree_element_css");
And perform the operations as usual like .click(), .getText().
shadowTreeElement.click()
This Looks simple when you have only one level of shadow DOM. But here, in this case we have multiple levels of shadow doms. So we have to access the element by reaching each shadow host and root.
Below is the snippet using the methods that mentioned above (getShadowElement and getShadowRoot)
// Locate shadowHost on the current dom
WebElement shadowHostL1 = driver.findElement(By.cssSelector("settings-ui"));
// now locate the shadowElement by traversing all shadow levels
WebElement shadowElementL1 = getShadowElement(driver, shadowHostL1, "settings-main");
WebElement shadowElementL2 = getShadowElement(driver, shadowElementL1,"settings-basic-page");
WebElement shadowElementL3 = getShadowElement(driver, shadowElementL2,"settings-section > settings-privacy-page");
WebElement shadowElementL4 = getShadowElement(driver, shadowElementL3,"settings-clear-browsing-data-dialog");
WebElement shadowElementL5 = getShadowElement(driver, shadowElementL4,"#clearBrowsingDataDialog");
WebElement clearData = shadowElementL5.findElement(By.cssSelector("#clearBrowsingDataConfirm"));
System.out.println(clearData.getText());
clearData.click();
You can achieve all the above steps in single js call as at mentioned at the beginning of the answer (added below just to reduce the confusion).
WebElement clearData = (WebElement) js.executeScript("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')");
Screenshot:

I had to do a similar test which required clearing browsing the chrome history. A minor difference was that I was clearing the data after going to the advanced section of the pop-up. As you are struggling to click only the "Clear data" button, I'm quite sure that you've missed one or two hierarchy elements mistakenly. Or got confused between sibling and parent elements probably. As per seeing your code, I assume that you already know that to access a particular shadow DOM element you need proper sequencing and it has been explained also quite nicely above.
Coming right at your problem now, here is my code snippet which is working correctly. The code waits until the data is cleaned and then will proceed to your next action-
public WebElement expandRootElement(WebElement element) {
WebElement ele = (WebElement) ((JavascriptExecutor) driver).executeScript("return arguments[0].shadowRoot",
element);
return ele;
}
public void clearBrowsingHistory() throws Exception {
WebDriverWait wait = new WebDriverWait(driver, 15);
driver.get("chrome://settings/clearBrowserData");
// Get shadow root elements
WebElement shadowRoot1 = expandRootElement(driver.findElement(By.xpath("/html/body/settings-ui")));
WebElement root2 = shadowRoot1.findElement(By.cssSelector("settings-main"));
WebElement shadowRoot2 = expandRootElement(root2);
WebElement root3 = shadowRoot2.findElement(By.cssSelector("settings-basic-page"));
WebElement shadowRoot3 = expandRootElement(root3);
WebElement root4 = shadowRoot3
.findElement(By.cssSelector("#advancedPage > settings-section > settings-privacy-page"));
WebElement shadowRoot4 = expandRootElement(root4);
WebElement root5 = shadowRoot4.findElement(By.cssSelector("settings-clear-browsing-data-dialog"));
WebElement shadowRoot5 = expandRootElement(root5);
WebElement root6 = shadowRoot5
.findElement(By.cssSelector("cr-dialog div[slot ='button-container'] #clearBrowsingDataConfirm"));
root6.click();
wait.until(ExpectedConditions.invisibilityOf(root6));
}
It should work properly in your case too if you don't intend to change any of the options selected by default in the pop-up (In that case, you will have to add a few more codes regarding selecting those checkboxes). Please tell me if this solves your issue. Hope this is helpful
I've added a snapshot of the the screen here too-
image

The Locator Strategy in #supputuri's answer using document.querySelector() works perfect through google-chrome-devtools
However, as the desired element opens from the shadow-dom you need to induce WebDriverWait for the elementToBeClickable() and you can you the following solution:
Code Block:
driver.get("chrome://settings/clearBrowserData");
new WebDriverWait(driver, 5).until(ExpectedConditions.elementToBeClickable((WebElement) ((JavascriptExecutor)driver).executeScript("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')"))).click();
System.out.println("Clear data Button Clicked");
Console Output:
Clear data Button Clicked

I was getting InvalidArgumentEXception when trying to identify shadowRoot element in DOM using Selenium 4.3.0 and Chrome Version 103.0.5060.134
The solution to this is
SearchContext se= driver.findElment(By.locator("...").getShadowRoot(); return type is SearchContext
in the above line try using locator as xpath
and secondly trying to locate element using SearchContext reference e.g.
WebElement we= se.findElement(By.locator("....."));
use locater as cssSelector
And boom it works like charm
Didn't find this solution available and took me half a day to figure out
Hope this helps!!!

Unable to Click button in Selenium

Im having difficulties in locating an element which is a finish button within a page. I have used driver.findElementById("finish").click (); and it does not work.
Below are different examples I used but with no success:
for (WebElement Element : driver.findElement(By.id("finish")).findElements(By.tagName("a"))) {
if (Element.getAttribute ("class").contains("criteria-filter")) { Element.click();
break;
}
----------------------------------------------------------------
WebElement click1 = new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.xpath("//*[#id=\"finish\"]")));
((JavascriptExecutor) driver).executeScript("arguments[0].click();",click1);
----------------------------------------------------------------
if(!driver.findElementById("finish").isEnabled())
{
driver.findElementById("finish").click();
}else{
System.out.println("False");
}
Element:
<a id="finish" access="" allof="PA.DEPLOYMENT_CONFIG" class="btn criteria-filter btn-success" href="" ng-click="verifyAllFields(sftpForm) && sftpForm.$valid && create()">Finish
</a>

Class selector is not the best idea as it will fail if there is more elements with the same class.
The best way is to open dev-tools, click ctrl + f, then try to cath your element by xpath like this:
//a[text()='Finish']
When you "cath" this browser will move to that element.

Selenium Webdriver: How to wait untill progressbar vanishes and click on the button [duplicate]

I used explicit waits and I have the warning:
org.openqa.selenium.WebDriverException:
Element is not clickable at point (36, 72). Other element would receive
the click: ...
Command duration or timeout: 393 milliseconds
If I use Thread.sleep(2000) I don't receive any warnings.
#Test(dataProvider = "menuData")
public void Main(String btnMenu, String TitleResultPage, String Text) throws InterruptedException {
WebDriverWait wait = new WebDriverWait(driver, 10);
driver.findElement(By.id("navigationPageButton")).click();
try {
wait.until(ExpectedConditions.elementToBeClickable(By.cssSelector(btnMenu)));
} catch (Exception e) {
System.out.println("Oh");
}
driver.findElement(By.cssSelector(btnMenu)).click();
Assert.assertEquals(driver.findElement(By.cssSelector(TitleResultPage)).getText(), Text);
}

WebDriverException: Element is not clickable at point (x, y)
This is a typical org.openqa.selenium.WebDriverException which extends java.lang.RuntimeException.
The fields of this exception are :
BASE_SUPPORT_URL : protected static final java.lang.String BASE_SUPPORT_URL
DRIVER_INFO : public static final java.lang.String DRIVER_INFO
SESSION_ID : public static final java.lang.String SESSION_ID
About your individual usecase, the error tells it all :
WebDriverException: Element is not clickable at point (x, y). Other element would receive the click
It is clear from your code block that you have defined the wait as WebDriverWait wait = new WebDriverWait(driver, 10); but you are calling the click() method on the element before the ExplicitWait comes into play as in until(ExpectedConditions.elementToBeClickable).
Solution
The error Element is not clickable at point (x, y) can arise from different factors. You can address them by either of the following procedures:
1. Element not getting clicked due to JavaScript or AJAX calls present
Try to use Actions Class:
WebElement element = driver.findElement(By.id("navigationPageButton"));
Actions actions = new Actions(driver);
actions.moveToElement(element).click().build().perform();
2. Element not getting clicked as it is not within Viewport
Try to use JavascriptExecutor to bring the element within the Viewport:
WebElement myelement = driver.findElement(By.id("navigationPageButton"));
JavascriptExecutor jse2 = (JavascriptExecutor)driver;
jse2.executeScript("arguments[0].scrollIntoView()", myelement);
3. The page is getting refreshed before the element gets clickable.
In this case induce ExplicitWait i.e WebDriverWait as mentioned in point 4.
4. Element is present in the DOM but not clickable.
In this case induce ExplicitWait with ExpectedConditions set to elementToBeClickable for the element to be clickable:
WebDriverWait wait2 = new WebDriverWait(driver, 10);
wait2.until(ExpectedConditions.elementToBeClickable(By.id("navigationPageButton")));
5. Element is present but having temporary Overlay.
In this case, induce ExplicitWait with ExpectedConditions set to invisibilityOfElementLocated for the Overlay to be invisible.
WebDriverWait wait3 = new WebDriverWait(driver, 10);
wait3.until(ExpectedConditions.invisibilityOfElementLocated(By.xpath("ele_to_inv")));
6. Element is present but having permanent Overlay.
Use JavascriptExecutor to send the click directly on the element.
WebElement ele = driver.findElement(By.xpath("element_xpath"));
JavascriptExecutor executor = (JavascriptExecutor)driver;
executor.executeScript("arguments[0].click();", ele);

In case you need to use it with Javascript
We can use arguments[0].click() to simulate click operation.
var element = element(by.linkText('webdriverjs'));
browser.executeScript("arguments[0].click()",element);

I ran into this error while trying to click some element (or its overlay, I didn't care), and the other answers didn't work for me. I fixed it by using the elementFromPoint DOM API to find the element that Selenium wanted me to click on instead:
element_i_care_about = something()
loc = element_i_care_about.location
element_to_click = driver.execute_script(
"return document.elementFromPoint(arguments[0], arguments[1]);",
loc['x'],
loc['y'])
element_to_click.click()
I've also had situations where an element was moving, for example because an element above it on the page was doing an animated expand or collapse. In that case, this Expected Condition class helped. You give it the elements that are animated, not the ones you want to click. This version only works for jQuery animations.
class elements_not_to_be_animated(object):
def __init__(self, locator):
self.locator = locator
def __call__(self, driver):
try:
elements = EC._find_elements(driver, self.locator)
# :animated is an artificial jQuery selector for things that are
# currently animated by jQuery.
return driver.execute_script(
'return !jQuery(arguments[0]).filter(":animated").length;',
elements)
except StaleElementReferenceException:
return False

You can try
WebElement navigationPageButton = (new WebDriverWait(driver, 10))
.until(ExpectedConditions.presenceOfElementLocated(By.id("navigationPageButton")));
navigationPageButton.click();

Scrolling the page to the near by point mentioned in the exception did the trick for me. Below is code snippet:
$wd_host = 'http://localhost:4444/wd/hub';
$capabilities =
[
\WebDriverCapabilityType::BROWSER_NAME => 'chrome',
\WebDriverCapabilityType::PROXY => [
'proxyType' => 'manual',
'httpProxy' => PROXY_DOMAIN.':'.PROXY_PORT,
'sslProxy' => PROXY_DOMAIN.':'.PROXY_PORT,
'noProxy' => PROXY_EXCEPTION // to run locally
],
];
$webDriver = \RemoteWebDriver::create($wd_host, $capabilities, 250000, 250000);
...........
...........
// Wait for 3 seconds
$webDriver->wait(3);
// Scrolls the page vertically by 70 pixels
$webDriver->executeScript("window.scrollTo(0, 70);");
NOTE: I use Facebook php webdriver

If element is not clickable and overlay issue is ocuring we use arguments[0].click().
WebElement ele = driver.findElement(By.xpath("//div[#class='input-group-btn']/input"));
JavascriptExecutor executor = (JavascriptExecutor)driver;
executor.executeScript("arguments[0].click();", ele);

The best solution is to override the click functionality:
public void _click(WebElement element){
boolean flag = false;
while(true) {
try{
element.click();
flag=true;
}
catch (Exception e){
flag = false;
}
if(flag)
{
try{
element.click();
}
catch (Exception e){
System.out.printf("Element: " +element+ " has beed clicked, Selenium exception triggered: " + e.getMessage());
}
break;
}
}
}

In C#, I had problem with checking RadioButton,
and this worked for me:
driver.ExecuteJavaScript("arguments[0].checked=true", radio);

Can try with below code
WebDriverWait wait = new WebDriverWait(driver, 30);
Pass other element would receive the click:<a class="navbar-brand" href="#"></a>
boolean invisiable = wait.until(ExpectedConditions
.invisibilityOfElementLocated(By.xpath("//div[#class='navbar-brand']")));
Pass clickable button id as shown below
if (invisiable) {
WebElement ele = driver.findElement(By.xpath("//div[#id='button']");
ele.click();
}

Element not found in cache - perhaps the page has changed since it has looked up

I'm testing a web app, in which after opening a page I need to select a radio button after that a pop-up will appear. In that pop-up I'm selecting an ID. Once selected, it will come to main window, in which I'm trying to select a text field with ID. But I receive the following error:
Element not found in the cache - perhaps the page has changed since it was looked up
Command duration or timeout: 50.14 seconds
driver.findElement(By.xpath("html/body/section/div/div[2]/div[3]/div/div/div/div/ul/li[1]/a")).click();
driver.findElement(By.xpath("html/body/section/div/div[12]/div[1]/div/div[2]/div[3]/div/div/div/div/button[1]")).click();
driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
String mainWindowHandle=driver.getWindowHandle();
driver.findElement(By.xpath("html/body/div[3]/div/div")).click();
driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
Set s = driver.getWindowHandles();
Iterator ite = s.iterator();
while(ite.hasNext())
{
String popupHandle=ite.next().toString();
if(!popupHandle.contains(mainWindowHandle))
{
driver.switchTo().window(popupHandle);
}
}
driver.findElement(By.xpath("html/body/div[3]/div/div/div[3]/div/div/div[1]/div[2]/div[4]/div[2]/a")).click();
driver.manage().timeouts().implicitlyWait(50, TimeUnit.SECONDS);
driver.switchTo().window( mainWindowHandle );
driver.findElement(By.id("Title")).sendKeys("auto title");
Problem is I cannot find the textfield with ID: Title" for inserting "auto title.
Thanks in Advance.

Instead of using sleep() with hardcoded time intervals, use an explicit wait:
WebDriverWait wait = new WebDriverWait(driver, 30);
WebElement title = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("Title")));
title.sendKeys("auto title");

Finally found a solution by adding
Thread.sleep(5000);
before finding the element and it solved.