Develop Reference

Finding Pseudo Elements in Selenium

I'm trying to find this element (screenshot below) I'm using Java, I tried using cssSelector and xpath none worked.
Technology.click();
wait.until(ExpectedConditions.presenceOfElementLocated(By.xpath("//*[#id=\"paletteItem:_8827283_1\"]/a/span")));
WebElement VS = driver.findElement(By.xpath("//*[#id=\"paletteItem:_8827283_1\"]/a/span"));
VS.click(); // all of the above works correctly, but after this point it keeps waiting and then erroring.
wait.until(ExpectedConditions.presenceOfElementLocated(By.cssSelector("#main-content > header > div > h1")));
System.out.println("Found");
JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript("document.querySelector(\"#session-0c5d29dffcf04299a02e217d457f7718 > div.item-list__item-details.click()");
HTML:
<div class="item-list__item-wrapper ng-isolate-scope" bb-session-list-item-content="session" tooltip-position="tooltipPosition" expand="sessionListItem.toggleExpand()" is-expanded="sessionListItem.isExpanded()" is-admin-view="isAdminView" can-edit="sessionListItem.canEdit()" can-show-options="sessionListItem.canShowOptions()" can-launch="sessionListItem.canLaunch()" can-launch-phone="sessionListItem.canLaunchPhone()" can-delete="sessionListItem.canDelete()" can-show-reports="sessionListItem.canShowReports()" can-copy-guest-link="sessionListItem.canCopyGuestLink()"><button class="item-list__item session-list-item-content item-list__item--active" id="session-a07f7478cf154c6ea4e9a56e5f1f1bca" aria-label="Show occurrences for session: الإسبوع الثالث_الطرقان" aria-describedby="session-a07f7478cf154c6ea4e9a56e5f1f1bca-details" ng-attr-aria-expanded="{{session.hasOccurrences ? (isExpanded ? 'true' : 'false') : undefined }}" ng-click="sessionListItemContent.sessionClicked()" ng-class="{'item-list__item--active': canLaunch}" aria-expanded="true">
Screen shots:
Website picture
, Code of element

How to interact with the elements within #shadow-root (open) while Clearing Browsing Data of Chrome Browser using cssSelector

I had been following the discussion How to automate shadow DOM elements using selenium? to work with #shadow-root (open) elements.
While in the process of locating the Clear data button within the Clear browsing data popup, which appears while accessing the url chrome://settings/clearBrowserData through Selenium I am unable to locate the following element:
#shadow-root (open)
<settings-privacy-page>
Snapshot:
Using Selenium following are my code trials and the associated errors encountered:
Attempt 1:
WebElement root5 = shadow_root4.findElement(By.tagName("settings-privacy-page"));
Error:
Exception in thread "main" org.openqa.selenium.JavascriptException: javascript error: b.getElementsByTagName is not a function
Attempt 2:
WebElement root5 = shadow_root4.findElement(By.cssSelector("settings-privacy-page"));
Error:
Exception in thread "main" org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"settings-privacy-page"}
Attempt 3:
WebElement root5 = (WebElement)((JavascriptExecutor)shadow_root4).executeScript("return document.getElementsByTagName('settings-privacy-page')[0]");
Error:
Exception in thread "main" java.lang.ClassCastException: org.openqa.selenium.remote.RemoteWebElement cannot be cast to org.openqa.selenium.JavascriptExecutor
Incase if it is helpful the initial code block (till the above line) works perfect:
driver.get("chrome://settings/clearBrowserData");
WebElement root1 = driver.findElement(By.tagName("settings-ui"));
WebElement shadow_root1 = expand_shadow_element(root1);
WebElement root2 = shadow_root1.findElement(By.cssSelector("settings-main#main"));
WebElement shadow_root2 = expand_shadow_element(root2);
WebElement root3 = shadow_root2.findElement(By.cssSelector("settings-basic-page[role='main']"));
WebElement shadow_root3 = expand_shadow_element(root3);
WebElement root4 = shadow_root3.findElement(By.cssSelector("settings-section[page-title='Privacy and security']"));
WebElement shadow_root4 = expand_shadow_element(root4);
PS: expand_shadow_element() works flawless.

If you are trying to get 'Clear Data' element then you can use the below js to get the element and then perform.
return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')
Here is the sample script.
driver.get("chrome://settings/clearBrowserData");
driver.manage().window().maximize();
JavascriptExecutor js = (JavascriptExecutor) driver;
WebElement clearData = (WebElement) js.executeScript("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')");
// now you can click on clear data button
clearData.click();
Edit 2: Explanation
Problem: Selenium does not provide explicit support to work with Shadow DOM elements, as they are not in the current dom. That's the reason why we will get NoSuchElementException exception when try to access the elements in the shadow dom.
Shadow DOM:
Note: We will be referring to the terms shown in the picture. So please go through the picture for better understanding.
Solution:
In order to work with shadow element first we have to find the shadow host to which the shadow dom is attached. Here is the simple method to get the shadow root based on the shadowHost.
private static WebElement getShadowRoot(WebDriver driver,WebElement shadowHost) {
JavascriptExecutor js = (JavascriptExecutor) driver;
return (WebElement) js.executeScript("return arguments[0].shadowRoot", shadowHost);
}
And then you can access the shadow tree element using the shadowRoot Element.
// get the shadowHost in the original dom using findElement
WebElement shadowHost = driver.findElement(By.cssSelector("shadowHost_CSS"));
// get the shadow root
WebElement shadowRoot = getShadowRoot(driver,shadowHost);
// access shadow tree element
WebElement shadowTreeElement = shadowRoot.findElement(By.cssSelector("shadow_tree_element_css"));
In order to simplify all the above steps created the below method.
public static WebElement getShadowElement(WebDriver driver,WebElement shadowHost, String cssOfShadowElement) {
WebElement shardowRoot = getShadowRoot(driver, shadowHost);
return shardowRoot.findElement(By.cssSelector(cssOfShadowElement));
}
Now you can get the shadowTree Element with single method call
WebElement shadowHost = driver.findElement(By.cssSelector("shadowHost_CSS_Goes_here));
WebElement shadowTreeElement = getShadowElement(driver,shadowHost,"shadow_tree_element_css");
And perform the operations as usual like .click(), .getText().
shadowTreeElement.click()
This Looks simple when you have only one level of shadow DOM. But here, in this case we have multiple levels of shadow doms. So we have to access the element by reaching each shadow host and root.
Below is the snippet using the methods that mentioned above (getShadowElement and getShadowRoot)
// Locate shadowHost on the current dom
WebElement shadowHostL1 = driver.findElement(By.cssSelector("settings-ui"));
// now locate the shadowElement by traversing all shadow levels
WebElement shadowElementL1 = getShadowElement(driver, shadowHostL1, "settings-main");
WebElement shadowElementL2 = getShadowElement(driver, shadowElementL1,"settings-basic-page");
WebElement shadowElementL3 = getShadowElement(driver, shadowElementL2,"settings-section > settings-privacy-page");
WebElement shadowElementL4 = getShadowElement(driver, shadowElementL3,"settings-clear-browsing-data-dialog");
WebElement shadowElementL5 = getShadowElement(driver, shadowElementL4,"#clearBrowsingDataDialog");
WebElement clearData = shadowElementL5.findElement(By.cssSelector("#clearBrowsingDataConfirm"));
System.out.println(clearData.getText());
clearData.click();
You can achieve all the above steps in single js call as at mentioned at the beginning of the answer (added below just to reduce the confusion).
WebElement clearData = (WebElement) js.executeScript("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')");
Screenshot:

I had to do a similar test which required clearing browsing the chrome history. A minor difference was that I was clearing the data after going to the advanced section of the pop-up. As you are struggling to click only the "Clear data" button, I'm quite sure that you've missed one or two hierarchy elements mistakenly. Or got confused between sibling and parent elements probably. As per seeing your code, I assume that you already know that to access a particular shadow DOM element you need proper sequencing and it has been explained also quite nicely above.
Coming right at your problem now, here is my code snippet which is working correctly. The code waits until the data is cleaned and then will proceed to your next action-
public WebElement expandRootElement(WebElement element) {
WebElement ele = (WebElement) ((JavascriptExecutor) driver).executeScript("return arguments[0].shadowRoot",
element);
return ele;
}
public void clearBrowsingHistory() throws Exception {
WebDriverWait wait = new WebDriverWait(driver, 15);
driver.get("chrome://settings/clearBrowserData");
// Get shadow root elements
WebElement shadowRoot1 = expandRootElement(driver.findElement(By.xpath("/html/body/settings-ui")));
WebElement root2 = shadowRoot1.findElement(By.cssSelector("settings-main"));
WebElement shadowRoot2 = expandRootElement(root2);
WebElement root3 = shadowRoot2.findElement(By.cssSelector("settings-basic-page"));
WebElement shadowRoot3 = expandRootElement(root3);
WebElement root4 = shadowRoot3
.findElement(By.cssSelector("#advancedPage > settings-section > settings-privacy-page"));
WebElement shadowRoot4 = expandRootElement(root4);
WebElement root5 = shadowRoot4.findElement(By.cssSelector("settings-clear-browsing-data-dialog"));
WebElement shadowRoot5 = expandRootElement(root5);
WebElement root6 = shadowRoot5
.findElement(By.cssSelector("cr-dialog div[slot ='button-container'] #clearBrowsingDataConfirm"));
root6.click();
wait.until(ExpectedConditions.invisibilityOf(root6));
}
It should work properly in your case too if you don't intend to change any of the options selected by default in the pop-up (In that case, you will have to add a few more codes regarding selecting those checkboxes). Please tell me if this solves your issue. Hope this is helpful
I've added a snapshot of the the screen here too-
image

The Locator Strategy in #supputuri's answer using document.querySelector() works perfect through google-chrome-devtools
However, as the desired element opens from the shadow-dom you need to induce WebDriverWait for the elementToBeClickable() and you can you the following solution:
Code Block:
driver.get("chrome://settings/clearBrowserData");
new WebDriverWait(driver, 5).until(ExpectedConditions.elementToBeClickable((WebElement) ((JavascriptExecutor)driver).executeScript("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')"))).click();
System.out.println("Clear data Button Clicked");
Console Output:
Clear data Button Clicked

I was getting InvalidArgumentEXception when trying to identify shadowRoot element in DOM using Selenium 4.3.0 and Chrome Version 103.0.5060.134
The solution to this is
SearchContext se= driver.findElment(By.locator("...").getShadowRoot(); return type is SearchContext
in the above line try using locator as xpath
and secondly trying to locate element using SearchContext reference e.g.
WebElement we= se.findElement(By.locator("....."));
use locater as cssSelector
And boom it works like charm
Didn't find this solution available and took me half a day to figure out
Hope this helps!!!

unable to click on Webelement on web Page

I'm trying to click on web element and enter text inside it.
Steps:
Launch "https://www.phptravels.net/"
Click on tours tab.
Perform send keys operation on search field.
1.I tried using click on search box and entering text via send keys but unable to do so, After that I performed click action and send keys using javaScript but this is also not working.
I have written different xpath for the same but no positive results.
//code is as below
public class HandlingDropDown2 {
static WebElement element;
static WebDriver driver;
public static void main(String[] args) throws Exception {
// TODO Auto-generated method stub
System.setProperty("webdriver.chrome.driver", "Driver/chromedriver.exe");
driver = new ChromeDriver();
driver.manage().window().maximize();
driver.get("https://www.phptravels.net/");
element = driver.findElement(By.xpath("//span[contains(text(),'Tours ')]"));
element.click();
Thread.sleep(2000);
element = driver.findElement(By.xpath("//button[contains(text(),'Got it!')]"));
element.click();
Thread.sleep(2000);
element = driver.findElement(By.xpath("//div[#id='s2id_autogen5']"));
JavascriptExecutor executor = (JavascriptExecutor)driver;
executor.executeScript("arguments[0].click();", element);
System.out.println("clicked on autogen box");
//element.click();
Thread.sleep(2000);
element = driver.findElement(By.xpath("//div[#class='select2-drop select2-display-none select2-with-searchbox select2-drop-active']"));
JavascriptExecutor executor2 = (JavascriptExecutor)driver;
executor2.executeScript("arguments[0].'value='Test';",element);
//element.sendKeys("test");
}
}
Expected Result: User must be able to enter some text via automation.
Actual Result: Unable to perform click and sendkeys using JavaScript and simple selenium methods.

Remember the functionality of sendKeys
Fir of all, your xPath is div element and you are trying to do sendKeys in div element which is wrong. If you observed there is span element named 'Search by Listing or City Name'. If you click there then your input element gets visible where you can click() and sendKeys("")
Try,
// click on below span element to get input visibled,
element = driver.findElement(By.xpath("//span[text()='Search by Listing or City Name']"));
element.click();
Then your input element is now available where you can click and sendkeys
element = driver.findElement(By.xpath("//div[#id='select2-drop']//input[#class='select2-input'][last()]"));
element.click();
element.sendKeys("test");

Unable to Click button in Selenium

Im having difficulties in locating an element which is a finish button within a page. I have used driver.findElementById("finish").click (); and it does not work.
Below are different examples I used but with no success:
for (WebElement Element : driver.findElement(By.id("finish")).findElements(By.tagName("a"))) {
if (Element.getAttribute ("class").contains("criteria-filter")) { Element.click();
break;
}
----------------------------------------------------------------
WebElement click1 = new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.xpath("//*[#id=\"finish\"]")));
((JavascriptExecutor) driver).executeScript("arguments[0].click();",click1);
----------------------------------------------------------------
if(!driver.findElementById("finish").isEnabled())
{
driver.findElementById("finish").click();
}else{
System.out.println("False");
}
Element:
<a id="finish" access="" allof="PA.DEPLOYMENT_CONFIG" class="btn criteria-filter btn-success" href="" ng-click="verifyAllFields(sftpForm) && sftpForm.$valid && create()">Finish
</a>

Class selector is not the best idea as it will fail if there is more elements with the same class.
The best way is to open dev-tools, click ctrl + f, then try to cath your element by xpath like this:
//a[text()='Finish']
When you "cath" this browser will move to that element.

Can't click the button - element not visible

I'm using selenium and chrome drive and i can't click the button,
I tried different approaches but nothing:
link = driver.findElement(By.xpath("//button[#class='btn buttonContainer arrow']"));
link = driver.findElement(By.xpath("//input[#class='btn buttonContainer arrow']"));
link = driver.findElement(By.cssSelector("input[type='button']"));
link = driver.findElement(By.cssSelector("input[type='button'][value='btn buttonContainer arrow']"));
error massage:
INFO: Detected dialect: OSS
Exception in thread "main" org.openqa.selenium.ElementNotVisibleException: element not visible
(Session info: chrome=58.0.3029.96)
(Driver info: chromedriver=2.29.461591 (62ebf098771772160f391d75e589dc567915b233),platform=Windows NT 10.0.10240 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 15 milliseconds
code:
<a class="ng-scope" ng-if="!loginCtrl.pageObject.isDebugMode" ng-click="rootCtrl.redirectToRegistration()">
<button class="btn buttonContainer arrow">
<span>Enter</span>
</button>
</a>
thanks Amir

Incase you want to click on Enter button try: driver.findElement(By.xpath("//button[#class='btn buttonContainer arrow']/span"));

I didnt reproduce it but you should be able to get it to work like this:
firefoxDriver.findElement(By.className("buttonContainer")).click();
This approach is working for me using Selenium V2.52.0

Use some implicit wait default in your script to locate the element by selenium before throwing the exception
Like
driver.get("your _site_url");
driver.manage().timeouts().implicitlyWait(45, TimeUnit.SECONDS);
Use explicit wait until your element get visible and then perform click
WebDriverWait wait = new WebDriverWait(driver, 60);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//button[#class='btn buttonContainer arrow']")));
driver.findElement(By.xpath("//button[#class='btn buttonContainer arrow']")).click();

You are automating a web-based application built using AngularJS.
In such a case, your click() events should be targeted to the web elements which have ng-click attribute.
Try clicking on the anchor link instead of the button.
link = driver.findElement(By.xpath("//button[#class='btn buttonContainer arrow']/.."));
link.click();

Looks like there are multiple web elements in the DOM with same xpath and the first element is not visible although Firebug would highlight visible element (It typically happens with Responsive websites)
You can try following:
public WebElement getDisplayedElement( WebDriver driver ) {
try {
WebElement visibleElement = null;
List<WebElement> elements = driver.findElements(locator);
for (WebElement element : elements) {
boolean isDisplayed = false;
boolean isEnabled = false;
System.out.println("Count of element(s) from locator " +locator+ " is " +elements.size());
isDisplayed = element.isDisplayed();
isEnabled = element.isEnabled();
if(isDisplayed && isEnabled ) {
System.out.println("Found enabled and displayed web element");
visibleElement = element;
break;
}
}
return visibleElement;
}