I am having a problem clicking an element using Selenium and Java. Here is what the table looks like.
<tr class="even">
<td class="linked-item">
<a title="Imported Gatekeeper" href="/rss-servlet/remotesupport/viewgatekeeper.action?siteId=20">SJHS_OK - Technology Center - (CWx Network Connectivity)</a>
</td>
</tr>
<tr class="odd">
<td class="linked-item">
<a title="Imported Gatekeeper" href="/rss-servlet/remotesupport/viewgatekeeper.action?siteId=26">SJHS_OK - Technology Center - (CWx Network Connectivity) - Backup</a>
</td>
</tr>
I am trying to select the first element SJHS_OK - Technology Center - (CWx Network Connectivity), which is a link to another page. This is only a segment of the code, there is much more than this.
My problem arises when I try to use contains, my xpath looks something like this:
//*[#title='Imported Gatekeeper'][contains(.,'SJHS_OK - Technology Center - (CWx Network Connectivity)')][not(contains(.,'Backup'))]
Sometimes when I put it into firepath it works and finds the element, sometimes it doesn't. Not sure why it doesn't always work.
my code in Java looks like this:
String gatekeeper = "SJHS_OK - Technology Center - (CWx Network Connectivity)"
DRIVER.findElement(By.xpath("//*[#title='Imported Gatekeeper']"+gatekeeper+"[not(contains(.,'Backup'))]")).click();
I am doing it this way so that I can easily change the gatekeeper variable and select a different gatekeeper. The gatekeeper will be pulled from an excel sheet.
Any help with what is wrong here would be appreciated, or maybe another way that I can go about finding the element to click it that can be changed by a variable.
If the xpath sometimes finds and sometimes does not, the problem probably is a missing wait. If the element does not exist or is not visible when you try to click, code will throw an error. Try it:
WebElement yourElement = (new WebDriverWait(driver, 10))
.until(ExpectedConditions.elementToBeClickable(By.xpath("//*[#title='Imported Gatekeeper'][contains(.,'SJHS_OK - Technology Center - (CWx Network Connectivity)')][not(contains(.,'Backup'))]")));
#Edit
As wait is not working and code stills sometimes working and sometimes no, when te page is renderized, there is no a constant number of spaces to fills the text. Try this xpath selector:
//a[#title='Imported Gatekeeper'][text()[contains(., 'SJHS_OK')]][contains(., 'Technology Center')][contains(., 'CWx')][contains(., 'Network')][contains(., 'Connectivity')][not(contains(.,'Backup'))]
Also, try all these selectors in google chrome console when test works and not in chrome console. You can test an xpath using an $x("your xpath here")
You don't have to use xpath for that. If you are using Java8 Try this instead:
driver.findElements(By.tagName("a")).stream()
.filter(td -> !td.getText().contains("Backup"))
.findFirst().get().click();
If the element can be found either way, there can be something underneath that, maybe element wasn't loaded completely?
Can you provide stacktrace when error occurs?
I found that when I copied the SJHS_OK - Technology Center - (CWx Network Connectivity directly out of Xpath then it would work, but if I typed it myself, even if it was exact when compared, it would not work. My conclusion is that it has to do with the spaces. The error would disappear if I cut it down without the spaces.
Related
I'm trying to fill in multiple forms that come after each other, all the forms get filled swiftly with no errors because I make sure to add
WebDriverWait wait = new WebDriverWait(driver, 20);
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("")));
before doing anything on a new page, and I know I'm on the correct page.
On the last form, I encounter this error :
Exception in thread "main" org.openqa.selenium.NoSuchElementException: Unable to locate element: //*[#id="formtovalidate"]/fieldset[1]/div/label/input For documentation on this error, please visit: https://www.seleniumhq.org/exceptions/no_such_element.html
So I went to check on the browser by taking a screenshot and the browser is on the correct page with the correct form, I also checked the xpath values and even tried other attributes.. nothing seemed to work.
So I went ahead and printed out the PageSource which showed a totally different page (not the previous page), I also noticed the this page flashed for a second before the final form appeared.
I also tried driver.navigate().refresh() but that didn't work. I kept searching and looking but nothing appeared. I also changed browsers, that did nothing..
This is the method I'm trying to execute:
private void method() {
WebDriverWait wait = new WebDriverWait(driver, 20);
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//*[#id=\"formtovalidate\"]/fieldset[1]/div/label/input")));
driver.findElement(By.xpath("//*[#id=\"formtovalidate\"]/fieldset[1]/div/label/input")).sendKeys(email); }
Update
Here's the form screenshot:
Here's the execution results:
Code:
String body_text = driver.findElement(By.tagName("body")).getText();
System.out.println(body_text);
Result: The form but in text
Code:
String body_innerHTML = driver.findElement(By.tagName("body")).getAttribute("innerHTML");
System.out.println(body_innerHTML);
Result: A different page :(
<zendesk-ticketing-form base-url="https://www.runescape.com/a=870/c=K0aO9WO69EI" css-cachebust="129" sitekey="6Lcsv3oUAAAAAGFhlKrkRb029OHio098bbeyi_Hv" grecaptcha="" has-valid-session="true" weblogin-url="https://secure.runescape.com/m=weblogin/a=870/c=K0aO9WO69EI/loginform?mod=www&ssl=1&dest=zendesk/support-form?form=360000065898">
<div class="x-display-none ie-error-display" data-js-ie-error="">
<section class="c-article">
<div class="c-article__content">
<h1>Error: Unsupported Browser</h1>
<p>
We do not support your web browser. Please use a supported web browser by choosing one below.
<br>
FireFox
<br>
Chrome
</p>
</div>
</section>
</div>
Code:
String pagesource = driver.getPageSource();
System.out.println(pagesource);
Result: Same as the previous one.. different page..
Firefox Page Source: https://pastebin.com/Kv15V2SK
Firefox Inspect Element of the page screenshot: http://prntscr.com/qvi6hc
This is weird, as the page source is different to the form!
I couldn't find time to solve your problem. If you want to do it on your own, please Search this on Google, "Shadow Root, Selenium", I had this kind of error before. What I know is, you cannot directly reach an element that stays inside of a shadow root, This is why you are not getting the source code inside of it.
What you need to do is go through the element step by step:
You have to expand the shadow root,
Here is shadow root expand function:
public static WebElement expand_shadow_element(WebElement element)
{
WebElement shadow_root = (WebElement)((JavascriptExecutor)driver).executeScript("return arguments[0].shadowRoot", element);
return shadow_root;
}
You can imagine this function like
.switchTo.frame()
for now..
After some researches you will understand the shadow root.
I hope I got the problem right..
Try this function, If you cannot, I will help you later on. Good Luck.
The PageSource from the <body> tag, containing...
<zendesk-ticketing-form base-url="https://www.runescape.com/a=870/c=K0aO9WO69EI" css-cachebust="129" sitekey="6Lcsv3oUAAAAAGFhlKrkRb029OHio098bbeyi_Hv" grecaptcha="" has-valid-session="true" weblogin-url="https://secure.runescape.com/m=weblogin/a=870/c=K0aO9WO69EI/loginform?mod=www&ssl=1&dest=zendesk/support-form?form=360000065898">
<div class="x-display-none ie-error-display" data-js-ie-error="">
<section class="c-article">
<div class="c-article__content">
<h1>Error: Unsupported Browser</h1>
<p>
We do not support your web browser. Please use a supported web browser by choosing one below.
<br>
FireFox
<br>
Chrome
</p>
</div>
</section>
</div>
...implies that the WebDriver driven Browsing Context was detected as a BOT and the navigation was blocked due to presence of reCAPTCHA.
There are different approaches to solve captcha / recaptcha. You can find a couple of relevant discussion in:
How to bypass Google captcha with Selenium
Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection
Update
From your comments now it is clear that you want to fill up the fields within the form:
At this point it is worth to mention that you had been redirected to this page for either of the following reasons:
You EmailID / UserID is banned / blocked from accessing the site.
You EmailID / UserID is black-listed from accessing the site.
As you have used a BOT to access/scrape the site which may have violated the T&C.
Solution
It would be tough to propose a solution to automatically fillup the fields as presumably the elements in the BAN APPEAL REQUEST page may be protected by Invisible reCAPTCHA and you may have to Programmatically invoke the challenge
As others have suggested, it appears RuneScape's website has detected that you're using a bot to interact with their site. It doesn't matter that you solved the captcha manually, as they can still detect automated behavior quite easily without one (and no, the navigator.webdriver flag is not their only way to detect this).
The captcha is meant to prevent automated interaction with their site, which means they don't want you using Selenium/WebDriver to interact with it. You should respect this, especially as it seems you want your account unbanned (going by the pasted snippets and screenshots), so trying to do exactly what they don't want won't win you any favors.
I am trying to locate an iframe by partial id. For this method, I used:
driver.switchTo().frame(driver.findElement(By.cssSelector("iframe[id*='card-fields-number']"))); I have also tried xpath.
driver.switchTo().frame(driver.findElement(By.xpath("//iframe[contains(#id,'card-fields-number')]")));
However, I still receive this exception:
org.openqa.selenium.NoSuchElementException: Returned node (null) was not a DOM element
I found out that when I enable javascript for HtmlUnit, it is able to locate the frame, and switch to it. I would rather not enable javascript as it runs very slow for me, and adds unneeded delay.
iFrame HTML code:
<iframe class="card-fields-iframe" frameborder="0" id="card-fields-number-7pbvqg7azsf00000" name="card-fields-number-7pbvqg7azsf00000" scrolling="no" src="https://checkout.shopifycs.com/number?identifier=438599641d0ed8fe61c161d72e62b5f8&location=https%3A%2F%2Fshopnicekicks.com%2F2192362%2Fcheckouts%2F438599641d0ed8fe61c161d72e62b5f8&dir=ltr&fonts[]=Lato" title="Field container for: Card number" style="height: 43px;"></iframe>
iFrame ID is dynamic, so that is why I resort to using partial ID.
Website link: https://shopnicekicks.com/checkout
You must fill everything out until you reach the last page, which is the credit card information page.
Update
The iFrame is inside of the parent frame.
Parent Frame:
<iframe srcdoc="<script>!function(){var e=function(e){var t={exports:{}};return e.call(t.exports,t,t.exports),t.exports},t=function(){function e(e,t){for(var n=0;n<t.length;n++){var i=t[n];i.enumerable=i.enumerable||!1,i.configurable=!0,"value"in i&&(i.writable=!0),Object.defineProperty(e,i.key,i)}}return function(t,n,i){return n&&e(t.prototype,n),i&&e(t,i),t}}(),n=function(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")},i=function(e){return e&&e.__esModule?e:{"default":e}},o=e(function(e,i){"use strict";Object.defineProperty(i,"__esModule",{value:!0});var o=function(){function e(){var t=this;n(this,e),this.calls=[],window.ga=function(){for(var e=arguments.length,n=Array(e),i=0;i<e;i++)n[i]=arguments[i];return t.gaCall(n)}}return t(e,[{key:"gaCall",value:function(e){var t=this;this.calls.push(e),clearTimeout(this.timeout),this.timeout=setTimeout(function(){t.calls.length>0&&t.sendMessage()},0)}},{key:"listen",value:function(){var e=this;window.addEventListener("message",function(t){return e.receiveMessage(t)},!1)}},{key:"sendMessage",value:function(){window.parent.postMessage({type:"analytics",calls:this.calls},this.origin),this.calls=[]}},{key:"receiveMessage",value:function(e){if(e.source===window.parent&&"checkout_context"===e.data.type){this.origin=e.origin,window.Shopify=e.data.Shopify,window.__st=e.data.__st;try{window.additionalScripts()}catch(e){console.error("User script error: ",e)}}}}]),e}();i["default"]=o});e(function(){"use strict";var e=i(o);!function(){(new e["default"]).listen()}()})}("undefined"!=typeof global?global:"undefined"!=typeof window&&window); window.additionalScripts = function () {};</script>" src="https://checkout.shopify.com/2192362/sandbox/google_analytics_iframe" onload="this.setAttribute('data-loaded', true)" sandbox="allow-scripts" id="google-analytics-sandbox" tabindex="-1" class="visually-hidden" style="display:none" aria-hidden="true" data-loaded="true"></iframe>
This error message...
org.openqa.selenium.NoSuchElementException: Returned node (null) was not a DOM element
...implies that there was no such element found as the returned node was null or was not a DOM element.
This is still a open issue with htmlunit-driver team.
However there are certain things which you need to take care as follows:
First and foremost, all the modern browsers come with built-in support for JavaScript.
HtmlUnitDriver is a WebDriver compatible driver for HtmlUnit headless browser. It has fairly good JavaScript support (which is constantly improving) and is able to work even with quite complex AJAX libraries, simulating Chrome, Firefox or Internet Explorer depending on the configuration used. So ideally while working with HtmlUnitDriver, JavaScript must be enabled, else HtmlUnitDriver may not ne able to detect the JavaScript based elements.
You can find a detailed discussion in HtmlUnitDriver does not load javascript when navigating a page from an url
The element seems to be a credit card field and historically Credit Card Number, etc resides within <iframes>.
You can find a detailed discussion in Unable to locate element of credit card number using selenium python
Whenever an <iframe> is in play src attribute of the <iframe> tag plays a vital role.
You can find a detailed discussion in Ways to deal with #document under iframe
As per best practices while switching <iframe> you need to induce WebDriverWait for the desired frame to be available and switch to it.
We have discussed this aspect in your previous question Selenium can't locate iframe
So you can use either of the following solutions:
cssSelector:
new WebDriverWait(driver, 10).until(ExpectedConditions.frameToBeAvailableAndSwitchToIt(By.cssSelector("iframe.card-fields-iframe[id^='card-fields-number-'][src*='shopifycs']")));
xpath:
new WebDriverWait(driver, 10).until(ExpectedConditions.frameToBeAvailableAndSwitchToIt(By.xpath("//iframe[#class='card-fields-iframe' and starts-with(#id,'card-fields-number-')][contains(#src, 'shopifycs')]")));
Update
See the snapshot of the CssSelector which identifies the element perfecto as per the HTML you have provided:
Just try with frame indexing "//iframe[index]", where index integer.
e.g. xpath : //iframe[1]
Frame id may change dynamically but in a few application structure remains same so indexing solves the problem.
Please let me know if it solves the issue.
As the iframe has tag iframe, you can switch to the iframe using the tagname like:
driver.switchTo().frame(driver.findElement(By.tagName("iframe")));
And if you want to again switch to the default content, then you can use driver.switchTo().defaultContent();
Well not to mention that I'm new on using the selenium web driver, I'm trying to automatize a mailbox so I can log myself in and to send a message, the thing is that I download the firebug plugin for Mozilla, that is the browser I'm currently working with, when I'm trying to find the element for the log in and the password everything goes well, because I do it by their name.
input class="_nb-input-controller" type="text" **name="login"** autocorrect="off" autocapitalize="off" value=""
input class="_nb-input-controller" type="password" **name="passwd"** value=""
but as soon as im at the mailbox entry the identificator changes it, i go to compose link and this is the code that shows up:
a class="b-toolbar__item b-toolbar__item_compose js-toolbar-item-compose daria-action" title="Compose (w, c)" href="#compose" data-action="compose.go" data-params="toolbar=1&toolbar.button=compose"
Since I don't find any method that could help on this, a friend came and told me about the xpath which was:
/html/body/div[2]/div/div[5]/div/div[3]/div/div[2]/div/div/div/div[2]/a[2]
by doing this selenium did find the element and I could click on the icon,
but again.. when I'm trying to get into the "TO" prompt so I can write inside
it is not finding the element, not by xpath, or anything is there a way or formula to get elements or translate them into a way that selenium can find them???
oh i see, i re try with the element and i noticed that the one i wanted was showing as hidden using firebug so i just try the next one up and the web driver could locate the item, it was a simple app to send an email the prompt that i wanted to locate was the 'TO' field. and thanks guys for giving me the correct way of using the Xpath property
Having locators such as :
/html/body/div[2]/div/div[5]/div/div[3]/div/div[2]/div/div/div/div[2]/a[2]
is generally a bad idea. If you want to build a stable test I would suggest you stay away from such.
Now I see you have attributes like #name and #class. XPath is provides you with a way to reach these elements directly by pointing the identifiers instead of the whole path to the element.
You can try something like:
//input[#name='login']
//input[#name='password']
More info at: http://www.w3schools.com/xsl/xpath_intro.asp
I have the HTML (partial) shown below. I want to find the element using:
org.jsoup.nodes.Element elem = doc.getElementById("date-2011-04-23");
But I always get a NULL. Can anyone help me? As a check, I've also code this using VB.NET and I can access this element.
<td class="" id="date-2011-04-23" data-week="3" data-wkday="6">...</td>
Assuming that your tag looks like:
<td class="" id="date-2011-04-23" data-week="3" data-wkday="6">...</td>
You can use the JSoup Selector API for this:
for( Element element : doc.select("#date-2011-04-23") )
{
// Do something here
}
If you need only the first Element:
Element element = doc.select("#date-2011-04-23").first();
The reason you're not finding that content in the HTML is that the schedule is loaded from a JSON file by the browser executing Javascript, then adding it to the browser DOM. Jsoup does not execute Javascript, so it only can see what is in the source HTML.
If you use a debugging proxy like Charles (or the debugging network pane in Chrome / Firefox), you can see all the requests a browser makes to render a page. In this example, the schedule data is coming from http://mlb.mlb.com/gen/schedule/phi/2011_4.json
I am trying to get a WebElement with Selenium:
driver.findElement(By.xpath("//input[#name='j_username']"))
But Selenium says: "Unable to find element with XPath ...".
The XPath is valid, I proofed it with FirePath.
But the input element has the following invalid code:
<input size="10" type="text" name="j_username" maxlength="8">
I can't change the html-file, despite the fact is there any solution to get the webElement?
Thanks in advance!
try select element with css selector. and also verify in firepath(firebug addon that element is located properly).
so your css selector be something like
input[name='j_username']
2nd approach is to use internal firebug mechanism for finding xPaths of elements.
See screen attached below
After these manipulations driver shoulda handle element properly.
Well I will suggest adding an id to your html code -
<input id="j_username"size="10" type="text" name="j_username" maxlength="8">
and findElement by id -
driver.findElement(By.id("j_username"));
I have faced similar issues with xpath(borwser issues??) but id never fails for me. ;)
By the way I feel your code should be -
driver.findElement(By.xpath(".//*[#name='j_username']"));
The best solution is to find out what selenium is doing wrong, but without a URL or sample page to test on it's a little hard. Is there anyway you could dump the HTML into a jsfiddle? If there is do that and paste the url into the question and I'm sure someone can find a solution.
If not however, another way to get the results is to do it with jQuery. If firebug is picking it up but not selenium, then there's no reason why jQuery wouldn't get it. Here's how to go about doing that if needed:
Step 1: Is jQuery already present on the page? If so then you don't need to do this bit, otherwise you will need to add it yourself by using driver.executeScript(addjQueryScript) where the script does something like this.
Step 2: call WebElement input = driver.executeScript(elementSelector); where the elementSelector script would be something like \"return $('input[name=\"j_username\"]')\");
My jQuery's not so good, but I believe that should work...
Best of luck!