My question is simple- Using Selenium, how do you keep on clicking links when each hyper link opens up in a new page or new window or opens in the same web page.
For example I have following links on a webpage:
Log in
Sign up
Forgot Password?
Signup with us
follow this link
Home
Terms
Privacy Policy
Here is the small snippet of code that I have written to click these mentioned links on webpage:
List<WebElement> elements = driver.findElements(By.tagName("a"));
//clicking all links
for (WebElement el : elements){
System.out.println("Link getting clicked" + el.getText());
el.click();
driver.navigate().back();
}
As you can see I am trying to get links and trying to click them one by one. However, I am getting an error after the first click itself. Console says: "org.openqa.selenium.StaleElementReferenceException: Element is no longer attached to the DOM".
I am petty sure I know the cause as the links are getting opened on the same webpage, but I am doing a back navigation, which is not helping me at all.
Any thoughts / suggestions?
This could be due to program execution advancing to the call to "driver.navigate().back();" before the page has loaded.
Try introducing an implicit wait, which tells the "WebDriver to poll the DOM for a certain amount of time when trying to find an element or elements if they are not immediately available"
e.g. When you create your web driver try:
WebDriver driver = new FirefoxDriver();
driver.manage().timeouts().implicitlyWait(2, TimeUnit.SECONDS);
You have to refind the element whenever a page is reloaded before you can interact with it, for your code, please try modify them to this:
driver.manage().timeouts().implicitlyWait(3000, TimeUnit.MILLISECONDS);
List<WebElement> elements = driver.findElements(By.tagName("a"));
//clicking all links
for (int i=0; i<elements.size(); i++){
WebElement el = driver.findElements(By.tagName("a")).get(i);
System.out.println("Link getting clicked" + el.getText());
el.click();
driver.navigate().back();
}
Related
I am trying to add multiple products in to the cart
WebElement ele = driver.findElement(By.xpath("//li[text()='Grocery ']"));
//Creating object of an Actions class
Actions action = new Actions(driver);
//Performing the mouse hover action on the target element.
action.moveToElement(ele).perform();
Thread.sleep(3000);
driver.findElement(By.xpath("//li[#title='staples']")).click();
Thread.sleep(5000);
List<WebElement> products = driver.findElements(By.xpath("//div[#class='product-grid-img']"));
for(int i=0;i<products.size();i++)
{
products.get(i).click();
Thread.sleep(3000);
driver.findElement(By.xpath("(//span[text()='ADD TO CART'])[1]")).click();
driver.navigate().back();
driver.navigate().refresh();
}
As we can see, by clicking on a product a new page is opened so to get back you are navigating to the previous page explicitly and refreshing the page.
When you leaving the original page or refreshing it the elements collected by Selenium on that page becoming no more relevant AKA Stale.
You can read here or at any other online resource about this issue.
To make your code work you will have to get the product again.
Something like this should work:
List<WebElement> products = driver.findElements(By.xpath("//div[#class='product-grid-img']"));
for(int i=0;i<products.size();i++)
{
List<WebElement> products = driver.findElements(By.xpath("//div[#class='product-grid-img']"));
products.get(i).click();
Thread.sleep(3000);
driver.findElement(By.xpath("(//span[text()='ADD TO CART'])[1]")).click();
driver.navigate().back();
driver.navigate().refresh();
Thread.sleep(3000);
}
Every time you navigate to anywhere, your browser constructs a map of a sorts of the page, in memory. This is called the DOM. When you navigate somewhere else, the DOM is replaced with a new DOM of the new page. The browser does not keep a history of the DOMs you visited, so when you navigate back to somewhere you have already been, the browser has to construct the DOM again.
Let's look at the relevant parts of your code:
List<WebElement> products = driver.findElements(By.xpath("//div[#class='product-grid-img']"));
Find a bunch of elements in the current DOM.
for(int i=0;i<products.size();i++)
{
products.get(i).click();
First time navigate to another page. So discard all the elements you just found above and rebuild the DOM.
Second time through the loop, you are trying to use an element that is no longer found in the current DOM. Which throws the StaleElementException.
Thread.sleep(3000);
driver.findElement(By.xpath("(//span[text()='ADD TO CART'])[1]")).click();
driver.navigate().back();
Navigate again somewhere. Another rebuild of the DOM.
driver.navigate().refresh();
Will not help.
}
You will need to restructure your loop to find the element each time. Perhaps something like:
for(int i=0;i<products.size();i++)
{
// You will have to find the correct XPath here!
WebElement product = driver.findElement(By.xpath("//div[#class='product-grid-img'][" + i + "]"));
product.click();
Thread.sleep(3000); // You might want to use a proper wait here.
driver.findElement(By.xpath("(//span[text()='ADD TO CART'])[1]")).click();
driver.navigate().back();
}
StaleElementExceptions are caused by going to a new page and accessing old elements that are no longer valid.
You could do what others did and regrab the values or if it's in an a tag you could just collect all the hrefs and then just driver.get() to them removing the need to driver.navigate().back();
.getAttribute("href")
I just searched for your query in Google to check , with the intention to learn more about what exactly is the question and topic ,
I got this :
"People also ask"
(Q)How do you fix stale element reference element is not attached to the page document in selenium?
Solution : The element in the DOM is not found because your page is not entirely loaded when Selenium is searching for the element. To solve that, you can put an explicit wait condition that tells Selenium to wait until the element is available to be clicked on.
https://www.google.com/search?q=his+error+%22stale+element+reference%3A+element+is+not+attached+to+the+page+document%22%2C&sourceid=chrome&ie=UTF-8
I'm trying to crawl a site which has title and comments. When the page is loaded 40 comments are rendered, but after clicking the button "Load comments" there are new 40 comments and so on. I want to load all the comments first, and then take them all.
The problem is that I'm getting only the first 40. This is my code:
WebDriver driver = new HtmlUnitDriver();
driver.get(www.website.com);
String title = driver.findElement(By.className("title")).getText();
while(driver.findElements(By.className("load-comments")).isDisplayed() || !driver.findElement(By.className("expand-loading")).isDisplayed()){
Thread.sleep(500);
if(!driver.findElements(By.className("loading")).isDisplayed()){
driver.findElements(By.className("load-comments")).click();
}
}
List<WebElement> comments = (List<WebElement>) driver.findElements(By.className("comment"));
for(WebElement comm:comments){
System.out.print(comm.getText());
}
So, if i need all 150 comments, in this situation i'm getting only the first 40 that are visible when the page is loaded.
I tried options.addArguments("--headless"); with ChromeDriver(options); but it's very slow.
PS: The load-comments button is hidden when all comments are loaded and the loading element is displayed if the page is loading new comments.
The website you provided din't display any comments. please provide exact web address to locate elements.
Why are you using "driver.findelements" instead of "driver.findelement" for is displayed condition? This logic will give you compilation error.
You need to add argument to set screen resolution while using headless.I suggest you to go with chrome with ui first then check the headless.
http://www.toysrus.com/family/index.jsp?categoryId=2535588&sr=1&origkw=watches
In the above web page lots of products are available, we need to click the product links randomly using java selenium. please help me out!
List<WebElement> links = driver.findElements(By.cssSelector('a.prodtitle'));
links.get(new Random().nextInt(links.size())).click();
Try this
The answer already posted will only click one link at random. If you eventually want to click every link randomly, then you will need to keep some sort of record of what links you have visited already. You will also need to go back after visiting a page.
List<WebElement> links = driver.findElements(By.cssSelector('a.prodtitle'));
List<WebElement> visited = new List<WebElement>();
WebElement random = links.get(new Random().nextInt(links.size()));
if( !visited.contains(random))
{
random.click();
visited.add(random);
driver.navigate().back();
}
For the example program on the webpage
http://www.qaautomation.net/?p=263
I carry out the following steps:
Run the program with the driver.close() line of code commented out.
The program opens a Firefox browser, then searches for the term "qa automation" on google.
Once the "Test passed." message has been printed to the screen (in the console), go to the google search results page.
Using the browser menu, go to Tools/Web Developer/Page Source.
On the page source page, search for the term "qaautomation.net".
Quit the Firefox application.
Open Firefox and a browser window manually i.e. not using the program.
Go to google.com and search manually for "qa automation".
When the results page has loaded, carry out Steps 4 and 5 above.
I get no search results in Step 5 but I do in Step 9. Why is this? Both page sources appear to derive from the same webpage. Any help on this would be appreciated.
Instead of searching through the HTML source, I would suggest that you just scrape the page. This method will let you more specifically target where you are seeing the text... in a link href vs text on the page vs ???, etc.
The code below demonstrates two different places that you can look for the "qaautomation.net" text. The first is in the heading link href and the second is in the citation link that occurs immediately below the heading href.
WebDriver driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.get("http://www.google.com");
driver.findElement(By.id("lst-ib")).sendKeys("qa automation\n"); // need the \n to simulate hitting ENTER to start the search
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until(ExpectedConditions.titleContains("Google Search")); // need to wait until the results page loads
List<WebElement> links = driver.findElements(By.cssSelector("h3.r > a"));
// System.out.println("links: " + links.size()); // for debugging
for (WebElement link : links)
{
// looking for qaautomation.net in the link href
if (link.getAttribute("href").contains("qaautomation.net"))
{
System.out.println("PASS: found in href");
}
}
List<WebElement> cites = driver.findElements(By.tagName("cite"));
// System.out.println("cites: " + cites.size()); // for debugging
for (WebElement cite : cites)
{
// looking for qaautomation.net in the citation (text below the heading link)
if (cite.getText().contains("qaautomation.net"))
{
System.out.println("PASS: found in cite");
}
}
System.out.println("DONE");
I am using Java and Firefox and Firebug
I am not able to locate the element on the second page. For example if I login to gmail then I am not able to locate and click on the sent items or any other button on the next page.
I tried with the xpath (both absolute and relative) but every time I am getting an error that element not found.
with the code I am successfully able to login but as soon as the page loads I get an error "Element not Found".
Please suggest any solution
Unless you are telling WebDriver to wait until the element on the 2nd page is loaded, WebDriver will simply try to click the element as soon as its able to run. This is bad because your element might not yet be loaded while WebDriver is already trying to click it... TIMEOUT mayhem ensues...
Try the following... use the WebDriverWait class to make WebDriver wait for the element on the page to be loaded before attempting to click it...:
WebDriverWait wait = new WebDriverWait(driver, 100);
WebElement element = wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("your xpath")));
element.click();
The '100' in WebDriverWait(driver, 100) is the maximum amount of seconds you want WebDriver to repeatedly attempt to locate the element before it times out...
I agree with the answer by CODEBLACK. Also you can go for Imlicit wait,which would make selenium wait implicitly for a given period of time.
Try following:-
driver.manage().timeouts().implicitlyWait(20,TimeUnit.SECONDS);
You can specify time as per your convenience.
Best O Luck. . .!