Develop Reference

Element Not Found Exception: HTMLUnit - Search By ID

I am attempting to use HTMLUnit (first time) to extract data from specific pages.
Specifically, I am currently trying to grab an HTML element by ID (a search box).
But I am running into:
Exception in thread "main" com.gargoylesoftware.htmlunit.ElementNotFoundException: elementName=[*] attributeName=[id] attributeValue=[space_search_keyword]
at com.gargoylesoftware.htmlunit.html.HtmlPage.getHtmlElementById(HtmlPage.java:1547)
at com.gargoylesoftware.htmlunit.html.HtmlPage.getHtmlElementById(HtmlPage.java:1517)
at Test.main(Test.java:33)
This is my code:
import java.util.List;
import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HTMLParserListener;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class Test {
public static void main(String[] args) {
HtmlPage page = null;
WebClient client = new WebClient();
client.setCssEnabled(false);
client.setJavaScriptEnabled(false);
try {
String searchUrl = "https://25live.collegenet.com/umassd/#space_search[0]";
page = client.getPage(searchUrl);
}catch(Exception e){
e.printStackTrace();
}
//System.out.println(page.asXml());
HtmlElement searchBox = (HtmlElement)page.getHtmlElementById("space_search_keyword");
}
}
Upon further inspection using the page.asXML(), it seems that the page isn't properly loading and that's why it can't find the item? I'm not sure why it isn't loading for HTMLUnit. There's no need to login, you can see the page come up for yourself by entering it in a browser.
Any help with debugging HTMLUnit issues like this would be greatly appreciated.

The site is a SPA (Single-Page Application) written in Angular.
You need JavaScript to run it.
Unfortunately the JavaScript capability of HtmlUnit is insufficient to run Angular, so your approach won't work.
You can try:
Reverse-engineer the page and fetch the underlying resource that the SPA is accessing
Try Selenium ChromeDriver (it actually opens Chrome and simulates button clicks on the page)

nosuchwindowexception Selenium with PhantomJS Java

I am using headless PhantomJS browser to automate the application using phantomjs driver with selenium. (selenium java version 3.5.2 and phantomjs.exe)
I have a scenario where i will fill the form and submit it and then the browser gets closed and after closing the browser I am reusing the driver reference to get the URL. It works well when I am using the firefox driver with selenium 2.47.0.
Now I switched to the selenium phontamjsdriver and phantombrowser. Here when I make a call to the driver.get(url);after the browser gets closed it is throwing the nosuchwindowexception saying window is closed or inactive. But, the same code is working with the firefox driver
example:
driver.get(url);// first time works
submitForm(driver);//browser window gets closed.
driver.get(url);
The last line is throwing exception as:
nosuchwindowexception(selenium java with 3.5.2 version and phantomjs.exe).
But works well with the firefoxbrowser with selenium 2.4.7.

First of all, as you migrated from Selenium v2.47.0 to Selenium v3.5.2 it's worth to mention a lot have been changed with the availability of Selenium 3.x. Now Selenium-WebDriver is a W3C Recommendation Candidate and is compliant with WebDriver W3C Editor's Draft
NoSuchWindowException
NoSuchWindowException class extends NotFoundException and is majorly thrown while attempting:
WebDriver.switchTo().window(String windowName);
Now, a bit more details about your usecase, relevant HTML and your code block would have given us some more ideas what is going wrong. Perhaps the defination of submitForm(driver) holds the key to the solution of your question.
Best Practices
Here some of the best practices you can follow to avoid NoSuchWindowException:
Always strore the Parent Window Handle in a variable so that you can traverse back to the Parent Window as generally required.
Before invoking driver.switchTo().window(windowHandle); always induce WebDriverwait in-conjunction with ExpectedConditions method numberOfWindowsToBe(int).
Once you invoke driver.switchTo().window(windowHandle); induce WebDriverWait in-conjunction with ExpectedConditions method titleContains(java.lang.String title) to wait for the Page Loading to get completed to continue your Test Steps on the newly opened window.
To switch back to the parent window use the previously stored windowhandle.
Here is a sample code block to demonstrate Window/Tab handling:
package demo;
import java.util.Set;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
public class WINDOW_HANDLE_ITERATE_Firefox
{
public static void main(String[] args) throws Exception
{
System.setProperty("webdriver.gecko.driver", "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
WebDriver driver = new FirefoxDriver();
driver.get("http://www.google.com");
String parent_window = driver.getWindowHandle();
System.out.println("Parent Window Handle is: "+driver.getWindowHandle());
System.out.println("Page Title is: "+driver.getTitle());
((JavascriptExecutor) driver).executeScript("window.open('http://facebook.com/');");
new WebDriverWait(driver,10).until(ExpectedConditions.numberOfWindowsToBe(2));
Set<String> allWindows_1 = driver.getWindowHandles();
System.out.println("Total Windows: "+allWindows_1.size());
for(String hand1:allWindows_1)
if(!parent_window.equals(hand1))
{
driver.switchTo().window(hand1);
new WebDriverWait(driver,10).until(ExpectedConditions.titleContains("Facebook"));
String first_child_window = driver.getWindowHandle();
System.out.println("First Child Window Handle is: "+first_child_window);
System.out.println("First Child Window Page Title is: "+driver.getTitle());
driver.close();
}
driver.switchTo().window(parent_window);
System.out.println("Current Window Handle is : "+driver.getWindowHandle()+ " which is same as "+parent_window +", which is the parent window handle" );
driver.quit();
}
}
Console Output:
1531917836983 geckodriver INFO geckodriver 0.20.1
1531917836993 geckodriver INFO Listening on 127.0.0.1:9993
1531917837915 mozrunner::runner INFO Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "-marionette" "-profile" "C:\\Users\\ATECHM~1\\AppData\\Local\\Temp\\rust_mozprofile.W5WqVulBNm9x"
1531917842220 Marionette INFO Listening on port 35364
1531917843126 Marionette WARN TLS certificate errors will be ignored for this session
Jul 18, 2018 6:14:03 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Parent Window Handle is: 4294967297
Page Title is: Google
Total Windows: 2
First Child Window Handle is: 4294967301
First Child Window Page Title is: Facebook – log in or sign up
Current Window Handle is : 4294967297 which is same as 4294967297, which is the parent window handle

This may be an issue to bring to the PhantomJS team here https://github.com/ariya/phantomjs/issues
Unfortunately screenshots in Selenium only capture the DOM and not address bar. If you're able to save the pagesource, you may be able to extract the URL. When viewing page source of this page I see tags which list various 3rd party apps, inside there is a content source which lists the URL.
<meta name="twitter:app:url:googleplay" content="http://stackoverflow.com/questions/51255939/nosuchwindowexception-selenium-with-phantomjs-java">
This may not be true for every site, but could be somewhere to look. You could also try and add this tag in yourself if you're the site owner.

selenium won't get URL?

this my code open fire fox and get google
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
public class auto {
public static void main(String[] args) {
System.setProperty("webdriver.gecko.driver", "//root//Desktop//jarselenium//geckodriver-v0.20.1-linux64/geckodriver");
WebDriver driver = new FirefoxDriver();
driver.manage().window().maximize();
//driver.get("https://www.easybooking.lk/login");
//driver.manage().timeouts().implicitlyWait(25, TimeUnit.SECONDS);
driver.get("https://www.google.com");
}
}
but when i run this code selenium open fire fox but wont get url im running this code in linux

As per the current implementation of Selenium v3.11.0, GeckoDriver 0.20.1 I don't see any major issue in your code block perhaps the trace logs would have helped us to understand your issue better. Having said that you need to follow the below mentioned steps :
As you are using Linux based System you need to pass the absolute path of the GeckoDriver within single forward slashes i.e. / as follows :
System.setProperty("webdriver.gecko.driver", "/root/Desktop/jarselenium/geckodriver-v0.20.1-linux64/geckodriver");
As GeckoDriver opens Firefox Browser client in maximized mode you need to omit the line of code :
driver.manage().window().maximize();

The driver executable does not exist: Selenium Firefox

I've been struggling with this for the past few hours. I'm trying to install Selenium web driver and have been running into a bunch of errors which prevent me from running the test page. I'm pretty sure my most recent issue is with this code:
public static void main(String[] args) throws InterruptedException{
System.setProperty("webdriver.gecko.driver","C:/Users/theone/Downloads/geckodriver 2.exe");
Would really appreciate any feedback on second steps!
package automationFramework;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
public class FirstTestCase {
public static void main(String[] args) throws InterruptedException{
System.setProperty("webdriver.gecko.driver","C:/Users/theone/Downloads/geckodriver 2.exe");
// Create a new instance of the Firefox driver
WebDriver driver = new FirefoxDriver();
//Launch the Online Store Website
driver.get("http://www.store.demoqa.com");
// Print a Log In message to the screen
System.out.println("Successfully opened the website www.Store.Demoqa.com");
//Wait for 5 Sec
Thread.sleep(5);
// Close the driver
driver.quit();
}
}

You can setup Selenium with GeckoDriver either with webdriver.gecko.driver property or using environment properties. It would be good if you the latest version of Firefox, GeckoDriver and Selenium 3.0
Check out this article which provides setup using both these ways -
http://automationtestinghub.com/selenium-3-0-launch-firefox-with-geckodriver/

C:/Users/theone/Downloads/geckodriver 2.exe
There's a space in the path, it may work if you rename your file geckodriver2.exe.

Unable to go for next Annotation of selenium browser automation with simple java codes

The thing what I wanted to do was automate the Firefox browser for search an item in the search bar. My java codes are as follows.
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
public class FlipkartTest {
public static void main(String[] args) {
CharSequence[] cs = {"Dell"};
//CharSequence[] d = String[]{"Dell"};
//String s = toString();
WebDriver driver = new FirefoxDriver();
driver.get("http://www.flipkart.com");
driver.findElement(By.id("fk-top-search-box")).sendKeys(cs);
driver.findElement(By.xpath("//input[#value=Search]")).click();
}
}
After debug or run the code a new Firefox window is appeared, But when click on next Annotation button, Nothing was happened. it doesn't go for next step such as opens the Flipcart.com site or searches the word.
After run the code, The console display a list. I found this line from it.
Caused by: org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms.
So how I fix this problem?

As been answered here the problem is in mismatch between selenium and Firefox. Updating them to the latest version should solve the problem.

I recommend you to first set the properties of the Firefox driver and then create an instance of it. Something like this -
Webdriver driver=null;
System.setProperty("webdriver.firefox.driver"."//your firefox driver path");
driver=new FirefoxDriver();
Also try with ChromeDriver and InternetExplorerDriver and check if the problem still persists.