Develop Reference

Selenium chromedriver opens a blank page instead of url on linux

I am trying to open a url in chromedriver(using RemoteWebDriver) on linux.
I took a screenshot after driver.get(url) is called. It displays a blank page.
east-northamptonshire_screenshot.jpg
I tried this(open a url using ChromeDriver) on my local machine(Windows). It is working fine.
This is the url I am trying to open. "https://publicaccess.east-northamptonshire.gov.uk/online-applications/search.do?action=weeklyList"
Main method:
public static void main(String[] args) throws Exception {
String OS = System.getProperty("os.name").toLowerCase();
WebDriver driver = null;
ChromeDriverService service = null;
boolean isWindows = OS.indexOf("win") >= 0;
logger.info("operating System : " + OS);
if (!isWindows) {
service = new ServerChromeDriver().loadService();
}
driver = new ServerChromeDriver().getPIDriver(service, isWindows);
String url = "https://publicaccess.east-northamptonshire.gov.uk/online-applications/search.do?action=weeklyList";
driver.get(url);
Thread.sleep(3000);
ScreenShot.takeScreenShot(driver);
driver.close();
driver.quit();
service.stop();
}
ServerChromeDriver Class:
public ChromeDriverService loadService() throws Exception {
Configuration configuration = new Configuration();
configuration.loadProperties();
Properties props = new Properties();
try {
props.load(new FileInputStream("config//log4j.properties"));
} catch (IOException e) {
logger.error(e);
}
PropertyConfigurator.configure(props);
service = new ChromeDriverService.Builder().usingDriverExecutable(new File(configuration.getChromeDriverPath()))
.usingAnyFreePort().withEnvironment(ImmutableMap.of("DISPLAY", configuration.getDisplay())).build();
service.start();
return service;
}
public WebDriver getPIDriver(ChromeDriverService service, boolean isWindows) {
WebDriver driver;
if (isWindows) {
driver = new LocalChromeDriver().getDriver();
} else {
driver = new ServerChromeDriver().getDriver(service.getUrl());
}
String hostName = new ServerChromeDriver().getHostName(driver);
logger.info("Running the application on host: " + hostName);
return driver;
}
public WebDriver getDriver(URL serviceUrl) {
Configuration configuration = new Configuration();
configuration.loadProperties();
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--headless");
chromeOptions.addArguments("--disable-gpu");
// chromeOptions.addArguments("--start-maximized");
chromeOptions.addArguments("--window-size=1800,1800");
// chromeOptions.addExtensions(new File(configuration.getAdBlockPath()));
System.setProperty("webdriver.chrome.driver", configuration.getChromeDriverPath());
System.setProperty("webdriver.chrome.logfile", configuration.getChromeDriverLogFilePath());
System.setProperty("webdriver.chrome.verboseLogging", configuration.getChromeVerboseLogging());
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability(ChromeOptions.CAPABILITY, chromeOptions);
capabilities.setJavascriptEnabled(true);
try {
driver = new RemoteWebDriver(serviceUrl, capabilities);
} catch (Exception e) {
logger.error("Error creating a new chrome instance");
throw new RuntimeException(e.getMessage());
}
driver.manage().timeouts().implicitlyWait(180, TimeUnit.SECONDS);
return driver;
}
Application is working fone for this url: https://eplanning.birmingham.gov.uk/Northgate/PlanningExplorer/GeneralSearch.aspx
birmingham_screenshot.jpg
I am using
Headless Chrome : 67.0.3396.62
chromedriver : 2.40.565383
This is what I found from the chromedriver.log file
[0617/144457.403693:ERROR:nss_ocsp.cc(601)] No URLRequestContext for NSS HTTP handler. host: crt.comodoca.com
[0617/144457.403801:ERROR:cert_verify_proc_nss.cc(980)] CERT_PKIXVerifyCert for publicaccess.east-northamptonshire.gov.uk failed err=-8179

I think because Chrome version in your Linux machine is not supported.

For people using newer versions of Selenium who are dealing with this issue, I was able to do the following (similar to Abhilash's answer which is using an older version of Selenium) to resolve the blank-page issue for uncertified domains with Chrome:
ChromeOptions options = new ChromeOptions();
...
options.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
options.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
...
driver = new ChromeDriver(options);
(In my case, webdriver worked fine on my local machine but not on the Linux box hosting CI tools)

After researching about certificate errors, I have added additional capabilities to chrome driver to ignore certificate related errors. Below is my solution.
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability(ChromeOptions.CAPABILITY, chromeOptions);
capabilities.setJavascriptEnabled(true);
capabilities.setCapability(CapabilityType.PROXY, proxy);
capabilities.setCapability("acceptSslCerts", true); // Added this additionally
capabilities.setCapability("acceptInsecureCerts", true); // Added this additionally
capabilities.setCapability("ignore-certificate-errors", true); // Added this additionally

Selenium: changing proxy in Firefox

i'm writing tests in selenium and want to change proxy to auto-detect in firefox, default is proxy from system settings. How to do it?
I have code below:
public class SodirRejestracja {
String baseUrl = "http://google.pl";
String driverPath= "C:\\geckodriver.exe";
WebDriver driver;
#BeforeTest
public void beforeTest() {
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("network.proxy.type", 2);
System.setProperty("webdriver.gecko.driver", driverPath);
driver=new FirefoxDriver(profile);
}
#Test
public void test(){
driver.get("http://google.com");
}
}
Code above is from How do I set a proxy for firefox using Selenium webdriver with Java?
but in line driver=new FirefoxDriver(profile) i get: "The constructor FirefoxDriver(FirefoxProfile) is undefined"

A sample (not tested but that compiles) that should do it
String proxyName = <yourProxyHost> + ":" + <yourProxyPort>;
Proxy proxy = new Proxy();
proxy.setHttpProxy(proxyName)
.setFtpProxy(proxyName)
.setSslProxy(proxyName);
DesiredCapabilities desiredCapabilities = DesiredCapabilities.firefox();
desiredCapabilities.setCapability(CapabilityType.PROXY, proxy);
FirefoxOptions options = new FirefoxOptions(desiredCapabilities);
driver = new FirefoxDriver(options);

This code works
Proxy proxy = new Proxy();
proxy.setProxyType(Proxy.ProxyType.AUTODETECT);
FirefoxOptions options = new FirefoxOptions();
options.setProxy(proxy);
driver = new FirefoxDriver(options);

Selenium -Core Java Error

I have the following code:
public static void main(String[] args) {
WebDriver wd = new FirefoxDriver();
wd.get("https://www.wordpress.com/");
}
There are no compile time errors with this but as soon as Mozilla opens, it says "This connection is untrusted". I wish to open the url that I've specified in the code.

Open Firefox driver like this -
ProfilesIni allProfiles = new ProfilesIni();
System.setProperty("webdriver.firefox.profile","your custom firefox profile name");
String browserProfile = stem.getProperty("webdriver.firefox.profile");
FirefoxProfile profile = allProfiles.getProfile(browserProfile);
profile.setAcceptUntrustedCertificates (true);
webdriver = new FirefoxDriver(profile);

How can I set the browser version in Selenium RemoteWebDriver?

When I use HtmlUnitDriver I can set my own browserVersion like:
private HtmlUnitDriver initDriver() {
BrowserVersion browserVersion = new BrowserVersion(
BROWSER_NAME,
BROWSER_OS,
USER_AGENT,
Float.parseFloat(BROWSER_VERSION));
browserVersion.setBrowserLanguage(BROWSER_LANGUAGE);
browserVersion.setHtmlAcceptHeader(HTML_ACCEPT_HEADER);
return new HtmlUnitDriver(browserVersion);
}
Is it possible to do the same with the RemoteWebDriver ?
WebDriver driver = new RemoteWebDriver(
new URL("http://localhost:4444/wd/hub"),
myCapabilities);
In Capabilities I can set myCapabilities.setBrowserName("htmlunit"). Is that all that I can do ?
EDIT:
To be clear, I need 3 things:
a) Selenium-server-standalone to be able to reuse the same old SessionID
b) To make browser that works only from console (so no firefox afaik)
c) To make http requests the same as the latest browsers so there will be no difference in the server logs.

You can also set a lot of parameters, for example:
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setBrowserName("firefox");
capabilities.setVersion("35.0");
capabilities.setPlatform(Platform.VISTA);
try {
driver = new RemoteWebDriver(new URL("http://192.168.63.109:5555/wd/hub"), capabilities);
} catch (MalformedURLException e) {
e.printStackTrace();
}
A bit more settings to skip dialog on file downloading using custom FirefoxProfile:
public static WebDriver setDriver() {
FirefoxProfile fxProfile = new FirefoxProfile();
fxProfile.setPreference("browser.download.folderList", 2);
fxProfile.setPreference("browser.helperApps.alwaysAsk.force", false);
fxProfile.setPreference("browser.download.manager.showWhenStarting", false);
fxProfile.setPreference("browser.download.dir", dir);
fxProfile.setPreference("browser.helperApps.neverAsk.saveToDisk", "application/vnd.ms-excel," +
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
return new FirefoxDriver(fxProfile);
}

Selenium 2 WebDriver to use Custom Profile

I'm trying to automate the interaction with a website that generates documents with MIME type application/vnd.wap.xhtml+xml. I am using Selenium 2, the WebDriver and the FirefoxProfile.
Because Firefox does not handle the above mentioned MIME type, I need to run Firefox with the XHTML Mobile Profile extension (https://addons.mozilla.org/en-US/firefox/addon/1345/).
After creating a FireFox profile -I named it 'selenium'- and installing the Mobile Profile extension, I tried to use the code snippets in the 'Tips and Tricks' section of the 'Selenium 2.0 and WebDriver' document (http://seleniumhq.org/docs/09_webdriver.html#htmlunit-driver).
Approach #1 looks like this:
ProfilesIni allProfiles = new ProfilesIni();
FirefoxProfile profile = allProfiles.getProfile("selenium");
profile.setPreference("general.useragent.override", "User Agent string to force application/vnd.wap.xhtml+xml content..");
FirefoxDriver driver = new FirefoxDriver(profile);
driver.get("http://www.mobilesite.com/");
WebElement element = driver.findElement(By.tagName("body"));
Approach #2 looks like this:
File profileDir = new File("/path/to/custom/profile/with/extension/ffprofile");
FirefoxProfile profile = new FirefoxProfile(profileDir);
profile.setPreference("general.useragent.override", "same user agent string as above");
FirefoxDriver driver = new FirefoxDriver(profile);
driver.get("http://www.mobilesite.com/");
No matter what code snippet I use, the browser instance that starts up is always unable to handle the generated content; the browser prompts me for an action to take on the content of the unrecognized MIME type as if the extension was not correctly configured.
Any ideas on what I could be doing wrong?
Thanks in advance,
Edit: Link to Selenium users group post.

Try starting with a blank profile and adding extensions/configurations at runtime:
public WebDriver getDriver() {
FirefoxProfile profile = new FirefoxProfile();
// add any custom firefox configurations...
profile.setPreference("general.useragent.override", "some UA string");
profile.setPreference("javascript.options.showInConsole", true);
profile.setPreference("dom.max_script_run_time", 0);
// might have to uninstall, search for *.xpi, then reinstall, then search
// again and compare to find the location on your system
// ...you should probably copy this into your selenium resources directory!
File modifyHeadersXpi = new File("/home/joecoder/.mozilla/firefox/dll8peh9.default/extensions/{b749fc7c-e949-447f-926c-3f4eed6accfe}.xpi");
if (modifyHeadersXpi.exists()) {
try {
profile.addExtension(modifyHeadersXpi);
profile.setPreference("modifyheaders.config.active", true);
profile.setPreference("modifyheaders.config.openNewTab", true);
profile.setPreference("modifyheaders.config.migrated", true);
profile.setPreference("modifyheaders.autocomplete.name.defaults",
"[\"Accept\",\"Cache-Control\",\"Cookie\",\"Content-Length\",\"Content-Type\",\"Date\",\"Host\",\"Pragma\",\"Referer\",\"User-Agent\",\"Via\",\"X-Requested-With\",\"X-Forwarded-For\",\"X-Do-Not-Track\"]");
}
catch (IOException e) { /* uh oh */ }
}
return new FirefoxDriver(profile);
}

Hope this will help you out:
public class Wap {
public static void main (String[] args) throws IOException{
FirefoxProfile profile = new FirefoxProfile();
String baseURL;
profile.addExtension(new File("C:\\Users\\Pandu\\Desktop\\WAP\\modify_headers-0.7.1.1-fx.xpi"));
profile.setPreference("modifyheaders.config.active", true);
profile.setPreference("modifyheaders.config.alwaysOn", true);
profile.setPreference("modifyheaders.headers.count", 2);
profile.setPreference("modifyheaders.headers.action0", "Add");
profile.setPreference("modifyheaders.headers.name0", "X-Nokia-msisdn");
profile.setPreference("modifyheaders.headers.value0", "123456789");
profile.setPreference("modifyheaders.headers.enabled0", true);
profile.setPreference("modifyheaders.headers.action1", "Add");
profile.setPreference("modifyheaders.headers.name1", "x-sec-pass");
profile.setPreference("modifyheaders.headers.value1", "sdp123");
profile.setPreference("modifyheaders.headers.enabled1", true);
Logger Log = Logger.getLogger(WebDriver.class.getName());
WebDriver driver = new FirefoxDriver(profile);
try{
driver.get("http://www.google.com");
driver.findElement(By.linkText("Telugu")).click();

You have to make sure you add the browser plugin as a DeploymentItem in your testsettings file.
Some examples (in this one we have added Firebug):
<Deployment>
<DeploymentItem filename="Selenium\firebug#software.joehewitt.com.xpi" />
<DeploymentItem filename="packages\Castle.Core.3.1.0\lib\net40-client\Castle.Core.dll" />
<DeploymentItem filename="Selenium\IEDriverServer.exe" />
<DeploymentItem filename="Selenium\chromedriver.exe" />
<DeploymentItem filename="Selenium\skipcerterror#foudil.fr.xpi" />
</Deployment>
You will then need to create a profile that looks something like this:
string firebugPath = Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) ?? "", "firebug#software.joehewitt.com.xpi");
FirefoxProfile firebugProfile = new FirefoxProfile() {AcceptUntrustedCertificates = true};
firebugProfile.AddExtension(firebugPath);
firebugProfile.SetPreference("extensions.firebug.currentVersion", "1.10.3");
firebugProfile.SetPreference("extensions.sce.bypass_domain_mismatch", true);
firebugProfile.SetPreference("webdriver_assume_untrusted_issuer", false);
Driver = new FirefoxDriver(firebugProfile);
Driver.Manage().Window.Maximize();
If you add the extension using AddExtension, it should be available within you selenium driver. I hope this helps.