Develop Reference

Selenium chromedriver opens a blank page instead of url on linux

I am trying to open a url in chromedriver(using RemoteWebDriver) on linux.
I took a screenshot after driver.get(url) is called. It displays a blank page.
east-northamptonshire_screenshot.jpg
I tried this(open a url using ChromeDriver) on my local machine(Windows). It is working fine.
This is the url I am trying to open. "https://publicaccess.east-northamptonshire.gov.uk/online-applications/search.do?action=weeklyList"
Main method:
public static void main(String[] args) throws Exception {
String OS = System.getProperty("os.name").toLowerCase();
WebDriver driver = null;
ChromeDriverService service = null;
boolean isWindows = OS.indexOf("win") >= 0;
logger.info("operating System : " + OS);
if (!isWindows) {
service = new ServerChromeDriver().loadService();
}
driver = new ServerChromeDriver().getPIDriver(service, isWindows);
String url = "https://publicaccess.east-northamptonshire.gov.uk/online-applications/search.do?action=weeklyList";
driver.get(url);
Thread.sleep(3000);
ScreenShot.takeScreenShot(driver);
driver.close();
driver.quit();
service.stop();
}
ServerChromeDriver Class:
public ChromeDriverService loadService() throws Exception {
Configuration configuration = new Configuration();
configuration.loadProperties();
Properties props = new Properties();
try {
props.load(new FileInputStream("config//log4j.properties"));
} catch (IOException e) {
logger.error(e);
}
PropertyConfigurator.configure(props);
service = new ChromeDriverService.Builder().usingDriverExecutable(new File(configuration.getChromeDriverPath()))
.usingAnyFreePort().withEnvironment(ImmutableMap.of("DISPLAY", configuration.getDisplay())).build();
service.start();
return service;
}
public WebDriver getPIDriver(ChromeDriverService service, boolean isWindows) {
WebDriver driver;
if (isWindows) {
driver = new LocalChromeDriver().getDriver();
} else {
driver = new ServerChromeDriver().getDriver(service.getUrl());
}
String hostName = new ServerChromeDriver().getHostName(driver);
logger.info("Running the application on host: " + hostName);
return driver;
}
public WebDriver getDriver(URL serviceUrl) {
Configuration configuration = new Configuration();
configuration.loadProperties();
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--headless");
chromeOptions.addArguments("--disable-gpu");
// chromeOptions.addArguments("--start-maximized");
chromeOptions.addArguments("--window-size=1800,1800");
// chromeOptions.addExtensions(new File(configuration.getAdBlockPath()));
System.setProperty("webdriver.chrome.driver", configuration.getChromeDriverPath());
System.setProperty("webdriver.chrome.logfile", configuration.getChromeDriverLogFilePath());
System.setProperty("webdriver.chrome.verboseLogging", configuration.getChromeVerboseLogging());
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability(ChromeOptions.CAPABILITY, chromeOptions);
capabilities.setJavascriptEnabled(true);
try {
driver = new RemoteWebDriver(serviceUrl, capabilities);
} catch (Exception e) {
logger.error("Error creating a new chrome instance");
throw new RuntimeException(e.getMessage());
}
driver.manage().timeouts().implicitlyWait(180, TimeUnit.SECONDS);
return driver;
}
Application is working fone for this url: https://eplanning.birmingham.gov.uk/Northgate/PlanningExplorer/GeneralSearch.aspx
birmingham_screenshot.jpg
I am using
Headless Chrome : 67.0.3396.62
chromedriver : 2.40.565383
This is what I found from the chromedriver.log file
[0617/144457.403693:ERROR:nss_ocsp.cc(601)] No URLRequestContext for NSS HTTP handler. host: crt.comodoca.com
[0617/144457.403801:ERROR:cert_verify_proc_nss.cc(980)] CERT_PKIXVerifyCert for publicaccess.east-northamptonshire.gov.uk failed err=-8179

I think because Chrome version in your Linux machine is not supported.

For people using newer versions of Selenium who are dealing with this issue, I was able to do the following (similar to Abhilash's answer which is using an older version of Selenium) to resolve the blank-page issue for uncertified domains with Chrome:
ChromeOptions options = new ChromeOptions();
...
options.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
options.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
...
driver = new ChromeDriver(options);
(In my case, webdriver worked fine on my local machine but not on the Linux box hosting CI tools)

After researching about certificate errors, I have added additional capabilities to chrome driver to ignore certificate related errors. Below is my solution.
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability(ChromeOptions.CAPABILITY, chromeOptions);
capabilities.setJavascriptEnabled(true);
capabilities.setCapability(CapabilityType.PROXY, proxy);
capabilities.setCapability("acceptSslCerts", true); // Added this additionally
capabilities.setCapability("acceptInsecureCerts", true); // Added this additionally
capabilities.setCapability("ignore-certificate-errors", true); // Added this additionally

How to Setup Private Proxy with Selenium?

I've been trying for days to setup a private proxy (with authentication) in Selenium using Firefox. However, no matter what I do I'v been unsuccessful.
Currently, I have tried the following two approaches and in both cases Firefox launches normally without any proxy.
Proxy proxy = new Proxy();
proxy.setHttpProxy(proxyHost + proxyPort);
proxy.setSocksUsername(proxyUsername);
proxy.setSocksPassword(proxyPass);
DesiredCapabilities cap = DesiredCapabilities.firefox();
cap.setCapability(CapabilityType.PROXY, proxy);
WebDriver driver = new FirefoxDriver(cap);
driver.get("http://google.com");
I have also tried the following:
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("network.proxy.type", 1);
profile.setPreference("network.proxy.http", proxyHost);
profile.setPreference("network.proxy.http_port", proxyPort);
profile.setPreference("network.proxy.http", "user:pass#1.1.1.1");
profile.setPreference("network.proxy.http_port", proxyPort);
WebDriver driver = new FirefoxDriver(profile);
driver.get("http://google.com");
How can I setup http private proxies (with user name and password in Selenium with Firefox)?
I am using Java.
Thanks

const {Builder, By, Key, until} = require('selenium-webdriver');
const proxy = require('selenium-webdriver/proxy');
(async function example(){
let driver = await new Builder().forBrowser('firefox').setProxy(proxy.manual({
http: 'zproxy.lum-superproxy.io:22225',
https: 'zproxy.lum-superproxy.io:22225'
})).build()
try {
await driver.get('http://lumtest.com/myip.json');
driver.switchTo().alert()
.sendKeys('lum-customer-USERNAME-zone-YOURZONE'+Key.TAB+'PASSWORD');
driver.switchTo().alert().accept();
} finally {
await driver.quit();
}
})();
Here is my sample code when I use Luminati proxy for Selenium.

You can try using browsermob proxy.
Here you go for example

Find solution:
proxy = Proxy({
'proxyType': ProxyType.MANUAL,
'httpProxy': PROXY_HOST,
'socksUsername': 'name',
'socksPassword': 'pass'
})
driver = webdriver.Firefox(proxy=proxy)

Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. when running selenium tests

I want to run my selenium tests on different browsers based on the browser name set in the Properties file.
I have a method named as initiateDriver() in which I get the browser name as set in the Properties file(valid values being ff, chrome or ie) and do the necessary settings for each of the web driver type. This method will return a WebDriver object to my methods.
public WebDriver initiateDriver()
{
// Created webdriver instance
WebDriver _drv = null;
String IEDriverPath, ChromeDriverPath;
try
{
//Get the Browser Name set in the properties file
String browserType = loadPropertiesFile("BrowserName");
Log4j.logger.warn("Browser name-----------"+browserType);
//Test if the browser is IE
if (browserType.equalsIgnoreCase("ie"))
{
//Currently, IEDriverServer.exe is copied on to Drivers folder of framework
IEDriverPath= "\\Drivers\\IEDriverServer.exe";
//Set the required properties to instantiate IE driver. Place any latest IEDriverServer.exe files under Drivers folder
System.setProperty("webdriver.ie.driver", IEDriverPath);
DesiredCapabilities cap= new DesiredCapabilities();
cap.setCapability("ignoreProtectedModeSettings", true);
_drv = new InternetExplorerDriver(cap);
}
//Check if BrowserType is set to Firefox
else if (browserType.equalsIgnoreCase("ff") || browserType.equalsIgnoreCase("firefox"))
{
//Getting the default Firefox with some settings
FirefoxProfile fp = new FirefoxProfile();
fp.setAcceptUntrustedCertificates(false);
//setting the Firefox preference "auto upgrade browser" to false and to prevent compatibility issues
fp.setPreference("app.update.enabled", false);
_drv = new FirefoxDriver();
}
//Check if BrowserType is set to Chrome
else if (browserType.equalsIgnoreCase("chrome"))
{
//Currently, chromedriver.exe is copied on to Drivers folder of framework
ChromeDriverPath= "\\Drivers\\chromedriver.exe";
//Set the required properties to instantiate Chrome driver. Place any latest Chromedriver.exe files under Drivers folder
System.setProperty("webdriver.chrome.driver", ChromeDriverPath);
ChromeOptions options = new ChromeOptions();
options.addArguments("--start-maximized");
_drv= new ChromeDriver(options);
}
else
{
Reporter.log("Invalid browser name. Please check the Resources/PropertiesLocation.properties file.");
System.out.println("Invalid browser name. Please check the Resources/PropertiesLocation.properties file.");
System.exit(0);
}
}
catch(Exception e)
{
System.out.println(e.getMessage());
Reporter.log("Enter valid browser name---");
System.exit(0);
}
return _drv;
}`
I am calling this method in the following class.
public class SmokeTest1
{
WebDriver d;
#BeforeClass
public void setUp() throws Exception
{
d = gm.initiateDriver();
d.manage().window().maximize();
d.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
}
#Test
public void firstSmokeTest() throws Exception
{
loginTest(d);
}
}
I am running my tests via ant build.xml file. However I am getting error as -
Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure."
Can somebody suggest what is going wrong or am I missing anything?

Firebug disabled in WebDriver after adding to profile

I can't seem to figure out why my Firebug extension shows that its disabled when I try to click on it within the Firefox WebDriver.
Here's what my code looks like, I got some of the code from this SO answer:
private final String firefoxExtPath = "/Users/[NAME]/Library/Application Support/Firefox/Profiles/4izeq9he.default/extensions/";
private final String firebugPath = firefoxExtPath + "firebug#software.joehewitt.com.xpi";
private final String firepathPath = firefoxExtPath + "FireXPath#pierre.tholence.com.xpi";
private WebDriver dummy;
private WebDriver driver;
...
#BeforeClass
public void addFirefoxExt() {
// Add extensions to FirefoxDriver
FirefoxProfile profile = new FirefoxProfile();
try {
profile.addExtension(new File(firebugPath));
profile.addExtension(new File(firepathPath));
profile.setPreference("extensions.firebug.currentVersion", "1.11.1");
profile.setPreference("extensions.firebug.onByDefault", true);
profile.setPreference("extensions.firebug.defaultPanelName", "net");
profile.setPreference("extensions.firebug.net.enableSites", true);
} catch (IOException e) {
e.printStackTrace();
}
dummy = new FirefoxDriver(profile);
driver = new FirefoxDriver(profile);
}
#BeforeClass
public void setup() {
dummy.get(BASE_URL);
dummy.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
...
}

That's what i could got from https://code.google.com/p/selenium/wiki/FirefoxDriver. They worked well.
Running with firebug
Download the firebug xpi file from mozilla and start the profile as follows:
File file = new File("firebug-1.8.1.xpi");
FirefoxProfile firefoxProfile = new FirefoxProfile();
firefoxProfile.addExtension(file);
firefoxProfile.setPreference("extensions.firebug.currentVersion", "1.8.1");
// Avoid startup screen
WebDriver driver = new FirefoxDriver(firefoxProfile);

Selenium 2 WebDriver to use Custom Profile

I'm trying to automate the interaction with a website that generates documents with MIME type application/vnd.wap.xhtml+xml. I am using Selenium 2, the WebDriver and the FirefoxProfile.
Because Firefox does not handle the above mentioned MIME type, I need to run Firefox with the XHTML Mobile Profile extension (https://addons.mozilla.org/en-US/firefox/addon/1345/).
After creating a FireFox profile -I named it 'selenium'- and installing the Mobile Profile extension, I tried to use the code snippets in the 'Tips and Tricks' section of the 'Selenium 2.0 and WebDriver' document (http://seleniumhq.org/docs/09_webdriver.html#htmlunit-driver).
Approach #1 looks like this:
ProfilesIni allProfiles = new ProfilesIni();
FirefoxProfile profile = allProfiles.getProfile("selenium");
profile.setPreference("general.useragent.override", "User Agent string to force application/vnd.wap.xhtml+xml content..");
FirefoxDriver driver = new FirefoxDriver(profile);
driver.get("http://www.mobilesite.com/");
WebElement element = driver.findElement(By.tagName("body"));
Approach #2 looks like this:
File profileDir = new File("/path/to/custom/profile/with/extension/ffprofile");
FirefoxProfile profile = new FirefoxProfile(profileDir);
profile.setPreference("general.useragent.override", "same user agent string as above");
FirefoxDriver driver = new FirefoxDriver(profile);
driver.get("http://www.mobilesite.com/");
No matter what code snippet I use, the browser instance that starts up is always unable to handle the generated content; the browser prompts me for an action to take on the content of the unrecognized MIME type as if the extension was not correctly configured.
Any ideas on what I could be doing wrong?
Thanks in advance,
Edit: Link to Selenium users group post.

Try starting with a blank profile and adding extensions/configurations at runtime:
public WebDriver getDriver() {
FirefoxProfile profile = new FirefoxProfile();
// add any custom firefox configurations...
profile.setPreference("general.useragent.override", "some UA string");
profile.setPreference("javascript.options.showInConsole", true);
profile.setPreference("dom.max_script_run_time", 0);
// might have to uninstall, search for *.xpi, then reinstall, then search
// again and compare to find the location on your system
// ...you should probably copy this into your selenium resources directory!
File modifyHeadersXpi = new File("/home/joecoder/.mozilla/firefox/dll8peh9.default/extensions/{b749fc7c-e949-447f-926c-3f4eed6accfe}.xpi");
if (modifyHeadersXpi.exists()) {
try {
profile.addExtension(modifyHeadersXpi);
profile.setPreference("modifyheaders.config.active", true);
profile.setPreference("modifyheaders.config.openNewTab", true);
profile.setPreference("modifyheaders.config.migrated", true);
profile.setPreference("modifyheaders.autocomplete.name.defaults",
"[\"Accept\",\"Cache-Control\",\"Cookie\",\"Content-Length\",\"Content-Type\",\"Date\",\"Host\",\"Pragma\",\"Referer\",\"User-Agent\",\"Via\",\"X-Requested-With\",\"X-Forwarded-For\",\"X-Do-Not-Track\"]");
}
catch (IOException e) { /* uh oh */ }
}
return new FirefoxDriver(profile);
}

Hope this will help you out:
public class Wap {
public static void main (String[] args) throws IOException{
FirefoxProfile profile = new FirefoxProfile();
String baseURL;
profile.addExtension(new File("C:\\Users\\Pandu\\Desktop\\WAP\\modify_headers-0.7.1.1-fx.xpi"));
profile.setPreference("modifyheaders.config.active", true);
profile.setPreference("modifyheaders.config.alwaysOn", true);
profile.setPreference("modifyheaders.headers.count", 2);
profile.setPreference("modifyheaders.headers.action0", "Add");
profile.setPreference("modifyheaders.headers.name0", "X-Nokia-msisdn");
profile.setPreference("modifyheaders.headers.value0", "123456789");
profile.setPreference("modifyheaders.headers.enabled0", true);
profile.setPreference("modifyheaders.headers.action1", "Add");
profile.setPreference("modifyheaders.headers.name1", "x-sec-pass");
profile.setPreference("modifyheaders.headers.value1", "sdp123");
profile.setPreference("modifyheaders.headers.enabled1", true);
Logger Log = Logger.getLogger(WebDriver.class.getName());
WebDriver driver = new FirefoxDriver(profile);
try{
driver.get("http://www.google.com");
driver.findElement(By.linkText("Telugu")).click();

You have to make sure you add the browser plugin as a DeploymentItem in your testsettings file.
Some examples (in this one we have added Firebug):
<Deployment>
<DeploymentItem filename="Selenium\firebug#software.joehewitt.com.xpi" />
<DeploymentItem filename="packages\Castle.Core.3.1.0\lib\net40-client\Castle.Core.dll" />
<DeploymentItem filename="Selenium\IEDriverServer.exe" />
<DeploymentItem filename="Selenium\chromedriver.exe" />
<DeploymentItem filename="Selenium\skipcerterror#foudil.fr.xpi" />
</Deployment>
You will then need to create a profile that looks something like this:
string firebugPath = Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) ?? "", "firebug#software.joehewitt.com.xpi");
FirefoxProfile firebugProfile = new FirefoxProfile() {AcceptUntrustedCertificates = true};
firebugProfile.AddExtension(firebugPath);
firebugProfile.SetPreference("extensions.firebug.currentVersion", "1.10.3");
firebugProfile.SetPreference("extensions.sce.bypass_domain_mismatch", true);
firebugProfile.SetPreference("webdriver_assume_untrusted_issuer", false);
Driver = new FirefoxDriver(firebugProfile);
Driver.Manage().Window.Maximize();
If you add the extension using AddExtension, it should be available within you selenium driver. I hope this helps.