Develop Reference

How to download a log file instead of viewing(some files are opened in chrome if memory is less than 200B) in chrome using Selenium Remote WebDriver

In the below code we are trying to capture the information when the chrome opens the file, but it is failing in some cases as some log files are getting downloaded and it's not viewable in chrome.
Let me know if any preference has to be added so that all files (irrespective of size) can be either viewed or downloaded.
public class DownloadFile {
String FILE_ADD = "file:///";
String LOG_PATH = "path";
String LOG_FILE = "sample.log";
String OUTPUT_FILE = "output.txt";
public static RemoteWebDriver driver;
#BeforeEach
public void setUp() throws Exception {
System.setProperty("webdriver.chrome.driver","chromedriver.exe");
ChromeOptions options = new ChromeOptions();
//Map<String, Object> prefs = new HashMap<String, Object>();
//prefs.put("download.prompt_for_download", false);
//prefs.put("log.disabled", true);
//options.setExperimentalOption("prefs", prefs);
//driver = new ChromeDriver(options);
//driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
//driver.manage().window().maximize();
String fileDownloadPath = "downnloadpath";
Map<String, Object> prefsMap = new HashMap<String, Object>();
prefsMap.put("profile.default_content_settings.popups", 0);
prefsMap.put("download.default_directory", fileDownloadPath);
prefsMap.put("txtjs.disabled", true);
ChromeOptions option = new ChromeOptions();
option.setExperimentalOption("prefs", prefsMap);
option.addArguments("--test-type");
option.addArguments("--disable-extensions");
//Any preference to be added to download all the files instead of viewing some files in chrome
driver = new ChromeDriver(option);
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
driver.manage().window().maximize();
}
#Test
public void fileDownload() throws AWTException, InterruptedException, IOException {
driver.get(FILE_ADD+LOG_PATH+LOG_FILE);
//String output = driver.findElement(By.xpath("/html/body/pre")).getText();
//String final_output = output.replaceAll("User.*>", "testing");
//FileUtils.writeFile(LOG_PATH+OUTPUT_FILE, output);
Thread.sleep(7000);
}
#AfterEach
public void tearDown() throws Exception {
driver.quit();
}
}

To my knowledge, you can not have this custom, where you can set it to view for 1 file, or download for the next, unless you override the ChromeDriver options below you download...
The below is if you wish to download the file/without a dialog box:
download.prompt_for_download is what you should use.
String downloadFilepath = "C:\\DownloadDirectoryPath"
HashMap<String, Object> chromePreferences = new HashMap<String, Object>();
chromePreferences.put("profile.default_content_settings.popups", 0);
chromePreferences.put("download.prompt_for_download", "false");
chromePreferences.put("download.default_directory", downloadFilepath);
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.setExperimentalOption("prefs", chromePreferences);
=== Edited ===
You could try to use the option:
chromePreferences.put("profile.content_settings.exceptions.automatic_downloads.*.setting", 1 );
Are you using setExperimentalOption? What is your ChromeDriver version?
Using: ChromeDriver 91, you can look into your Preferences file under the directory: C:\Users<yourName>\AppData\Local\Google\Chrome\User Data\Default
The below URLs, Chrome does not popup a Download Dialog box.
In my Preferences file, I have the following as an example:
"automatic_downloads": {
"https://folkets-lexikon.csc.kth.se:443,*": {
"last_modified": "13182517278621031",
"setting": 1
},
"https://mail.google.com:443,*": {
"last_modified": "13160578764124505",
"setting": 1
}
},
As a reference, the below are helpful:
Disable chrome download multiple files confirmation
Could not set download.prompt_for_download false for avoiding popup when downloading a file in an Electron application
How to use chrome webdriver in selenium to download files in python?

WebDriver save problems in headless mode

I'm using Java+ChromeDriver on Mac for my project.
Here is some code:
System.setProperty("webdriver.chrome.driver", TestConfig.driverURL());
ChromeOptions options = new ChromeOptions();
options.addArguments("headless").addArguments("window-size=2560x1440");
String downloadFilePath = "some file path";
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("download.default_directory", downloadFilePath);
options.setExperimentalOption("prefs", chromePrefs);
driver = new ChromeDriver(options);
driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
When I comment this line:
options.addArguments("headless").addArguments("window-size=2560x1440");
all is working fine and I download a couple of files, but Chrome doesn't work in headless mode.
When this line is not commented files don't download.
Can anybody tell my how I can download files using ChromeDriwer in headless mode?
Thanks.

Here is what worked for me:
options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
prefs = {'download.default_directory' : out_path}
options.add_experimental_option('prefs', prefs)
download_path = './output/'
driver = webdriver.Chrome('./chromedriver', chrome_options=options)
driver.command_executor._commands["send_command"] = ("POST",
'/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow',
'downloadPath': download_path}}
command_result = driver.execute("send_command", params)

Getting error as "NoSuchSessionException" when trying to open an website through FirefoxProfile

I am running the below code, to open a URL. However, I am getting error as "NoSuchSessionException". Kindly suggest.
Is it because of the below versions I am using.
Selenium--> 3.12.0, Firefox Setup 50.0 and geckodriver-v0.21.0-win64
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.firefox.FirefoxProfile;
import org.openqa.selenium.firefox.internal.ProfilesIni;
public class Gmail {
public static void main(String[] args){
System.setProperty("webdriver.gecko.driver", "D:\\Drivers\\geckodriver.exe");
FirefoxOptions options = new FirefoxOptions();
ProfilesIni allProf = new ProfilesIni();// all profiles
FirefoxProfile prof = allProf.getProfile("Abhi_Selenium");
options.setProfile(prof);
//FirefoxDriver driver = new FirefoxDriver(options);
WebDriver driver = new FirefoxDriver(options);
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("http://gmail.com");
}
}

You have 2 ways to use a existing Firefox Profile to access a Web Application as follows:
Using DesiredCapabilities() and FirefoxOptions():
public class FirefoxProfile_dc_opt {
public static void main(String[] args) {
System.setProperty("webdriver.gecko.driver", "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
ProfilesIni profile = new ProfilesIni();
FirefoxProfile testprofile = profile.getProfile("Abhi_Selenium");
DesiredCapabilities dc = DesiredCapabilities.firefox();
dc.setCapability(FirefoxDriver.PROFILE, testprofile);
FirefoxOptions opt = new FirefoxOptions();
opt.merge(dc);
WebDriver driver = new FirefoxDriver(opt);
driver.get("https://www.google.com");
}
}
Using FirefoxOptions():
public class FirefoxProfile_opt {
public static void main(String[] args) {
System.setProperty("webdriver.gecko.driver", "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
ProfilesIni profile = new ProfilesIni();
FirefoxProfile testprofile = profile.getProfile("Abhi_Selenium");
FirefoxOptions opt = new FirefoxOptions();
opt.setProfile(testprofile);
WebDriver driver = new FirefoxDriver(opt);
driver.get("https://www.google.com");
}
}
Note: Ensure that you have already created a Firefox Profile as Abhi_Selenium before you trigger your Test.
Update
As you are still seeing the exception as no such session, perform the following upgradation/cleanup steps:
Upgrade JDK to recent levels JDK 8u181.
Upgrade Selenium to current levels Version 3.13.0.
Upgrade GeckoDriver to GeckoDriver v0.20.1 level.
Ensure GeckoDriver is present in the specified location.
Ensure GeckoDriver is having executable permission for non-root users.
Upgrade Firefox version to Firefox v61.0.1 levels.
Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
(WindowsOS only) Use CCleaner tool to wipe off all the OS chores before and after the execution of your Test Suite.
(LinuxOS only) Free Up and Release the Unused/Cached Memory in Ubuntu/Linux Mint before and after the execution of your Test Suite.
If your base Web Client version is too old, then uninstall it through Revo Uninstaller and install a recent GA and released version of Web Client.
Take a System Reboot.
Execute your Test as a non-root user.
Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.

You can use FireFoxProfile class and FirefoxOptions class to set a profile.
FirefoxOptions options = new FirefoxOptions();
FirefoxProfile firefoxProfile = new FirefoxProfile(pathToProfile);
options.setProfile(firefoxProfile);

On the first look the path to firefox.exe is missing. There is my setup:
public class foo{
private static WebDriver driver;
#BeforeClass
public static void setUpClass() {
FirefoxOptions options = new FirefoxOptions();
ProfilesIni allProfiles = new ProfilesIni();
FirefoxProfile selenium_profile = allProfiles.getProfile("selenium_profile");
options.setProfile(selenium_profile);
options.setBinary("C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe");
System.setProperty("webdriver.gecko.driver", "C:\\Users\\pburgr\\Desktop\\geckodriver-v0.20.0-win64\\geckodriver.exe");
driver = new FirefoxDriver(options);
driver.manage().window().maximize();}
// #Before, #After, #AfterClass and #Test
}

How to disable JavaScript in browser using Selenium (Java)?

In my feature automation, I need to disable JavaScript in browser and run the flow. How to disable JavaScript?
Tried DesiredCapabilities for firefox and Chrome.
DesiredCapabilities dc = new DesiredCapabilities();
dc.setCapability(CapabilityType.SUPPORTS_JAVASCRIPT, false)
And
DesiredCapabilities dc = new DesiredCapabilities();
dc.setJavascriptEnabled(false);
For firefox, tried
1) Setting up profile for firefox
2) Adding add-on - noScript.xpi
3) profile.setPreference("javascript.enabled", false);
4) Through UI, tried changing the flag - "javascript.enabled" in "about:config" to false. Here, opened firefox and gave "about:config" getting a warning - "This might void your warranty!". There is a button - "I'll be careful, I promise!" with id - warningButton. This button should be clicked to proceed further. To click this button, used driver.findElement(By.id("warningButton")).click(); but it not work.
All the above options are not working. Any advice will be helpful.

I don't know Java, but maybe a solution for Python 3 will help you.
in Python, you can use Options() instead of FirefoxProfile() to deactivate JavaScript:
from selenium.webdriver.firefox.options import Options
options = Options()
options.preferences.update({"javascript.enabled": False})
driver = webdriver.Firefox(options=options)
driver.get('about:config')
Maybe Java this:
FirefoxOptions options = new FirefoxOptions();
options.preferences.update({"javascript.enabled": False});
WebDriver driver = new FirefoxDriver(options);
driver.get('about:config')

You change the preference value using profile with lots of options:
DesiredCapabilities capabilities = new DesiredCapabilities();
// setCapability(SUPPORTS_JAVASCRIPT, javascriptEnabled);
capabilities.setJavascriptEnabled(false);
FirefoxBinary binary = new FirefoxBinary( new File( binaryPath ) );
FirefoxProfile profile = new FirefoxProfile();
//profile.setPreference("preferenceName", "Value");
profile.setPreference("javascript.enabled", false);
RemoteWebDriver driver = new FirefoxDriver(binary, profile, capabilities);
To view the preferences, you can visit the URL about:config
#See
Chrome driver to disable JavaScript issue
chromium-command-line-switches

Truse me this was random trial but works perfectly for me
from selenium import webdriver
options= webdriver.ChromeOptions()
chrome_prefs = {}
options.experimental_options["prefs"] = chrome_prefs
chrome_prefs["profile.default_content_settings"] = {"javascript": 2}
chrome_prefs["profile.managed_default_content_settings"] = {"javascript": 2}
driver = webdriver.Chrome("your chromedriver path here",options=options)
driver.get('https://google.com/search?q=welcome to python world')
Example image here:-https://i.stack.imgur.com/DdKZQ.png

As per Selenium 3.6 Java Client Release, the easiest way to disable Javascript in the browser would be to set the setJavascriptEnabled argument through an instance of DesiredCapabilities to False and merge it through an instance of FirefoxOptions as follows:
package demo;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.DesiredCapabilities;
public class Q46883024_setJavascriptEnabled
{
public static void main(String[] args)
{
System.setProperty("webdriver.gecko.driver", "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
DesiredCapabilities dc = new DesiredCapabilities();
dc.setJavascriptEnabled(false);
FirefoxOptions op = new FirefoxOptions();
op.merge(dc);
WebDriver driver = new FirefoxDriver(op);
driver.get("https://google.com");
driver.quit();
}
}
While execution, the browser you are using may override the setJavascriptEnabled settings.

this works:
FirefoxOptions options = new FirefoxOptions();
options.addPreference("javascript.enabled", false);

This is how you can do it for Chrome in Java.
// import org.openqa.selenium.chrome.ChromeOptions;
ChromeOptions options = new ChromeOptions();
options.addArguments("user-agent=\"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\"");
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("profile.default_content_setting_values.javascript", 2);
options.setExperimentalOption("prefs", chromePrefs);
new ChromeDriver(options);
And it worked for me with ChromeDriver 2.41.578706. As a bonus I am also setting Googlebot as user-agent.
In case you need to do something with DesiredCapabilities you can also convert the options above to capabilities:
// import static org.openqa.selenium.chrome.ChromeOptions.CAPABILITY;
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability(CAPABILITY, options);
new ChromeDriver(capabilities);

How to set Proxy Authentication in seleniumWebdriver for Chrome Browser

I'm trying to Automate a web application selenium 2.0 [webdriver+java].The web application is currently deployed in our UAT servers on our local network.My test cases are executing, but I have to manually enter the Proxy Authentication details for my Chrome instance at the start of the test execution. I have tried all the solutions provided on stack overflow but still, the authentication message pops out.
This is the code I'm using in my driver initializing process
package com.misyn.ess.ui;
import java.util.Arrays;
import org.openqa.selenium.Proxy;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.DesiredCapabilities;
/**
*
* #author User
*/
public class DriverClass {
private String baseUrl;
private String driverPath;
private String driverName;
private static WebDriver driver;
private static DriverClass driverClass;
private DriverClass() {
try {
baseUrl = "http://192.168.0.10:8282/ess";
driverPath = "E:\\Work_Folder\\SelTools\\chromedriver.exe";
driverName = "webdriver.chrome.driver";
//Set the location of the ChromeDriver
System.setProperty(driverName, driverPath);
//Create a new desired capability
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
// Create a new proxy object and set the proxy
Proxy proxy = new Proxy();
proxy.setHttpProxy("192.168.0.200:3128");
proxy.setSocksUsername("avishka");
proxy.setSocksPassword("12345678");
//Add the proxy to our capabilities
capabilities.setCapability("proxy", proxy);
//Start a new ChromeDriver using the capabilities object we created and added the proxy to
driver = new ChromeDriver(capabilities);
//Navigation to a url and a look at the traffic logged in fiddler
driver.navigate().to(baseUrl);
// System.setProperty(driverName, driverPath);
// driver = new ChromeDriver();
// driver.get(baseUrl);
} catch (Exception e) {
e.printStackTrace();
}
}
}
Can anyone give me a solution how to give this proxy username and password thing from the application itself than manually entering details on the pop-up(Authentication), any help would be much appreciated.Thanks
the currently answered one is only for
As of Selenium 3.4 it is still in beta
Right now implementation is only done for InternetExplorerDriver
Where I'm using selenium 3.0 and Google Chrome as my web browser.

You can do via MultiPass for HTTP basic authentication
Download the extension from
https://chrome.google.com/webstore/detail/multipass-for-http-basic/enhldmjbphoeibbpdhmjkchohnidgnah
Download the extension as crx. You can get it as crx from chrome-extension-downloader
After that the config is simple.
import java.io.File;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
/**
*
* #author Phystem
*/
public class ChromeAuthTest {
WebDriver driver;
public ChromeAuthTest() {
System.setProperty("webdriver.chrome.driver", "chromedriver.exe");
}
private void initDriver() {
ChromeOptions cOptions = new ChromeOptions();
cOptions.addExtensions(new File("MultiPass-for-HTTP-basic-authentication_v.crx"));
driver = new ChromeDriver(cOptions);
configureAuth(
"https://the-internet.herokuapp.com/basic_auth",
"admin",
"admin");
}
private void configureAuth(String url, String username, String password) {
driver.get("chrome-extension://enhldmjbphoeibbpdhmjkchohnidgnah/options.html");
driver.findElement(By.id("url")).sendKeys(url);
driver.findElement(By.id("username")).sendKeys(username);
driver.findElement(By.id("password")).sendKeys(password);
driver.findElement(By.className("credential-form-submit")).click();
}
public void doTest() {
initDriver();
driver.get("https://the-internet.herokuapp.com/basic_auth");
System.out.println(driver.getTitle());
driver.quit();
}
public static void main(String[] args) {
new ChromeAuthTest().doTest();
}
}
I have used a sample site for testing.
Provide your url,username and password in the configure Auth function and try

public class DriverClass {
private String baseUrl;
private String driverPath;
private String driverName;
private static WebDriver driver;
private static DriverClass driverClass;
public DriverClass() {
try {
baseUrl = "http://192.168.0.10:8282/ess";
driverPath = "E:\\Work_Folder\\SelTools\\chromedriver.exe";
driverName = "webdriver.chrome.driver";
System.setProperty(driverName, driverPath);
Proxy proxy = new org.openqa.selenium.Proxy();
proxy.setSslProxy("192.168.0.200" + ":" + 3128);
proxy.setFtpProxy("192.168.0.200" + ":" + 3128);
proxy.setSocksUsername("avishka");
proxy.setSocksPassword("12345678");
DesiredCapabilities desiredCapabilities = DesiredCapabilities.chrome();
desiredCapabilities.setCapability(CapabilityType.PROXY, proxy);
driver = new ChromeDriver(desiredCapabilities);
driver.get(baseUrl);
} catch (Exception e) {
e.printStackTrace();
}
}
}
The proxy setting has been added with desired capabilities to pass values to proxy authentication,worked finally

This code (from Avishka Perera's answer) does not work for me:
proxy.setSocksUsername("avishka");
proxy.setSocksPassword("12345678");
The username and password set in this way do not take effect for the http/https proxy - the Proxy Authentication box still popped up.
I'm using Selenium java 3.141.0, ChromeDriver 2.33 and chrome 70. What works for me is to follow Mike's answer here Selenium using Python: enter/provide http proxy password for firefox .
Create the zip file, then add the extension like this:
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addExtensions(new File("src/test/resources/proxy.zip"));
WebDriver driver = new ChromeDriver(chromeOptions);
One catch is that the above code will run into error if you set "--headless" argument because chrome in headless mode cannot have extension (Is it possible to run Google Chrome in headless mode with extensions?). If your Chrome runs in Docker container and cannot show the UI, then to get this solution work, you'll need to run with Xvfb instead of in headless mode.

Simple method to add authenticated proxy using selenium wire in Both firefox and chrome
In python
Step:1
pip3 install selenium-wire
Step:2
from seleniumwire import webdriver
from selenium import webdriver
step:3
Add proxy in below-mensioned format
proxy= "username:password#ip:port"
options = {'proxy': {'http': proxy, 'https': proxy, 'no_proxy': 'localhost,127.0.0.1,dev_server:8080'}}
step:4
pass proxy as an argument
CHROME
driver = webdriver.Chrome(options=chrome_options, executable_path="path of chrome driver", seleniumwire_options=options)
Firefox
driver = webdriver.Firefox(seleniumwire_options=options, executable_path="path of firefox driver", options=firefox_options)
step:5
Verify proxy applied by requesting the url https://whatismyipaddress.com/
time.sleep(20)
driver.get("https://whatismyipaddress.com/")
Note:
But selenium log shows it runs in without proxy because we are using an external package to apply proxy.

I know this is an old thread, still leaving a solution which worked for me using browsermob proxy, for someone who still needs an option.
Maven dependency:
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.0.0</version>
</dependency>
<dependency>
<groupId>net.lightbody.bmp</groupId>
<artifactId>browsermob-core</artifactId>
<version>2.1.5</version>
</dependency>
Java Code:
// I am using firefox
System.setProperty("webdriver.gecko.driver", "C:\\Selenium\\geckodriver.exe");
BrowserMobProxy browsermobProxy = new BrowserMobProxyServer();
browsermobProxy.setChainedProxy(new InetSocketAddress(PROXY_HOSTNAME, PROXY_PORT));
browsermobProxy.chainedProxyAuthorization(PROXY_USERNAME, PROXY_PASSWORD, AuthType.BASIC);
browsermobProxy.start(0);
FirefoxBinary firefoxBinary = new FirefoxBinary();
FirefoxOptions firefoxOptions = new FirefoxOptions();
firefoxOptions.setBinary(firefoxBinary );
firefoxOptions.setProfile(firefoxProfile);
firefoxOptions.setProxy(ClientUtil.createSeleniumProxy(browsermobProxy));
WebDriver webDriverWithProxy = new FirefoxDriver(firefoxOptionsWithProxy);
webDriverWithProxy.get("https://stackoverflow.com/");

The approach that worked perfectly fine for me is by using AutoIT.
Install autoIT and prepare a simple script as shown in the picture attached and execute the script file from your testscript using Runtime.getRuntime().exec("\YOUR_SCRIPT.exe") before navigating to the baseURL.