I'm trying to get response and request using BMP's RequestFilter and ResponseFilter. However, when the webpage loads, nothing gets printed in the console.
Everything else seems to work though. Maybe BMP is not watching GeckoDriver?
I'm using Firefox 50.0, BrowserMobProxy 2.1.2, Selenium 3.0.1, and GeckoDriver 0.11.1
The testing code is below. Could someone please help me?
Thank you very much!
BrowserMobProxy server = new BrowserMobProxyServer();
server.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);
server.start();
int port = server.getPort();
server.addRequestFilter((request, content, info) -> {
String q = URLDecoder.decode(info.getOriginalUrl(), "UTF-8");
System.out.println("Request: "+q);
return null;
});
server.addResponseFilter((response, content, info) -> {
String type = response.headers().get("Content-Type");
System.out.println("Response: "+info.getOriginalRequest());
System.out.println(type);
});
Proxy proxy = ClientUtil.createSeleniumProxy(server);
DesiredCapabilities capabilities = DesiredCapabilities.firefox();
capabilities.setCapability(CapabilityType.PROXY, proxy);
capabilities.setCapability("marionette", true);
capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
capabilities.setCapability(CapabilityType.SUPPORTS_JAVASCRIPT, true);
FirefoxProfile fp = new FirefoxProfile();
capabilities.setCapability(FirefoxDriver.PROFILE, fp);
String gecko = "d:/Programming/java/geckodriver.exe";
System.setProperty("webdriver.gecko.driver", gecko);
driver = new FirefoxDriver(capabilities);
driver.get("https://google.com");;
In Firefox 51 and lower, there is a bug/missing feature in Selenium 3's GeckoDriver that prevents Firefox from picking up the proxy settings when setting CapabilityType.PROXY on the DesiredCapabilities object.
However, you can still set the proxy settings directly on the FirefoxProfile. There's an example of this in one of BMP's tests. Since you're already using a FirefoxProfile object, this would probably be a sensible solution for you. It would look something like this (replace localhost with a hostname/ip address as appropriate):
FirefoxProfile fp = new FirefoxProfile();
fp.setPreference("network.proxy.http", "localhost");
fp.setPreference("network.proxy.http_port", server.getPort());
fp.setPreference("network.proxy.ssl", "localhost");
fp.setPreference("network.proxy.ssl_port", server.getPort());
fp.setPreference("network.proxy.type", 1);
fp.setPreference("network.proxy.no_proxies_on", "");
This geckodriver issue also discusses a few other alternatives to using CapabilityType.PROXY on the DesiredCapabilities object.
UPDATE
According to the mozilla bug report, this issue is fixed in Firefox 52, which is scheduled to be released on March 7, 2017. In the meantime, the solution with the FirefoxProfile should work with 51 (and lower), and should also continue to work with 52+.
Related
Proxy not working in Selenium
First of all i use free proxys from proxyfish (maybe thats the problem alrdy) https://hidemyna.me/en/proxy-checker/ tells me theyre working.
I found lots of code/solutions, most is outdated (capabilities=options..) or not working for me.
Tried several code/proxys chrome/ff however https://www.iplocation.net/ and https://whatismyipaddress.com showing actual IP address (IPv4 and IPv6).
What I tried..
System.setProperty("webdriver.chrome.driver", "C:\\selenium\\chromedriver.exe");
System.getProperties().put("74.208.112.***", "8080"); //1st try
Proxy proxy = new Proxy();
proxy.setHttpProxy("74.208.112.***:8080"); //2nd try
ChromeOptions options = new ChromeOptions();
options.setCapability("proxy", proxy);
options.setProxy(proxy);
options.addArguments("--proxy-server=74.208.112.***:8080"); //3rd try
ChromeDriver driver = new ChromeDriver(options);
driver.get("https://www.iplocation.net/");
driver.get("https://whatismyipaddress.com/");
Solution:
proxy.setSslProxy("74.208.112.***:8080");
Is there a way to test proxy connection before using proxy?
I am having a problem with geckodriver (FF) in java-selenium.
I am running multiple test scenarios via TEST-NG parallel execution (6+ threads).
Sometimes happens that at the start (opening new browser) I get unreachableBrowserException. So i try-catch it, waited 100ms and try to create it again. it works, but it keeps the first failed browser opened, with a blank page.
Problem is that the first window got opened BEFORE it got initialized as a driver, so it cannot be closed by driver.close() or something (it is null). So I have to close them manually.
So, my question is, has anyone encountered such behaviour?
How can i close such browsers, without influencing the other threads?
while(true) {
try {
setDriver(DriverFactory.createInstance(getBrowserInstance()));
break;
} catch (UnreachableBrowserException e) {
try {
Thread.sleep(100);
log.info("UnreachableBrowserException! Needed to wait for 100ms ");
removeDriver();
counter++;
} catch (Exception e1) {
log.info("Thread could not wait!");
e1.printStackTrace();
}
}
if(counter>100){
log.info("Was not able to create a browser session!");
break;
}
}
and the part of .createInstance :
final ProfilesIni profilesIni = new ProfilesIni();
System.setProperty("webdriver.gecko.driver", "drivers/geckodriver.exe");
System.setProperty(FirefoxDriver.SystemProperty.DRIVER_USE_MARIONETTE,"false");
firefoxProfile = new FirefoxProfile();
firefoxProfile.setAcceptUntrustedCertificates(true);
firefoxProfile.setAssumeUntrustedCertificateIssuer(false);
firefoxProfile.setPreference("app.update.auto", false);
firefoxProfile.setPreference("app.update.enabled", false);
firefoxProfile = profilesIni.getProfile("selenium_profile");
capabilities = DesiredCapabilities.firefox();
capabilities.setCapability(FirefoxDriver.PROFILE, firefoxProfile);
return new FirefoxDriver(capabilities); //Here it throws the exception.
Selenium/Java/geckodriver versions are most recent. FF version is unfortunately 44.02 (cannot get newer).
I would be grateful for any tips/hints.
Kind regards,
Martin
As you are using Mozilla Firefox version 44.02 and cannot get newer you have to consider a lot of things.
To use Selenium 3.x you have to mandatory use geckodriver. See why here.
Firefox 44.02 is not marionette enabled, so you have to set marionette to false through DesiredCapabilities class. See why here
There are couple of Selenium version dependencies and GeckoDriver version dependencies. See them here
You cannot create a new Firefox Profile and then again try to use a existing Firefox Profile.
Assuming you have a existing Firefox Profile by the name selenium_profile, the following code would open the Firefox Profile in a new browser session:
System.setProperty("webdriver.gecko.driver", "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
ProfilesIni profile = new ProfilesIni();
FirefoxProfile firefoxProfile = profile.getProfile("selenium_profile");
firefoxProfile.setAcceptUntrustedCertificates(true);
firefoxProfile.setAssumeUntrustedCertificateIssuer(false);
firefoxProfile.setPreference("app.update.auto", false);
firefoxProfile.setPreference("app.update.enabled", false);
DesiredCapabilities dc = DesiredCapabilities.firefox();
dc.setCapability(FirefoxDriver.PROFILE, firefoxProfile);
dc.setCapability("marionette", false);
WebDriver driver = new FirefoxDriver(dc);
driver.get("http://www.google.com");
Now you can easily return the capabilities.
My Firefox version is 46.0.1 and Selenium version is 3.0.1.
I am getting error:
Your connection is not secure
while executing following code:
#Test
public void test() {
ProfilesIni profile = new ProfilesIni();
FirefoxProfile ffProfile = profile.getProfile("newCretedProfile");
ffProfile.setAcceptUntrustedCertificates(true);
ffProfile.setAssumeUntrustedCertificateIssuer(false);
System.setProperty("webdriver.gecko.driver", "D:\\SELENUIUM\\Drivers\\geckodriver.exe");
FirefoxDriver driver = new FirefoxDriver(ffProfile);
driver.get("http://www.google.com");
driver.quit();
}
I have created new firefox profile and followed steps from this url
Nevertheless it's not working and giving me same error while I launching any site.
Download Firefox 55 beta and set
capabilities.setCapability("acceptInsecureCerts", true);
Here is my code that works for Firefox 55 beta:
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setBrowserName("firefox");
capabilities.setCapability("acceptInsecureCerts", true);
RemoteWebDriver driver = new RemoteWebDriver(Environment.remoteWebDriverURL, capabilities);
I have tried this approach and it worked well for me.
Create new firefox profile by following below step.
Close all your firefox windows
In the Run dialog box, type in: ‘firefox.exe -p' and then Click OK.
Click “Create Profile”
Create a name for your new profile(say Selenium)
Click “Choose Folder”
Pick location something easy to find — like “C:\NewFirefoxProfile”
Click Finish
Now after selecting newly created profile, start Firefox. Open the specific url you were getting 'Secure Connection Issue', accept SSL certificates for this profile.
Now use the newly created firefox profile to run your selenium test. Modify below code as per your requirement.
System.setProperty("webdriver.firefox.marionette","D:\\SELENUIUM\\Drivers\\geckodriver.exe");
ProfilesIni profile = new ProfilesIni();
FirefoxProfile myprofile = profile.getProfile("C:\\NewFirefoxProfile");//location of your new firefox profile
WebDriver driver = new FirefoxDriver(myprofile);
driver.get("https://cacert.org/");
With FF v53+ and Se 3.x (summer 2017), advice from before (May?) 2017 is no longer true.
You have to use Marionette and set capability to True.
Took me few days to sort out all old and obsolete advice, yours for free. :-)
Looks like it is not supported yet by geckodriver/Marionette.
You can check below bugs for more information:-
https://github.com/mozilla/geckodriver/issues/93
https://bugzilla.mozilla.org/show_bug.cgi?id=1103196
If you want to run the tests on Firefox with Selenium 3.0 set the Firefox driver capability “marionette” to false.
#Test
public void test() {
DesiredCapabilities d = new DesiredCapabilities();
d.setCapability("marionette", false);
WebDriver driver = new FirefoxDriver(d);
driver.get("http://www.google.com");
driver.quit();
}
In my automation project I need to set proxy server. I tried with system variable settings and profile setting for firefox browser. But those technique does not work for me. Please any one help me in this regard.
Note: I also tried with executing shell command using java but I got stuck when password is asked.
You definitely don't need to set any System-level properties. This is one way to do it in Firefox:
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("network.proxy.type", 1); // Manual proxy config
profile.setPreference("network.proxy.http", "proxy3.proxy.net");
profile.setPreference("network.proxy.http_port", 3128);
profile.setPreference("network.proxy.ssl", "proxy3.proxy.net");
profile.setPreference("network.proxy.ssl_port", 3128);
WebDriver driver = new FirefoxDriver(profile);
Or a more flexible, less browser-specific alternative:
org.openqa.selenium.Proxy proxy = new org.openqa.selenium.Proxy();
proxy.setHttpProxy("proxy3.proxy.net:3128");
proxy.setSslProxy("proxy3.proxy.net:3128");
DesiredCapabilities caps = DesiredCapabilities.firefox(); // or chrome() etc.
caps.setCapability(CapabilityType.PROXY, proxy);
WebDriver driver = new FirefoxDriver(caps);
I am using selenium with PhantomJs to scrape the URL. I initialized the driver as below
final DesiredCapabilities caps = DesiredCapabilities.chrome();
caps.setCapability(
PhantomJSDriverService.PHANTOMJS_EXECUTABLE_PATH_PROPERTY,
"PhantomJsPath");
caps.setCapability("page.settings.loadImages", false);
caps.setCapability("trustAllSSLCertificates", true);
RemoteWebDriver driver = new PhantomJSDriver(caps);
driver.setLogLevel(Level.OFF);
driver.get("https://.......")
The pagesource obtained from the driver is empty
Am I missing anything?
Recently the POODLE vulnerability forced websites to remove SSLv3 support. Since PhantomJS < v1.9.8 uses SSLv3 by default, the page cannot be loaded. To fix this, you would need to run PhantomJS with --ssl-protocol=tlsv1 or --ssl-protocol=any. See this answer for plain PhantomJS.
caps = DesiredCapabilities.phantomjs(); // or new DesiredCapabilities();
caps.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS,
new String[] {"--ssl-protocol=tlsv1"});
// other capabilities
driver = new PhantomJSDriver(caps);
If this doesn't solve the issue, you can also add
"--web-security=false", "--ignore-ssl-errors=true"
to the String array of cli args as seen in SiKing's answer here.