I use BrowserStack with Selenium-webdriver to run tests on different types of devices and browsers. So actually tests are running by RemoteWebDriver.
I know that it's possible to capture network within Selenium tests using BrowserMobProxy, but as i understand it's working only if test is running on local machine.
Is there a way to capture network while running test on cross-platform base like BrowserStack?
UPDATE
I managed to get capture of network in har file (from link "localhost:8080/proxy/8081/har"), using standalone BrowserMobProxy and standalone local BrowserStack, as I was advised.
I tried to do the same automatically from code:
BrowserMobProxy proxy = new BrowserMobProxyServer();
proxy.start(8080);
System.out.println("Proxy port: " + port);
System.setProperty("java.net.useSystemProxies", "true");
System.setProperty("http.proxyHost", "127.0.0.1");
System.setProperty("http.proxyPort", "8080");
System.setProperty("https.proxyHost", "127.0.0.1");
System.setProperty("https.proxyPort", "8080");
Local l = new Local();
Map<String, String> options = new HashMap<String, String>();
options.put("key", accessKey);
options.put("forcelocal", "true");`
//when I uncomment it i get an exception:
//com.browserstack.local.LocalException: Could not connect to www.browserstack.com!
// options.put("forceproxy", "true");
// options.put("proxyHost", "localhost");
// options.put("proxyPort", "8080");
l.start(options);
}
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
proxy.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);
driver = new RemoteWebDriver(new URL("http://"+username+":"+accessKey+"#"+config.get("server")+"/wd/hub"), capabilities);'
proxy.newHar("testHar.com");
driver.get(testUrl);
Thread.sleep(15000);
Har har = proxy.getHar();
FileOutputStream fos = new FileOutputStream("C:\\LoadingPage\\network\\testHar.har");
har.writeTo(fos);
The connection to the url is working, I could see it and make screenshouts.
BUT! In the har file I see only request to "hub-cloud.browserstack.com/wd/hub/...", not the requests from page itself.
How to get correct har from code?
What in the code is not correct?
From my experience I would like to add small modification to the binary command given in BrowserStack link (Shared by Mikhail). The cmd given in doc should work well for private URLs but may not work for public ones.
Steps for Standalone binary:
1 - Download the BrowserStackLocal binary from 'https://www.browserstack.com/local-testing#command-line'.
Launch the binary by running the below command to enable your proxy to monitor the traffic.
- BrowserStackLocal.exe --key
--local-proxy-host --local-proxy-port --local-proxy-user --local-proxy-pass --force-proxy --force-local
More details on all the modifiers are available at 'https://www.browserstack.com/local-testing#modifiers'.
2 - Include "browserstack.local" capability in your test script.
"browserstack.local" = true
Steps for Java (BrowserStack Local) bindings:
1 - Follow these steps for using local bindings.
2 - Using this you can use newer options available in latest versions of the binary. For instance if you wish to add --local-proxy-* options, for which there is no existing wrapper (like this which is internally mapped to this), try using below:
bsLocalArgs.put("-local-proxy-host", "Your BrowserMob proxy IP");
bsLocalArgs.put("-local-proxy-port", "Your BrowserMob proxy Port");
bsLocalArgs.put("-local-proxy-user", "Your BrowserMob proxy Username");
bsLocalArgs.put("-local-proxy-pass", "Your BrowserMob proxy Password");
3 - Include "browserstack.local" capability in your test script.
"browserstack.local" = true
How it works:
BrowserStack, by default, will resolve all the public URLs from their network.
Using --force-local option will force the binary to resolve all the traffic (even public URLs) via your network and not from BrowserStack's network.
Adding --local-proxy-* options will let the binary know that the traffic needs to be routed via your local proxy as well.
Now your local BrowserMob can capture all the traffic in HAR.
I see 2 solutions for this problem
BrowserMobProxy - there are 2 ways to run it: 1. from your code(adding library) 2. standalone proxy(controlled by REST API). In both cases you need to provide proxy to webdriver and control your proxy. One more improtant thing here to understand is that you need to redirect all the traffic from browsermob through the machine where proxy is located, please refer to this article for browserstack local execution. As I understand problem describes the case when proxy is being created on one machine and browserstack is simply not able to reach it.
Using browsermob you can get ALL required information: like request, params, response code, response time, etc. And even wait for requests to finish.
Examine performance logs. There is an option for ChromeDriver to capture performance logs. This option is easier since you don't need to care about proxy. However there are certain limitations of this approach as well: You won't be able to get request statistics like response time and response code and maybe some other that one may require. It would allow you to get only basic info like: request type, request url and of course you can parse params from url.
Related
I am trying to automate test to webrtc application and I'm trying to do it with multiple users. I created a setUp as below.
`
ArrayList<String> prefs = new ArrayList<String>();
prefs.add("--use-fake-device-for-media-stream");
prefs.add("--use-fake-ui-for-media-stream");
System.setProperty("webdriver.chrome.driver", "C:\\....\\resources\\chromedriver.exe");
ChromeOptions options = new ChromeOptions();
options.addArguments(prefs);
driver = new ChromeDriver(options);
driver.get("https://......");`
but when I use "--use-fake-ui-for-media-stream", the following remote address appears in the logs of the media server of the app.(I used this to disable the security popup for camera and mic.)
remote address looks like: 79beeb9e-ff01-4e69-906c-5be9cab979e6
when I don't use it, the remote address looks like this: 172.17.x.x
Therefore, I cannot connect to the meeting room, the server refuses the remote address.
When I remove "--use-fake-ui-for-media-stream" and put "--user-data-dir=C:\Users....\Local\Temp\...", I overcome this problem, but this time I can only connect to a single chromedriver on the Jmeter, the other chromedrivers are not working. I integrated testcases to Jmeter with Junit request.
I want to use this code for multiple users but I only could do it for a user or I could not connected.
How can i overcome this problem?
When first ChromeDriver instance is launched the profile directory gets locked and it cannot be re-used by 2nd, 3rd, etc. instance.
You can do something like:
prefs.add("--profile-directory=User" + org.apache.jmeter.threads.JMeterContextService.getContext().getThreadNum());
so each instance will have it's own separate profile folder.
More information:
open multiple chrome profile with selenium
JMeterContextService JavaDoc
How to Use JUnit With JMeter
In my Selenium Tests I need to test a webpage where I use a basic Authen,
Knowing that I am using Chrome Headless Java and Selenium WebDriver.
On my machine 'locally' It works perfectly using driver.get("https://admin:admin#localhost..");
and then
driver.get("https://localhost..") for example.
I know that Chrome doesn't support this function anymore but I managed to make it work based on someone's solution here by passing the first URL with credentials and the second without.
But when I run it on remote (Jenkins) On Linux servers I get the following Error
the configuration of your browser does not accept cookies
. I don't have vision on the servers when I can configure Chrome ..any ideas how to make it work without facing that problem.
I know a lot of people asked that question before, But I didn't find any valide answer for my issue.
try ChromeDriver 2.45 (changelog) or change the location, where it is supposed to save the cookies:
ChromeOptions options = new ChromeOptions();
options.addArguments("user-data-dir=/path/to/your/custom/profile");
otherwise (per default) it would create a new temporary directory each time it starts a session.
ChromeOptions options = new ChromeOptions();
//Command line flag for enabling account consistency. The default mode is disabled.
options.addArguments("--account-consistency");
//Chrome that will start logging to a file from startup
options.addArguments("--log-net-log=C:/some_path/resource/log.json");
//Sets the granularity of events to capture in the network log.
options.addArguments("--net-log-capture-mode=IncludeCookiesAndCredentials");
Try this, basically, it saves the logs on startup of chrome browser, then it will set the account consistency. Anywhere from the log, you can debug the issue.
Hello I managed to fix the problem (I forgot to mention that our website is protected by Siteminder) so I did the following following:
1-We inject the USER and the PASSWORD on the URL :
The issue we faced was that the displayed prompt wasn’t part of the page’s HTML and it was hard for us to catch it using Selenium. We managed this by directly injecting the user login and the user password in the URL as follow :
‘https://USERNAME:PASSWORD#basicAuthentURL’
This will launch the Chrome session. Beware, this is only the first step of the process. The user identification have not been performed yet.
2- We create a new cookie :
After launching the URL, we have to manually create a cookie called « SMCHALLENGE » and add it to current session with Selenium, for example in JAVA :
new Cookie("SMCHALLENGE", "YES");
3- Call the URL without the user credencials :
As the SMCHALLENGE cookie is now set, the last step is to call the URL again (https://basicAuthentURL ). The SMCHALLENGE cookie will be deleted once the authentication succeed and a SMSESSION cookie will be generated by Siteminder.
The SMSESSION cookie now allows us to call the application and sucessfully pass Siteminder as if normally logged in (via SSO).
Let me know if you try this out.
I need to run Jmeter programmatically using Java from behind a proxy.
The problem lies in the fact I need to do it using HTTPS .
I have read the manual at: http://jmeter.apache.org/usermanual/get-started.html
I have using Jmeter for a few months now and feel comfortable with it but the problem started when I needed to switch to HTTPS.
I have tried the following: (both separately and all together)
Added a HTTP Request Defaults configuration with the proxy server details
Added the proxy server details to each HTTP Request
Added both https.proxyHost & https.proxyPort with the proxy server details to the system.properties file found at ...\apache-jmeter-3.1\bin
I am aware of the fact that I can run JMETER using the command line with -H -P as parameters (That works) but that isn't how I work with JMeter - I only use it programmatically therefore this is not an option.
This is a snippet describing a known JAVA bug related to my problem:
The Java HTTP implementation has some limitations:
There is no control over how connections are re-used. When a connection is >released by JMeter, it may or may not be re-used by the same thread.
The API is best suited to single-threaded usage - various settings are defined >via system properties, and therefore apply to all connections.
There is a bug in the handling of HTTPS via a Proxy (the CONNECT is not handled >correctly). See Java bugs 6226610 and 6208335.
It does not support virtual hosts.
It supports only the following methods: GET, POST, HEAD, OPTIONS, PUT, DELETE and >TRACE
It does not support client based certificate testing with Keystore Config.
Bug link: http://bugs.java.com/view_bug.do?bug_id=6226610
I've read the bug and saw that "Java SE Development Kit 8u131" would solve this problem - so I downloaded it and alas it didn't help at all.
I would appreciate any help.
Thank,
Yigal
Given you run JMeter from a separate Java program you need to pass proxy arguments to this Java program. There are several ways on how this could be done, check our Java Networking and Proxies article for comprehensive information, you seem to be a fan of doing everything through the code so just add the following lines before launching a JMeter test:
System.setProperty("http.proxyHost","your_proxy_host");
System.setProperty("http.proxyPort", "your_proxy_port");
System.setProperty("https.proxyHost","your_proxy_host");
System.setProperty("https.proxyPort","your_proxy_port");
System.setProperty("http.nonProxyHosts","");
Just in case complete code demonstrating how you can run an existing JMeter test using JMeter API through JMeter's HTTP(S) Test Script Recorder as a proxy
import org.apache.jmeter.engine.StandardJMeterEngine;
import org.apache.jmeter.reporters.ResultCollector;
import org.apache.jmeter.reporters.Summariser;
import org.apache.jmeter.save.SaveService;
import org.apache.jmeter.util.JMeterUtils;
import org.apache.jorphan.collections.HashTree;
import java.io.File;
public class JMeterFromCode {
public static void main(String[] argv) throws Exception {
//Define JVM Proxy Settings
System.setProperty("http.proxyHost", "localhost");
System.setProperty("http.proxyPort", "8888");
System.setProperty("https.proxyHost", "localhost");
System.setProperty("https.proxyPort", "8888");
System.setProperty("http.nonProxyHosts", "");
// JMeter Engine
StandardJMeterEngine jmeter = new StandardJMeterEngine();
// Initialize Properties, logging, locale, etc.
JMeterUtils.loadJMeterProperties("/tmp/jmeter/bin/jmeter.properties");
JMeterUtils.setJMeterHome("/tmp/jmeter");
JMeterUtils.initLocale();
// Initialize JMeter SaveService
SaveService.loadProperties();
// Load existing .jmx Test Plan
HashTree testPlanTree = SaveService.loadTree(new File("/tmp/jmeter/bin/test.jmx"));
Summariser summer = null;
String summariserName = JMeterUtils.getPropDefault("summariser.name", "summary");
if (summariserName.length() > 0) {
summer = new Summariser(summariserName);
}
ResultCollector logger = new ResultCollector(summer);
logger.setFilename("/tmp/jmeter/test.jtl");
testPlanTree.add(testPlanTree.getArray()[0], logger);
// Run JMeter Test
jmeter.configure(testPlanTree);
jmeter.run();
}
}
I am looking for a way to programmatically change Windows Internet Options (proxy settings more specific). I saw in C# there is a class called InternetSetOption which I believe will do want I need. I was wondering is there a Java equivalent?
If not is there anyway I can in Java change Windows network proxy settings INSTANTLY because changing registry would work but you need to reboot or restart explorer.exe which aren't options for this application.
I think I found what you are looking for here.
Here is some example code:
//Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setPropery("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.setProperty("http.proxyHost", null);
// From now on http connections will be done directly.
Now,this works reasonably well, even if a bit cumbersome, but it can get tricky if your application is multi-threaded. Remember, system properties are “VM wide” settings, so all threads are affected. Which means that the code in one thread could, as a side effect, render the code in an other thread inoperative.
EDIT:
Here is some more specific examples:
Let's look at a few examples assuming we're trying to execute the main method of the GetURL class:
$ java -Dhttp.proxyHost=webcache.mydomain.com GetURL
All http connections will go through the proxy server at webcache.mydomain.com listening on port 80 (we didn't specify any port, so the default one is used).
$ java -Dhttp.proxyHost=webcache.mydomain.com -Dhttp.proxyPort=8080
-Dhttp.noProxyHosts=”localhost|host.mydomain.com” GetURL
In that second example, the proxy server will still be at webcache.mydomain.com, but this time listening on port 8080. Also, the proxy won't be used when connecting to either localhost or host.mydonain.com.
EDIT 2:
Perhaps something along these lines then:
System.setProperty( "http.proxyHost", "webcache.mydomain.com" );
System.setProperty( "http.proxyPort", "8080" );
System.setProperty( "https.proxyHost", "webcache.mydomain.com" );
System.setProperty( "https.proxyPort", "8080" );
So i have a java app that uses Google Analytics API to gather some info. I am putting this application to run in my oracle cloud managed server which has a firewall and blocks any web calls to work. So, they setup a proxy for me to use....I've never set up a proxy to work with a java application before, I've been reading at tutorials like this: http://docs.oracle.com/javase/1.5.0/docs/guide/net/proxies.html
And i have no idea how to set this up...anyone want to point me in the right direction?
You must tell your application that there's a proxy somewhere.
As the documentation says, you must set some properties in your virtual machine. You can do it programatically:
//Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setPropery("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.clearProperty("http.proxyHost");
// From now on http connections will be done directly.
Or use https.proxy... if the connection is HTTPS.
Besides, if you have access to the application server start script, you could add those properties as VM properties with -Dhttp.proxyHost....
The solution in my case was to configure the JVM with a HTTPS proxy:
System.setProperty("https.proxyHost", "proxy");
System.setProperty("https.proxyPort", "3128");