I am looking for a way to programmatically change Windows Internet Options (proxy settings more specific). I saw in C# there is a class called InternetSetOption which I believe will do want I need. I was wondering is there a Java equivalent?
If not is there anyway I can in Java change Windows network proxy settings INSTANTLY because changing registry would work but you need to reboot or restart explorer.exe which aren't options for this application.
I think I found what you are looking for here.
Here is some example code:
//Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setPropery("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.setProperty("http.proxyHost", null);
// From now on http connections will be done directly.
Now,this works reasonably well, even if a bit cumbersome, but it can get tricky if your application is multi-threaded. Remember, system properties are “VM wide” settings, so all threads are affected. Which means that the code in one thread could, as a side effect, render the code in an other thread inoperative.
EDIT:
Here is some more specific examples:
Let's look at a few examples assuming we're trying to execute the main method of the GetURL class:
$ java -Dhttp.proxyHost=webcache.mydomain.com GetURL
All http connections will go through the proxy server at webcache.mydomain.com listening on port 80 (we didn't specify any port, so the default one is used).
$ java -Dhttp.proxyHost=webcache.mydomain.com -Dhttp.proxyPort=8080
-Dhttp.noProxyHosts=”localhost|host.mydomain.com” GetURL
In that second example, the proxy server will still be at webcache.mydomain.com, but this time listening on port 8080. Also, the proxy won't be used when connecting to either localhost or host.mydonain.com.
EDIT 2:
Perhaps something along these lines then:
System.setProperty( "http.proxyHost", "webcache.mydomain.com" );
System.setProperty( "http.proxyPort", "8080" );
System.setProperty( "https.proxyHost", "webcache.mydomain.com" );
System.setProperty( "https.proxyPort", "8080" );
Related
i checked this page and got some usefull code for using a proxy in java code when connecting to a webpage.
I can confirm that pages like whatsmyip do indeed tell me that proxy is working - it is showing proxy ip. The problem is that the page i am accesing to in java code, somehow detects my true ip and blocks content. I do know how it does that (header, return ip, etc.), what i do not know is how to bypass that.
Maybe another interesting thing is that this page works with no problems using 1 of the best known online proxy sites - it shows content. Now what is even more interesting is that i tried taking that sites ip and used it as proxy in my program, but there it didn't work - true ip got detected, which is really strange.
edit: This is my new code:
System.setProperty("java.net.useSystemProxies","false");
System.setProperty("http.proxyHost", "94.230.208.147");
System.setProperty("http.proxyPort", "9001");
System.setProperty("http.nonProxyHosts", "localhost|127.0.0.1");
System.setProperty("https.proxyHost", "94.230.208.147");
System.setProperty("https.proxyPort", "9001");
System.setProperty("https.nonProxyHosts", "localhost|127.0.0.1");
I can confirm that https://whatsmyip.com/ isn't fooled by this proxy and can see my true ip. What did i forget to include ?
Add this at the end of your code:
System.setProperty("http.nonProxyHosts", "localhost|127.0.0.1");
That indicates the hosts that should be accessed without going through the proxy. Typically this defines internal hosts. The value of this property is a list of hosts, separated by the '|' character.
I use BrowserStack with Selenium-webdriver to run tests on different types of devices and browsers. So actually tests are running by RemoteWebDriver.
I know that it's possible to capture network within Selenium tests using BrowserMobProxy, but as i understand it's working only if test is running on local machine.
Is there a way to capture network while running test on cross-platform base like BrowserStack?
UPDATE
I managed to get capture of network in har file (from link "localhost:8080/proxy/8081/har"), using standalone BrowserMobProxy and standalone local BrowserStack, as I was advised.
I tried to do the same automatically from code:
BrowserMobProxy proxy = new BrowserMobProxyServer();
proxy.start(8080);
System.out.println("Proxy port: " + port);
System.setProperty("java.net.useSystemProxies", "true");
System.setProperty("http.proxyHost", "127.0.0.1");
System.setProperty("http.proxyPort", "8080");
System.setProperty("https.proxyHost", "127.0.0.1");
System.setProperty("https.proxyPort", "8080");
Local l = new Local();
Map<String, String> options = new HashMap<String, String>();
options.put("key", accessKey);
options.put("forcelocal", "true");`
//when I uncomment it i get an exception:
//com.browserstack.local.LocalException: Could not connect to www.browserstack.com!
// options.put("forceproxy", "true");
// options.put("proxyHost", "localhost");
// options.put("proxyPort", "8080");
l.start(options);
}
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
proxy.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);
driver = new RemoteWebDriver(new URL("http://"+username+":"+accessKey+"#"+config.get("server")+"/wd/hub"), capabilities);'
proxy.newHar("testHar.com");
driver.get(testUrl);
Thread.sleep(15000);
Har har = proxy.getHar();
FileOutputStream fos = new FileOutputStream("C:\\LoadingPage\\network\\testHar.har");
har.writeTo(fos);
The connection to the url is working, I could see it and make screenshouts.
BUT! In the har file I see only request to "hub-cloud.browserstack.com/wd/hub/...", not the requests from page itself.
How to get correct har from code?
What in the code is not correct?
From my experience I would like to add small modification to the binary command given in BrowserStack link (Shared by Mikhail). The cmd given in doc should work well for private URLs but may not work for public ones.
Steps for Standalone binary:
1 - Download the BrowserStackLocal binary from 'https://www.browserstack.com/local-testing#command-line'.
Launch the binary by running the below command to enable your proxy to monitor the traffic.
- BrowserStackLocal.exe --key
--local-proxy-host --local-proxy-port --local-proxy-user --local-proxy-pass --force-proxy --force-local
More details on all the modifiers are available at 'https://www.browserstack.com/local-testing#modifiers'.
2 - Include "browserstack.local" capability in your test script.
"browserstack.local" = true
Steps for Java (BrowserStack Local) bindings:
1 - Follow these steps for using local bindings.
2 - Using this you can use newer options available in latest versions of the binary. For instance if you wish to add --local-proxy-* options, for which there is no existing wrapper (like this which is internally mapped to this), try using below:
bsLocalArgs.put("-local-proxy-host", "Your BrowserMob proxy IP");
bsLocalArgs.put("-local-proxy-port", "Your BrowserMob proxy Port");
bsLocalArgs.put("-local-proxy-user", "Your BrowserMob proxy Username");
bsLocalArgs.put("-local-proxy-pass", "Your BrowserMob proxy Password");
3 - Include "browserstack.local" capability in your test script.
"browserstack.local" = true
How it works:
BrowserStack, by default, will resolve all the public URLs from their network.
Using --force-local option will force the binary to resolve all the traffic (even public URLs) via your network and not from BrowserStack's network.
Adding --local-proxy-* options will let the binary know that the traffic needs to be routed via your local proxy as well.
Now your local BrowserMob can capture all the traffic in HAR.
I see 2 solutions for this problem
BrowserMobProxy - there are 2 ways to run it: 1. from your code(adding library) 2. standalone proxy(controlled by REST API). In both cases you need to provide proxy to webdriver and control your proxy. One more improtant thing here to understand is that you need to redirect all the traffic from browsermob through the machine where proxy is located, please refer to this article for browserstack local execution. As I understand problem describes the case when proxy is being created on one machine and browserstack is simply not able to reach it.
Using browsermob you can get ALL required information: like request, params, response code, response time, etc. And even wait for requests to finish.
Examine performance logs. There is an option for ChromeDriver to capture performance logs. This option is easier since you don't need to care about proxy. However there are certain limitations of this approach as well: You won't be able to get request statistics like response time and response code and maybe some other that one may require. It would allow you to get only basic info like: request type, request url and of course you can parse params from url.
So i have a java app that uses Google Analytics API to gather some info. I am putting this application to run in my oracle cloud managed server which has a firewall and blocks any web calls to work. So, they setup a proxy for me to use....I've never set up a proxy to work with a java application before, I've been reading at tutorials like this: http://docs.oracle.com/javase/1.5.0/docs/guide/net/proxies.html
And i have no idea how to set this up...anyone want to point me in the right direction?
You must tell your application that there's a proxy somewhere.
As the documentation says, you must set some properties in your virtual machine. You can do it programatically:
//Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setPropery("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.clearProperty("http.proxyHost");
// From now on http connections will be done directly.
Or use https.proxy... if the connection is HTTPS.
Besides, if you have access to the application server start script, you could add those properties as VM properties with -Dhttp.proxyHost....
The solution in my case was to configure the JVM with a HTTPS proxy:
System.setProperty("https.proxyHost", "proxy");
System.setProperty("https.proxyPort", "3128");
I've had to update a previous java application that requests a SOAP response from an external web service. This service is outside our firewall which now requires us to go through a proxy instead of hitting the URL directly.
Currently the Java App uses URLEndpoint that takes a string for the URL. Usually when I am getting to a URL through a proxy I create a URL like so:
URL url = new URL("http", "theproxy.com", 5555, finalUrl);
The problem is URLEndpoint only takes a string for the url, I tried to convert URL to string using toExternalForm() but it malformed the URL.
Any ideas as to a way around this?
EDIT: I can't use System.setProperty as this runs with a whole heap of other Java applications in tomcat.
second edit: I can't set a system properties as it will override all other applications running on the server, I can't use jsocks as the proxy we run through squid proxy which does not support socks4/5
Any help appreciated.
That's not how proxy's work. The way a proxy works is that you take your normal URL:
http://example.com/service
and instead of looking up "example.com" and port 80, the message is sent to your proxy host instead (http://theproxy.com:5555).
Java has built in proxy handling using http.proxyHost and http.proxyPort System properties.
So in your case you would need to do:
System.setProperty("http.proxyHost", "theproxy.com");
System.setProperty("http.proxyPort", "5555");
Then your code should, ideally, "Just Work".
Here is a page documenting the proxy properties.
Use Apache HttpClient and do as show in this example.
About the URL constructor with individual proxy setting:
http://edn.embarcadero.com/article/29783
(sorry don't have privileges to comment)
I'm trying to find a way to force any connection attempts a Jar makes to an external IP through my proxy server, which is running on localhost(Also a Java application).
Once the proxy server receives the connection it will open a connection with the external IP and begin routing the IO to and from the client/server.
I've been Googling this for 2 days, and I haven't had any luck, I believe I'm using the wrong terms in my search attempts.
If you've got any ideas, please let me know, I'll try anything.
Thanks in advance. - Sean.
If is that a "real" Proxy the you could specify the proxy to use using java system properties.
You have two alternatives:
Specify the proxy in the command line
Hardcode it into your app
Well you actually have three
Specify a .properties file, and read from there, and set it as System property ( which is pretty much option 2 but more dynamic )
From command line you'll use:
java -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8080 -jar YourJar.jar
With that all the http connections you perform will go through localhost at port 8080
The second is add this at the main method of your program:
public static void main( String [] args ) {
System.setProperty("http.proxyHost", "localhost");
System.setProperty("http.proxyPort", "8080");
.....
}
Which does the same.
Finally loading from myapp.properties
public static void main( String [] args ) {
try { // there are cleaner ways of course
ResorceBundle bundle = ResourceBundle.getBundle("myapp");
System.setProperty("http.proxyHost", bundle.getString("proxy.server"));
System.setProperty("http.proxyPort", bundle.getString("proxy.port"));
} catch( MissingResourceException missingResourceException ){}
....
}
You just have to make sure myapp.properties is available from the classpath
More information about this functionality here
If you are asking about general (NOT HTTP / FTP specific!) proxying of Socket connections, then the simple answer is that it is not supported by Java.
When you configure a proxy for HTTP and FTP traffic, the proxying happens at the application protocol level. The Java-side proxy properties tell the URLConnection layer to connect to your designated proxy rather than the IP address from the URL your application is trying to connect to. The Java Socket level is unaware that this is happening. It just sees a requests to connect to the proxy.
This work because the HTTP and FTP protocols specifically support proxying. For instance, the first 'line' of an HTTP GET request message gives the full URL of the page that the client is requesting. If the GET request goes to a proxy, the proxy can figure out where is has to send it.
Looking at the problem of proxying at the Socket level, the first observation is that the standard Java class libraries don't support this. The second observation is that it is actually unimplementable ... unless you implement this as an alternative transport layer. The reason is that IP and TCP/IP simply do not support the notion of explicitly proxying or relaying messages / streams. And even if you did implement such a transport, it doesn't fit into the standard Socket model.
So, if you are really asking about proxying all of the network traffic for a Java application, this can only be implemented outside of the JVM; i.e. at the network transport level of the JVM's (physical or virtual) host operating system.
If it's HTTP traffic or FTP traffic, you could try the following system properties:
http.proxyHost (default: )
http.proxyPort (default: 80 if http.proxyHost specified)
http.nonProxyHosts (default:
See this link for details:
http://java.sun.com/docs/books/tutorial/networking/urls/_setProxy.html