I am new in HTMLunit and trying to set HTTPS proxy for HTMLunit.
I tried to use https:// just before the HOST IP, but I got Exception.
Anyone can help me to solve this issue?
Update: My Code is:
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_3_6,"https://199.127.100.13", 11888);
Update 2:
I asked the developer team, The said that it is a bug in the framework. They will fix it.
You should not be putting http:// or https:// behind the ip address of the proxy server.
If your http proxy server supports https then htmlunit would automatically use it. Here is an example of how to use proxy with htmlunit
For HTTP proxy
ProxyConfig pc = new ProxyConfig();
pc.setSocksProxy(false); //Set to false if it is a http server
pc.setProxyHost("192.168.1.200"); //your proxy IP
pc.setProxyPort(proxyPort);
webClient.getOptions().setProxyConfig(pc);
and of course if you are using socks proxy than set the setSocksProxy to true.
Related
I am new to Windows authentication and am facing a weird issue.
I have setup an application with SPNEGO filter library for Java.
All settings as per the documentations have been set.
Now when i open the URL of my application from another machine in the same domain, using any browser, i get a negotiation header as
TlRMTVNTUAABAAAAl4II...
This means that it is an NTLM negotiation request.
if i start fiddler and then try to run the same request for testing, i am getting a kerberos authentication request.
YIIGgwYGKwYBBQUCoIIGdzCCBnOgMDAuBg...
This means that when I am calling using fiddler, the browser is assuming that the system is on same network.
I am unable to figure out why this is happening..??
I need the kerberos ticket even in normal execution.
Server: JBoss 4.3.2 GA
anybody has any idea...??
thanks in advance
I am getting this error in chrome:
The page at 'https://www.SERVER_ONE.com/' was loaded over HTTPS, but
requested an insecure XMLHttpRequest endpoint
'http://SERVER_TWO.com/someAPI'. This request has been blocked; the
content must be served over HTTPS.
Both SERVER_ONE, and SERVER_TWO are owned by me.
But the problem is that the HTTPS certificate I hold is only for server_ONE.
Is there anything I can do to resolve this error, can I introduce some mode_proxy in SERVER_TWO to redirect all https to http, or is there any way in which I can write some proxy in java side and put it on server_one which can act as an adapter for https to http?
Please guide me with some snippet code if any such adapter code is possible.
You should not call directly SERVER_TWO, you should configure or implement a proxy on SERVER_ONE so that every call can be done over HTTPS.
Just enable https on server2 and change your call from http to https.
For the certificate you can use https://letsencrypt.org/ for server2 for free.
I am using the great Charles Web Proxy do make a reverse proxy and debug an exchange between my client and a secure server.I accepted the Charles certificate as a trusted authority and it works generally very well when using it in Proxy Mode, the problem is when I try to make a reverse proxy.
I am using an external library over which I have no control that uses an HttpUrlConnection and lets me specify the URL. Following the Charles documentation, I set my URL to https://mybox.test.com:100 (note that I used a hostname, added to my host file, instead of localhost in the hope that I can have a valid certificate).
The problem is that as part of the Https validation, HttpUrlConnection verifies the host name (mybox.test.com since it is pointing to my reverse proxy) against the one in the certificate (the one of the secure site I am connecting to). Obviously that does not match and causes a
Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative DNS name matching localhost found.
I realize that Charles cannot generate a certificate for mybox.test.com, since it only listens to a particular port and has no knowledge of the URL used in the HttpUrlConnection. I do not see how it could generate a correct certificate that would be accepted.
My question is: is there a better way to use a reverse proxy with Https or do I need to disable the host name and certificate validation ?
I was expecting this code to return a 404, however it produces the output :
"Response code is 200"
Would it be possible to learn how to differentiate between existent and non-existent web pages . . . thanks so much,
try
{
// create the HttpURLConnection
URL url = new URL("http://www.thisurldoesnotexist");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
System.out.println("Response code is " + connection.getResponseCode());
}
EDIT: I see you've call openConnection() but not connect() - could that be the problem? I would expect getResponseCode() to actually make the request if it hasn't already, but it's worth just trying that...
That suggests you've possible got some DNS resolver which redirects to a "helper" (spam) page, or something like that.
The easiest way to see exactly what's going on here is to use Wireshark - have that up and capturing traffic (HTTP-only, to make life easier) and then run your code. You should be able to see what's going on that way.
Note that I wouldn't have expected a 404 - because that would involve being able to find a web server to talk to to start with. If you're trying to go to a host which doesn't involve, there shouldn't be an HTTP response at all. I'd expect connect() to throw an exception.
try adding a "connection.connect();" or look at the contents returned...
it could be a dns issue, ie: your dns is being sent to a parking place... for example: freedns does this.
You could:
Resolve the IP from the host of the page
Try to connect to port 80 on the resolved IP using plain sockets
This is a bit low level however and will add complexity since you will need to make a simple GET request through the socket. Then validate the response so you're sure that its actually a HTTP server running on port 80.
NMap might be able to help you here.
Ideally you should be getting this error:
java.net.UnknownHostException: www.thisurldoesnotexist
But it looks like your URL is resolved by you DNS provider.
For instance on my company's network running your code with URI "http://profile/" displays
the employee profile.
Please also check etc.home file if you are on windows to check if any settings have been changed.
Like #spgennard - I think this is most likely a DNS issue.
The URL you have chosen is owned by a DNS speculator.
The URL you have chosen is "parked" by a DNS provider.
Your ISP is messing with your DNS results to send your browser to some search page.
It is also possible that you are accessing the web via a proxy, and the proxy is doing something strange.
The way to diagnose this is to look at the other information in the HTTP responses you are getting, particularly the response body.
I've had to update a previous java application that requests a SOAP response from an external web service. This service is outside our firewall which now requires us to go through a proxy instead of hitting the URL directly.
Currently the Java App uses URLEndpoint that takes a string for the URL. Usually when I am getting to a URL through a proxy I create a URL like so:
URL url = new URL("http", "theproxy.com", 5555, finalUrl);
The problem is URLEndpoint only takes a string for the url, I tried to convert URL to string using toExternalForm() but it malformed the URL.
Any ideas as to a way around this?
EDIT: I can't use System.setProperty as this runs with a whole heap of other Java applications in tomcat.
second edit: I can't set a system properties as it will override all other applications running on the server, I can't use jsocks as the proxy we run through squid proxy which does not support socks4/5
Any help appreciated.
That's not how proxy's work. The way a proxy works is that you take your normal URL:
http://example.com/service
and instead of looking up "example.com" and port 80, the message is sent to your proxy host instead (http://theproxy.com:5555).
Java has built in proxy handling using http.proxyHost and http.proxyPort System properties.
So in your case you would need to do:
System.setProperty("http.proxyHost", "theproxy.com");
System.setProperty("http.proxyPort", "5555");
Then your code should, ideally, "Just Work".
Here is a page documenting the proxy properties.
Use Apache HttpClient and do as show in this example.
About the URL constructor with individual proxy setting:
http://edn.embarcadero.com/article/29783
(sorry don't have privileges to comment)