Network environment:
Https Client<=============>Proxy Server<==============>Https Server
192.168.17.11<-----extranet------>192.168.17.22
10.100.21.10<----intranet----->10.100.21.11
ps: Http Client without default gateway, but it can ping to 10.100.21.11
Description:
OS: Ubuntu 12.04 on 3 hosts
Https Client: Implement with java(openjdk-6).Have one network-interface.
Proxy Server: Apache2.2.Have two network-interfaces.
Https Server: Tomcat6.Have one network-interface.
I use two method to implement httpsurlconnection through proxy:
(For facilitate I do not write down about ssl handle function for checking serverTrusted and hostnameVerifier issue.If need I will update.)
1.Proxy class
InetSocketAddress proxyInet = new InetSocketAddress("10.100.21.11",80);
Proxy proxy = new Proxy(Proxy.Type.HTTP, proxyInet);
URL httpsUrl = new URL("https://192.168.17.22:8443/test");
HttpsURLConnection httpsCon = (HttpsURLConnection) httpsUrl.openConnection(proxy);
httpsCon.setDoOutput(true);
httpsCon.setDoInput(true);
httpsCon.setRequestMethod("POST");
OutputStream out = httpsCon.getOutputStream();
OutputStreamWriter owriter = new OutputStreamWriter(out);
owriter.write("<request>test</request>");
owriter.flush();
owriter.close();
...
This method workable and I observed packets flow also met my expectation.
HttpClient ---> ProxyServer ---> HttpServer
But when I use set Property method:
2.setProperty
System.setProperty("http.proxySet", "true");
System.setProperty("http.proxyHost",10.100.21.11);
System.setProperty("http.proxyPort","80");
URL httpsUrl = new URL("https://192.168.17.22:8443/test");
HttpsURLConnection httpsCon = (HttpsURLConnection)httpsUrl.openConnection();
httpsCon.setDoOutput(true);
httpsCon.setDoInput(true);
httpsCon.setRequestMethod("POST");
OutputStream out = httpsCon.getOutputStream();
OutputStreamWriter owriter = new OutputStreamWriter(out);
owriter.write("<request>test</request>");
owriter.flush();
owriter.close();
...
I got a NoRouteToHostException: Network is unreachable.
It make me confused.I did not see any packets between HttpClient and ProxyServer.
But HttpClient can ping to ProxyServer(10.100.12.10 ping 10.100.21.11)
So I remove proxy setting(as without using proxy):
Also got NoRouteToHostException: Network is unreachable.
I thought this is reasonable.Because there is no route to extranet.
I guess it seems like to setProperty method that the inner function of httpsUrlConnection will to check this url can be reachable or not.
But it is weird. 1st method can be success.
Have any idea? Or what are different between 1st and 2nd method?
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Update
System.setProperty("https.proxyHost",10.100.21.11);
System.setProperty("https.proxyPort","80");
It can work and packets flow are correct what I expect for.
But set https.proxyPort=443 is not workable for me
System.setProperty("https.proxyPort","443");
It will thorow a exception as bellow:
java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:770)
....
So I thought Apache Proxy have also to be modified to the right configuration.
Your URL connection is https whereas you are only setting the http proxy.
Try setting the https proxy.
//System.setProperty("https.proxySet", "true");
System.setProperty("https.proxyHost",10.100.21.11);
System.setProperty("https.proxyPort","443");
EDIT
#EJP is correct. There is no https.proxySet .. I copied your original question and included in the answer.
You will need to create a Proxy object for it. Create one as below:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(proxyServer, Integer.parseInt(proxyPort)));
Now use this proxy to create the HttpURLConnection object.
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection(proxy);
If you have to set the credentials for the proxy, set the Proxy-Authorization request property:
String uname_pwd = proxyUsername + ":" + proxyPassword
String authString = "Basic " + new sun.misc.BASE64Encoder().encode(uname_pwd.getBytes())
connection.setRequestProperty("Proxy-Authorization", authString);
And finally, you connect:
connection.connect();
thank you #divinedragon!
Same code on kotlin:
fun testProxy(login: String, pass: String, proxyData: ProxyData): String {
val url = URL("http://api.ipify.org")
val proxy = Proxy(Proxy.Type.HTTP, InetSocketAddress(proxyData.ip, proxyData.port))
val connection = url.openConnection(proxy) as HttpURLConnection
val loginPass = "$login:$pass"
val encodedLoginPass = Base64.getEncoder().encodeToString(loginPass.toByteArray())
val authString = "Basic $encodedLoginPass"
connection.setRequestProperty("Proxy-Authorization", authString);
with(connection) {
requestMethod = "GET" // optional default is GET
connectTimeout = 2000
readTimeout = 2000
return inputStream.bufferedReader().readText()
}
}
Related
I am using Restful Services on server and trying to maintain the same session after performing login:
URL endpoint = new URL(getString(R.string.url_login));
// Create connection
HttpURLConnection myConnection = (HttpURLConnection) endpoint.openConnection();
myConnection.setRequestMethod("POST");
myConnection.setRequestProperty("User-Agent", "my-rest-app-v0.1");
// Create the data
String myData = "username=" + username + "&password=" + pwd;
// Enable data writing
myConnection.setDoOutput(true);
// Write the data
myConnection.getOutputStream().write(myData.getBytes());
rc = myConnection.getResponseCode();
if (rc == 200) {
String Cookie= myConnection.getHeaderField("Set-Cookie");
URL endpoint2 = new URL(getString(R.string.url_checkUserType));
HttpURLConnection myConnection2 =
(HttpURLConnection)endpoint2.openConnection();
myConnection2.setRequestMethod("GET");
myConnection2.setRequestProperty("User-Agent", "my-rest-app-v0.1");
myConnection2.setRequestProperty("Cookie", Cookie);
rc2 = myConnection2.getResponseCode();
....
}....
The problem is that rc2 is every time 401 as WebService doesn't recognize that requiest is part of the same session. What am I doing wrong?
You may do as follows
on the server side, it checks if the incoming request has a "sessionId" in cookie: if not, create one and return it in response; if yes, it knows the request belongs to a known session
on the client side, after a successful login, retrieve the "sessionId" from the response, and set it in the following requests
==
connection.setRequestProperty("Cookie", "JSESSIONID=" + URLEncoder.encode(jSessionId, "UTF-8"));
The solution was to use Spring RESTful services for Android that worked great for me.
I have a remote server that use basic authentication (HTTP) and i need several calls to different URLs. A example of calling:
URL address = new URL ("hhtp://localhost:8080");
HttpURLConnection connection;
// open connection
connection = (HttpURLConnection) address.openConnection();
String encoding = DatatypeConverter.printBase64Binary((username + ":" + password).getBytes());
connection.setRequestMethod("POST");
connection.setDoOutput(true);
connection.setRequestProperty("Authorization", "Basic " + encoding);
// transform result code into an exception
int code = connection.getResponseCode();
if ((code < 200) || (code >= 300))
{
throw new HttpRetryException(connection.getResponseMessage(), code, address.toString());
}
return connection;
The problem appear that the server take a lot of time authenticating the user. After this first call, we need another page (a different URL) to the same server. How I can keep the session between different calls?
Edit:
A clarification, I'm trying to connect to a Jenkins server.
I was trying to hit the https url for google api.
Using the code below but its giving some errors.
but i can hitting one google api http url and its working very well without any error
DataInputStream di = null;
FileOutputStream fo = null;
byte[] b = new byte[1];
// PROXY
System.setProperty("https.proxyHost", "my proxy");
System.setProperty("https.proxyPort", "8080");
Authenticator.setDefault(new PasswordAuthenticator());
URL u = new URL("https://maps.googleapis.com/maps/api/place/autocomplete/json?input=Vict&types=geocode&language=fr&sensor=true&key=keyforuse");
HttpURLConnection con = (HttpURLConnection) u.openConnection();
sun.misc.BASE64Encoder encoder = new sun.misc.BASE64Encoder();
String encodedUserPwd = encoder.encode("domain\\user:password"
.getBytes());
con.setRequestProperty("Proxy-Authorization", "Basic "
+ encodedUserPwd);
di = new DataInputStream(con.getInputStream());
// result = parseJSON(di);
while (-1 != di.read(b, 0, 1)) {
System.out.print(new String(b));
}
but getting error below
java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authorization Required"
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:1648)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:164)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1172)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:234)
at kf.store.locator.googlemaps.test.main(test.java:58)
any help????
You may not need the domain in front of the username. I had a very similar issue, and I fixed it when I removed the domain in front of the username. Since the proxy server is on a domain, I think it implies that your under the same domain.
I fixed it by authenticating to the proxy server with my user name and password. If you are on your company's network , it can be your sso cred too.
The equivalent of this will be like this :
-Dhttp.proxyUser=myusername
-Dhttp.proxyPassword=mypassword
-Dhttp.proxyHost=myproxyserver.com
-Dhttp.proxyPort=9000
I have some working java code which does the following:
URL myUrl = new URL("http://localhost:8080/webservice?user=" + username + "&password=" + password + "&request=x");
HttpURLConnection myConnection = (HttpURLConnection) myUrl.openConnection();
myConnection.setRequestMethod("POST");
// code continues to read the response stream
However, I noticed that my webserver access log contained the plaintext password for all of the users who connected. I would like to get this out of the access log, but the webserver admins claim that this needs to be changed in my code and not via webserver config.
I tried changing the code to the following:
URL myUrl = new URL("http://localhost:8080/webservice");
HttpURLConnection myConnection = (HttpURLConnection) myUrl.openConnection();
myConnection.setRequestMethod("POST");
// start of new code
myConnection.setDoOutput(true);
myConnection.addRequestProperty("username", username);
myConnection.addRequestProperty("password", password);
myConnection.addRequestProperty("request", "x");
// code continues to read the response stream
Now the access log does not contain the username/password/request method. However, the webservice now throws an exception indicating that it didn't receive any username/password.
What did I do wrong in my client code? I also tried using "setRequestProperty" instead of "addRequestProperty" and it had the same broken behavior.
I actually found the answer in another question on stackoverflow.
The correct code should be:
URL myUrl = new URL("http://localhost:8080/webservice");
HttpURLConnection myConnection = (HttpURLConnection) myUrl.openConnection();
myConnection.setRequestMethod("POST");
myConnection.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(myConnection.getOutputStream ());
wr.writeBytes("username=" + username + "&password="+password + "&request=x");
// code continues to read the response stream
My Tomcat instance is listening to multiple IP addresses, but I want to control which source IP address is used when opening a URLConnection.
How can I specify this?
This should do the trick:
URL url = new URL(yourUrlHere);
Proxy proxy = new Proxy(Proxy.Type.DIRECT,
new InetSocketAddress(
InetAddress.getByAddress(
new byte[]{your, ip, interface, here}), yourTcpPortHere));
URLConnection conn = url.openConnection(proxy);
And you are done.
Dont forget to handle exceptions nicely and off course change the values to suit your scenario.
Ah and I omitted the import statements
Using the Apache commons HttpClient I have also found the following to work (removed try/catch for clarity):
HostConfiguration hostConfiguration = new HostConfiguration();
byte b[] = new byte[4];
b[0] = new Integer(192).byteValue();
b[1] = new Integer(168).byteValue();
b[2] = new Integer(1).byteValue();
b[3] = new Integer(11).byteValue();
hostConfiguration.setLocalAddress(InetAddress.getByAddress(b));
HttpClient client = new HttpClient();
client.setHostConfiguration(hostConfiguration);
GetMethod method = new GetMethod("http://remoteserver/");
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
System.err.println("Method failed: " + method.getStatusLine());
}
byte[] responseBody = method.getResponseBody();
System.out.println(new String(responseBody));");
However, I still wonder what would happen if the gateway of the IP is down (192.168.1.11 in this case). Will the next gateway be tried or will it fail?
The obvious portable way would be to set a Proxy in URL.openConnection. The proxy can be in local host, you can then write a very simple proxy that binds the local address of the client socket.
If you can't modify the source where the URL is connected, you can replace the URLStreamHandler either when calling the URL constructor or globally through URL.setURLStreamHandlerFactory. The URLStreamHandler can then delegate to the default http/https handler, modifying the openConnection call.
A more extreme method would be to completely replace the handler (perhaps extending the implementation in your JRE). Alternatively, alternative (open source) http clients are available.
Setting manually socket work fine ...
private HttpsURLConnection openConnection(URL src, URL dest, SSLContext sslContext)
throws IOException, ProtocolException {
HttpsURLConnection connection = (HttpsURLConnection) dest.openConnection();
HttpsHostNameVerifier httpsHostNameVerifier = new HttpsHostNameVerifier();
connection.setHostnameVerifier(httpsHostNameVerifier);
connection.setConnectTimeout(CONNECT_TIMEOUT);
connection.setReadTimeout(READ_TIMEOUT);
connection.setRequestMethod(POST_METHOD);
connection.setRequestProperty(CONTENT_TYPE, SoapConstants.CONTENT_TYPE_HEADER);
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setSSLSocketFactory(sslContext.getSocketFactory());
if ( src!=null ) {
InetAddress inetAddress = InetAddress.getByName(src.getHost());
int destPort = dest.getPort();
if ( destPort <=0 )
destPort=SERVER_HTTPS_PORT;
int srcPort = src.getPort();
if ( srcPort <=0 )
srcPort=CLIENT_HTTPS_PORT;
connectionSocket = connection.getSSLSocketFactory().createSocket(dest.getHost(), destPort, inetAddress, srcPort);
}
connection.connect();
return connection;
}