My Tomcat instance is listening to multiple IP addresses, but I want to control which source IP address is used when opening a URLConnection.
How can I specify this?
This should do the trick:
URL url = new URL(yourUrlHere);
Proxy proxy = new Proxy(Proxy.Type.DIRECT,
new InetSocketAddress(
InetAddress.getByAddress(
new byte[]{your, ip, interface, here}), yourTcpPortHere));
URLConnection conn = url.openConnection(proxy);
And you are done.
Dont forget to handle exceptions nicely and off course change the values to suit your scenario.
Ah and I omitted the import statements
Using the Apache commons HttpClient I have also found the following to work (removed try/catch for clarity):
HostConfiguration hostConfiguration = new HostConfiguration();
byte b[] = new byte[4];
b[0] = new Integer(192).byteValue();
b[1] = new Integer(168).byteValue();
b[2] = new Integer(1).byteValue();
b[3] = new Integer(11).byteValue();
hostConfiguration.setLocalAddress(InetAddress.getByAddress(b));
HttpClient client = new HttpClient();
client.setHostConfiguration(hostConfiguration);
GetMethod method = new GetMethod("http://remoteserver/");
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
System.err.println("Method failed: " + method.getStatusLine());
}
byte[] responseBody = method.getResponseBody();
System.out.println(new String(responseBody));");
However, I still wonder what would happen if the gateway of the IP is down (192.168.1.11 in this case). Will the next gateway be tried or will it fail?
The obvious portable way would be to set a Proxy in URL.openConnection. The proxy can be in local host, you can then write a very simple proxy that binds the local address of the client socket.
If you can't modify the source where the URL is connected, you can replace the URLStreamHandler either when calling the URL constructor or globally through URL.setURLStreamHandlerFactory. The URLStreamHandler can then delegate to the default http/https handler, modifying the openConnection call.
A more extreme method would be to completely replace the handler (perhaps extending the implementation in your JRE). Alternatively, alternative (open source) http clients are available.
Setting manually socket work fine ...
private HttpsURLConnection openConnection(URL src, URL dest, SSLContext sslContext)
throws IOException, ProtocolException {
HttpsURLConnection connection = (HttpsURLConnection) dest.openConnection();
HttpsHostNameVerifier httpsHostNameVerifier = new HttpsHostNameVerifier();
connection.setHostnameVerifier(httpsHostNameVerifier);
connection.setConnectTimeout(CONNECT_TIMEOUT);
connection.setReadTimeout(READ_TIMEOUT);
connection.setRequestMethod(POST_METHOD);
connection.setRequestProperty(CONTENT_TYPE, SoapConstants.CONTENT_TYPE_HEADER);
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setSSLSocketFactory(sslContext.getSocketFactory());
if ( src!=null ) {
InetAddress inetAddress = InetAddress.getByName(src.getHost());
int destPort = dest.getPort();
if ( destPort <=0 )
destPort=SERVER_HTTPS_PORT;
int srcPort = src.getPort();
if ( srcPort <=0 )
srcPort=CLIENT_HTTPS_PORT;
connectionSocket = connection.getSSLSocketFactory().createSocket(dest.getHost(), destPort, inetAddress, srcPort);
}
connection.connect();
return connection;
}
Related
I'm trying to write a simple test where I submit a request to http://localhost:12345/%, knowing that this is an illegal URI, because I want to assert that my HTTP Server's error-handling code behaves correctly. However, I am having a hard time forcing Java to do this.
If I try to create a Java 11 HttpRequest with URI.create("localhost:12345/%"), I get a URISyntaxException, which is correct and not helpful.
Similarly, using a ws-rs WebTarget:
ClientBuilder.newBuilder().build().target("http://localhost:12345").path("/%")
builds me a WebTarget pointing to /%25, which would normally be very helpful, but is not what I want in this particular situation.
Is there a way to test my error-handling behavior without resorting to low-level bytestream manipulation?
Another possibility is just to use plain Socket - it's easy enough if you know the protocol (especially if using the new text-block feature). This will allow you to misformat the request in any way you like. Reading the response and analysing the result is - of course - a bit more involved:
String request = """
GET %s HTTP/1.1\r
Host: localhost:%s\r
Connection: close\r
\r
""".formatted("/%", port);
try (Socket client = new Socket("localhost", port);
OutputStream os = client.getOutputStream();
InputStream in = client.getInputStream()) {
os.write(request.getBytes(StandardCharsets.US_ASCII));
os.flush();
// This is optimistic: the server should close the
// connection since we asked for it, and we're hoping
// that the response will be in ASCII for the headers
// and UTF-8 for the body - and that it won't use
// chunk encoding.
byte[] bytes = in.readAllBytes();
String response = new String(bytes, StandardCharsets.UTF_8);
System.out.println("response: " + response);
}
Noah's comment lead me down the right path; I was able to do this with the URL class:
#Test
public void testUriMalformed() throws Exception {
final URL url = new URL(TEST_URI + "/%");
final HttpURLConnection connection = (HttpURLConnection)url.openConnection();
final int code = connection.getResponseCode();
final String contentType = connection.getHeaderField("Content-Type");
final String entity = IOUtils.toString(connection.getErrorStream(), Charsets.UTF_8);
assertEquals(500, code);
assertEquals(MediaType.APPLICATION_JSON, contentType);
assertTrue(entity.contains("error_id"));
}
I am trying to send an HTTP request from an APP Engine endpoint, from experiments on Postman I know the result is quite big, and the the request usually takes about a minute.
here is my Code:
void testRequest() {
String test = getConnectionString();
URL url = new URL(YARDI_URL);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setInstanceFollowRedirects(false);
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "text/xml");
connection.setConnectTimeout(1000000);
OutputStream os = connection.getOutputStream();
PrintWriter p = new PrintWriter(os);
p.print(test);
p.close();
YardiResponse response = new
YardiResponse(connection.getInputStream().toString());
System.out.println(response.getResponse());
connection.disconnect();
}
I am getting two errors,
the first is: java.net.ProtocolException: Cannot write output after reading input.
and after a long time I am getting a java.net.SocketException: Connection reset message.
Obviously I am mishandling the steams, and the way I send them.
I higly recomend http-request built on apache http api.
private static final HttpRequest<String.class> HTTP_REQUEST =
HttpRequestBuilder.createPost(YARDI_URL, String.class)
.responseDeserializer(ResponseDeserializer.ignorableDeserializer())
.contentTypeOfBody(ContentType.TEXT_XML)
.connectTimeout(someIntValue)
.socketTimeOut(someIntValue)
.connectionRequestTimeout(someIntValue).
.build();
void testRequest() {
ResponseHadler<String> yardiHandler = HTTP_REQUEST.executeWithBody(yourXml);
int statusCode = yardiHandler.getStatusCode();
String content = yardiHandler.get(); //returns response body as String in this case
}
Note: I recomend see javadocs of connectTimeout, socketTimeOut and connectionRequestTimeout methods.
Network environment:
Https Client<=============>Proxy Server<==============>Https Server
192.168.17.11<-----extranet------>192.168.17.22
10.100.21.10<----intranet----->10.100.21.11
ps: Http Client without default gateway, but it can ping to 10.100.21.11
Description:
OS: Ubuntu 12.04 on 3 hosts
Https Client: Implement with java(openjdk-6).Have one network-interface.
Proxy Server: Apache2.2.Have two network-interfaces.
Https Server: Tomcat6.Have one network-interface.
I use two method to implement httpsurlconnection through proxy:
(For facilitate I do not write down about ssl handle function for checking serverTrusted and hostnameVerifier issue.If need I will update.)
1.Proxy class
InetSocketAddress proxyInet = new InetSocketAddress("10.100.21.11",80);
Proxy proxy = new Proxy(Proxy.Type.HTTP, proxyInet);
URL httpsUrl = new URL("https://192.168.17.22:8443/test");
HttpsURLConnection httpsCon = (HttpsURLConnection) httpsUrl.openConnection(proxy);
httpsCon.setDoOutput(true);
httpsCon.setDoInput(true);
httpsCon.setRequestMethod("POST");
OutputStream out = httpsCon.getOutputStream();
OutputStreamWriter owriter = new OutputStreamWriter(out);
owriter.write("<request>test</request>");
owriter.flush();
owriter.close();
...
This method workable and I observed packets flow also met my expectation.
HttpClient ---> ProxyServer ---> HttpServer
But when I use set Property method:
2.setProperty
System.setProperty("http.proxySet", "true");
System.setProperty("http.proxyHost",10.100.21.11);
System.setProperty("http.proxyPort","80");
URL httpsUrl = new URL("https://192.168.17.22:8443/test");
HttpsURLConnection httpsCon = (HttpsURLConnection)httpsUrl.openConnection();
httpsCon.setDoOutput(true);
httpsCon.setDoInput(true);
httpsCon.setRequestMethod("POST");
OutputStream out = httpsCon.getOutputStream();
OutputStreamWriter owriter = new OutputStreamWriter(out);
owriter.write("<request>test</request>");
owriter.flush();
owriter.close();
...
I got a NoRouteToHostException: Network is unreachable.
It make me confused.I did not see any packets between HttpClient and ProxyServer.
But HttpClient can ping to ProxyServer(10.100.12.10 ping 10.100.21.11)
So I remove proxy setting(as without using proxy):
Also got NoRouteToHostException: Network is unreachable.
I thought this is reasonable.Because there is no route to extranet.
I guess it seems like to setProperty method that the inner function of httpsUrlConnection will to check this url can be reachable or not.
But it is weird. 1st method can be success.
Have any idea? Or what are different between 1st and 2nd method?
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Update
System.setProperty("https.proxyHost",10.100.21.11);
System.setProperty("https.proxyPort","80");
It can work and packets flow are correct what I expect for.
But set https.proxyPort=443 is not workable for me
System.setProperty("https.proxyPort","443");
It will thorow a exception as bellow:
java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:770)
....
So I thought Apache Proxy have also to be modified to the right configuration.
Your URL connection is https whereas you are only setting the http proxy.
Try setting the https proxy.
//System.setProperty("https.proxySet", "true");
System.setProperty("https.proxyHost",10.100.21.11);
System.setProperty("https.proxyPort","443");
EDIT
#EJP is correct. There is no https.proxySet .. I copied your original question and included in the answer.
You will need to create a Proxy object for it. Create one as below:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(proxyServer, Integer.parseInt(proxyPort)));
Now use this proxy to create the HttpURLConnection object.
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection(proxy);
If you have to set the credentials for the proxy, set the Proxy-Authorization request property:
String uname_pwd = proxyUsername + ":" + proxyPassword
String authString = "Basic " + new sun.misc.BASE64Encoder().encode(uname_pwd.getBytes())
connection.setRequestProperty("Proxy-Authorization", authString);
And finally, you connect:
connection.connect();
thank you #divinedragon!
Same code on kotlin:
fun testProxy(login: String, pass: String, proxyData: ProxyData): String {
val url = URL("http://api.ipify.org")
val proxy = Proxy(Proxy.Type.HTTP, InetSocketAddress(proxyData.ip, proxyData.port))
val connection = url.openConnection(proxy) as HttpURLConnection
val loginPass = "$login:$pass"
val encodedLoginPass = Base64.getEncoder().encodeToString(loginPass.toByteArray())
val authString = "Basic $encodedLoginPass"
connection.setRequestProperty("Proxy-Authorization", authString);
with(connection) {
requestMethod = "GET" // optional default is GET
connectTimeout = 2000
readTimeout = 2000
return inputStream.bufferedReader().readText()
}
}
This is my code:
public static void downloadZipFile() {
String saveTo = "C:\\Users\\aria\\Downloads\\Temp";
try {
URL url = new URL("http://www.bcfi.be/download/files/R1112B2_BcfiHtm.zip");
URLConnection conn = url.openConnection();
InputStream in = conn.getInputStream();
FileOutputStream out = new FileOutputStream(saveTo + "BcfiHtm.zip");
byte[] b = new byte[1024];
int count;
while ((count = in.read(b)) >= 0) {
out.write(b, 0, count);
}
out.flush(); out.close(); in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
**When i compile it i get following error but if i use url directly in browsers evrything is ok.
How can i fix it? or is there any other way to download zip file?**
java.net.UnknownHostException: www.bcfi.be
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:195)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:395)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:234)
at sun.net.www.http.HttpClient.New(HttpClient.java:307)
at sun.net.www.http.HttpClient.New(HttpClient.java:324)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:970)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:911)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:836)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1172)
at be.azvub.ext.prismaFlex.Exterahelp.download.DownloadFile.downloadZipFile(DownloadFile.java:72)
at be.azvub.ext.prismaFlex.Exterahelp.download.DownloadFile.main(DownloadFile.java:37)
Quoting from the Java Docs:
Thrown to indicate that the IP address of a host could not be
determined.
Make sure your program is not blocked by the Firewall or a proxy.
UPDATE:
To configure you proxy, do as Peter Liljenberg suggested:
You could pass the proxy information to the openConnection call in
your code like this:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new
InetSocketAddress("pro",9999)); URLConnection conn =
url.openConnection(proxy);
Since you're behind a proxy you can try some different approaches:
1) Add the proxy information to the JVM when starting:
java -Dhttp.proxyHost=proxyhostURL -Dhttp.proxyPort=proxyPortNumber
-Dhttp.proxyUser=someUserName -Dhttp.proxyPassword=somePassword javaClassToRun
In your case it would probably be:
java -Dhttp.proxyHost=pro -Dhttp.proxyPort=9999 javaClassToRun
2) You could pass the proxy information to the openConnection call in your code like this:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("pro",9999));
URLConnection conn = url.openConnection(proxy);
I'm trying to communicate with a RESTful API over SSL. The whole client application relies on a basic connection method which looks something like this
URL url = null;
HttpsURLConnection connection = null;
BufferedReader bufferedReader = null;
InputStream is = null;
try {
url = new URL(TARGET_URL);
HostnameVerifier hv = new HostnameVerifier() {
public boolean verify(String urlHostName, SSLSession session) {
return true;
}
};
HttpsURLConnection.setDefaultHostnameVerifier(hv);
connection = (HttpsURLConnection) url.openConnection();
connection.setRequestMethod(requestType);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("Content-Language", "en-US");
connection.setSSLSocketFactory(sslSocketFactory);
is = connection.getInputStream();
bufferedReader = new BufferedReader(new InputStreamReader(is));
String line;
StringBuffer lines = new StringBuffer();
while ((line = bufferedReader.readLine()) != null) {
lines.append(line).append(LINE_BREAKER);
}
return lines.toString();
} catch (Exception e) {
System.out.println("Exception is :"+e.toString());
return e.toString();
}
}
This works well, but is there a more efficient way? We've tried Apache HTTPClient. It has an awesomely simple API but when we compared the performance of the above code vs Apache HTTPClient with YourKit, the latter one was creating more objects than the first. How do I optimize this?
I've used HTTPClient (but not for HTTPS) but I think this applies to your case. The recommendation is to create a single HTTPClient for your server, and a new HTTPGet object per call. You'll want to specify the multi-threaded client connection manager with an appropriate number of connections to allocate per host, and max total connections when you initialize your HTTPClient.
You could also try to use HttpCore instead of HttpClient. HttpCore is a set of HTTP transport components HttpClient is based upon. It has been specifically optimized for low memory footprint as well as performance. It lacks higher level HTTP functionality provided by HttpClient (connection management, cookie & state management, authentication) but should be quite efficient in terms of memory utilization.
http://hc.apache.org/httpcomponents-core-ga/examples.html