I've had to update a previous java application that requests a SOAP response from an external web service. This service is outside our firewall which now requires us to go through a proxy instead of hitting the URL directly.
Currently the Java App uses URLEndpoint that takes a string for the URL. Usually when I am getting to a URL through a proxy I create a URL like so:
URL url = new URL("http", "theproxy.com", 5555, finalUrl);
The problem is URLEndpoint only takes a string for the url, I tried to convert URL to string using toExternalForm() but it malformed the URL.
Any ideas as to a way around this?
EDIT: I can't use System.setProperty as this runs with a whole heap of other Java applications in tomcat.
second edit: I can't set a system properties as it will override all other applications running on the server, I can't use jsocks as the proxy we run through squid proxy which does not support socks4/5
Any help appreciated.
That's not how proxy's work. The way a proxy works is that you take your normal URL:
http://example.com/service
and instead of looking up "example.com" and port 80, the message is sent to your proxy host instead (http://theproxy.com:5555).
Java has built in proxy handling using http.proxyHost and http.proxyPort System properties.
So in your case you would need to do:
System.setProperty("http.proxyHost", "theproxy.com");
System.setProperty("http.proxyPort", "5555");
Then your code should, ideally, "Just Work".
Here is a page documenting the proxy properties.
Use Apache HttpClient and do as show in this example.
About the URL constructor with individual proxy setting:
http://edn.embarcadero.com/article/29783
(sorry don't have privileges to comment)
Related
I created a soap client with wsimport and a given wsdl. I also used SoapUI to test the service. Using SoapUI I had no problem but when using my Java client I get
java.net.ConnectException: Connection timed out: connect
The default values I have in the requestContext are as follows
com.sun.xml.internal.ws.connect.timeout=100000
javax.xml.ws.service.endpoint.address=[fully qualified domain name endpoint]
com.sun.xml.internal.ws.request.timeout=100000
javax.xml.ws.soap.http.soapaction.use=null
com.sun.xml.internal.ws.client.ContentNegotiation=none
javax.xml.ws.soap.http.soapaction.uri=null
I've tried increasing the timeout but it still doesn't connect.
Has anyone else had a similar problem?
As you mentioned the problem is of proxy, it has been answered in below links.
How to use an HTTP proxy in java
Proxy settings in a java program
If you are using proxy with authentication then you have set authenticator along with the proxy. This is answered here.
Authenticated HTTP proxy with Java
EDIT:
As correctly mentioned by William Burnham, you have set to set the properties before calling them.
Morever, I recommend you to clear the property soon after getting response using System.clearProperty(key) as the property is set for complete instance of jvm till it is restarted and hence can cause problems for other outgoing connections.
The problem was I was behind a proxy. I did different tests and found that using a web browser (or SoapUI) I was able to access the resource but from the command line it wasn't working.
After much searching, it was a simple fix: either passing the property as a jvm argument or manually setting it in the code with System.setProperty("java.net.useSystemProxies", "true"). The JVM wasn't using the proxy on its own.
EDIT
As I used wsimport I have a jax-ws client. It's important that proxy settings be configured prior to instantiantion.
ANOTHER EDIT
If by chance you're having problems and you're using an application server to make the soap request through the proxy, you may have to specify java.net.useSystemProxies=true (or similar) in the server's configuration--for example catalina.properties if using tomcat.
So i have a java app that uses Google Analytics API to gather some info. I am putting this application to run in my oracle cloud managed server which has a firewall and blocks any web calls to work. So, they setup a proxy for me to use....I've never set up a proxy to work with a java application before, I've been reading at tutorials like this: http://docs.oracle.com/javase/1.5.0/docs/guide/net/proxies.html
And i have no idea how to set this up...anyone want to point me in the right direction?
You must tell your application that there's a proxy somewhere.
As the documentation says, you must set some properties in your virtual machine. You can do it programatically:
//Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setPropery("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.clearProperty("http.proxyHost");
// From now on http connections will be done directly.
Or use https.proxy... if the connection is HTTPS.
Besides, if you have access to the application server start script, you could add those properties as VM properties with -Dhttp.proxyHost....
The solution in my case was to configure the JVM with a HTTPS proxy:
System.setProperty("https.proxyHost", "proxy");
System.setProperty("https.proxyPort", "3128");
I have a java app that talks to some REST services, and I want to look at the HTTP traffic using Fiddler.
Fiddler acts as a proxy on localhost:8888, so the following Java VM options are supposed to configure java to use this proxy:
-Dhttp.proxyHost=localhost -Dhttp.proxyPort=8888
However, if I pass these parameters when running the java app that I want to debug, I see no traffic in Fiddler.
I wrote a test Java app that simply performs an HTTP GET using HttpURLConnection.
I can view the HTTP traffic from this app in fiddler, if I specify the above-mentioned command-line parameters when debugging it from Eclipse.
What are the reasons that http.proxyHost/Port might not work for all java HTTP operations?
You can tell HttpClient to honor the JDK system arguments using the below code (HttpClient 4.x).
public static final DefaultHttpClient HTTP = new DefaultHttpClient();
ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(HTTP.getConnectionManager().getSchemeRegistry(),
ProxySelector.getDefault());
HTTP.setRoutePlanner(routePlanner);
As mentioned above you need to do something like this:
CloseableHttpClient httpClient = HttpClients.custom()
.setRoutePlanner(new SystemDefaultRoutePlanner(ProxySelector.getDefault())
But that will not be enough.
In TL;DR fashion - you also need all three of these system properties:
-Dhttp.proxyHost=localhost -Dhttp.proxyPort=8888 -Dhttp.nonProxyHosts=
The long answer can be found here:
As mentioned by Gerard Davison at the link above:
It can be very convenient when developing to server based application
to run them using "localhost" in order to maintain consistency between
developer machines. This is normally a good idea but there is a small
case where this can cause problems.
Consider if you are running a local http proxy on your machine in
order to capture your HTTP traffic. (Cough perhaps even the one in
JDeveloper I work on). Then you might run into Java bug 6737819.
Basically by default JDK 1.6 was hard coded not to send any request to
localhost via a proxy which of course was a bit of a pain. Luckily a
workaround was put in where you could put the string "~localhost" in
your nonProxyHosts entry to turn of this feature:
java -client -classpath classes
-Dhttp.proxyHost=localhost
-Dhttp.proxyPort=8099 -Dhttp.nonProxyHosts=~localhost
-Dhttps.proxyHost=localhost
-Dhttps.proxyPort=8099 client.Example
Now moving forward to JDK 1.7 this workaround no longer works; but you
need to take care to define nonProxyHosts as an empty string:
java -client -classpath classes
-Dhttp.proxyHost=localhost
-Dhttp.proxyPort=8099
-Dhttp.nonProxyHosts=
-Dhttps.proxyHost=localhost
-Dhttps.proxyPort=8099 client.Example
If you define this any anything other than an empty string the
DefaultProxySelector though beware because internally it will append /
or use the http.nonProxyHosts value from ../jre/lib/net.properties".
Just a minor complication that is not obvious from the published API.
For 4.3.6 I used
HttpHost proxy = new HttpHost(System.getProperty("http.proxyHost"), Integer.parseInt(System.getProperty("http.proxyPort")), "http");//System.getProperty("http.proxyHost")
DefaultRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpClient = HttpClients.custom().setRoutePlanner(routePlanner).build();
With Apache httpclient 4.3.6, it seems that you should use SystemDefaultRoutePlanner.
SystemDefaultRoutePlanner routePlanner = new SystemDefaultRoutePlanner(null);
CloseableHttpClient httpClient = HttpClientBuilder.create()
.setRoutePlanner(routePlanner)
.build();
This will pick up the proxy settings from system properties http.proxyHost and http.proxyPort, and will use the current settings of those values each time a new http request is sent.
I was expecting this code to return a 404, however it produces the output :
"Response code is 200"
Would it be possible to learn how to differentiate between existent and non-existent web pages . . . thanks so much,
try
{
// create the HttpURLConnection
URL url = new URL("http://www.thisurldoesnotexist");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
System.out.println("Response code is " + connection.getResponseCode());
}
EDIT: I see you've call openConnection() but not connect() - could that be the problem? I would expect getResponseCode() to actually make the request if it hasn't already, but it's worth just trying that...
That suggests you've possible got some DNS resolver which redirects to a "helper" (spam) page, or something like that.
The easiest way to see exactly what's going on here is to use Wireshark - have that up and capturing traffic (HTTP-only, to make life easier) and then run your code. You should be able to see what's going on that way.
Note that I wouldn't have expected a 404 - because that would involve being able to find a web server to talk to to start with. If you're trying to go to a host which doesn't involve, there shouldn't be an HTTP response at all. I'd expect connect() to throw an exception.
try adding a "connection.connect();" or look at the contents returned...
it could be a dns issue, ie: your dns is being sent to a parking place... for example: freedns does this.
You could:
Resolve the IP from the host of the page
Try to connect to port 80 on the resolved IP using plain sockets
This is a bit low level however and will add complexity since you will need to make a simple GET request through the socket. Then validate the response so you're sure that its actually a HTTP server running on port 80.
NMap might be able to help you here.
Ideally you should be getting this error:
java.net.UnknownHostException: www.thisurldoesnotexist
But it looks like your URL is resolved by you DNS provider.
For instance on my company's network running your code with URI "http://profile/" displays
the employee profile.
Please also check etc.home file if you are on windows to check if any settings have been changed.
Like #spgennard - I think this is most likely a DNS issue.
The URL you have chosen is owned by a DNS speculator.
The URL you have chosen is "parked" by a DNS provider.
Your ISP is messing with your DNS results to send your browser to some search page.
It is also possible that you are accessing the web via a proxy, and the proxy is doing something strange.
The way to diagnose this is to look at the other information in the HTTP responses you are getting, particularly the response body.
I'm trying to find a way to force any connection attempts a Jar makes to an external IP through my proxy server, which is running on localhost(Also a Java application).
Once the proxy server receives the connection it will open a connection with the external IP and begin routing the IO to and from the client/server.
I've been Googling this for 2 days, and I haven't had any luck, I believe I'm using the wrong terms in my search attempts.
If you've got any ideas, please let me know, I'll try anything.
Thanks in advance. - Sean.
If is that a "real" Proxy the you could specify the proxy to use using java system properties.
You have two alternatives:
Specify the proxy in the command line
Hardcode it into your app
Well you actually have three
Specify a .properties file, and read from there, and set it as System property ( which is pretty much option 2 but more dynamic )
From command line you'll use:
java -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8080 -jar YourJar.jar
With that all the http connections you perform will go through localhost at port 8080
The second is add this at the main method of your program:
public static void main( String [] args ) {
System.setProperty("http.proxyHost", "localhost");
System.setProperty("http.proxyPort", "8080");
.....
}
Which does the same.
Finally loading from myapp.properties
public static void main( String [] args ) {
try { // there are cleaner ways of course
ResorceBundle bundle = ResourceBundle.getBundle("myapp");
System.setProperty("http.proxyHost", bundle.getString("proxy.server"));
System.setProperty("http.proxyPort", bundle.getString("proxy.port"));
} catch( MissingResourceException missingResourceException ){}
....
}
You just have to make sure myapp.properties is available from the classpath
More information about this functionality here
If you are asking about general (NOT HTTP / FTP specific!) proxying of Socket connections, then the simple answer is that it is not supported by Java.
When you configure a proxy for HTTP and FTP traffic, the proxying happens at the application protocol level. The Java-side proxy properties tell the URLConnection layer to connect to your designated proxy rather than the IP address from the URL your application is trying to connect to. The Java Socket level is unaware that this is happening. It just sees a requests to connect to the proxy.
This work because the HTTP and FTP protocols specifically support proxying. For instance, the first 'line' of an HTTP GET request message gives the full URL of the page that the client is requesting. If the GET request goes to a proxy, the proxy can figure out where is has to send it.
Looking at the problem of proxying at the Socket level, the first observation is that the standard Java class libraries don't support this. The second observation is that it is actually unimplementable ... unless you implement this as an alternative transport layer. The reason is that IP and TCP/IP simply do not support the notion of explicitly proxying or relaying messages / streams. And even if you did implement such a transport, it doesn't fit into the standard Socket model.
So, if you are really asking about proxying all of the network traffic for a Java application, this can only be implemented outside of the JVM; i.e. at the network transport level of the JVM's (physical or virtual) host operating system.
If it's HTTP traffic or FTP traffic, you could try the following system properties:
http.proxyHost (default: )
http.proxyPort (default: 80 if http.proxyHost specified)
http.nonProxyHosts (default:
See this link for details:
http://java.sun.com/docs/books/tutorial/networking/urls/_setProxy.html