i checked this page and got some usefull code for using a proxy in java code when connecting to a webpage.
I can confirm that pages like whatsmyip do indeed tell me that proxy is working - it is showing proxy ip. The problem is that the page i am accesing to in java code, somehow detects my true ip and blocks content. I do know how it does that (header, return ip, etc.), what i do not know is how to bypass that.
Maybe another interesting thing is that this page works with no problems using 1 of the best known online proxy sites - it shows content. Now what is even more interesting is that i tried taking that sites ip and used it as proxy in my program, but there it didn't work - true ip got detected, which is really strange.
edit: This is my new code:
System.setProperty("java.net.useSystemProxies","false");
System.setProperty("http.proxyHost", "94.230.208.147");
System.setProperty("http.proxyPort", "9001");
System.setProperty("http.nonProxyHosts", "localhost|127.0.0.1");
System.setProperty("https.proxyHost", "94.230.208.147");
System.setProperty("https.proxyPort", "9001");
System.setProperty("https.nonProxyHosts", "localhost|127.0.0.1");
I can confirm that https://whatsmyip.com/ isn't fooled by this proxy and can see my true ip. What did i forget to include ?
Add this at the end of your code:
System.setProperty("http.nonProxyHosts", "localhost|127.0.0.1");
That indicates the hosts that should be accessed without going through the proxy. Typically this defines internal hosts. The value of this property is a list of hosts, separated by the '|' character.
Related
I have a web application which is hosted at https://example.com. I would like to share a session cookie between the main domain https://example.com and sub-domain https://www.example.com. Thus, there should be no need for a user to re-login if they switch from one domain to another. How do I achieve this in springboot 2.2.6?
This is what I have tried:
I went to application.properties and set server.servlet.session.cookie.domain=.example.com
Now, this does not help. I get an error:
java.lang.IllegalArgumentException: An invalid domain [.example.com] was specified for this cookie
at org.apache.tomcat.util.http.Rfc6265CookieProcessor.validateDomain(Rfc6265CookieProcessor.java:210) ~[tomcat-embed-core-9.0.33.jar:9.0.33]
at org.apache.tomcat.util.http.Rfc6265CookieProcessor.generateHeader(Rfc6265CookieProcessor.java:145) ~[tomcat-embed-core-9.0.33.jar:9.0.33]
at org.apache.catalina.connector.Response.generateCookieString(Response.java:973) ~[tomcat-embed-core-9.0.33.jar:9.0.33]
If I set server.servlet.session.cookie.domain=example.com, then the cookie is not visible for http://www.example.com and if I set server.servlet.session.cookie.domain=www.example.com, then the cookie is not visible for http://example.com
I have read discussions about Rfc6265CookieProcessor and LegacyCookieProcessor, but I don't know the right way to fix this issue.
Springboot 2.2.6 uses tomcat version 9.0.*
So, how do I fix this issue?
EDIT:
I was trying the above changes on localhost only and not on production. Instead of accessing http://www.example.com, I was doing https://www.localhost and instead of accessing http://example.com, I was doing http://localhost
The right value is:
server.servlet.session.cookie.domain=example.com
What I was trying was that I was making changes on localhost and they were not working for me. I was modifying the values in chrome console manually and expecting to see cookies set on https://localhost with domain localhost to be visible in another tab for domain https://www.localhost and that was not happening.
I read the answer here: Share cookie between subdomain and domain and #Cesc 's comment on that answer which was :
I am not sure where to put this so I am choosing the comments of the
accepted answer. It took long time and failed experiments to prove the
above on my localhost, until it occurred to me that I should call the
localhost with a dot in the name. Like "localhost.com" or something
like that. Then all the "set cookies" behaviours started following the
explanations written here in this answer. Hoping this might help
somebody.
So, I tried my changes on production directly and they worked fine. I am still not able to get it to work on localhost. The way I access my website on localhost is:
https://localhost and https://www.localhost. Based on #Cesc 's comment, I probably need to access the website on localhost as https://www.localhost.com or https://localhost.com and then it will work. But, I have not tried that.
Rather than testing with 'localhost' on your dev machine, try using your machine's fully-qualified host name. I've had a similar challenge with testing authentication against our single-signon platform.
So I have this situation: I try to download an image from somedomain.com using HTTPS. The domain is probably misconfigured, but unfortunately I can't change that. What exactly is happening:
When I browse to https://somedomain.com/animage.jpg I get a valid certificate issued for somedomain.com, which is perfect. But when I call the same site using it's IP address, say https://123.123.123.123 - I get a (also valid) certificate for *.hostingcompany.com - the certificate of the hosting company.
Now, I try to download the contents of the file using Java's HttpsUrlConnection, nothing special:
var urlConnection = new URL(imageUrl).openConnection();
((HttpURLConnection) urlConnection).getResponseCode();
(I want to first check the response code, but it's not important here.)
This code runs inside a Spring Boot App and is run on request. It works fine for the first request since booting the app. Each subsequent request fails with java.security.cert.CertificateException: No subject alternative DNS name matching somedomain.com found. It's because on each subsequent request the SSL Handshake is sent to the IP, not hostname, and get's the hosting company's certificate.
I was trying to find different settings for the SSL classes, but to no avail. I know there is a workaround where I could supply my own HostnameVerifier which could just return true, but that won't be secure, so I don't want to do that.
Did anyone encounter such problem? Maybe I'm searching in the wrong places? Maybe it's something with DNSes? I will appreciate any help.
Turns out it is a bug in Java 11.01. It is fixed since 11.02. After switching to 11.03. the behaviour I described above is gone. Each request gets a proper certificate.
Here are the details of the bug: https://bugs.openjdk.java.net/browse/JDK-8211806
I created a soap client with wsimport and a given wsdl. I also used SoapUI to test the service. Using SoapUI I had no problem but when using my Java client I get
java.net.ConnectException: Connection timed out: connect
The default values I have in the requestContext are as follows
com.sun.xml.internal.ws.connect.timeout=100000
javax.xml.ws.service.endpoint.address=[fully qualified domain name endpoint]
com.sun.xml.internal.ws.request.timeout=100000
javax.xml.ws.soap.http.soapaction.use=null
com.sun.xml.internal.ws.client.ContentNegotiation=none
javax.xml.ws.soap.http.soapaction.uri=null
I've tried increasing the timeout but it still doesn't connect.
Has anyone else had a similar problem?
As you mentioned the problem is of proxy, it has been answered in below links.
How to use an HTTP proxy in java
Proxy settings in a java program
If you are using proxy with authentication then you have set authenticator along with the proxy. This is answered here.
Authenticated HTTP proxy with Java
EDIT:
As correctly mentioned by William Burnham, you have set to set the properties before calling them.
Morever, I recommend you to clear the property soon after getting response using System.clearProperty(key) as the property is set for complete instance of jvm till it is restarted and hence can cause problems for other outgoing connections.
The problem was I was behind a proxy. I did different tests and found that using a web browser (or SoapUI) I was able to access the resource but from the command line it wasn't working.
After much searching, it was a simple fix: either passing the property as a jvm argument or manually setting it in the code with System.setProperty("java.net.useSystemProxies", "true"). The JVM wasn't using the proxy on its own.
EDIT
As I used wsimport I have a jax-ws client. It's important that proxy settings be configured prior to instantiantion.
ANOTHER EDIT
If by chance you're having problems and you're using an application server to make the soap request through the proxy, you may have to specify java.net.useSystemProxies=true (or similar) in the server's configuration--for example catalina.properties if using tomcat.
I was expecting this code to return a 404, however it produces the output :
"Response code is 200"
Would it be possible to learn how to differentiate between existent and non-existent web pages . . . thanks so much,
try
{
// create the HttpURLConnection
URL url = new URL("http://www.thisurldoesnotexist");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
System.out.println("Response code is " + connection.getResponseCode());
}
EDIT: I see you've call openConnection() but not connect() - could that be the problem? I would expect getResponseCode() to actually make the request if it hasn't already, but it's worth just trying that...
That suggests you've possible got some DNS resolver which redirects to a "helper" (spam) page, or something like that.
The easiest way to see exactly what's going on here is to use Wireshark - have that up and capturing traffic (HTTP-only, to make life easier) and then run your code. You should be able to see what's going on that way.
Note that I wouldn't have expected a 404 - because that would involve being able to find a web server to talk to to start with. If you're trying to go to a host which doesn't involve, there shouldn't be an HTTP response at all. I'd expect connect() to throw an exception.
try adding a "connection.connect();" or look at the contents returned...
it could be a dns issue, ie: your dns is being sent to a parking place... for example: freedns does this.
You could:
Resolve the IP from the host of the page
Try to connect to port 80 on the resolved IP using plain sockets
This is a bit low level however and will add complexity since you will need to make a simple GET request through the socket. Then validate the response so you're sure that its actually a HTTP server running on port 80.
NMap might be able to help you here.
Ideally you should be getting this error:
java.net.UnknownHostException: www.thisurldoesnotexist
But it looks like your URL is resolved by you DNS provider.
For instance on my company's network running your code with URI "http://profile/" displays
the employee profile.
Please also check etc.home file if you are on windows to check if any settings have been changed.
Like #spgennard - I think this is most likely a DNS issue.
The URL you have chosen is owned by a DNS speculator.
The URL you have chosen is "parked" by a DNS provider.
Your ISP is messing with your DNS results to send your browser to some search page.
It is also possible that you are accessing the web via a proxy, and the proxy is doing something strange.
The way to diagnose this is to look at the other information in the HTTP responses you are getting, particularly the response body.
I'm trying to find a way to force any connection attempts a Jar makes to an external IP through my proxy server, which is running on localhost(Also a Java application).
Once the proxy server receives the connection it will open a connection with the external IP and begin routing the IO to and from the client/server.
I've been Googling this for 2 days, and I haven't had any luck, I believe I'm using the wrong terms in my search attempts.
If you've got any ideas, please let me know, I'll try anything.
Thanks in advance. - Sean.
If is that a "real" Proxy the you could specify the proxy to use using java system properties.
You have two alternatives:
Specify the proxy in the command line
Hardcode it into your app
Well you actually have three
Specify a .properties file, and read from there, and set it as System property ( which is pretty much option 2 but more dynamic )
From command line you'll use:
java -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8080 -jar YourJar.jar
With that all the http connections you perform will go through localhost at port 8080
The second is add this at the main method of your program:
public static void main( String [] args ) {
System.setProperty("http.proxyHost", "localhost");
System.setProperty("http.proxyPort", "8080");
.....
}
Which does the same.
Finally loading from myapp.properties
public static void main( String [] args ) {
try { // there are cleaner ways of course
ResorceBundle bundle = ResourceBundle.getBundle("myapp");
System.setProperty("http.proxyHost", bundle.getString("proxy.server"));
System.setProperty("http.proxyPort", bundle.getString("proxy.port"));
} catch( MissingResourceException missingResourceException ){}
....
}
You just have to make sure myapp.properties is available from the classpath
More information about this functionality here
If you are asking about general (NOT HTTP / FTP specific!) proxying of Socket connections, then the simple answer is that it is not supported by Java.
When you configure a proxy for HTTP and FTP traffic, the proxying happens at the application protocol level. The Java-side proxy properties tell the URLConnection layer to connect to your designated proxy rather than the IP address from the URL your application is trying to connect to. The Java Socket level is unaware that this is happening. It just sees a requests to connect to the proxy.
This work because the HTTP and FTP protocols specifically support proxying. For instance, the first 'line' of an HTTP GET request message gives the full URL of the page that the client is requesting. If the GET request goes to a proxy, the proxy can figure out where is has to send it.
Looking at the problem of proxying at the Socket level, the first observation is that the standard Java class libraries don't support this. The second observation is that it is actually unimplementable ... unless you implement this as an alternative transport layer. The reason is that IP and TCP/IP simply do not support the notion of explicitly proxying or relaying messages / streams. And even if you did implement such a transport, it doesn't fit into the standard Socket model.
So, if you are really asking about proxying all of the network traffic for a Java application, this can only be implemented outside of the JVM; i.e. at the network transport level of the JVM's (physical or virtual) host operating system.
If it's HTTP traffic or FTP traffic, you could try the following system properties:
http.proxyHost (default: )
http.proxyPort (default: 80 if http.proxyHost specified)
http.nonProxyHosts (default:
See this link for details:
http://java.sun.com/docs/books/tutorial/networking/urls/_setProxy.html