This is a question regarding an exception that is occurring in my code which makes a call to an https server.
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
It internally uses an instance of CloseableHttpClient to execute the PUT call.
Also, this code is a functional test that would be running on a remote machine as a CI job. I have seen some solutions with the SSL certificate error that mention how we can disable the SSL certificate validation in Java or add the certificate in the local JVM, one of them being here -
'peer not authenticated' SSL certificate error usng DefaultHttpClient
Unfortunately, it doesn't seem to be working as this is a remote machine and we cannot import certificates into that machine.
String endPoint = "https://" + hostName + ":" + port + "/v1/service/data/put";
endPoint is set in the code that is called from a jar. So, there is no scope that we'd be able to change it either.
If I am running the code that makes a PUT call to the endPoint from a standalone class (through the main method), it seems to be running fine, returning a 200/OK. Currently, the exception occurs if it is being run as a TestNG class from the .xml file.
The code added as a Github gist is here.
Let me know if you need more details.
there's a lot going on there and most of it isn't really related to the problem (the caching, for example or the other boilerplate code to set up the call).
what i usually do in this kind of situation is reduce your problem to a smaller and smaller chunk of code that can still reproduce the problem. for ex, using these HttpClient components, can you make any SSL call? try this code, which requires HttpClient 4.4 and will work on sites that don't have valid certificates:
CloseableHttpClient client = HttpClients.custom()
.setSSLContext(sslContext)
.setSSLHostnameVerifier(new NoopHostnameVerifier())
.build();
HttpGet httpGet = new HttpGet(<your https URL here>);
httpGet.setHeader("Accept", <whatever appropriate for URL above>);
HttpResponse response = client.execute(httpGet);
System.out.println(response.getStatusLine().getStatusCode());
As it was mentioned in the question, the code works fine if it were running in a standalone class, through the main method.
I was able to resolve the issue by placing my code in a static block. It might be related to the certificates being disabled during class load and thus, works fine now.
Related
So I have this situation: I try to download an image from somedomain.com using HTTPS. The domain is probably misconfigured, but unfortunately I can't change that. What exactly is happening:
When I browse to https://somedomain.com/animage.jpg I get a valid certificate issued for somedomain.com, which is perfect. But when I call the same site using it's IP address, say https://123.123.123.123 - I get a (also valid) certificate for *.hostingcompany.com - the certificate of the hosting company.
Now, I try to download the contents of the file using Java's HttpsUrlConnection, nothing special:
var urlConnection = new URL(imageUrl).openConnection();
((HttpURLConnection) urlConnection).getResponseCode();
(I want to first check the response code, but it's not important here.)
This code runs inside a Spring Boot App and is run on request. It works fine for the first request since booting the app. Each subsequent request fails with java.security.cert.CertificateException: No subject alternative DNS name matching somedomain.com found. It's because on each subsequent request the SSL Handshake is sent to the IP, not hostname, and get's the hosting company's certificate.
I was trying to find different settings for the SSL classes, but to no avail. I know there is a workaround where I could supply my own HostnameVerifier which could just return true, but that won't be secure, so I don't want to do that.
Did anyone encounter such problem? Maybe I'm searching in the wrong places? Maybe it's something with DNSes? I will appreciate any help.
Turns out it is a bug in Java 11.01. It is fixed since 11.02. After switching to 11.03. the behaviour I described above is gone. Each request gets a proper certificate.
Here are the details of the bug: https://bugs.openjdk.java.net/browse/JDK-8211806
I've been experimenting with the HttpClient stuff in the Java 9/10 incubator, and have the following trivial code (virtually stolen from the project home page!):
URI uri = URI.create("http://192.168.1.102:8080/");
HttpRequest getRequest = HttpRequest.newBuilder()
.uri(uri)
.GET()
.build();
HttpResponse<String> response = client.send(getRequest,
HttpResponse.BodyHandler.asString());
System.out.println("response to get: " + response.body());
I find it works fine if it's pointed at a URL that is not the localhost, but fails if I ask for the localhost (whether by the name "localhost", by 172.0.0.1, or by the actual IP address of the local host). The error is very strange, and the entire stack trace does not mention any of my code.
WARNING: Using incubator modules: jdk.incubator.httpclient
Exception in thread "main" java.io.EOFException: EOF reached while reading
at jdk.incubator.httpclient/jdk.incubator.http.Http1AsyncReceiver$Http1TubeSubscriber.onComplete(Http1AsyncReceiver.java:507)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadSubscription.signalCompletion(SocketTube.java:551)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:728)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowTask.run(SocketTube.java:171)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:675)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:829)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:243)
at jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:769)
at jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:731)
There is a server running locally, and I can connect to it just fine using a simple request from a web browser.
Any thoughts?
[EDIT]I found, I beleive, the mail list for this project. It's "obfuscated" (which fooled me completely!) but shown as: net dash dev at openjdk dot java dot net I'll post there too, and see if they have any input.
[EDIT 2]I'm pretty sure that this has nothing to do with localhost (per original title) but is something in the protocol negotiation with node.js/express (which is the server I'm using because it's easy to experiment with). Node occasionally (e.g. with a last line of text that's not LF terminated) seems to report the wrong content-length, but this isn't the problem, as the failure still occurs with correct length. I think it's possibly a bug in the attempt to upgrade the connection to HTTP/2.0 but don't know yet...
[EDIT 3]After wasting way too much of my life experimenting, I'm fairly sure that there's something in the way node.js 8.11.1 (and express 4.13.4 and body-parser 1.15.1) handle a request to upgrade a to HTTP 2.0 that's causing the problem. But I have no idea what. I'm giving up, and will continue the learning process for httpClient using a different server.
Updated. I finally got curl built with http 2.0 support, and the blame is entirely on node/express. When this server sees an upgrade request (node 8.something) it simply fails to create any output.Consequently, the client correctly fails with an EOF error.
As a side note, node/express also sets the content-length header "off by one" on occasions (not always!?)
try this
HttpRequest request = HttpRequest.newBuilder()
.uri(new URI("http://localhost:3000"))
.POST(BodyPublisher.fromString("hello"))
.version(Version.HTTP_1_1).build();
One of our devs reported the following error.
HttpGet foo = new HttpGet("http://www.example.com/path/to/file.xml");
works fine.
However, if the port is specified,
HttpGet foo = new HttpGet("http://www.example.com:80/path/to/file.xml");
the server returns a HTTP 500 error.
I've already verified that the website runs on the standard HTTP port 80. What could be the reason of this behavior? It looks like it's server side, as both lines of code work fine towards other websites.
A look into the server's log should bring up more information what exactly is going wrong there (status code 500 means that the server ran into a problem) but my guess is that there is some kind of script configured behind the URL that processes that value of the HTTP-request-header Host, doesn't expect the port-specification and runs into an error because of this.
Another reason might be a proxy between you and the server that ran into an error but I found that more hard to believe than the above theory.
Please provide the error-log of the server in order to be able to say more about this.
I am writing a routine to access a remote server. This server I am connecting to requires mutual authentication so I have to provide a keystore, and while I'm at it I'd like to put a proper truststore in place as well.
I can find plenty of tutorials on how to create a keystore with keytool and multiple ways to get an Apache HTTP client to recognize it, but not where to store it in a Tomcat environment so that the application can find it. Somehow putting it in the application's war file seems like a bad idea to me.
Again, this is not to permit Tomcat to handle inbound https connections - I have a reverse proxy set up by our admin team for that. I'm creating outgoing https connections that require mutual authentication, i.e., both accepting a self-signed destination server certificate, and providing my server's self-signed client certificate.
Where do you store the actual keystore and truststore files in a Tomcat environment for use by a web application?
You can put your keystore wherever you want, as long as you know how to tell httpclient where to load the keystore.
That, of course, is the trick.
Using Apache httpclient for https
Buried in all that mess of code in the accepted answer is the key (ha!) to using httpclient with your own custom keystore. It's unfortunate that httpclient doesn't have a simple API like "here's the path to my keystore file, now use it" or "here are the bytes for my keystore, use those" (if you wanted to load the keystore from the ClassLoader or whatever), but that seems to be the case.
The honest truth is that using keystores and truststores in Java is messy business, and there's usually no way around it. Having written a client-cert-capable HTTP client myself using nothing other than HttpsURLConnection and then also adding raw-socket components to that, I know how painful it is.
The code in the above-linked article is fairly straightforward if a bit verbose. Unfortunately, you're going to need to make it a lot messier for production-quality code because you've got to do error-checking, etc. for every step of the process to make sure your service doesn't fall-over when you are trying to set up the various stores and make your connection.
This is basically a comment to Christopher Schultz's answer, but since it involves some code snippets please excuse my putting it here
It's unfortunate that httpclient doesn't have a simple API like
"here's the path to my keystore file, now use it" or "here are the
bytes for my keystore, use those" (if you wanted to load the keystore
from the ClassLoader or whatever), but that seems to be the case
This is how one can configure Apache HttpClient 4.3 to use a specific trust store for SSL context initialization.
SSLContext sslContext = SSLContexts.custom()
.loadTrustMaterial(trustStore)
.build();
CloseableHttpClient client = HttpClients.custom()
.setSslcontext(sslContext)
.build();
One can load trust material from a resource like that
URL resource = getClass().getResource("/com/mycompany/mystuff/my.truststore");
KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
InputStream inputStream = resource.openStream();
try {
trustStore.load(inputStream, null /*usually not password protected*/);
} finally {
inputStream.close();
}
I have a java app that talks to some REST services, and I want to look at the HTTP traffic using Fiddler.
Fiddler acts as a proxy on localhost:8888, so the following Java VM options are supposed to configure java to use this proxy:
-Dhttp.proxyHost=localhost -Dhttp.proxyPort=8888
However, if I pass these parameters when running the java app that I want to debug, I see no traffic in Fiddler.
I wrote a test Java app that simply performs an HTTP GET using HttpURLConnection.
I can view the HTTP traffic from this app in fiddler, if I specify the above-mentioned command-line parameters when debugging it from Eclipse.
What are the reasons that http.proxyHost/Port might not work for all java HTTP operations?
You can tell HttpClient to honor the JDK system arguments using the below code (HttpClient 4.x).
public static final DefaultHttpClient HTTP = new DefaultHttpClient();
ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(HTTP.getConnectionManager().getSchemeRegistry(),
ProxySelector.getDefault());
HTTP.setRoutePlanner(routePlanner);
As mentioned above you need to do something like this:
CloseableHttpClient httpClient = HttpClients.custom()
.setRoutePlanner(new SystemDefaultRoutePlanner(ProxySelector.getDefault())
But that will not be enough.
In TL;DR fashion - you also need all three of these system properties:
-Dhttp.proxyHost=localhost -Dhttp.proxyPort=8888 -Dhttp.nonProxyHosts=
The long answer can be found here:
As mentioned by Gerard Davison at the link above:
It can be very convenient when developing to server based application
to run them using "localhost" in order to maintain consistency between
developer machines. This is normally a good idea but there is a small
case where this can cause problems.
Consider if you are running a local http proxy on your machine in
order to capture your HTTP traffic. (Cough perhaps even the one in
JDeveloper I work on). Then you might run into Java bug 6737819.
Basically by default JDK 1.6 was hard coded not to send any request to
localhost via a proxy which of course was a bit of a pain. Luckily a
workaround was put in where you could put the string "~localhost" in
your nonProxyHosts entry to turn of this feature:
java -client -classpath classes
-Dhttp.proxyHost=localhost
-Dhttp.proxyPort=8099 -Dhttp.nonProxyHosts=~localhost
-Dhttps.proxyHost=localhost
-Dhttps.proxyPort=8099 client.Example
Now moving forward to JDK 1.7 this workaround no longer works; but you
need to take care to define nonProxyHosts as an empty string:
java -client -classpath classes
-Dhttp.proxyHost=localhost
-Dhttp.proxyPort=8099
-Dhttp.nonProxyHosts=
-Dhttps.proxyHost=localhost
-Dhttps.proxyPort=8099 client.Example
If you define this any anything other than an empty string the
DefaultProxySelector though beware because internally it will append /
or use the http.nonProxyHosts value from ../jre/lib/net.properties".
Just a minor complication that is not obvious from the published API.
For 4.3.6 I used
HttpHost proxy = new HttpHost(System.getProperty("http.proxyHost"), Integer.parseInt(System.getProperty("http.proxyPort")), "http");//System.getProperty("http.proxyHost")
DefaultRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpClient = HttpClients.custom().setRoutePlanner(routePlanner).build();
With Apache httpclient 4.3.6, it seems that you should use SystemDefaultRoutePlanner.
SystemDefaultRoutePlanner routePlanner = new SystemDefaultRoutePlanner(null);
CloseableHttpClient httpClient = HttpClientBuilder.create()
.setRoutePlanner(routePlanner)
.build();
This will pick up the proxy settings from system properties http.proxyHost and http.proxyPort, and will use the current settings of those values each time a new http request is sent.