Develop Reference

Java SSL handshake always get "PKIX path building failed" for any domain

I get this error message when build a gradle project in idea:
> Could not resolve io.spring.gradle:dependency-management-plugin:1.0.6.RELEASE.
> Could not get resource 'https://plugins.gradle.org/m2/io/spring/gradle/dependency-management-plugin/1.0.6.RELEASE/dependency-management-plugin-1.0.6.RELEASE.pom'.
> Could not HEAD 'https://plugins.gradle.org/m2/io/spring/gradle/dependency-management-plugin/1.0.6.RELEASE/dependency-management-plugin-1.0.6.RELEASE.pom'.
> sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
I found that all ssl handshack in java get the same error.
I use SSLPoke to check some domains, such as stackoverflow.com, github.com, all of them return the same error message: "PKIX path building failed". But I can visit this sites on the browser with no error.
I try to change the jdk from jdk11 to jdk8, and try to reinstall the jdk, but also get the same result. I check the default jdk keystore by keytool -list command, and it looks like no problem.
I try to debug and found that the certificate looks weird, it only has one cert in the cert chain, and the Issuer is always CN=GlobalSign Root CA, C=EN, no matter which domain. Such as stackoverflow.com:
[
[
Version: V3
Subject: CN=*.stackexchange.com
Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11
Key: Sun RSA public key, 1024 bits
modulus: 129343236870246922217917428341848371602010941604981692235450252202393431416169367447480541321401904173442212978999107322095875009215075266308069463921433338673265672174736174633404814882397952490528363553362969976277321592285699339620492251079789709609773064124868826755702755848122392099215387700370904957487
public exponent: 65537
Validity: [From: Fri Jul 26 22:38:33 CST 2019,
To: Thu Oct 24 22:38:33 CST 2019]
Issuer: CN=GlobalSign Root CA, C=EN
SerialNumber: [ bae8be0e 04cb0e2b 0e83d26f c22ba1e7]
Certificate Extensions: 3
[1]: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
CA:false
PathLen: undefined
]
[2]: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
serverAuth
clientAuth
]
[3]: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
DNSName: *.askubuntu.com
DNSName: *.blogoverflow.com
DNSName: *.mathoverflow.net
DNSName: *.meta.stackexchange.com
DNSName: *.meta.stackoverflow.com
DNSName: *.serverfault.com
DNSName: *.sstatic.net
DNSName: *.stackexchange.com
DNSName: *.stackoverflow.com
DNSName: *.stackoverflow.email
DNSName: *.superuser.com
DNSName: askubuntu.com
DNSName: blogoverflow.com
DNSName: mathoverflow.net
DNSName: openid.stackauth.com
DNSName: serverfault.com
DNSName: sstatic.net
DNSName: stackapps.com
DNSName: stackauth.com
DNSName: stackexchange.com
DNSName: stackoverflow.blog
DNSName: stackoverflow.com
DNSName: stackoverflow.email
DNSName: stacksnippets.net
DNSName: superuser.com
]
]
Algorithm: [SHA256withRSA]
Signature:
0000: B5 35 45 AA 8D 99 FF F1 3F 5D CA 94 6D 5F 6A 12 .5E.....?]..m_j.
0010: D2 39 0E 66 1D 11 63 80 12 0C A1 2D A7 CA A7 39 .9.f..c....-...9
0020: 36 4B A4 12 45 AD A8 4D E5 1E DD 7B AF A9 10 CD 6K..E..M........
0030: ED 5B 15 76 F2 49 41 F8 AB 82 67 5D E8 09 0A 65 .[.v.IA...g]...e
0040: 7D BC 22 C5 53 7D DD 32 15 9E 88 92 FB 35 C2 C8 ..".S..2.....5..
0050: 86 E0 53 BF 32 72 DA FA CE 27 A0 BA 78 5F DA B2 ..S.2r...'..x_..
0060: CA C3 8B 14 0B C5 EF E1 4D 96 8F BF 4A AC B0 DB ........M...J...
0070: 24 5E 20 7C 32 51 58 93 36 0B 1A 2A BB 88 A3 9B $^ .2QX.6..*....
0080: DF 6F B4 F1 25 CD B8 C6 C1 1D 19 BD A7 54 27 73 .o..%........T's
0090: 56 A8 5D 78 13 E6 86 00 59 E2 32 34 34 28 6D 4F V.]x....Y.244(mO
00A0: 30 39 F6 3A 2E 43 1F E6 7B 43 57 C2 79 E5 87 4C 09.:.C...CW.y..L
00B0: CB 9E 95 6D 99 6D 46 AD FA 7D 74 BA 12 D9 D0 8B ...m.mF...t.....
00C0: 93 B7 49 E4 61 FD 4B 73 00 FA 0E 61 9A 4E DA C1 ..I.a.Ks...a.N..
00D0: D3 B9 45 B1 79 13 BB 90 02 98 24 E7 4D 31 01 52 ..E.y.....$.M1.R
00E0: 1F 38 47 0B 4E 4C E0 91 2A 8A 05 6E 20 89 81 E3 .8G.NL..*..n ...
00F0: 3B E3 60 D5 70 DF 28 D3 58 E7 D6 FF A6 CA 1D B6 ;.`.p.(.X.......
]
Different to the normal cert:
Key length. Normal is 2048 bits, not 1024 bits.
Issuer. Normal is CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US
Certificate Extensions. Normal has 9 extensions, not 3.
modulus, SerialNumber, Signature. Their value is different to the normal.
Normal cert has a intermediate certificate in the cert chain, Issuer is CN=DST Root CA X3, O=Digital Signature Trust Co.
So, how can I find the real reason of the problem, and how to solve the problem.

Meta: not an answer yet but need space to respond to data
Aside: I assume you are in timezone +8. The PCAPs are 09-18 01:43:00-01 and 01:49:52-54 Z
while the Java-log timestamps are 09-18 09:42:59-43:01 and 09:49:51-54 CST.
First, the Java-logs. In the success log at line 3652, the ServerHello selects TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
and agrees to extensions renegotiation_info server_name ec_point_formats status_request extended_master_secret;
this is consistent with the real stackoverflow servers in my testing. In the fail log at 3651,
it selects TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 and agrees to only renegotiation_info ec_point_formats,
which is substantially different behavior. We then see the different certificates,
with success.txt getting the correct leaf cert (under Let's Encrypt) and chain cert (under DST),
and fail.txt getting the bogus cert under "CN=GlobalSign Root CA, C=EN" with most extensions missing, and no chain.
But as to the PCAPs:
See line 3638 in the fail log, the raw read bytes is different to the first five bytes of Transport Layer Security in the server hello response. In success log, they are the same.
That's not the only difference. Although the time matches,
your fail.PCAP doesn't match your fail.txt at all. Most importantly, it doesn't fail --
it receives the correct cert chain (Let's Encrypt + DST) and completes the handshake successfully.
In more detail:
the ClientHello is very different. It offers max TLS1.2, not TLS1.3 as offered by (your) Java.
It has a different list of ciphersuites, and different extensions (necessarily so for 1.2 vs 1.3),
among which are Heartbeat and ticket neither of which Java supports.
the ServerHello is shorter (record header 16 03 03 00 45 vs 16 03 03 00 59 in the fail.txt or 16 03 03 00 65 in the success.txt,
as you noticed) because it offered ticket and the server agreed, and thus the server does not provide a session-id.
(as above) it gets the good certs, as well as CertStatus ServerKX ServerDone,
and proceeds to send ClientKX CCS Finished and receive ticket CCS Finished -- i.e. success.
your client soon (but not instantly) RSTs without sending (or receiving) any data; this is the behavior of a Windows program that closes without doing shutdown (perhaps due to being killed) or a Unix program that explicitly sets linger=0 (which is unusual).
I also notice the fail.PCAP connection goes to 151.101.65.69 but the success.PCAP goes to 151.101.193.69.
Both of those addresses (and two others) are valid for stackoverflow.com, but usually connections
from the same machine within a short time window like this would resolve to and use the same address.
Look carefully at what you did for the fail.PCAP and see if you can get a PCAP that actually is a failure case.
Actually, an idea occurs here. DO YOU HAVE AN ANTIVIRUS OR OTHER 'ENDPOINT PROTECTION'? That might intercept the outgoing connection before it leaves your machine, so that the program (Java) sees the bogus cert even though the correct cert was actually fetched from the network. But there would still need to be a reason it intercepts one connection and not another.

Where does InstallCert.java get the server certificates

I'm attempting to follow the instructions on this page:
http://www.mkyong.com/webservices/jax-ws/suncertpathbuilderexception-unable-to-find-valid-certification-path-to-requested-target/
to create a certificate for my localhost in which to do some development testing.
When running InstallCert for localhost:8443, the following two certificates are generated:
Server sent 2 certificate(s):
1 Subject CN=localhost4.localdomain4, O=example.com, C=US
Issuer CN=Certificate Shack, O=example.com, C=US
sha1 f4 2a a9 09 32 a6 ee 41 9d 9c 44 e6 4a bc 31 79 17 cb 88 fd
md5 e0 78 65 83 30 33 78 c5 80 17 e7 7a a2 91 85 52
2 Subject CN=Certificate Shack, O=example.com, C=US
Issuer CN=Certificate Shack, O=example.com, C=US
sha1 b8 87 d6 2d ac d8 36 06 7c 58 68 10 3e 21 39 6a a0 33 a1 25
md5 07 24 57 5f f8 35 1e 97 70 ff 54 aa 13 e6 6b 12
The trouble is that my system needs the CN to be localhost. I have no idea where the localhost4.localdomain4 comes from. How can I change this to be simply localhost?

The certificate comes from the server, during the handshake.
The CN is inside the certificate.
You can't change it without creating a new server certificate.

Getting EOFException error while establishing a SSL connection

I’m writing a program in java that send official invoice information to the Fiscal Administration. This public service provided certificates to use in the SSL connection to the web services and to encrypt some especial data fields inside the request body message.
I’m having an EOFException error during the handshake phase after client and server have agreed to communicate using the agreed cipher suite that in this case is TLS_RSA_WITH_AES_128_CBC_SHA.
Following the SSL protocol the client perform with success a test using the new cipher and send the test data to the server so the server can also repeat the same test and confirm that it is also capable of encrypt and decrypt data. And in this point the server send the EOFException.
Here is the last part of the SSL communication log:
Send a quick confirmation to the server verifying that we know the private key corresponding to the client certificate we just sent...
* CertificateVerify
[write] MD5 and SHA1 hashes: len = 262
binary data here too large not displayed
main, WRITE: TLSv1 Handshake, length = 262
[Raw write]: length = 267
binary data here too large not displayed
*Tell the server we're changing to the newly established cipher suite. All further messages will be encrypted using the parameters we just established. *
main, WRITE: TLSv1 Change Cipher Spec, length = 1
[Raw write]: length = 6
0000: 14 03 01 00 01 01
... and finishes with success
..Finished
We send an encrypted Finished message to verify everything worked.
verify_data: { 221, 96, 47, 110, 19, 170, 244, 8, 37, 152, 160, 40 }
...
The client encrypt the test data..
[write] MD5 and SHA1 hashes: len = 16
0000: 14 00 00 0C DD 60 2F 6E 13 AA F4 08 25 98 A0 28 .....`/n....%..(
Padded plaintext before ENCRYPTION: len = 48
0000: 14 00 00 0C DD 60 2F 6E 13 AA F4 08 25 98 A0 28 .....`/n....%..(
0010: 10 7F 85 11 EC 6D 5D ED 21 70 27 F4 DC 23 C0 9B .....m].!p'..#..
0020: A7 6F C2 80 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B .o..............
main, WRITE: TLSv1 Handshake, length = 48
* ...and send the test data to the server so that the server can do the same test and confirm that encrypted communication can be established (53 bytes = 48 from the test data + 5 from header) *
[Raw write]: length = 53
0000: 16 03 01 00 30 1C 17 08 0F 49 C9 6A 7A 8B 8C 48 ....0....I.jz..H
0010: BA 57 2D CB 06 46 1E 65 61 7C 5F 74 F2 08 AB 12 .W-..F.ea._t....
0020: 91 47 72 8C 8F 84 0A CB D7 29 E2 FD 84 B2 FD 9E .Gr......)......
0030: 47 DC 13 60 B4 G..`.
...and the server respond with the EOFException error
main, received EOFException: error
main, handling exception: javax.net.ssl.SSLHandshakeException: Remote host closed
connection during handshake
%% Invalidated: [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA]
main, SEND TLSv1 ALERT: fatal, description = handshake_failure
Padded plaintext before ENCRYPTION: len = 32
0000: 02 28 BC 65 1A CA 68 87 79 84 5F 64 16 F5 28 72 .(.e..h.y._d..(r
0010: F7 8A 69 72 93 D8 09 09 09 09 09 09 09 09 09 09 ..ir............
main, WRITE: TLSv1 Alert, length = 32
[Raw write]: length = 37
0000: 15 03 01 00 20 0D 9A 35 18 B7 98 4B 7B AF 82 4E .... ..5...K...N
0010: 1A EE 7D AC 5D D5 49 05 4E 74 B9 77 E4 CD 87 61 ....].I.Nt.w...a
0020: 23 03 5C 9C 7E #...
main, called closeSocket()
main, called close()
main, called closeInternal(true)
I have no idea on what might be the cause for such a failure and how to programmatically influence the outcome of this step in the process. I’ve tried force the use of other ciphers recognized by both client and server such as SSL_RSA_WITH_RC4_128_MD5 but the error remain.
Any thoughts on how to solve this problem?

Any thoughts on how to solve this problem?
I suggest that you get in contact with the people who run that service, and get them to look at their logs to see why their server is closing the connection during SSL setup.
(Strictly speaking, the server does not "respond with [an] EOFException error". It is actually closing the TCP connection and the client-side Java libraries are throwing the exception. You are likely to get a more helpful response from the maintainers of the service if you use correct terminology.)

Java SSL Streaming - splitted applicationdata

I try to send a byte[] () over a established SSL Connection (handshake etc is done).
The result: The byte[] is spitted into two packets (see debug below):
First packet: just the first byte of the application data (**01**) .
Second packet: the rest (fe db 01 00 ...) 650 Bytes
Is there a way to commit all application data bytes in one packet?
Stream to send 651 Bytes:
**01** fe db 01 00 00 02 83 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 22 31 2e 30 22 20 65 6e 63 6f 64 69 6e 67 3d 22 75 73 2d 61 73 63 69 69 22 20 73 74 61 6e 64 61 6c 6f 6e 65 3d 22 6e 6f 22 3f 3e …
javax.net.debug output
Padded plaintext before ENCRYPTION: len = 32
0000: **01** 06 03 06 46 7F 7F AE D4 E8 30 5D B7 DB 3C 44 ....F.....0]..<D
0010: 02 08 C9 2A A1 0A 0A 0A 0A 0A 0A 0A 0A 0A 0A 0A ...*............
1, WRITE: TLSv1 Application Data, length = 32
[Raw write]: length = 37
0000: 17 03 01 00 20 B3 4E EE CE 5B 69 EC A5 4A 80 7F .... .N..[i..J..
0010: D6 03 35 AF 6A 7B 85 17 B7 46 A2 31 B2 EF 7E D0 ..5.j....F.1....
0020: EA 1B 67 7E ED ..g..
Padded plaintext before ENCRYPTION: len = 672
0000: FE DB 01 00 00 02 83 3C 3F 78 6D 6C 20 76 65 72 .......<?xml ver
0010: 73 69 6F 6E 3D 22 31 2E 30 22 20 65 6E 63 6F 64 sion="1.0" encod
0020: 69 6E 67 3D 22 75 73 2D 61 73 63 69 69 22 20 73 ing="us-ascii" s
0030: 74 61 6E 64 61 6C 6F 6E 65 3D 22 6E 6F 22 3F 3E tandalone="no"?>
[…]

Sun's impl comments:
By default, we counter chosen plaintext issues on CBC mode
ciphersuites in SSLv3/TLS1.0 by sending one byte of application
data in the first record of every payload, and the rest in
subsequent record(s). Note that the issues have been solved in
TLS 1.1 or later.
Experiment with SSLEngine.wrap( largePlainText ) shows that it produces 2 SSL records, the 1st record contains 1 byte of plain text, the 2nd record contains 15846 bytes of plain text.
The receiver API probably handle record-by-record, so it'll return 1 byte for the 1st read.
We can also observe this behavior in other SSL impls, e.g. HTTPS requests from web browsers.
OpenSSL inserts empty records against the attack. If the receiver is Java SSL socket, the input stream cannot return 0 bytes for read(), so the record is skipped. Other receivers may not be prepared for a 0-length record and may break.

The assumption you're making about reading the byte[] exactly as you write them on the other end is a classic TCP mistake. It's not actually specific to SSL/TLS, but could also happen with a TCP connection.
There is no guarantee in TCP (and in SSL/TLS) that the reader's buffer will be filled with the exact same packet length as the packets in the writer's buffer. All TCP guarantees is in-order delivery, so you'll eventually get all your data, but you have to treat it as a stream.
This is why protocols that use TCP rely on indicators and delimiters to tell the other end when to stop reading certain messages.
For example, HTTP 1.1 uses a blank line to indicate when the headers end, and it uses the Content-Length header to tell the recipient what entity length to expect (or chunked transfer encoding). SMTP also uses line returns and . at the end of a message.
If you're designing your own protocol, you need to define a way for the recipient to know when what you define as meaningful units of data are delimited. When you read the data, read such indicators, and fill in your read buffer until you get the amount of bytes you expect or until you find the delimiter that you've defined.

I had the same problem until I saw this page:
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=7157903
So, I run the JVM with -Djsse.enableCBCProtection=false parameter and now the data is not splitted.
Best regards

JMeter proxy and java serialization in HTTP/POST?

We have an applet-servlet communication that we'd like to record with JMeter's HTTP proxy.
It works with GET messages until the applet sends an HTTP POST message which includes some serialized Java objects (built-in types), then we get this error in the Applet:
alt text http://img339.imageshack.us/img339/9238/appletservletjmeterhttp.png
OK, so there's some JVM version conflict somewhere in the queue. But where?
The communication runs OK without JMeter, that is: Applet -> Tomcat -> Servlet. All on my local machine.
But it doesn't work through JMeter: Applet -> JMeter proxy -> Tomcat -> Servlet. Also all on my machine.
It is as if JMeter was modifying the POST message content...
I tested it with the Apache proxy as well, working fine.
Even funnier thing is that I have only one version of Java installed, one JDK and one JRE. Both 1.6.0_07...
Thought I'd ask before starting digging deeper in the rabbit hole ;-)
Here is the hex dump of the POST data sent directly to Tomcat:
00000348 ac ed 00 05 73 72 00 11 6a 61 76 61 2e 6c 61 6e ....sr.. java.lan
00000358 67 2e 49 6e 74 65 67 65 72 12 e2 a0 a4 f7 81 87 g.Intege r.......
00000368 38 02 00 01 49 00 05 76 61 6c 75 65 78 72 00 10 8...I..v aluexr..
00000378 6a 61 76 61 2e 6c 61 6e 67 2e 4e 75 6d 62 65 72 java.lan g.Number
00000388 86 ac 95 1d 0b 94 e0 8b 02 00 00 78 70 00 00 01 ........ ...xp...
00000398 7b {
And here is the data when sent through JMeter:
00000128 ac ed 00 05 73 72 00 11 6a 61 76 61 2e 6c 61 6e ....sr.. java.lan
00000138 67 2e 49 6e 74 65 67 65 72 12 e2 a0 a4 f7 3f 3f g.Intege r.....??
00000148 38 02 00 01 49 00 05 76 61 6c 75 65 78 72 00 10 8...I..v aluexr..
00000158 6a 61 76 61 2e 6c 61 6e 67 2e 4e 75 6d 62 65 72 java.lan g.Number
00000168 3f ac 3f 1d 0b 3f e0 3f 02 00 00 78 70 00 00 01 ?.?..?.? ...xp...
00000178 7b {
A lot of "3f"s in the second dump...
So this is definitely some kind of an encoding problem.
The content type is set correctly in the header:
POST /ABCOrder/ABCServlet?cmd=getNetworkConnection HTTP/1.1
Connection: keep-alive
Content-Type: application/octet-stream
Host: 109.107.148.164:8443
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
User-Agent: Mozilla/4.0 (Windows Vista 6.0) Java/1.6.0_14
Content-Length: 81

Here is the solution:
JMeter has a config file, bin/jmeter.properties.
Here you can find an option where you can set the binary content types:
# Binary content-type handling
# These content-types will be handled by saving the request in a file:
proxy.binary.types=application/x-amf,application/x-java-serialized-object
Now I don't know why application/octet-stream isn't included by default, but you can simply add it to the list, and you are done.
proxy.binary.types=application/x-amf,application/x-java-serialized-object,application/octet-stream
This is how I found it out:
https://issues.apache.org/bugzilla/show_bug.cgi?id=44808
Did a search on JMeter closed bugs... :-)

Someone else is reporting a very similar: http://markmail.org/message/pl5erin2isehm5q6. I can't find any issue related to this problem in their bug tracker though. It looks like you won the privilege to dig deeper in the rabbit hole :)

The accepted answer only allows recording static requests.
This will not be realistic as it will not allow any variabilisation of requests (for example changing the searched word, ...) so you will always be stress testing the same bunch of data.
To make it a real test, you need to use a third party plugin.
A commercial JMeter plugin allows this, see:
http://ubikloadpack.com/
To make your tests realistic, you will need to variabilize content in the serialized objects.
This Java Serialization plugin will allow the following:
Easy recording of traffic with JMeter Proxy Server, a Test Plan using custom Sampler will be created
Easy variabilization of requests (which will appear as XML) through as easy syntax as for example ${searchedWord} where searchedWord can come from a CSV or any user defined variable.
Easy extraction of data from responses using JMeter standard Post Processors
Easy debugging of Request/Responses through standard JMeter View Results Tree element
Disclaimer :I work for this company.