I wrote a routine for the programmatic retrieval of certificates for SSL connections (to be used for example with restTemplate) and the programmatic import of these certificates to import them into cacerts of the currently used jre.
What I would like to replicate programmatically is the following manual procedure:
keytool -printcert -sslserver {host}:{port} -rfc >> {host}.crt
keytool -importcert -alias {host} -keystore /usr/local/openjdk-8/jre/lib/security/cacerts -file {host} -storepass changeit -noprompt
This is the code I use:
InputStream in = new FileInputStream(keystore);
KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
ks.load(in, passphrase);
in.close();
SSLContext context = SSLContext.getInstance("TLS");
TrustManagerFactory tmf = TrustManagerFactory.getInstance("X509");
tmf.init(ks);
X509TrustManager defaultTrustManager = null; //(X509TrustManager) tmf.getTrustManagers()[0];
TrustManager[] tms = tmf.getTrustManagers();
for (TrustManager tm : tms) {
if (tm instanceof X509TrustManager) {
defaultTrustManager = (X509TrustManager) tm;
break;
}
}
if (defaultTrustManager == null) {
if (isLog) log.error("No Trust Manager found!");
return false;
}
SaveTrustManager tm = new SaveTrustManager(defaultTrustManager);
context.init(null, new TrustManager[]{tm}, null);
SSLSocketFactory factory = context.getSocketFactory();
if (isLog) log.info("Opening connection -> {}:{} ...", host, port);
// Initiate socket
SSLSocket socket = null;
Socket proxiedSocket = null;
if (proxyHost != null) {
if (isLog) log.info("Using Proxy -> {}:{} ...", proxyHost, proxyPort);
try {
if (proxyUser == null || proxyPass == null) {
Proxy prx = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(proxyHost, proxyPort));
proxiedSocket = new Socket(prx);
proxiedSocket.connect(new InetSocketAddress(host, port));
socket = (SSLSocket) factory.createSocket(proxiedSocket, host, port, true);
} else {
String proxyUserPass = String.format("%s:%s", proxyUser, proxyPass);
String proxyConnect = "CONNECT " + host + ":" + port + " HTTP/1.0\r\n"
+ "Proxy-Authorization: Basic "
+ Base64.getEncoder().encodeToString(proxyUserPass.getBytes(ASCII7)).replace("\r\n", "") + "\r\n"
+ "Connection: close\r\n"
+ "\r\n";
if (isLog) log.info("Socket: {}", proxyConnect);
proxiedSocket = new Socket(proxyHost, proxyPort);
proxiedSocket.getOutputStream().write(proxyConnect.getBytes(ASCII7));
proxiedSocket.getOutputStream().flush();
this.readSocketConnection(proxiedSocket);
proxiedSocket.getOutputStream().close();
socket = (SSLSocket) factory.createSocket(proxiedSocket, host, port, true);
}
} catch (Throwable t) {
t.printStackTrace();
if (isLog) log.info("Proxy Error: {}", t.getMessage());
}
} else {
socket = (SSLSocket) factory.createSocket(host, port);
}
socket.setSoTimeout(10000);
try {
if (isLog) log.info("Initiating SSL handshake...");
socket.startHandshake();
socket.close();
if (proxiedSocket != null) proxiedSocket.close();
if (isLog) log.info("Certificate is already trusted...");
} catch (SSLException e) {
//e.printStackTrace(System.out);
success = false;
if (isLog) log.warn("Socket Warning: {}", e.getMessage());
}
X509Certificate[] chain = tm.chain;
if (chain == null) {
log.error("Could not obtain server certificate chain");
return false;
}
if (isLog) log.info("Server sent " + chain.length + " certificate(s):");
MessageDigest sha1 = MessageDigest.getInstance("SHA1");
MessageDigest md5 = MessageDigest.getInstance("MD5");
for (int i = 0; i < chain.length; i++) {
X509Certificate cert = chain[i];
try {
cert.checkValidity();
if (isLog) log.info("\tValid certificate...");
} catch (Throwable t) {
log.info("\tCertificate invalid!");
continue;
}
sha1.update(cert.getEncoded());
md5.update(cert.getEncoded());
if (isLog) {
log.info("\tIndex: {} Subject: {}", (i + 1), cert.getSubjectDN());
log.info("\tIssuer: {}", cert.getIssuerDN());
log.info("\tSHA-1: {}", toHexString(sha1.digest()));
log.info("\tMD5: {}", toHexString(md5.digest()));
}
String alias = host + "-" + (i + 1);
ks.setCertificateEntry(alias, cert);
if (isLog) {
log.info("{}", cert);
log.info("Added certificate to keystore '{}' using alias '{}'", keystore, alias);
}
}
OutputStream out = new FileOutputStream(keystore);
ks.store(out, passphrase);
out.close();
On one particular VM I get this error:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
The weird thing is that locally it works and it works on other VMs too.
Also the other strange thing is that if I convert the downloaded certificate (although the handshake fails) in PEM format (base64) and I compare it with the one exported by keytool it's identical!
Also even if I try to load the downloaded PEM file with keytool using the Keystore programmatic procedure in java, I don't get any errors, but when I go to use a restTemplate to make that call.., I get the same error:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
In particular this VM does not run behind a proxy...
I did a lot of tests and I tried with -Djavax.net.debug=ssl,handshake to get some more information, but unfortunately I didn't come up with it.
I tried importing the PEM programmatically from a keytool downloaded file manually and got the same error.
I tried to verify to reach the portal in https with wget/curl/openssl and I reach it correctly.
There are no proxies behind the VM.
I tried giving full access (chmod) to cacerts file.
The portal only downloads 1 certificate from the chain of the 3 certificates but it is the same one that it downloads with keytool (the PEMs downloaded with keytool and the one downloaded programmatically are identical).
Everything is dockerized and runs under tomcat:9.0.35-jdk8-openjdk.
On other machines and locally it works correctly even behind proxies.
The VM it doesn't work on is an Ubuntu 20 (Debian).
I think I solved the issue.
I have a routine that programmatically downloads certificates from SSL protected servers through a socket that handshake and retrieve certificates.
Currently the certificate SSL server has some problems in returning the whole chain of certificates (including the intermediary certificates of the CAs that signed the certificate).
It is likely that they are problems related to a misconfiguration.
This problem can be verified with the command:
openssl s_client -showcerts -connect host:port
These are the intermediary CAs irrecoverable certs:
certs
certs
These are the intermediary certificates of the CAs:
Intermediate cert CAs chain
On a programmatic level, when JAVA persists the trusted certificates in the Trust-Store, it writes information related to the fact that it was not possible to recover the whole CA chain.
It is as if the certificate is self-signed for the trust-store when it is not. In fact, the certificate is not actually trusted (unless you enter it "manually" in the trust store).
When making an external call to an application-side https server, the client is not fully aware of the certificate.
The error was: "PKIX path building failed".
It was necessary to use a custom built HTTP client to accept single certificates of which it was not possible to download the whole CA chain.
Obviously it is not a client that can accept self-signed certificates.
This client is "satisfied" with the single certificate but it is actually a necessary and sufficient condition to perform a secure SSL handshake.
In other words to make an external call with RestTemplate what you have to do is change it from:
restTemplate = new RestTemplate();
to:
restTemplate = new RestTemplate();
HttpClient httpClient = HttpClients.custom().build();
restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory(httpClient));
I think I have given an explanation. What do you think about it?
Related
I have an ssl connection(2 way handshake) and I am unable to understand the why the following code procedures 400(openJdk 11, p12 file & password provided by the server , cer file provided by the server) ,
I have created the jks file from the cer file via the following command:
keytool -importcert -file example-api.cer -keystore example-api.jks
The code
File keyFile = new File(Objects.requireNonNull(exampleController.class.getClassLoader().
getResource("example-client-api1.p12")).getFile());
File trustFile = new File(Objects.requireNonNull(exampleController.class.getClassLoader().
getResource("example-api.jks")).getFile());
KeyStore keyStore = KeyStore.getInstance("PKCS12");
try(FileInputStream inStream = new FileInputStream(keyFile)) {
keyStore.load(inStream, "password".toCharArray());
}
SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(trustFile, "password".toCharArray() ,new TrustAllStrategy()).
loadKeyMaterial(keyStore , "password".toCharArray()).build();
HostnameVerifier hostnameVerifier = new NoopHostnameVerifier();
SSLConnectionSocketFactory socketFactory =
new SSLConnectionSocketFactory(sslContext, hostnameVerifier);
CloseableHttpClient httpclient = HttpClients.custom()
.setSSLHostnameVerifier(hostnameVerifier)
.setSSLSocketFactory(socketFactory)
.useSystemProperties()
.build();
HttpGet httpget = new HttpGet("https://example-api/link?token=#Secret_Token#");
System.out.println("executing request" + httpget.getRequestLine());
return httpclient.execute(httpget);
The code above always returns 400 (No required SSL certificate was sent).
but the following curl works(on IOS):
curl https://example-api/link?token=#secret_token# --cacert ./example-api-ca.crt --cert ./example-client-api1.p12:password
Any help would be greatly appreciated
OK So after Some frustrating days:
I do not know the reason why but The problem was with the hyphen char ('-') in the hostName of the URL, removing hyphen sign fix the issue , not sure why , but posting it anyway, maybe someone could explain this phenomenon.
Example(using the code above):
example-api -> not working
example.api -> Works OK
I want to secure my SSL connection to the socket. But unfortunately, I have a problem with CA certificate.
javax.net.ssl.SSLHandshakeException: Handshake failed
at com.android.org.conscrypt.OpenSSLEngineImpl.unwrap(OpenSSLEngineImpl.java:441)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:1014)
at com.koushikdutta.async.AsyncSSLSocketWrapper$5.onDataAvailable(AsyncSSLSocketWrapper.java:194)
at com.koushikdutta.async.Util.emitAllData(Util.java:23)
at com.koushikdutta.async.AsyncNetworkSocket.onReadable(AsyncNetworkSocket.java:152)
at com.koushikdutta.async.AsyncServer.runLoop(AsyncServer.java:789)
at com.koushikdutta.async.AsyncServer.run(AsyncServer.java:627)
at com.koushikdutta.async.AsyncServer.access$700(AsyncServer.java:41)
at com.koushikdutta.async.AsyncServer$13.run(AsyncServer.java:569)
Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:337)
at com.android.org.conscrypt.TrustManagerImpl.checkServerTrusted(TrustManagerImpl.java:231)
at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:115)
at com.android.org.conscrypt.OpenSSLEngineImpl.verifyCertificateChain(OpenSSLEngineImpl.java:666)
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake_bio(Native Method)
at com.android.org.conscrypt.OpenSSLEngineImpl.unwrap(OpenSSLEngineImpl.java:426)
... 8 more
Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
My code:
String type = KeyStore.getDefaultType();
KeyStore trustStore = KeyStore.getInstance(type);
trustStore.load(null, null);
trustStore.setCertificateEntry("ca", new CA().getCert());
String tmfAlg = TrustManagerFactory.getDefaultAlgorithm();
TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlg);
tmf.init(trustStore);
try {
SSLContext context = SSLContext.getInstance("TLS");
context.init(null, tmf.getTrustManagers(), null);
SSLEngine engine = context.createSSLEngine();
AsyncSSLSocketWrapper.handshake(socketNormal, url, port, engine, tmf.getTrustManagers(), new NoopHostnameVerifier(), true, (e, socket1) -> {
// ... more
My cert is in my assets catalog:
public X509Certificate getCert() throws CertificateException, IOException {
CertificateFactory certFactory;
certFactory = CertificateFactory.getInstance("X.509");
InputStream inputStream = new BufferedInputStream(context.getAssets().open("pem.pem"));
return (X509Certificate) certFactory.generateCertificate(inputStream);
}
My certificate is signed using a private key and it is self-signed.
Obviously, it is working without cert... but it's not secure.
EDIT:
I have tried to put existing BKS Keystore instead of dynamically adding at run-time:
KeyStore trustStore = KeyStore.getInstance("BKS");
BufferedInputStream is = new BufferedInputStream(c.getAssets().open("key.bks"));
trustStore.load(is, "12345".toCharArray());
Log.i("Cert", "ca " + (new CA().getCert()).getSubjectDN());
String tmfAlg = TrustManagerFactory.getDefaultAlgorithm();
TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlg);
tmf.init(trustStore);
try {
SSLContext context = SSLContext.getInstance("TLS");
context.init(null, tmf.getTrustManagers(), null);
SSLEngine engine = context.createSSLEngine();
AsyncSSLSocketWrapper.handshake(socketNormal, url, port,
engine, tmf.getTrustManagers(), new NoopHostnameVerifier(), true, (e, socket1) -> {
But unfortunetly the error is the same.
EDIT 2:
More info:
Server PHP generates this cert using:
function createSSLCert($pem_file, $pem_passphrase, $pem_dn) {
// //create ssl cert for this scripts life.
//Create private key
$privkey = openssl_pkey_new();
//Create and sign CSR
$cert = openssl_csr_new($pem_dn, $privkey);
$cert = openssl_csr_sign($cert, null, $privkey, 365 * 99);//365
//Generate PEM file
$pem = array();
openssl_x509_export($cert, $pem[0]);
openssl_pkey_export($privkey, $pem[1], $pem_passphrase);
$pem = implode($pem);
//Save PEM file
//echo $pem;
file_put_contents($pem_file, $pem);
//chmod($pem_file, 0600);
}
$pem_passphrase = "XXXXX"; //Set a password here
$pem_file = "cert.pem"; //Set a path/filename for the PEM SSL Certificate which will be created.
//The following array of data is needed to generate the SSL Cert
$pem_dn = array(
"countryName" => "PL", //Set your country name
"localityName" => "City", //Ser your city name
"organizationName" => "Firm name", //Set your company name
"commonName" => "CN", //Set your full hostname.
"emailAddress" => "admin#email.pl" //Set your email address
);
//create ssl cert for this scripts life.
$this->createSSLCert($pem_file, $pem_passphrase, $pem_dn);
Keystore was created using Portecle tool like:
New -> BKS -> Import Trusted Certificate -> My *pem file -> save
openssl s_client -debug -connect 10.100.0.24:5678 return:
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1630 bytes and written 451 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Verify return code: 18 (self signed certificate)
plus some private informations about cert
ACRA set up with standard options:
#ReportsCrashes(
formUri = "https://XXXXXXXXXX.php",
mode = ReportingInteractionMode.TOAST,
resToastText = R.string.str_acra_crash_report_info)
Tried to copy the server certificate to assets and create a custom KeyStore:
try {
KeyStore ksTrust = KeyStore.getInstance("BKS");
InputStream instream = new BufferedInputStream(getAssets().open("keystore.bks"));
ksTrust.load(instream, "ez24get".toCharArray());
ConfigurationBuilder configurationBuilder = new ConfigurationBuilder(this);
configurationBuilder.setKeyStore(ksTrust);
final ACRAConfiguration config = configurationBuilder.build();
ACRA.init(this, config);
} catch (KeyStoreException | IOException | NoSuchAlgorithmException | CertificateException e) {
e.printStackTrace();
}
or another way:
CertificateFactory cf = CertificateFactory.getInstance("X.509");
InputStream caInput = new BufferedInputStream(getAssets().open("ssl-cert-snakeoil.pem"));
Certificate ca = cf.generateCertificate(caInput);
String keyStoreType = KeyStore.getDefaultType();
KeyStore keyStore = KeyStore.getInstance(keyStoreType);
Unfortunately after hours of tests, still no luck, still getting an exception:
java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
Any hints?
EDIT: Created another certificate, with CA:TRUE (standard ssl-cert-snakeoil.pem had CA:FALSE), but still no luck.
EDIT 2: Certificates made as they should be: main CA cert. + server cert., but still the same exception.
#Matthew you will need to use the head of the ACRA's master as it has this https://github.com/ACRA/acra/pull/388 pull request added.
We'll probably cut another release within a week or so.
I am developing a client GUI that accepts self signed server certificates and adds them to the trust store just like any browser would do.
The problem is that my client application asks for the certificate every time it is started, in other words it does not remember that the certificate is already in the trust store. How do I implement this?
This is how I am writing my trust store files:
public void WriteTrustStore(String alias, X509Certificate c){
char[] password = "changeit".toCharArray();
char SEP = File.separatorChar;
keystoreFile = new File(System.getProperty("java.home")
+ SEP + "lib" + SEP + "security" + SEP + "cacerts");
try {
setTrustStore(trustStore);
FileInputStream in = new FileInputStream(keystoreFile);
trustStore.load(in, password);
in.close();
trustStore.setCertificateEntry(alias, c);
FileOutputStream out = new FileOutputStream(keystoreFile);
trustStore.store(out, password);
out.close();
} catch (KeyStoreException | NoSuchAlgorithmException | CertificateException | IOException e) {
e.printStackTrace();
}
}
Then I have another method where I am initializing my SSL Context and also creating dynamic alias names by doing something like:
string alias = getHostname() + "-" + getPortname();
In the end I have an alias name like:
"myhost-5001"
And then I call the WriteTrustStore(alias,certificate) method.
But in the next execution run of the program if I try to find the certificate with this alias name, I always end up getting a Null Pointer Exception.
I know that a truststore file has a property like:
trustStore.containsAlias(alias)
I tried,
if(trustStore.containsAlias(alias) == false){
WriteTrustStore(alias, (X509Certificate) cert)
}
else {
System.out.Println("Certificate already in trust store!");
}
But still I get a Null-Pointer exception. And also I know that the certificate with alias name myhost-5001 is in the Java trust store, I crossed checked using keytool and portecle.
Thanks for your help!
I figured it out, there are two ways this can be done.
First Method
I found this here: Check for trusted certificates, where you enumerate over aliases like this:
Enumeration en = keystore.aliases();
String ALIAS = "" ;
X509Certificate signingcert = null;
while (en.hasMoreElements())
{
X509Certificate storecert = null;
String ali = (String)en.nextElement() ;
if(keystore.isCertificateEntry(ali))
{
storecert = (X509Certificate)keystore.getCertificate(ali);
if( (storecert.getIssuerDN().getName()).equals(issuerdn))
{
try{
System.out.println("Found matching issuer DN cert in keystore:\r\nChecking signature on cert ...") ;
cert.verify(storecert.getPublicKey()) ;
System.out.println("Signature verified on certificate") ;
signingcert = storecert;
break;
}
catch(Exception exc){
System.out.println("Failed to verify signature on certificate with matching cert DN");
}
}
}
else
if(keystore.isKeyEntry(ali))
System.out.println(ali + " **** key entry ****");
}
Second Method
Just create a duplicate certificate that looks in the trust store for a certificate with the alias name you are passing.
X509Certificate DuplicateCert = (X509Certificate) trustStore.getCertificate(alias);
First method is safer because you are also looking at the Issuer DN but takes longer, second method is simple and shorter.
Second method works like a charm for me, you can find the complete GUI code review here and see how I am using it: JAX-WS client SSL code review
I am in the process of learning SSL and in this process I am attempting to create an SSL connection between a .NET server with a Java client. I use a self signed certificate for this purpose. I don't want to use the standard keystore in Java so I create a custom keystore and load that instead.
I use the following steps to generate the certificate and the pfx file to use at the .NET server end.
Generated a cerficiate using the following command on windows.
makecert -r -pe -sr "localhost" -$ individual -n "CN=localhost" -sv
.pvk -r localhost.cer
Converted this to a .pfx so that I can load this certificate on the .NET server app.
Exported the .cer file as a .pem (Base64 format).
Took the .cer file (the public component of the above mentioned certificate) and created a .jks file (JavaKeyStore) to use as the java client using the following command.
keytool \
-import \
-v \
-trustcacerts \
-alias 0 \
-file <(openssl x509 -in localhost.pem) \
-keystore mystore.jks\
-storetype JKS\
-storepass ez24get
Loaded this .jks in the Java client app and initiated the connection with the following code
FileInputStream fis = new FileInputStream("res/myjksstore.jks");
KeyStore trusted = KeyStore.getInstance("JKS");
trusted.load(fis, "ez24get".toCharArray());
TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(trusted);
SSLContext context = SSLContext.getInstance("SSL");
context.init(null, trustManagerFactory.getTrustManagers(), new SecureRandom());
Socket socket = context.getSocketFactory().createSocket("localhost", 443);
String str = "abc123";
socket.getOutputStream().write(GeneralUtil.toByta(str.length()));
socket.getOutputStream().write(str.getBytes());
socket.setKeepAlive(true);
But when I try to write data to the socket, I get the following error at the server end
System.ComponentModel.Win32Exception: The client and server cannot communicate, because they do not possess a common algorithm
My server code looks as follows...
X509Certificate cert = new X509Certificate("localhost.pfx", "abc123");
TcpListener listener = new TcpListener(IPAddress.Loopback, 443);
listener.Start();
while (true) {
try { TcpClient tcpClient = listener.AcceptTcpClient();
NetworkStream networkStream = tcpClient.GetStream();
SslStream sslStream = new SslStream(networkStream);
sslStream.AuthenticateAsServer(cert, false, SslProtocols.Default,
false);
byte[] data1 = new byte[4];
sslStream.Read(data1, 0, data1.Length);
int len = BitConverter.ToInt32(data1, 0);
String message = "Length of incoming data " +
BitConverter.ToInt32(data1, 0);
byte[] data2 = new byte[len];
sslStream.Read(data2, 0, data2.Length);
message += " Message: " + ASCIIEncoding.ASCII.GetString(data2);
Thread.Sleep(1000);
} catch (Exception ex)
{
}
}
The exception occurs at the line
sslStream.AuthenticateAsServer(cert, false, SslProtocols.Default, false);
What could be the reason for this and how can I fix it ?
Any help would be highly appreciated.
SSLEngine (obtained from SSLContext) must be set to client mode (setUseClientMode)