Develop Reference

using KeyStore with .crt

I need to integrate an application with external web service which forces to use https. Authors of this web service provided me with .crt file which I should use for making https requests. After some investigation I've found the following code which uses KeyStore class for secured https access:
KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
FileInputStream instream = new FileInputStream(new File(file));
try {
trustStore.load(instream, password.toCharArray());
} finally {
instream.close();
}
SSLContext sslcontext =
SSLContexts.custom().loadTrustMaterial(trustStore, new TrustSelfSignedStrategy()).build();
SSLConnectionSocketFactory sslsf =
new SSLConnectionSocketFactory(sslcontext, new String[]{"TLSv1.2"}, null,
BROWSER_COMPATIBLE_HOSTNAME_VERIFIER);
HttpClients.custom().setSSLSocketFactory(sslsf).build();
In this code KeyStore requires input stream along with password trustStore.load(instream, password.toCharArray());. However, as I understand, we don't need password when using .crt file. So this way of loading certificate is not suitable for me. At the same time, from what I've discovered so far, the code I provided here is the only way to configure HttpClient for using SSL certificate. Is there any workaround to configure HttpClient for using .crt certificate?
Thanks,
Andrey

I assume the Web service is providing you with a self-signed certificate (i.e. not signed by a well-known CA). If it is already signed by a well-known CA that is in Java's cacerts file, then you don't need to do anything.
Otherwise, you have a couple of options:
Import the certificate into global cacerts keystore
Launch your application with application-specific keystore
In either case, you first need to convert crt file into jks keystore that Java uses. You can do this by:
$ keytool -import -keystore mykeystore.jks -storepass horsestaple
Note that keytool requires the password (horsestaple above) to be supplied for creating a jks store. You can put anything there; as you mention, public website certificates do not need password protection, they are public after all.
If you are doing option 1, make a backup of your cacerts and supply the cacerts file instead of mykeystore.jks. See the link below for the location of cacerts. For this option, you are all set, your application should be connecting to the Web service via HTTPS without any additional configuration, since Java loads cacerts by default.
If you are doing option 2, which is probably preferred at least for testing phase, you need to run your application with this parameter:
-Djavax.net.ssl.trustStore=mykeystore.jks
This is a JVM parameter, so supply it appropriately. This depends on how you are running you application.
Note that you will only have the imported certificate in this case, so your other HTTPS connections will not work. You can avoid this by first copying the standard cacerts to a temporary location, importing the key into it and using that in the command above. That will give you all the standard certificates, plus the one you need.
A slight downside of option 2 is that if new certificates are added or revoked, your application-specific keystore will not be updated. If this is a concern, you can merge keystores on the fly, for example:
Registering multiple keystores in JVM
In either case, you should now be able to just do a standard URL fetch, such as in example given here:
https://blogs.oracle.com/java-platform-group/entry/diagnosing_tls_ssl_and_https
i.e.:
final URL url = new URL("https://example.com");
try(final InputStream in = url.openStream()){
//…
}
More information here:
Keytool and general certificates info
https://docs.oracle.com/cd/E19830-01/819-4712/ablqw/index.html
Cacerts location
http://docs.oracle.com/javase/7/docs/technotes/tools/solaris/keytool.html
More options for importing self-signed certificates
How to properly import a selfsigned certificate into Java keystore that is available to all Java applications by default?

Unless they provided you with their .crt file which should be used for validating connections to them, which would indicate they are using a self-signed certificate, they haven't provided you with anything useful. If you need a certificate as a client, the first thing you need is a public/private key pair, from which you generate a CSR, which you get signed. Nobody else can securely provide any of that except the signed certificate.
If they've provided their own (self-signed?) certificate, you need to load it into a truststore, not a keystore, via the keytool with the -trustcacerts option, and then tell Java to use that truststore, either via the javax.net.ssl.trustStore system property or by constructing your own TrustManager and feeding it to a custom SSLContext, as described in the JSSE Reference Guide.

Multiple certificates in keystore for Mysql SSL Client authentication and JMX over SSL setup

My Java application needs to authenticate to Google cloud Mysql instance with SSL client authentication. Its client-key and certificate are provided by Google. I also need to setup JMX agent with SSL on same application whose certificates are provided by a private CA.
How to prevent Mysql from presenting JMX certificate and vice-versa in case I add both private certificates into single keystore provided to JVM at startup
Is there any other way to authenticate SSL certificates with Mysql beside putting then in 'javax.net.ssl.keyStore'? If not, are there any aliases that Mysql or JMX agent prefer over other names?

You can look at using the Cloud SQL MySQL socket factory which uses temporary SSL certificates to authenticate to Cloud SQL (only supported for Second Generation instances):
https://github.com/GoogleCloudPlatform/cloud-sql-mysql-socket-factory

MySQL Connecting Securely Using SSL
For SSL support to work, you must have the following:
A JDK that includes JSSE (Java Secure Sockets Extension), like JDK-1.4.1 or newer. SSL does not currently work with a JDK that you can add JSSE to, like JDK-1.2.x or JDK-1.3.x due to the following JSSE bug: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=4273544
A MySQL server that supports SSL and has been compiled and configured to do so, which is MySQL 4.0.4 or later. For more information, see Building MySQL with Support for Secure Connections.
A client certificate (covered in this section)
How to work with multiple keystore?
The test certificates reside in keystores named node1.keystore … node100.keystore, which were created following the steps described in Creating Self-Signed Test Certificates.
Export the test certificate for node1.example.com:
$ keytool -exportcert -keystore node1.keystore -alias node1 \
-storepass changeme -file node1.cer
Import the test certificate into the custom truststore:
keytool -importcert -keystore custom.truststore -alias node1 \
-storepass trustchangeme -file node1.cer -noprompt
Here we specify the -noprompt option to suppress the prompt asking
you to confirm that the certificate is trustworthy. Since you
created the certificate yourself, this confirmation is unnecessary.
Repeat Steps 1 and 2 for node2.keystore … node100.keystore.
Resource Link:
Creating Java Keystores and Truststores
Keystore and truststore details:
A keystore is used in one of two distinct ways:
The keystore contains private keys and certificates used by TLS/SSL servers to authenticate themselves to TLS/SSL clients. By convention, such files are referred to as keystores.
When used as a truststore, the file contains certificates of trusted TLS/SSL servers, or of Certificate Authorities trusted to identify servers. There are no private keys in the truststore.
Because keystores contain private keys, while truststores do not, the security requirements for keystores are more stringent. In particular:
Hadoop TLS/SSL requires that truststores and the truststore password be stored, in plaintext, in a configuration file that is readable by all.
Keystore and key passwords are stored, in plaintext, in a file that is readable only by members of the appropriate group.
These considerations should inform your choice of which keys and certificates to store in the keystores and truststores you will deploy across your cluster.
Keystores should contain a minimal set of keys and certificates. A reasonable strategy would be to create a unique keystore for each host, which would contain only the keys and certificates needed by the Hadoop TLS/SSL services running on the host. In most cases, the keystore would contain a single key/certificate entry.
Modifying Keystores: CDH services and processes must be restarted in case changes are made to a keystore. However, this is relatively rare since keystores do not need to be updated when hosts are added or deleted from a cluster.
Because truststores do not contain sensitive information, it is reasonable to create a single truststore for an entire cluster. On a production cluster, such a truststore would often contain a single CA certificate (or certificate chain), since you would typically choose to have all certificates issued by a single CA.
Important: Do not use the same password for truststores and keystores/keys.
Since truststore passwords are stored in the clear in files readable by all, doing so would compromise the security of the private keys in the keystore.

How to load SSL Certificate in Java

I am creating a Java program to get information from a server but I have to perform a ssl handshake with the server from the Java program.
I have myfilercert.cer file certificate for authentication purpose but I have no idea how I can load that certificate in java so that the java program can perform 'handshake' with the server where I want to get information from. Where to begin?

What you need is the java keystore. The keystore is a repository of security certificates used in SSL encryption.
You can read here about the Server Authentication During SSL Handshake. This is a keystore tutorial.
As an alternative to keytool, i would suggest a tool with a Graphical User Interface called Portecle. You can use it to browse the contents of your .cer file and see what's in it.
It can be useful to know about the various certificate encodings. Also read about the X.509 standard.
This is an article on java keytool essentials (which is the oracle tool that works with the java keystore).
You can google and find a lot of resources that instruct you how to generate. I think you will want to keep the certificate at the application level.
Some SO questions that helped me along the way:
Trust Store vs Key Store - creating with keytool - important to know the difference between the trust manager and keymanager
Java HTTPS client certificate authentication
How to export private key from a keystore of self-signed certificate
What is difference between cacerts and keystore
How to connect to a secure website using SSL in Java with a pkcs12 file?
Received fatal alert: handshake_failure through SSLHandshakeException
How to configure trustStore for javax.net.ssl.trustStore on windows?
Good luck!

You can use Apache HttpClient (or just use the required classes from it to use SslContextBuilder, really), and then it'd be like so:
SSLContextBuilder sslContextBuilder = SSLContextBuilder.create();
sslContextBuilder.loadTrustMaterial(new File("yourTrustStore.jks"), "thePassWord");
SSLContext sslContext = sslContextBuilder.build();
HttpsURLConnection httpsURLConnection = (HttpsURLConnection) (new URL("https://thesite.com").openConnection());
httpsURLConnection.setSSLSocketFactory(sslContext.getSocketFactory());
But you need to create a keystore for your certificate, which can be done with keytool. If you need this for android, you'll need SpongyCastle library, and use that as a provider for KeyTool to create a BKS keystore instead of a JKS keystore; and you will need to explicitly open the KeyStore in Java.
KeyStore keyStore = KeyStore.getInstance("BKS",
BouncyCastleProvider.PROVIDER_NAME);
byteArrayInputStream = new ByteArrayInputStream(keyStoreBytes);
keyStore.load(byteArrayInputStream, keyStorePassword);
Certificate[] certificates = keyStore.getCertificateChain("theCertAlias");
Certificate certificate = certificates[0];

When to install keystore & when to install only certificate wrapped in keystore [duplicate]

This question already has answers here:
Truststore and Keystore Definitions
(7 answers)
Closed 6 years ago.
I have a PKCS#12 file which I considered as a keystore file since it contains one key entry & one certificate entry.
In Android, I see people programmatically install keystore in the following way (The code is from Android developer blog):
byte[] keystore = . . (read from a PKCS#12 keystore)
Intent installIntent = KeyChain.createInstallIntent();
installIntent.putExtra(KeyChain.EXTRA_PKCS12, keystore);
startActivityForResult(installIntent, INSTALL_KEYSTORE_CODE);
I also see people programmatically install only the certificate wrapped inside keystore:
Intent intent = KeyChain.createInstallIntent();
intent.putExtra(KeyChain.EXTRA_CERTIFICATE, cert);
startActivity(intent);
Besides, I also see people install both the keystore & the certificate wrapped in keystore. For example, this article shows us how to first install keystore & then install the certificate wrapped in keystore programmatically.
I really get confused about when should I install keystore only & when should I install certificate (wrapped inside keystore) only ? And when should I install both ?? Could someone make me clear about this please?
For example, my keystore PKCS#12 file (mycert.p12) contains key/certificate pair, it is used to connect to VPN server. When should my android client install both keystore and certificate wrapped in the keystore ? When should client install only certificate wrapped in keystore? What are the differences ? I am quite confused about this.

I have a PKCS#12 file which I considered as a keystore file since it contains one key entry & one certificate entry.
Correct.
In Android, I see people programmatically install keystore in the following way ...
This is done when you have a keystore, i.e. a keypair and certificate.
I also see people programmatically install only the certificate wrapped inside keystore
This is done when you have someone else's certificate, typically a self-signed one, that isn't trusted by any of the default CA's (certificate authorities) that are already installed. You should never have to do this.
So note that you never do both with the same certificate, because the cases (the ownerships) are different. There can never be any doubt about which process is appropriate. If it's yours, import the keystore. If it's someone else's, import the certificate.
The ultimate normative reference for all this stuff is ITU Recommendation X.509.
Finally, some notes on the poor quality blog articles you have linked.
From Unifying key store access in ICS:
In the past, it was common practice for apps to maintain their own key
store if they needed to authenticate a secure SSL web server, or
authenticate the user to a server via a client certificate.
This is already incorrect.
To authenticate a web server you shouldn't need anything, if it has a CA-signed certificate. If it has a self-signed certificate you will need to import it into your truststore.
To authenticate yourself to a web server, you need a keystore containing your own private key and a certificate, preferably a CA-signed one. Otherwise the server has to import your self-signed certificate into its truststore, i.e. the converse of (1) above. Don't go down this path. Self-signed certificates are far more trouble than they are worth, which is nothing, as you can tell from the price you pay for them.
From Using ICS keychain API:
We first get the private key and certificate chain using the key alias
and then create and verify a signature to check if the key is actually
usable.
Complete nonsense. We already have the private key, the public key, and the certificate. They are already usable. Creating a signature and verifying it locally is just a complete waste of time.
Installing a CA certificate is not very different from installing a
PKCS#12 file: you load the certificate in a byte array and pass it as
an extra to the install intent.
The difference being that you use KeyChain.EXTRA_CERTIFICATE in the CA certificate case, and KeyChain.EXTRA_PKCS12 in the keystore case.

Since no one has answered you yet, I hope I can at least clarify some points from the blog article you have linked to.
In the past, it was common practice for apps to maintain their own key
store if they needed to authenticate a secure SSL web server, or
authenticate the user to a server via a client certificate.
These are the two basic use-cases right here:
If you are authenticating with a client certificate (which proves to the server that you are an authorized client), then you would only install a certificate.
If you are trying to verify the server's identity, then you will want to validate the server's certificate. In this case you will need a keystore installed (possibly a chain if you're not using self-signed certs). The private key in the keystore will be used to validate the server's certificate.
That second bit of code you have in your question was intended for creating a certificate chain (when you're NOT using self-signed certs):
We first get the private key and certificate chain using the key alias
and then create and verify a signature to check if the key is actually
usable. Since we are using a self-signed certificate the 'chain'
consists of a single entry, but for a certificate signed by a CA you
will need to find the actual end entity certificate in the returned
array.
Installing a CA certificate is not very different from installing a
PKCS#12 file: you load the certificate in a byte array and pass it as
an extra to the install intent.
Intent intent = KeyChain.createInstallIntent();
intent.putExtra(KeyChain.EXTRA_CERTIFICATE, cert);
startActivity(intent);
I hope this explanation helps! :)

With respect to an android or any other 'client', that is, an application, the following hold -
A truststore (public key only) is required whenever it needs to validate the certificate (or a certificate chain) that is sent across by the server during SSL communication (in case of ssl communication the server will always present its certificate to the client).
If the server's certificate is already signed by a trusted certificate authority (implying that certificate is already present in the java-runtime-truststore that can usually be found under $JAVA_HOME/jre/lib/security/cacerts), then this step is not required unless a customized SSLContext is being used (which also means that a customized TrustManager is being used).
For example SO's current certificate is signed by DigiCert identified by SHA1-Thumbprint : 5F:B7:EE:06:33:E2:59:DB:AD:0C:4C:9A:E6:D3:8F:1A:61:C7:DC:25 and would likely be present in the 'cacerts' truststore under the alias 'digicerthighassuranceevrootca'. If a java client were to make a request to https://stackoverflow.com then by default there would not be any specific keystore or truststore required for communication.
A keystore (private and public key) is generally required when the client is required to digitally-sign some data which is being posting to the server.
A common example is xml-signing, you can find a mention here
It is also required if the server expects the client to present its own certificate for authentication as part of two-way ssl handshake. From what I have come across this is not common.
Links :
Two Way SSL
SO's own Keystore and truststore post

Does anyone know how to use StartCom SSL without keystore in Java OR use a custom keystore?

I have found the following post, which I think I may help because I need a way of verifying my website (that my java application connects to) without having to use the java keystore:
http://www.mombu.com/programming/java/t-ssl-for-java-without-keystores-1366416.html
However, I'm quite new to the world of SSL and don't know which files and passwords I need to use, so if anybody can point me in the right direction that would be great. If you didn't guess, I got my SSL certificate from StartSSL, who are ran/owned/something by StartCom Ltd. Here is a post I used for putting the details into the keystore:
https://forum.startcom.org/viewtopic.php?t=1390
Thanks in Advance!
Alternitively, is there any way for me to use a custom keystore. i.e I'll do what I need with the default keystore and then copy the keystore into the .JAR so that I can tell my application to use the one in the .JAR instead of the one in the Java install directory, etc...

Yes, you can create and use your own keystore for your app using the keytool utility which comes with the JDK.
Basically, a keystore is a datastore for your keys & certificates, which will be used in your app either to authenticate yourself to another entity or to digitally sign your messages or any other data.
A distinct keystore called trustore is used to keep the public key certificates of the entities you trust.
You can have these keystores placed in your classpath and specify the path in your app in code like (these are VM params which will affect all apps running in this VM):
Properties properties = System.getProperties();
properties.put("javax.net.ssl.keyStore", path/to/keystorefile);
properties.put("javax.net.ssl.keyStorePassword", keyStorePassword);
properties.put("javax.net.ssl.trustStore", path/to/truststore);
properties.put("javax.net.ssl.trustStorePassword", trustStorePassword);
EDIT:
Yes, you can specify the keystore & trustore locations and their passwords in a property file and load the properties file as myProperties.load(myProperties.getClass().getResourceAsStream("/path/to/proertyfile.properties")); and then use it in your code as (exception handling omitted)(this will not affect any other app):
KeyStore mykeystore = KeyStore.getInstance("JKS");
InputStream is = new FileInputStream("/path/from/myproperties");
mykeystore.load(is, myKeystorePasswordFromProperties.toCharArray());