Develop Reference

How to establish TLS connection between stunnel and Android app with my own certs from my CA

I have stunnel running on my server with the following configuration:
[myservice]
accept = 12345
connect = 9999
verifyPeer = yes
cert = /etc/stunnel/stunnel.pem
CAfile = /etc/stunnel/androidApp.crt
Both cert and CAfile has been issued by the same private CA.
I want to achieve a secure communication between stunnel (on port 12345) and my Android application. Moreover, I want stunnel to verify the peer (that its certificate has been issued by the same CA as the stunnel's one) and on the other hand, the Android application should also verify the identity of the stunnel (server) part.
In my application I have the following code
// ...
InputStream caInputStream = ctx.getResources().openRawResource(R.raw.android_app); //PKCS12
KeyStore keyStore;
KeyManagerFactory keyManagerFactory;
SSLContext sslContext;
SSLSocketFactory sslSocketFactory;
Socket socket;
keyStore = KeyStore.getInstance("PKCS12");
keyStore.load(caInputStream, "password".toCharArray());
keyManagerFactory = KeyManagerFactory.getInstance("X509");
keyManagerFactory.init(keyStore, "password".toCharArray());
sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManagerFactory.getKeyManagers(), null, SecureRandom.getInstance("SHA1PRNG"));
sslSocketFactory = sslContext.getSocketFactory();
socket = sslSocketFactory.createSocket("hostname", 12345);
// ...
When the socket is created, I get the following logs from stunnel:
2021.05.13 17:01:21 LOG5[2]: Service [myservice] accepted connection from XXX.XXX.XXX.XXX:YYYYY
2021.05.13 17:01:21 LOG6[2]: Peer certificate required
2021.05.13 17:01:25 LOG3[2]: SSL_accept: 1417C0C7: error:1417C0C7:SSL routines:tls_process_client_certificate:peer did not return a certificate
2021.05.13 17:01:25 LOG5[2]: Connection reset: 0 byte(s) sent to TLS, 0 byte(s) sent to socket
At this stage I am fully aware that I am doing something fundamentally wrong (like I do not send the peer certificate), but I am a bit confused how to do that. Could you please give me a hand with this?
Cheers

This is an assumption, but it looks like the PKCS12 file you are opening does not contain a private key.
Add private key -> create CSR -> sign with CA -> import chain to key store.
Everything else looks in order.

connect android device via MQTT with ssl

I have created an MQTT Broker and a client in java. It works perfectly using SSL too.
With a broker server and a client both written in java using paho libs, enabling SSL is easy. We need just to swich the protocol in the url from tcp to ssl IE: ssl://.messaging.internetofthings.ibmcloud.com:8883 and setting some props in te src code:
java.util.Properties sslClientProps = new java.util.Properties();
sslClientProps.setProperty("com.ibm.ssl.protocol", "TLSv1.2");
options.setSSLProperties(sslClientProps);
on the SSLContext
SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
sslContext.init(null, null, null);
Than a secure and encrypted connection is created (checked sniffing the packets with WireShark).
Using a specific CA trusted certificate works well too (messaging.pem file)
CertificateFactory cf = CertificateFactory.getInstance("X.509");
InputStream certFile = MqttHandler.class.getResourceAsStream("messaging.pem");
Certificate ca = cf.generateCertificate(certFile);
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
keyStore.load(null, null);
keyStore.setCertificateEntry("ca", ca);
TrustManager trustManager = TrustManagerUtils.getDefaultTrustManager(keyStore);
SSLContext sslContext = SSLContextUtils.createSSLContext("TLSv1.2", null, trustManager);
options.setSocketFactory(sslContext.getSocketFactory());
I need to use an Android Client and a custom MQTT Java Server Broker (without using SSL, MQTT publish and subscribe works well from Android too).
The trouble seems related with the creation of the SSLSocketFactory from Android.
I did this tests:
1) Setting SSL props (as I did in the src of the java client as reported above)
2) passing the CA trusted certificate on Android client (as I did in the src of the java client as reported above)
3) generate the key store from the trusted CA in BouncyCastle format (compatible with Android) as reported here http://rijware.com/accessing-a-secure-mqtt-broker-with-android/ and pass the key store on Android client:
CertificateFactory cf = CertificateFactory.getInstance("X.509");
InputStream keyStoreFile = getAssets().open("raw_key_file");
//keystore trusted
KeyStore keystoreTrust = KeyStore.getInstance("BKS");//Bouncy Castle format for Android
keystoreTrust.load(keyStoreFile, "mykeystorepassword".toCharArray());
TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(keystoreTrust);
SLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, trustManagerFactory.getTrustManagers(), new SecureRandom());
options.setSocketFactory(sslContext.getSocketFactory());
4) using local trust store (CA) and client certificate:
// use local trust store (CA)
TrustManagerFactory tmf;
CertificateFactory cf = CertificateFactory.getInstance("X.509");
InputStream srvIn = getAssets().open("messaging.pem");
Certificate ca = cf.generateCertificate(srvIn);
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
keyStore.load(null, null);
keyStore.setCertificateEntry("ca", ca);
tmf = TrustManagerFactory.getInstance("X509");
tmf.init(keyStore);
// load client certificate
KeyStore clientKeyStore = null;
InputStream clIn = getAssets().open("raw_key_file");
clientKeyStore = KeyStore.getInstance("BKS");
clientKeyStore.load(clIn, "mykeystorepassword".toCharArray());
KeyManagerFactory kmf = KeyManagerFactory.getInstance("X509");
kmf.init(clientKeyStore, "mykeystorepassword".toCharArray());
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
options.setSocketFactory(sslContext.getSocketFactory());
Unfortunally with all the tests I still getting this error: MqttException (0) - javax.net.ssl.SSLException: Connection closed by peer
MqttException (0) - javax.net.ssl.SSLException: Connection closed by peer
at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:38)
org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:604)
at java.lang.Thread.run(Thread.java:841)
Caused by: javax.net.ssl.SSLException: Connection closed by peer
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)
at com.android.org.conscrypt.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:405)
at org.eclipse.paho.client.mqttv3.internal.SSLNetworkModule.start(SSLNetworkModule.java:89)
at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:590)
Maybe I still confusing with the certificates process. How to create these from scratch and using on server and client side (creating the keystore in BouncyCastle format compatible with Android) ?
Or Maybe I did something wrong using the SSLSocketFactory classes in Android...
Thanks, any suggestion is really approciated.

Java JSSE TLS - Is this connection safely encrypted in both directions?

In Java using JSSE with TLS. I have created a secure socket between the server and client. After finally getting the sockets to connect securely, I still have a fundamental question about my existing code's security. I followed instructions in a tutorial, and sometimes the documentation in the JavaDoc is very precise but a little vague unless you speak the Swaheli dialect of Jargon....
I have been network programming for quite awhile now in C++. The transition to Java was easy. Recently, however, I have found it prudent to make the traffic secure. This being said:
I want to create a secure socket in the same way a web browser creates a secure socket, so traffic in both direction is encrypted. A client can see their personal account information sent from the server (very bad if intercepted), and a client can send their username and password to the server securely (also very bad if intercepted).
I know all about how public key cryptography works, but there's a side effect to public key cryptography alone. You send your public key to a client, the client encrypts with the public key, and sends data to the server only the server can decrypt. Now from what I understand, the server uses the private key to encrypt messages going to the client, and another layer of security needs to be added to prevent anyone with the public key from being able to decrypt it.
I have a public / private key pair stored in files public.key and private.key (i made these using JSSE's keytool utility
I included public.key in the client
I included private.key in the server
Client Class:
KeyStore keyStore;
TrustManagerFactory tmf;
KeyManagerFactory kmf;
SSLContext sslContext;
SecureRandom secureRandom = new SecureRandom();
secureRandom.nextInt();
keyStore = KeyStore.getInstance("JKS");
keyStore.load(this.getClass().getClassLoader().getResourceAsStream("server.public"),"public".toCharArray());
tmf = TrustManagerFactory.getInstance("SunX509");
tmf.init(keyStore);
kmf = KeyManagerFactory.getInstance("SunX509");
kmf.init(keyStore, "public".toCharArray());
sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), secureRandom);
SSLSocketFactory sslsocketfactory = sslContext.getSocketFactory();
SSLSocket sslsocket = (SSLSocket)sslsocketfactory.createSocket("localhost", 9999);
Server Class:
String passphrase = "secret"
KeyStore keyStore;
TrustManagerFactory tmf;
KeyManagerFactory kmf;
SSLContext sslContext;
SecureRandom secureRandom = new SecureRandom();
secureRandom.nextInt();
keyStore = KeyStore.getInstance("JKS");
keyStore.load(this.getClass().getClassLoader().getResourceAsStream("server.private"),passphrase.toCharArray());
tmf = TrustManagerFactory.getInstance("SunX509");
tmf.init(keyStore);
kmf = KeyManagerFactory.getInstance("SunX509");
kmf.init(keyStore, passphrase.toCharArray());
sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), secureRandom);
SSLServerSocketFactory sslserversocketfactory = sslContext.getServerSocketFactory();
SSLServerSocket sslserversocket =
(SSLServerSocket)sslserversocketfactory.createServerSocket(9999);
/ ******* THE QUESTION ********/
Everything works! I attach the sockets to BufferedReader and BufferedWriter and begin talking beautifully back and forth after accept(); ing the connection from the client and starting my client and server send / receive loops.
Now I know that at this point client to server communication is secure. Only the server key can decrypt traffic coming from the client. But what about server to client communication? The client's key can decrypt messages coming from the server, but in Public Key Crypto 101 you learn that the client is now supposed to send a public key to the server. Is this happening behind the scenes in this code? Did SSLContext take care of this? Or now that I have an encrypted connection from the client to the server, am I now expected to generate a private/public key pair for the client as well?
Let me know if the traffic being sent and received in the above code is actually secure in both directions.

The certificates (and their private keys) in SSL/TLS are only used for authenticating the parties in SSL/TLS (often, only the server uses a certificate).
The actual encryption is done using shared/symmetric keys that are negotiated during the handshake, derived from the pre master key exchanged using a form of authenticated key exchange (see TLS Specification, Section F.1.1.
How this authenticated key exchange is done depends on the cipher suite, but the end result is the same: a shared pre-master secret between the two parties, guaranteed to be known only to the client and the server with the private key for its certificate.
Following that pre-master secret exchange, the master secret itself is calculated, from which a pair of secret keys is derived (as described in the Key Calculation section): one for the client to write (and for the server to read) and one for the server to write (and for the client to read). (MAC secrets are also generated, to guarantee the connection integrity.)
In principle, not all cipher suites provide encryption and authenticated key exchange (see Cipher Suite Definitions section), but all those enabled by default in JSSE with the SunJSSE provider do (see Cipher Suite tables in the SunJSSE provider documentation). In short, don't enable cipher suites with anon or NULL in their names.
Regarding your code:
There are multiple example of code around that fix the Key/TrustManagerFactory algorithm like this ("SunX509"). This is typically code that hard-codes the Java 1.4 defaults. Since Java 5, the default TMF algorithm is PKIX (see Customization section of the JSSE Reference Guide). The best way to get around this is to use TrustManagerFactory.getDefaultAlgorithm() (same for KMF), which will also allow your code to run on other JREs that don't support SunX509 (e.g. IBM's).
Since you're not using client-certificate authentication, there's no point having a KeyManagerFactory on the client side. Your initialising it with a keystore that probably doesn't have a private key anyway, which makes it pointless. You might as well use sslContext.init(null, tmf.getTrustManagers(), null). (Same thing for the secure random in both cases, let the JSSE use its default values.)

You do have an understanding of how PKI works, but you are missing two key pieces of SSL implementation. First most PKI algorithms allow for encrypting traffic both ways. You can send encrypt message using public key and only whoever has a private key can read it, this is called encryption. You can also encrypt the message using a private key and anybody who has a public key can decrypt it, this is called digital signature.
Another missing piece is that SSL doesn't use PKI to send network traffic between client and server. It uses symmetric encryption algorithm. However the key for the symmetric encryption (called session key) is establish using rather complicated challenge-response protocol that employs PKI, and certificates. During this phase server proves to the client that it is not man in in the middle, client can optionally proved it's certificate to the server if it has any for stronger authentication, and the symmetric session key is established. More details are here https://www.rfc-editor.org/rfc/rfc5246
The symmetric key is used for encrypting traffic using algorithms like RC5 or AES

How can I effect 'known_hosts' vs self-certified servers for my app's SSL usage?

Oh wise and noble Oracle,
I'm adding SSL to a TCP client I've written on my Android phone. I can
successfully connect to servers with properly signed certificates, and I can
connect to self-certifying hosts by cooking up a TrustManager implementation
that always thinks everything is fine.
I now have a decorator TrustManager capturing the certificates (before
delegating to its decoratee) for self-certifying hosts and presenting them for
my breathless perusal, but what I can't work out is how to implement ssh's
behaviour of warning that a host is unknown and offering to remember it for
next time - and doing so.
I presumed all I needed was to store the public key - as ssh does with
known_hosts - and re-represent it, but with this code and 'sslTrust' holding
the public key:
TrustManagerFactory tmf = TrustManagerFactory.getInstance(
TrustManagerFactory.getDefaultAlgorithm());
KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
ks.load(null, null); // initialise!
ks.setKeyEntry("dbentry", Base64.decode(sslTrust, Base64.NO_WRAP), null);
tmf.init(ks);
tms = tmf.getTrustManagers();
ss.stm = new SnoopyTrustManager((X509TrustManager) tms[0]);
// ...
SLContext context = SSLContext.getInstance("SSL");
context.init(null, new TrustManager[] { ss.stm } , null);
ss.factory = context.getSocketFactory();
// ...
SocketFactory factory = ss.getFactory();
mSocket = factory.createSocket(host, port);
attempting to establish a connection results in
SSLHandshakeException: InvalidAlgorithmParameterException: trustAnchors.isEmpty()
which is fair enough: I don't know how to cook things up from the certificate
offered by the remote server. I'm also fairly sure this isn't how I tell a
TrustManager about a remote server's public key anyway.
Since the site is self-certifying, I imagine could probably just verify that
the public keys match in a trivial TrustManager, but I'd like to understand
how this 'should' be done - adding a CA on a per-connection basis, since
I won't trust that CA for anything else.

You need to use your own trust store on pre-ICS version, and add the serer's certificates to it on first error. Subsequent connections will load it from the trust store and thus trust the remote certificate. This is not a complete solution, but here's one way to do it (code on Github), along with some discussion:
http://nelenkov.blogspot.jp/2011/12/using-custom-certificate-trust-store-on.html

using Certificate in https

I write android application.
How can I use Certificate in https connection when I initialize certificate from directory file and not from packages?
When I have packages file with password, this code works:
KeyStore keyStore = KeyStore.getInstance("PKCS12");
keyStore.load(certificateIs, pass.toCharArray());
KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(keyStore, pass.toCharArray());
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(kmf.getKeyManagers(), trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
But I have certificate initialized from der file:
CertificateFactory cf = CertificateFactory.getInstance("X.509");
X509Certificate certificate = (X509Certificate) cf.generateCertificate(certBytes);
I do not know how use this certificate over https connection.

You seem to be talking about client-certificate authentication (where your Android device is the client).
Firstly, you need the client to have the private key matching the public key in the certificate you're trying to use (that's the whole point, otherwise, it wouldn't authenticated anything). PKCS#12 is one of the usual formats for containing the private key and the certificate. If you only have the certificate in a der file, you probably won't have the private key in it, hence it won't work.
It's not quite clear from your question what you do with your certificate variable, with respect to the KeyManagerFactory (if you have a custom X509KeyManager, it should return the private key in its getPrivateKey method, otherwise it won't work).
Secondly, client-certificate authentication is always initiated by the server, so you'd need the server to be set up accordingly too (it seems to be the case already, if your test based on a PKCS#12 keystore works).