Develop Reference

Securing TCP traffic and certificate pinning?

We develop a Java application that serves requests over TCP. We also develop client libraries for the application, where each library supports a different language/platform (Java, .NET, etc.).
Until recently, TCP traffic was confined to a secure network. To support usage on an insecure network we implemented TLS using the recipes in java-plain-and-tls-socket-examples. There are recipes here for both server and client, and a script to generate an X.509 certificate. Below is a summary of the recipe for TLS with server-only authentication:
Create an X.509 root certificate that is self-signed.
Configure the server with a keystore file that contains the certificate's identifying data plus the public and private keys.
Configure the client with a trust store file that contains the same identifying data plus only the public key.
When connecting, the client validates the certificate received from the server by comparing its identifying data with the corresponding data in the client's trust store. (This looks like certificate pinning.)
We assume for now that this approach is valid for securing TCP traffic. Signing by a certificate authority seems unnecessary because we control both server and client.
Initial testing shows that the implementation is working in our Java server and Java client:
Client accepts a server certificate that matches data in the client's trust store.
Client rejects a non-matching server certificate.
TCP packets captured by tcpdump contain encrypted data.
.NET Client
We use SslStream to encrypt TCP traffic. As the documentation suggests, we do not specify a TLS version; instead we throw an exception if the version is below 1.2.
We're not confident about how to use X509Chain.ChainPolicy.CustomTrustStore correctly, because the documentation omits information like use cases for this type, and for option types like X509KeyStorageFlags and X509VerificationFlags.
The code below aims to mimic the recipe outlined above, i.e. configure a trust store data structure for the client to use when validating a server certificate. This approach seems equivalent to importing the certificate into the operating system's trust store.
// Import the trust store.
private X509Certificate2Collection GetCertificates(string storePath, string storePassword)
{
byte[] bytes = File.ReadAllBytes(storePath);
var result = new X509Certificate2Collection();
result.Import(bytes, storePassword, X509KeyStorageFlags.EphemeralKeySet);
return result;
}
// Callback function to validate a certificate received from the server.
// fCertificates stores the result of function GetCertificates.
private bool ValidateServerCertificate(
object sender,
X509Certificate certificate,
X509Chain chain,
SslPolicyErrors sslPolicyErrors)
{
// Do not allow this client to communicate with unauthenticated servers.
//
// With a self-signed certficate, sslPolicyErrors should be always equal to
// SslPolicyErrors.RemoteCertificateChainErrors.
var result = (SslPolicyErrors.RemoteCertificateChainErrors == sslPolicyErrors);
if (result)
{
// The values below are default values: set them to be explicit.
chain.ChainPolicy.VerificationFlags = X509VerificationFlags.NoFlag;
chain.ChainPolicy.RevocationMode = X509RevocationMode.NoCheck;
chain.ChainPolicy.TrustMode = X509ChainTrustMode.CustomRootTrust;
chain.ChainPolicy.CustomTrustStore.AddRange(fCertificates);
result = chain.Build((X509Certificate2)certificate);
}
return result;
}
// Initialize SslStream.
private SslStream GetStream(TcpClient tcpClient, string targetHost)
{
SslStream sslStream = new SslStream(
tcpClient.GetStream(),
false,
new RemoteCertificateValidationCallback(ValidateServerCertificate),
null
);
try
{
sslStream.AuthenticateAsClient(targetHost);
// require TLS 1.2 or higher
if (sslStream.SslProtocol < SslProtocols.Tls12)
{
throw new AuthenticationException($"The SSL protocol ({sslStream.SslProtocol}) must be {SslProtocols.Tls12} or higher.");
}
}
catch (AuthenticationException caught)
{
sslStream.Dispose();
throw caught;
}
return sslStream;
}
Initial testing has yielded results that vary depending on the operating system:
When deployed to Ubuntu on WSL2, this code:
Accepts a valid server certificate.
Rejects an invalid server certificate.
Encrypts TCP packets.
Automatically uses TLS 1.3.
Given a valid server certificate, the callback function argument sslPolicyErrors is equal to SslPolicyErrors.RemoteCertificateChainErrors (expected).
When deployed to MacOS:
This code automatically uses TLS 1.2.
Given a valid server certificate, the callback function argument sslPolicyErrors includes these values:
SslPolicyErrors.RemoteCertificateNameMismatch (unexpected).
SslPolicyErrors.RemoteCertificateChainErrors (expected).
Questions
In what ways could security be compromised with this .NET code?
Upon reviewing discussions about "certificate name mismatch" (see SslPolicyErrors.RemoteCertificateNameMismatch above), it seems that our server certificate should include a subjectAltName field to specify allowed DNS names. Is this necessary, or would it be reasonable, as we are using certificate pinning, to ignore sslPolicyErrors when validating the server certificate?

I can't answer your specific questions but here is some thoughts:
You have not mentioned anything about how the server authenticates clients. So you might consider implementing something like client certificates. If you control both you probably want some way to ensure random attackers cannot connect.
You might consider creating a threat model. In many cases it is the things that you haven't thought about that cause problems.
If you are handling national security data or financial data you might want an external audit. Such might even be required in some cases.
If there is no way an attacker could sell, use, or ransom the data, then you will probably not be directly targeted. So you might worry more about mass attacks against known vulnerabilities, i.e. keep all your software up to date.
Consider other ways to mitigate risks. Are your server/client running in least privilege possible? Are you using a DMZ? Are firewalls correctly configured? Are credentials for support etc well managed?

How to check whether Android phone supports TEE?

I have read this two posts: One and Two, but I still have question.
I use KeyStore (Android 9) to generate an AES key, and use isInsideSecureHardware() method to check whether the key isInsideSecureHardware. I got return False. Sample code can be found here, and here.
public boolean isInsideSecureHardware ()
Returns true if the key resides inside secure hardware (e.g., Trusted Execution Environment (TEE) or Secure Element (SE)). Key material of such keys is available in plaintext only inside the secure hardware and is not exposed outside of it.
Thus, I want to further confirm whether my phone device (Huawei P20) supports TEE.
Question:
If the phone supports TEE, the key generated by KeyStore will be store into TEE automatically? Do I Need any manually configuration in Java? I heard that keys will be automatically stored in TEE, as long as you use KeyStore.getInstance(), KeyGenerator
.getInstance(algorithm, KeyStore Name). But I am not sure this is True or Not?
If the answer of Q1 is "Need manually configuration", it becomes the reason of isInsideSecureHardware() returns False, right? If the answer of Q1 is "automatically", ignore Q2.
Any method to directly check whether the phone supports TEE, in Java?

#JensV is correct: if you set setIsStrongBoxBacked on the keyGenParameterSpecBuilder, key generation will fail with a StrongBoxUnavailableException if StrongBox is not supported. However, the intermediate case - where there is a TEE (i.e. keys are generated and used within secure HW), but no support for StrongBox - is more tricky to discern.
In general, the way to go is to actually generate a key on the device, and then perform HW key attestation on it at the server - consulting the signed key properties to examine the exact degree of HW backing:
generate a nonce (random byte string) ON The SERVER, pass it to the device
generate a key on the device, requesting HW attestation by calling setAttestationChallenge on the KeyGenParameterSpec builder and passing in the nonce you get from the server (DO NOT USE A NONCE PRODUCED ON THE DEVICE)
request the attestation chain for the key from the Android Key Store
pass the attestation data (cert chain) to your server
verify the attestation (signature) chain on your server
confirm that the root cert matches a published Google root cert
confirm that no cert in the chain hasn been revoked (check against CRL # https://android.googleapis.com/attestation/status)
examine the properties of the Google Key Attestation extension (OID 1.3.6.1.4.1.11129.2.1.17) of the leaf cert
confirm the nonce matches (attestationChallenge)
consult the attestationSecurityLevel of KeyDescription
SecurityLevel ::= ENUMERATED {
Software (0),
TrustedEnvironment (1),
StrongBox (2),
}
TrustedEnvironment and StrongBox both correspond to hardware-backed keys and crypto operations.

From the Android keystore system docs:
Supported devices running Android 9 (API level 28) or higher installed can have a StrongBox Keymaster, an implementation of the Keymaster HAL that resides in a hardware security module. The module contains the following:
[...]
* Secure storage.
[...]
When checking keys stored in the StrongBox Keymaster, the system corroborates a key's integrity with the Trusted Execution Environment (TEE).
[...]
When generating or importing keys using the KeyStore class, you indicate a preference for storing the key in the StrongBox Keymaster by passing true to the setIsStrongBoxBacked() method.
In my understanding that means when you generate a Key and call keyGenParameterSpecBuilder.setIsStrongBoxBacked(true) for the key configuration you can ensure that it's backed by a TEE. If there is no TEE available, it'll throw a StrongBoxUnavailableException.
So to check if there's a TEE available you could just attempt to generate a key this way and see if it works.

Android - validating self-signed certificates in addition to normal SSL certificates

I have an Android application that calls web services over SSL. In production we will have normal SSL certificates that are signed by a trusted CA. However, we need to be able to support self-signed certificates (signed by our own CA).
I have successfully implemented the suggested solution of accepting self-signed certificates but this will not work due to the risk of man in the middle attacks. I then created a trustmanager that validates that the certificate chain was in fact signed by our CA.
The problem is I have to bypass the normal SSL validation - the application will now only speak to a server that has one of our self-signed certificates installed.
I am a bit lost, I've googled extensively but can't find anything. I was hoping to find a way of programmatically adding our CA to the trust store on the device as this would be the least intrusive way of dealing with the issue.
What I want to achieve:
1. Full standard support for normal SSL certificates.
2. Additional support for self-signed certificates signed by our own CA.
Any advice?

You haven't posted any code, so I can't be sure what you actually did. However, I'll assume that you're setting up a SSLContext using only your custom X509TrustManager subclass. That's fine, but what you can do is have your custom trust manager implementation also chain to the built-in trust managers. You can do this while setting up your trust manager; something like this should work:
private List<X509TrustManager> trustManagers = new ArrayList<X509TrustManager>();
public MyCustomTrustManager() {
TrustManagerFactory tmFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmFactory.init((KeyStore)null);
for (TrustManager tm : tmFactory.getTrustManagers()) {
if (tm instanceof X509TrustManager)
trustManagers.add((X509TrustManager)tm);
}
}
So now your custom trust manager has a list of all the built-in trust managers. In your override of checkServerTrusted(), you'll want to loop through the built-in trust managers and check each one by calling checkServerTrusted() on each one in turn. If none of them trust the certificate, you can apply your own cert checking. If that passes, you can return normally. If not, just throw a CertificateException as you'd otherwise do.
EDIT: Adding the below about doing things like host name verification.
You can also verify that the hostname in the certificate matches what you expect it to be. You'll want to pass in the valid hostname in your constructor for your custom trust manager and stash it in the class. Your checkServerTrusted() method will get passed an array of X509Certificate. Many "chains" will consist of just a single cert, but others will have several, depending on how the cA signed your cert. Either way, the first cert in the array should be "your" cert that you want to compare against.
After you check for basic cert validity using the trust managers, you'll then want to do something like this:
Principal subjectDN = chain[0].getSubjectDN();
String subjectCN = parseDN(subjectDN.getName(), "CN");
if (this.allowedCN.equals(subjectCN)) {
// certificate is good
}
The implementation of parseDN() is left up to you. subjectDN.getName() will return a comma-separated list of key-value pairs (separated by =), something like C=US,ST=California,L=Mountain View,O=Google Inc,CN=www.google.com. You want the CN ("Common Name") value for your hostname comparison. Note that if you have a wildcard cert, it'll be listed as something like *.example.com, so you'll need to do more than a simple equals match in that case.

Validate X509 certificates using Java APis

I am trying to validate a certificate against java key store and this is the code I am using is as below. If it completes succesfully then I assume the validation has gone through correctly, else if an exception is thrown, then the validation fails.
My concern is:
Is the code below sufficient to validate a certificate? As in is there something I am missing here (Like checking the data signed by the computer sending me the certificate?)?
2. Should the signature contained within the certificate be verified? If yes, how?
Thanks in advance for the response!
pradeep
// To check the validity of the dates
cert.checkValidity();
//Check the chain
CertificateFactory cf = CertificateFactory.getInstance("X.509");
List<X509Certificate> mylist = new ArrayList<X509Certificate>();
mylist.add(cert);
CertPath cp = cf.generateCertPath(mylist);
PKIXParameters params = new PKIXParameters(getTrustStore());
params.setRevocationEnabled(false);
CertPathValidator cpv =
CertPathValidator.getInstance(CertPathValidator.getDefaultType());
PKIXCertPathValidatorResult pkixCertPathValidatorResult =
(PKIXCertPathValidatorResult) cpv.validate(cp, params);

Normally, a certificate will be issued by an intermediate issuing authority, not a "root" authority (which is all that should be in your trust store). Most protocols encourage sending a "chain" of certificates, not just the entity's certificate.
You should add all of the intermediate certs so that a complete chain can be formed.
In order to be certain that the certificate is still valid, you should not disable revocation checks. If you don't want to retrieve a CRL (which can be large), the issuer may offer OCSP support. But, this has to be enabled in the Java runtime by setting certain system properties.
If the path validator returns successfully, you don't need to check anything else. If the certificate is not valid, an exception will be raised.
Also, an explicit check on the validity date is unnecessary. This occurs during validation (using the current time, unless you specify a time via the PKIXParameters).
For a more extensive discussion of validation, including sample code, see a previous answer of mine.

If you're happy with the default trust settings (as they would be used for the default SSLContext), you could build an X509TrustManager independently of SSL/TLS and use if to verify your certificate independently.
It would look like this:
TrustManagerFactory trustManagerFactory =
TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init((KeyStore)null);
// you could use a non-default KeyStore as your truststore too, instead of null.
for (TrustManager trustManager: trustManagerFactory.getTrustManagers()) {
if (trustManager instanceof X509TrustManager) {
X509TrustManager x509TrustManager = (X509TrustManager)trustManager;
x509TrustManager.checkServerTrusted(...);
}
}
(You should also check the server's identity and the certificate match, see RFC 6125 (Representation and Verification of Domain-Based Application Service Identity within Internet Public Key Infrastructure Using X.509 (PKIX) Certificates in the Context of Transport Layer Security (TLS)).)

What you are doing here is verifying if a certificate (in your example cert) has been signed (directly) by any of the trusted CA's in the truststore.
Additionally you check for expiration but no revocation checking is performed.
So if the cert has not been signed by any of the trusted CA's you will get an exception.
So the code is sufficient to verify if cert has been signed by any of the trusted CAs
If you are refering to server authentication, then the code in the post is not sufficient.
This code just verifies that a specific certificate is signed by a trusted CA.
You have no indication though if the "entity" that send you this certificate is actually the owner of the certificate (i.e. they own the private key associated with this certificate).
This is part of the SSL authentication, where e.g. the client sends the ClientKeyExchange message encrypted with the remote server's public key and is certain that if the other party is fake then it will not be possible to decrypt the message

Digitally signing a web-http response in Java (using pgp)

I am trying to digitally sign an http - web response. Essentially, I create the HTML and multipart content-type response, sign the response then append the digital signature to the response. I think I am close but off a few steps as this is not a true PGP signature since the appended signature is actually HEXtoString. Big thing is to be able to represent the signature correctly so that response can be interpreted correctly. Could use some suggestions here as I am fairly green with this. Thanks in advance.. below is snippets of code I am using now.
StringBuffer myResponse = new StringBuffer("");
myResponse.append(getHttpHeader());
KeyPair pair2 = loadKeyPair();//loads a key pair from generated files
if (signer==null)
signer = Signature.getInstance("MD5withRSA");
signer.initSign(pair2.getPrivate());
signer.update(message.getBytes());
byte[] b = signer.sign();
FileOutputStream sigfos = new FileOutputStream(getFileLocation(0,localTest));
sigfos.write(b);
sigfos.close();
//verify
signer.initVerify(pair2.getPublic());//pubKey);
signer.update(message.getBytes());
if (signer.verify(b)){
myResponse.append(message);
}
StringBuffer signed= new StringBuffer("");
signed.append(boundary);
signed.append(CRLF);
signed.append("content-type: application/pgp-signature");
signed.append(CRLF);
signed.append("-----BEGIN PGP MESSAGE-----");
signed.append(CRLF);
signed.append("Version: 1");//update this
signed.append(CRLF);
signed.append(CRLF);
signed.append(digSignature);//generated as HexString representation of signed file from above
signed.append(CRLF);
signed.append("-----END PGP MESSAGE-----");
signed.append(CRLF);
signed.append(boundary+"--");
myResponse.append (signed);
ServletOutputStream.println(myResponse);
The resulting "signature" that is transmitted is a byte-hashing hexToString representation of the signed files. I am using standard java classes, but not sure if other libraries would give me a true PGP representation with characters outside of the 0-9a-f representation. ideas??

How is the verification code downloaded to the client? More details about the application in question? If it's a verification script downloaded via HTTP then the scheme is fundamentally broken. You probably need to use SSL, especially if you already argued as such.
Without knowing more about your system, it sounds like an adversary in a man-in-the-middle attack need only to:
Replace the public key in the verification code with their own.
Resign all "secure" communications with their own signature.
Your script sees nothing wrong because the public key it checks was modified by the adversary.
Not to mention all communication is in plain-text (so hopefully no personal/sensitive information being transmitted?)
SSL works around this problem because all the certificates have to be signed by a root certificate authority trusted by / installed with the web browser. CAs are supposed to only issue certificates for domains to people that control/own them; therefore, the previous attack would not work.
Now, if your client is installed in a trusted fashion such that an adversary cannot tamper with it, then you can continue with your scheme and still be secure. For example, if the client is installed on a client PC by hand, or delivered securely some other way (like via SSL, and/or using code signing).
(I did notice a reference to MD5 hashing. Do not use MD5 hashes; MD5 has been broken.)

This issue is due to a NAESB-EDI standard. Where a file has been submitted in an http request and we are required to produce a particular response. We are using SSL and the original payload is supposed to be encrypted. The response is plain html (of 4 items) with an additional digital signature of the response. What I have figured to do is to create the response, have existing pgp software create the signature based upon the generated response and then append the signature to the response. Thus I am not using MD5 anymore and I am not exposing keys to public use (except to those that we specifically trade). So James answer is partially correct and without SSL, this offers little if any protection against sniffing since the response is clear text. Yet without the required information in the request, they would not even get a proper response. Likely wouldnt get a response (let alone a proper one).