I am building an AWS Lambda service for a small PoC. The flow in PoC is :
take a (text) input via POST,
performs a small string manipulation +
store the manipulated value into DynamoDB, and then
send the same (manipulated) value to a particular URL via HTTP POST
Seems like a simple lambda tutorial example, but the tricky part for me was the authorization. The URL that I have to POST to only allows requests that are mutually authenticated via a SSL cert. How can I achieve this in Lambda ?
I could not find enough answers to make this work. I looked at using the AWS API gateway 2-way ssl cert option. However, For that to work, I need to install the receiving part cert into cert store. Is the even possible ? Or the only way is to use a micro-EC2 box ?
At Lambda, I am okay to use Node.JS, Java, or Python.
How to implement mutual TLS in AWS Lambda?
First big applause for Hakky54 for this good tutorial on mutual TLS.
https://github.com/Hakky54/mutual-tls-ssl
I followed his tutorial to understand and implement MTLS for AWS Lambdas. You can also test your implementation locally before deploying to AWS by just running the spring-boot app which saves a lot of time.
Steps (all commands are documented on the above link)
Export server cert and import it to client trust store
Load your client key store and trust store, I saved both in s3 bucket
Create TLS Context
SSLContext sslContext = SSLContexts.custom()
.loadKeyMaterial(keyStore, stores.getKeyStorePassword().toCharArray())
.loadTrustMaterialtrustStore, (X509Certificate[] chain, String authType) -> true)
.build();
Create a new Jersey client
Client client = ClientBuilder.newBuilder()
.withConfig(new ClientConfig())
.sslContext(sslContext.get())
.trustStore(trustStore)
.keyStore(keyStore, keyStorePassword)
.build();
Make the call to the API
client.target(endpoint).get();
I am storing my keystore credentials in parameter store.
Related
I want to connect to AWS DocumentDB cluster from AWS Lambda (using Java). TLS is enabled for cluster so I need to import the certificates to truststore. Not able to find any document around this on how to proceed.
You need to store https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem file to certstore before connecting to documentDB otherwise it will not work.
Their are many ways to import certificates using code during runtime.
Ref :
How to import a .cer certificate into a java keystore?
After importing cert, you can connect to documentDB, reference code can be found here :-
https://docs.aws.amazon.com/documentdb/latest/developerguide/connect_programmatically.html
I encourage you to avoid packaging the cert as part of your Lambda code. Instead you can get it dynamically from Amazon S3. This will avoid future issues in the future when the cert is rotate. Following a python example:
#Function to download the current docdb certificate
def getDocDbCertificate():
try:
print('Certificate')
clientS3.Bucket('rds-downloads').download_file('rds-combined-ca-bundle.pem', '/tmp/rds-combined-ca-bundle.pem')
except botocore.exceptions.ClientError as e:
if e.response['Error']['Code'] == "404":
print("The object does not exist.")
else:
raise
For you to do that, the role of your Lambda needs permissions to get the object from S3 and S3 access via the Internet or a VPC endpoint.
I am quite new to working with certificates and security, so pardon me if this is a no-brainer to others. I have followed this guide to set up my Grails application to run on HTTPS with self-signed certificates.
I am trying to establish 2-way SSL with another HTTPS network (a Nifi standalone instance) running on the same machine. I can get the Nifi instance to talk to Grails over HTTPS, but I am having issues with Grails talking to Nifi (specifically to a ListenHTTP processor).
I was hoping someone could advise how to use certificate authentication in Grails when posting over HTTPS.
Nifi uses certificate authentication; however per the above guide Grails only specifies a single keystore (for receiving requests?) so I'm a bit thrown off. I can successfully CURL to Nifi's REST API by specifying the --cert and --key properties, but since the final product will be a WAR on a client machine I want to set this up the 'right way', and I believe leaving those files on the client machine is a really big no-no for security.
During early development RestBuilder was sufficient for 2-way comms over HTTP, however, I am unable to find any mention of using it with certificate authentication (only basic authentication is covered in the documentation?).
HTTPBuilder shows up a lot when I looked for alternatives, however looking at the relevant documentation (line 139 'certificate()') it states that it takes a whole keystore JKS and password. I think this is close but not quite what I am looking for considering I only have one keystore; I am open to correction here.
Please note that I will be unavailable to respond until at least the day after this question was posted.
When making an outgoing HTTPS connection, if the remote endpoint (in this case Apache NiFi) requires client certificate authentication, the originating endpoint (Grails) will attempt to provide a certificate. The certificate that Grails is using to identify itself as a service is fine to use in this scenario, provided:
The certificate either does not have the ExtendedKeyUsage extension set, or if it is set, both ServerAuth and ClientAuth values are present. If ClientAuth is missing, the system will not allow this certificate to be used for client authentication, which is the necessary role in this exchange.
The certificate has a valid SubjectAlternativeName value which matches the hostname it is running on. RFC 6125 prescribes that SAN values should be used for certificate identity rather than Distinguished Name (DN) and Common Name (CN). So if the Grails app is running on https://grails.example.com, the SAN must contain values for grails.example.com or *.example.com.
The certificate must be imported into NiFi's truststore in order to allow NiFi to authenticate a presenter of this certificate.
NiFi must have ACL permissions in place for this "user". This can be done through the UI or by modifying the conf/authorizers.xml file before starting NiFi for the first time. See NiFi Admin Guide - Authorizers Configuration for more information.
Your concern for leaving the cert.pem and key.key files on the client machine is understandable, but the sensitive information contained therein is the same data that's in your keystore. At some point, the private key must be accessible by the Grails app in order to perform HTTPS processes, so having it in the keystore is functionally equivalent (you don't mention having a password on the *.key file, but obviously you should have a password on the keystore).
I am writing a NodeJS proxy that will replace a Java Jetty Proxy. I am using node-http-proxy. The only piece remaining is to have the original client certificate passed along to the proxied server.
From my understanding, the Java Servlet specification requires that a Servlet container pull the Client Certificate from an HTTPS request and store that as an attribute on the HttpServletRequest.
I am not sure how the Servlet Container handles the Attributes when proxying the request to a new server. I presume that it is attaching them somehow either as headers or by some other means.
Does anyone know how those attributes (specifically the javax.servlet.request.X509Certificate) are passed on a proxied HTTPS request? And two, how do I achieve the same functionality using NodeJS.
In the event that is helps someone else out... The issue turned out to be the node module I was using (node-http-proxy) wasn't reusing the HTTP server connection certificates. That is, when attempting to create a connection with the proxy server, it was using a default (generated) certificate.
To properly connect with the proxy server, I had to pass the ca, pfx, and passphrase to the proxy connector.
const ca = ...
const pfx = ...
const passphrase = ...
// proxy connection
server.web(req, res, { ca: ca, pfx: pfx, passphrase: passphrase }, function(err) {});
After doing so, the Proxy server was able to pull and validate the certificate.
My project use xfire as a web service client api. My project is in legacy Servlet/JSP. We used XFire eclipse plugin to generate client stub.
Web-service has Migrated to SLL (HTTPS). Is there any easy way to consume Webservice over SSL in XFire.
I found some code at http://docs.codehaus.org/display/XFIRE/HTTP+Transport.
I have some confusion there too. It motivates to use not-so-common-ssl which is in Alpha and I don't know if it is stable enough to be used in production.
// Technique similar to http://juliusdavies.ca/commons- ssl/TrustExample.java.html
HttpSecureProtocol protocolSocketFactory = new HttpSecureProtocol();
// "/thecertificate.cer" can be PEM or DER (raw ASN.1). Can even be several PEM certificates in one file.
TrustMaterial trustMaterial = new TrustMaterial(getClass().getResource("/thecertificate.cer"));
// We can use setTrustMaterial() instead of addTrustMaterial() if we want to remove
// HttpSecureProtocol's default trust of TrustMaterial.CACERTS.
protocolSocketFactory.addTrustMaterial(trustMaterial);
// Maybe we want to turn off CN validation (not recommended!):
protocolSocketFactory.setCheckHostname(false);
Protocol protocol = new Protocol("https", (ProtocolSocketFactory) protocolSocketFactory, 8443);
Protocol.registerProtocol("https", protocol);
Now above is a way to create a Protocol factory and getting it registered with Apache HTTPclient api. But id doesnot say what to do further with the generated stub.
Please feel free to ask more information if any.
We can't move to other web-service client api so that is not an option.
Managed to solve my own problem.
This is how I did it. XFire use Apache Http client internally so setting Security certifect detail on this Api will do the job. We will use no-yet-common-ssl.jar for this purpose.
First we will create org.apache.commons.ssl.TrustMaterial using commons and then set it in HttpSecureProtocol which is a child of javax.net.ssl.SSLSocketFactory.
Suppose XYZ.cer is the client certifect provided by service provider.
HttpSecureProtocol protocolSocketFactory = new HttpSecureProtocol();
protocolSocketFactory.addTrustMaterial(TrustMaterial.DEFAULT); //for trusting all the certifects in java trusted Store.
protocolSocketFactory.addTrustMaterial(new TrustMaterial(getClass().getResource("/XYZ.cer")));
Protocol protocol = new Protocol("https", (ProtocolSocketFactory)protocolSocketFactory, 443);
Protocol.registerProtocol("https", protocol);
If this is a web Application you can do this in ServletContextListener or in any part of code that executes when application boots.
Now you can use any ssl service using Xfire client stub. Any service which implement the above certifect.
Now why this work. Because XFire uses Apache Http Client as a connection api and we are telling Http client to use the above TrustManager when HTTPS is used.
I am writing a routine to access a remote server. This server I am connecting to requires mutual authentication so I have to provide a keystore, and while I'm at it I'd like to put a proper truststore in place as well.
I can find plenty of tutorials on how to create a keystore with keytool and multiple ways to get an Apache HTTP client to recognize it, but not where to store it in a Tomcat environment so that the application can find it. Somehow putting it in the application's war file seems like a bad idea to me.
Again, this is not to permit Tomcat to handle inbound https connections - I have a reverse proxy set up by our admin team for that. I'm creating outgoing https connections that require mutual authentication, i.e., both accepting a self-signed destination server certificate, and providing my server's self-signed client certificate.
Where do you store the actual keystore and truststore files in a Tomcat environment for use by a web application?
You can put your keystore wherever you want, as long as you know how to tell httpclient where to load the keystore.
That, of course, is the trick.
Using Apache httpclient for https
Buried in all that mess of code in the accepted answer is the key (ha!) to using httpclient with your own custom keystore. It's unfortunate that httpclient doesn't have a simple API like "here's the path to my keystore file, now use it" or "here are the bytes for my keystore, use those" (if you wanted to load the keystore from the ClassLoader or whatever), but that seems to be the case.
The honest truth is that using keystores and truststores in Java is messy business, and there's usually no way around it. Having written a client-cert-capable HTTP client myself using nothing other than HttpsURLConnection and then also adding raw-socket components to that, I know how painful it is.
The code in the above-linked article is fairly straightforward if a bit verbose. Unfortunately, you're going to need to make it a lot messier for production-quality code because you've got to do error-checking, etc. for every step of the process to make sure your service doesn't fall-over when you are trying to set up the various stores and make your connection.
This is basically a comment to Christopher Schultz's answer, but since it involves some code snippets please excuse my putting it here
It's unfortunate that httpclient doesn't have a simple API like
"here's the path to my keystore file, now use it" or "here are the
bytes for my keystore, use those" (if you wanted to load the keystore
from the ClassLoader or whatever), but that seems to be the case
This is how one can configure Apache HttpClient 4.3 to use a specific trust store for SSL context initialization.
SSLContext sslContext = SSLContexts.custom()
.loadTrustMaterial(trustStore)
.build();
CloseableHttpClient client = HttpClients.custom()
.setSslcontext(sslContext)
.build();
One can load trust material from a resource like that
URL resource = getClass().getResource("/com/mycompany/mystuff/my.truststore");
KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
InputStream inputStream = resource.openStream();
try {
trustStore.load(inputStream, null /*usually not password protected*/);
} finally {
inputStream.close();
}