We have a dropwizard microservice which accesses a MySQL database. We currently stored encrypted credentials for database access in a configuration file (env.properties) so that it can be changed at deploy time by an ansible script per environment.
The encryption key is something we need to decrypt the credentials, and there is some debate about how to store it.
Some options:
Store the key in the config file. The key is environment specific as well, and because we are deploying the code with ansible, the credentials and keys will not be stored in the microservice repository. The downside here is that the encrypted password is stored in the same file as the key to decrypt it.
Store the key in a startup script, which loads the value into environment variables. Ops didn't like this because they would have to manage two files. currently they only want to manage a single environment properties file. This is understandable because we have enough issues managing the one file, as it is.
What is the best practice for managing this decryption key?
Related
I've read following tutorial:Vault Configuration
Ok we installed the Vault server and put 2 pairs of secret properties:
$ vault kv put secret/gs-vault-config example.username=demouser example.password=demopassword
$ vault kv put secret/gs-vault-config/cloud example.username=clouduser example.password=cloudpassword
Spring boot application has following properties(bootstrap.properties):
spring.application.name=gs-vault-config
spring.cloud.vault.token=00000000-0000-0000-0000-000000000000
spring.cloud.vault.scheme=http
spring.cloud.vault.kv.enabled=true
So based on spring.cloud.vault.token application able to read secure properties(name and password) but spring.cloud.vault.token is stored in the insecure place - bootstrap.properties which is stored in the code repository. Could you please explain why it is safe?
P.S.
As we found out it is insecure. How to make it secure ? I understand that there are might be several solutions to make it secure but single simplified example would be enough for me.
Could you please explain why it is safe?
The answer is that it is NOT safe ... if you do it that way. For example, the Spring Vault reference manual says:
"Consider carefully your security requirements. Static token authentication is fine if you want quickly get started with Vault, but a static token is not protected any further. Any disclosure to unintended parties allows Vault use with the associated token roles."
You should either protect your static tokens, or only grant them access to "secrets" in the vault which you are happy to be widely known.
Alternatively, have your application use an authenticated method to generate short-term dynamic tokens.
As I understand initial problem it is bad to store passwords in an application.properties file on Github.
And storing a static Vault token in an application.properties file on Github is equally as bad.
What is the difference ?
There is almost no difference1. This is simply the wrong way to use Vault.
1 - There is a small advantage in that you could invalidate the token if you discover it leaked by accident. But this doesn't mean that it is sensible to publish it deliberately.
So how do you do things securely?
First, you MUST secure the machines where the secrets are going to be used. Even if you are not going to store the actual secrets on disk, you will need to store a different secret (securely) on each of your machines so that they can authenticate themselves to the place where the real secrets are kept.
Here is an example using Chef.
Set up a secure Chef server that holds the configs for your machines; i.e. recipes for all of the things that need to be installed, node descriptions to say what recipes to apply, etc.
When you bootstrap a machine as a node, a keypair is generated for the machine and registered with the Chef server. The keypair is also held on the machine, and has to be held securely.
Then you use the Chef client to run the recipes that install and configure your server.
Note that this relies on having a properly secured system to run the Chef server. It also relies on each of the nodes being sufficiently secure to protect their own keys.
There are other ways to do this, but nothing will work if you cannot secure your host sufficiently.
Storing spring.cloud.vault.token in the application.properties that is checked into VCS (e.g. Git) might compromise all secrets stored in Vault.
The solution is not to store Vault token in application.properties as a plain text. There are several options.
Remove Vault token from application.properties
Simply remove spring.cloud.vault.token from application.properties and instead provide it via system property -Dspring.cloud.vault.token=00000000-0000-0000-0000-000000000000 (when starting the application) or environment variable SPRING_CLOUD_VAULT_TOKEN. Environment variable is especially convenient if you use containers (Docker or Kubernetes).
Store encrypted Vault token in application.properties
You can keep spring.cloud.vault.token property in application.properties if it is encrypted.
Spring Cloud Config supports decrypting properties with values starting with {cipher}:
spring.cloud.vault.token={cipher}encrypted_vault_token
To use properties encryption and decryption you will need the following dependencies (example for Gradle):
implementation 'org.springframework.cloud:spring-cloud-context:2.2.2.RELEASE'
implementation 'org.bouncycastle:bcprov-jdk15on:1.64'
Symmetric encryption
The simplest way to encrypt the properties is to use symmetric encryption.
Come up with a symmetric key (for example s3cr3t).
To encrypt Vault token you can use Spring Boot CLI with Spring Boot Cloud CLI:
curl "https://repo.spring.io/release/org/springframework/boot/spring-boot-cli/2.2.2.RELEASE/spring-boot-cli-2.2.2.RELEASE-bin.tar.gz" -o spring-boot-cli-bin.tar.gz
tar -xf spring-boot-cli-bin.tar.gz
cd spring-2.2.2.RELEASE
bin/spring install org.springframework.cloud:spring-cloud-cli:2.2.1.RELEASE
bin/spring encrypt 00000000-0000-0000-0000-000000000000 --key s3cr3t
# 507cd1614682535ab8237b448ca73dc74058d3ae9145d63a7381ee67f3046eb1598da6960abdbf2dbf22c47206db5222e45fc74fd6122bc707b61c62f5051e0f
bin/spring decrypt 507cd1614682535ab8237b448ca73dc74058d3ae9145d63a7381ee67f3046eb1598da6960abdbf2dbf22c47206db5222e45fc74fd6122bc707b61c62f5051e0f --key s3cr3t
# 00000000-0000-0000-0000-000000000000
Pass the symmetric key to the application in ENCRYPT_KEY environment variable.
Symmetric encryption key must be never checked into a VCS.
Asymmetric encryption
Consider asymmetric encryption using public and private key pair as a more secure alternative to symmetric encryption.
Instead of a symmetric encryption key you need to generate a keystore (using the keytool utility that comes with the JDK or openssl).
In the bootstrap.properties specify the following properties:
encrypt.keyStore.location
encrypt.keyStore.password
encrypt.keyStore.alias
encrypt.keyStore.type
The keystore has to be mounted at the location specified in encrypt.keyStore.location and never checked into a VCS.
Also, password that unlocks the keystore better to pass in ENCRYPT_KEYSTORE_PASSWORD environment variable.
Read about key management in Spring Cloud Config.
Answering your questions
Could you please explain why it is safe?
It is not safe! Never put a plain secret in source control.
As we found out it is insecure. How to make it secure?
Some ways to increase security:
Use environment variables instead of properties file;
Restrict the access at the network level to the Vault server only to workload servers. This guarantees that nobody outside that network can exchange the token;
Vault works generating real but temporary credentials each time the token is used. Reduce the scope of the real credentials to read-only whenever be possible;
Rotate the token regularly.
Regarding the Spring specifics
The bootstrap properties should contain only non-critical properties. For critical properties, you can pass them to the application using environment variables.
spring.cloud.vault.token = ${SPRING_CLOUD_VAULT_TOKEN}
Summary
The question remains "Who guards the keys?". But the Vault Token serves actually to protect the real sensitive data. In case the vault token is leaked, you can invalidate just the token.
Improve the constraints from where applications can access the vault server and reduce the scope of the real credentials are additional ways to ensure that only the servers that run the application can exchange the tokens by the real credentials and the real credentials have the minimum privilege as possible.
I have some secret api keys for my server and i have to use in my project. But i am not feeling safe to keep in properties file or any physical location in server. Can any one tell me what are possible ways to keep it secret and use wherever it's require in application.
If you are really considered about the passwords you've got, the best way is using HSM (Hardware security module). This way you will be assured that keeping your private key in a secure place is not part of your consideration anymore.
If that is expensive, then you can think of storing your private key in JKS or PKCS12 and choose a strong password.
Generally you are looking for a vault. In PKI you can have a vault to store your secret in it. there are couple of ways I can think of. One of them is HSM as I described above and the other one is following:
For storing your password you need to salt it or store it in another server w/ salt and whenever you need the password, it's better to have a secure connection between your real server and password only stored server to request the password.
Also you can take a look at this project from Hashicorp
https://github.com/hashicorp/vault
There are many options here depending on your actual security requirements.
You've mentioned java and spring boot, so I'll mainly specify solutions related to this technology stack.
Since you're already using Spring Boot, consider spring-cloud-config server.
It has some interesting "encryption" features:
Step 1
Create a Key (public/private pair or shared depending on your requirements)
Specify it when starting the config server (via -- or env variable):
encrypt.key=ABC123ABC123ABC123
Step 2
Encode value of the password by calling a rest request of this server (you do it once)
curl localhost:8888/encrypt -d mysecretdbpassword
>>AZXCASDAZXC341234ZXCASDFedr453
Step 3
In application.properties specify that the value is "encrypted" with a special syntax
The file will be stored on a configuration server and not on the microservice. The
property will look like this
db.password={cipher}AZXCASDAZXC341234ZXCASDFedr453
The property will be kept encrypted in files but decrypted in memory. So the microservice will get a decrypted value.
The documentation is available here. This is by far the fastest way to implement the requirements. You can keep it in a dedicated microservice or embedded into the already-existing application.
In addition, it allows integration with JKS, so its possible to store passwords there.
Complementary/Alternatives
Spring cloud config is also integrated with Hashicorp Vault which can be a good solution for your situation on its own (even without spring boot cloud config).
Yet Another possible solution that comes from Spring Universe is called Spring Vault
Its intended for working with secrets.
I use onboardbase to hide secret i would recommend that they are few other out there to check like doppler
You can setup your application.properties as below
application.properties
secret.key1=${SECRET_KEY1}
secret.key2=${SECRET_KEY2}
While building the application we can pass it as an environment variable
mvn clean install -dSECRET_KEY1=123456 -dSECRET_KEY2=abcdef
We can pass the environment variables in docker as well by using -e SECRET_KEY1=123456
Where should I store credentials for my java application to access third party services?
The credentials are not specific per user on my application. They are for accessing a web service my application is consuming. I know enough not to hard code them into my application, but where and how do I store them? I also assume they will need to be encrypted.
.jar file is best way to store all credentials.
Create interface where store your credentials as a final String
convert interface to jar file
Add that jar file in your build path
Implement this interface where u use credentials, and access String object in which u stored credentials.
Db
.properties file
configuration class with constant
Spring have nice functionality with #Value annotation that can auto-magically inject value from .properties file (under resources folder) with a given key.
I use that because in my case I have different key values in multiple app instances and db would require little more complexity, and furthermore I don't make unnecessary queries to db.
On security basis if attacker can read files on your server than he can easily read your db so that don't play a part here. It can be stored in any file on the system.
On the other hand you can have configuration class with
public static final String SECRET_KEY = "someKey"
To build upon #Zildyan's answer, comments and references to other answers.
There are a few options for where to store:
Database
Properties file
Constant (hard coded)
File system (away from application)
As for how to store:
Depending upon sensitivity. Credentials could be stored in plain text (low sensitivity) or should be encrypted (high sensitivity).
It should also be noted that using a combination of encryption and separating the credentials from the source you would restrict internal access to the credentials.
Some examples
a password stored in plain text may be added to source control and read by anyone with access to the source control.
An encrypted password with decryption code would be easily available to anyone able to run the code.
A plain text file stored on the server may be accessible to anyone with access to the server.
An encrypted file stored on the file system may only be accessible to sys admins and the decryption method available to devs.
The same goes for storing in a database and who has access to that database.
JNDI
Per Wikipedia:
The Java Naming and Directory Interface (JNDI) is a Java API for a directory service that allows Java software clients to discover and look up data and resources (in the form of Java objects) via a name.
Your enterprise likely has a JNDI-compatible directory service established. You would ask the sysadmin to include an entry for your particular credentials.
If you are self-administering, then your Java EE (now Jakarta EE) should have a JNDI-compatible server built-in. Learn to configure it, and add the entry for your particular credentials.
We have run into a requirement to encrypt certain sensitive data before storing it in a PostgreSQL database. Our application is a Spring app running on top of Tomcat. We need to store the keys so that our web app can encrypt data before inserting it and decrypt data after retrieving it.
We would like to avoid storing this type of thing in our config files (or files that are filtered by Maven) since those files are checked into source control and readable by anyone with access. We seem to have the same issue with database credentials but it looks like JNDI datasources might solve those.
Are there best practices for this when using Tomcat? Things like environment variables might work but would be cumbersome to maintain.
EDIT: To clarify, we're trying to prevent sensitive data loss due to unauthorized access to the actual database file. Our db lives on a different physical box from our app server so we feel comfortable keeping the keys on the server that the application server lives on.
As far as I have seen, all the best practices around credentials optimize ease-of-use (for the operations team) and code simplification (for the development team), rather than optimize for security. This is very disappointing.
I would state the requirement as: Do not ever store any credentials or encryption keys in an unencrypted form, whether embedded in properties/config files, source code, or files managed separately from the application.
Some effects of this requirement:
JNDI data sources do not work for DB credentials, because the configuration of the JNDI data source resides in a configuration file in the container.
Passwords/passphrases for encryption/decryption (e.g., for the data in a database) cannot reside in the code or a properties file.
You must implement a way for the operations team to pass the secrets to the application as it is instantiated.
How might you design a system to meet this requirement?
Store all of your secrets in an encrypted file in a centralized location (e.g., in Amazon S3 or a Hardware Security Module). The passphrase/credentials to read this encrypted data must never be stored digitally - it is provided (e.g., in the body of a POST) when the app spins up. In addition to the passphrase to the secrets file, you may also need to dynamically provide credentials to get access to the file itself.
When the application receives the passphrase to the secrets file, it cracks it open and initializes it's internals (e.g., create DB data sources, initialize the encryption services, etc.).
You don't need encryption. You need access control.
Best practice? Yes. Cheap? No:
Don't use a single web server user account to log in to the db. This allows privilege escalation and data theft via SQL Injection. Use a unique connection per user. This kills connection pooling and allows maybe 500 simultaneous users max per db box.
Use a database that supports Row Level Security and Column Level Security. Right now, this is just Oracle. PG9.5 will have it. You can mimic it in 9.4
Put the db in its own network zone. Only the web server can connect.
Use a DAM (database activity monitor) to look for bad queries in and too much data out
A guide: https://securosis.com/assets/library/reports/Securosis_Understanding_DBEncryption.V_.1_.pdf
We are using Java + H2 Database in server mode, because we do not want users from accessing database file.
To add more protection to database file, we plan to use AES encryption (add CIPHER=AES to database URL) in case the storage is stolen.
However, each user will also need to supply file protection password when connecting ([file password][space][user password]).
Although users do not have access to database file, knowing the encryption key (file protection password) will make the encryption quite useless.
Any idea to keep the database file secure (encrypted) without exposing file encryption key to users?
Thank you.
There is currently no way to do that within H2.
One solution is to use file system encryption that is independent of H2.
But please note at some point you would have to provide the (database file or file system) password. This could be when starting the server (prompting for the password to be entered manually). Unfortunately, because somebody would have to enter the password, you couldn't fully automate starting the server.
One clever approach I've heard of is to write a simple webservice that blocks all sites but your webapp's server. Use SSL with certificate-based authentication.
This webservice provides the encryption key.
It sounds really stupid since it seems to provide the key without authentication but the certificate-based authentication covers that. It provides encryption of the key during transit (and if you're really paranoid you could use a shared key to wrap the database key). It can only be called from the webapp's server and it's a lot harder to trigger a webservice call than to do SQL injection or even looking at the filesystem.
If your risk model allows it you could even run this webservice on the same box as your webapp.
A second approach, if you have the necessary permissions, is to create a virtual disk. Put the key on the virtual disk.
During startup you mount the virtual disk, read the encryption key, then unmount the virtual disk. In some designs you could drop the operating system permissions that allow you to mount the virtual disk - it would then be literally impossible for an attacker to read the key via your webapp.
This comes from a much older strategy that read sensitive information from a CD-ROM (or even floppy disk). The app would read the key and then eject the media. It works but requires manual intervention to reload the media before the next restart. It also doesn't work in modern environments where servers don't have CD-ROMs, much less floppy drives.