Microsoft Windows 2000 and later versions expose the Data Protection API (DPAPI) that encrypts data for a per-user or per-system context. The caller does not provide a key with which to encrypt the data. Rather, the data is encrypted with a key derived from the user or system credentials.
This API is conveniently exposed in .NET via the ProtectedData class:
// Encrypts the data in a specified byte array and returns a byte array
// that contains the encrypted data.
public static byte[] Protect(
byte[] userData,
byte[] optionalEntropy,
DataProtectionScope scope
)
// Decrypts the data in a specified byte array and returns a byte array
// that contains the decrypted data.
public static byte[] Unprotect(
byte[] encryptedData,
byte[] optionalEntropy,
DataProtectionScope scope
)
Is there an equivalent API on Linux? A bonus would be that it integrates conveniently with Java.
What are my alternatives if there isn't one?
There are two options for user-level key stores on Linux:
GnomeKeyring
KWallet
This does not address the need for a system-level key store.
https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/introduction?view=aspnetcore-5.0
Here!
This documentation is so good, that I won't even bother to explain more.
Do not be discouraged with no immediate code samples on the first page. There are examples in links there. For all scenarios. DI, ASP.NET, console. Windows and Linux.
As the others before me said - AFAIK you don't have default keys for users and system in Linux. But a key is a key. You can create them and on Linux it's your (as the administrator / root) responsibility to protect the key files (meaning make them accessible only to authorized users).
The good part is you do not rely on system specific keys. You just use separate keys, your application keys.
If that is what you need - the linked API is just for you. I wish Linux had built in, default keys for users, but well... It's just one extra step for increased app-level security. Do you want one step more? Use Azure Key Vault, they have nice REST API you can use anywhere, not necessarily in Dotnet. Yes, AKV requires locally stored user password, but you can disable the access remotely, so it's a good additional security layer. If your user / machine was compromised, you just disable the secret and the target app is disabled until you provide the user with the new key. I use it in my sensitive apps a lot. Works as charm.
BTW, my minimalistic example of Linux DPAPI usage:
using System;
using System.IO;
using Microsoft.AspNetCore.DataProtection;
var keyDirectory = Directory.GetCurrentDirectory();
var dataProtectionProvider = DataProtectionProvider.Create(new DirectoryInfo(keyDirectory));
var protector = dataProtectionProvider.CreateProtector("Test");
var password = "Secret1337";
var protectedPassword = protector.Protect(password);
Console.WriteLine($"Protected: {protectedPassword}");
var decodedPassword = protector.Unprotect(protectedPassword);
Console.WriteLine($"Decoded: {decodedPassword}");
Of course in real world app you won't store keys in current directory, but that's the simplest example I could think of.
It doesn't look any more (or less) advanced than PGP, or Pretty Good Privacy. There are APIs available for PGP, and the one that I recall others speaking kindly of is Bouncy Castle.
Here's an example of how someone used Bouncy Castle.
Better APIs or solutions may be available, depending on your specific needs.
DPAPI does not exist on Linux.
Windows uses a special machine-id to deviate a machine key. You can emulate this behavior by looking into "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\MachineGuid" reading this key and deviate your special key by using any encryption library you want.
Under Linux on the other hand this machine-id is stored in the file "/etc/machine-id". You can read it's contents and deviate your special key from it. Be aware this key may be the same when using fast deployment VMs.
Encrypt your data with this special machine-id and it cannot not be read across other machines. Read at first the machine-id (Linux or Windows) and then try to decrypt the contents of your data. On another machine the result will obviously be different and not correct.
You can code your platform independent wrapper class by using the information from above.
Hope this helps someone in the future.
Cheers
Related
I'm writing an app in Java where users must be able to share encrypted messages, and communication happens exclusively via a database where all data is publicly visible.
Requirements are roughly:
Each user may publish some information (e.g. a public key) to the public database but should only do this once
Other users must be able to encrypt a message for a target user and publish it publicly on the same database
The intended recipient must be able to decrypt the message, but no other user should be able to do so
The encryption must be strong enough that no plausible brute force computation should be able to decrypt the message (now or in the future)
Messages are arbitrary length, but usually quite small (think short emails, tweets etc.)
Must be easy to implement in regular Java (libraries like Bouncy Castle are fine if needed)
Users already have Ed25519 key pairs for digital signatures if necessary
What crypto algorithm or combination of algorithms would be best to meet these requirements? I'm assuming some form of asymmetric encryption algorithm with a public/private key pair for each user but alternative ideas welcome. I definitely want to avoid the "roll your own crypto" trap though....
You (almost) always roll your own protocols. Which is usually where the hacks are.
Now, if you were to roll your own algorithm (as in, you avoid, say, AES-256, and write something on your own. We'll just XOR every byte with a repeated application of 'OhWowThisIsTheBestSecurityEvar', it's like a one-time pad right, can't be cracked! - that kinda thinking is what the whole 'dont roll your own crypto' meme is all about. Don't do that.
Even if you use off the shelf 'protocols', it's real easy to mess it up and create holes. The protocol side (HOW you use the crypto algorithms) is by its nature not easily abstracted into a single, hard-to-impossible to abuse prebuilt library.
So, roll your own. Protocol that is.
It seems nearly trivial here - but it's not. The basic job is to use Public/Private key crypto, in the usual fashion, encrypting the message with symmetric crypto (say, AES-256), generating a random IV and random key, and storing both in the DB, but the key is stored encrypted - encrypted using Public/Private key crypto.
That is, at its core, enough to do what your requirements say you want. But, what about replay attacks? Possibly overzealous or implausible, but what if I can write stuff into your DB? I could replay the message: Store the exact same ball of encrypted bytes but with different timestamps, and you'd think it was real.
That's exactly one of those protocol thingies: It helps if you include the timestamp, sender, etc all inside the blob to be encrypted - you want a replay attack to be innocuous, and generally, if the message (including the metadata) is the exact, precise same, it should be. Maybe. Usually. It depends on what your users expect and what they'll be using it for. There is no such thing as perfect crypto, after all. But if even that is not acceptable, there are solutions to that too, though the easy route is simply to ensure nobody has raw SQL-level write access to the DB.
If your system gets 'hacked' it'll almost always be in the second-level 'protocol' bits. Can I just call your help desk and inpersonate a user? Can I just say I lost my password and get a new one mailed to me, and 'just' hack the user's email instead? Can I stick a keylogger in their computer? Maybe litter some USB sticks with RAT worms around the parking lot; a RAT worm specifically written to find their key files, catch them in entering the password for this key file, and send it all to me? No 'off the shelf java library' is ever going to protect you against any of this. You can't do security in a 'I dont really know what I am doing, but I at least I know I don't know so I'll make sure to get a community and expert recommended library and try to follow its manual as well as I can and surely I'll be fine!' - kind of fashion. Not if you take it seriously.
Some of the algorithms you should probably use:
BouncyCastle supports ElGamal, a Public/Private key encryption scheme. The idea is that all users have a public and private key; the public key is known to your server (and all users on demand; your server is a clearinghouse for them and will have to vouch for the truth, that is, if you go: "here is the public key for user Foo", you're vouching that this is true, and presumably vouching that the personal info of user Foo on your site is correct. How - that's on you, and has nothing to do with crypto, but with politics and local processes. Do you call them? Do they identify themselves with a passport? What? The private key is known only to them. You'll need to handroll some sort of invalidation scheme. Perhaps users can anoint a few other users - they then get the right to invalidate their key. The idea being, if a user feels their private key is compromised, they ask one of their buddies to log in and authenticate themselves to the system, and tell the system to mark your public key as no longer valid for any further communications.
Note that you'll have to find a way to use that Ed25519 key pair you have if you want to use that for the vouching system as above.
Use AES-256 along with your plane jane basic new SecureRandom() to generate a key for each and every message you want to store in the system. To store a message, you take the data, generate a random key, encrypt the data using that key and the AES-256 algorithm, store the encrypted data, then you encrypt the key using ElGamal and the user's public key, and store that too. To decrypt this data, the user fetches the (with their public key encrypted) key data (which can be public), and the encrypted data (also public), and can undo the job on their end by first using their private key + ElGamal to derive the randomly generated AES-256 key used, and then use that. You don't encrypt the entire message with ElGamal; that's quite slow and not the common way to do it. AES-256 is blazingly fast. But symmetric. You'll need a 'block mode' and a 'padding mode' for your encryption in addition to an algorithm (Which will be AES-256). block mode should probably be GCM; you may read about CBC; that's outdated (worse, and slower). Definitely don't pick ECB, that's straight up insecure. Padding probably doesn't matter, depends, as usual, on so many factors.
You mentioned nothing about signatures. If user "Foo" wants to send a message to user "Bar" in a way that nobody but Bar can read it, all they have to do is the above. But if they do, "Bar" has absolutely no idea who sent it. GCM has some built in support for MACs, which is what you need to tag a message in a way that senders can prove they were, in fact, the sender, and also to tag date and time, though this is not easy; basically you as a server would tag any message with 'I, server, decree, and you are going to have to trust me, that this was present in the DB at this point in time and appeared recently; to me anyway. Signed, server'.
That'll give you a few terms (MAC, GCM, AES-256, ElGamal, Bouncy Castle, signatures, and a few more) to search the internet for and read up on.
It sounds to me that you are developing a chat system.
The way to fulfil your requirements you need to combine two crypto systems.
The first one is a Diffie-Hellman key exchange - in short: each party generates a private/public key pair. The public key is stored on the server. If I try to send a message to Bob I'm using your database for "Bob" and get his public key. Next I'm generating a "shared secret" with my private key and Bob's public key - this shared secret is usually 32 bytes long.
Now the second phase begins - I'm encrypting my message with an AES algorithm (best one could be "GCM"-mode) and save the encrypted message (encoded as Base64-string) in your public database (of course with any hint that the counterpart is me).
Third phase: Bob is getting the encrypted message with the note it's from Michael. Now Bob is searching in the database for Michael's public key, builds the shared secret with his (Bob's) private key and Michael's public. Some kind of magic will happen - the shared secret is the same key I used for encryption. Now Bob is been able to decrypt my message.
Just a note: beware the risks because if Bob will lose his private key (maybe stored on his stolen smartphone) he will no longer read any messages for him, so a backup of his private key is needed.
I am trying to encrypt my sensitive data like user email address to the database using aws kms cmk. I am using aws SDK in java.
I have encrypted the field and stored in data base.
I can also decrypt the data.
But the main problem comes when I have to query the data...
As I am getting back the aws sdkbytes and I am converting it to base 68 and storing it.
Why does aws always return the different SDK bytes for the same data after encryption?
I can't search simply through the data.
I have to bring all the list and decrypt the data and then match the data.
It is taking a lot of time.
Please suggest me the good way to encrypt the data using aws cmk keys.
Thanks
Ankit
The encryption, when done properly, is never deterministic. Being deterministic and encrypting always abc#abc.com to fa3GeFi2nb52JH0 would be a weakness, as it would leak information (if you find two equal ciphertexts, then you know that they hold the same plain text. You don't know the exact value, but you know it is the same). Technically you do it with a random value named IV (initialization vector) being passed to the encryption process, but let's not dig deeper.
So, how to deal with your case. You can either:
accept it the way it is now,
say you want your encryption to be deterministic even at the cost of being weaker and leaking data.
How to make encryption deterministic? Use constant IV value. I don't know if this can be done in AWS and how, but that would be the easiest way to go.
By using KMS in this way you're performing client side encryption which would put the responsibility on you to decrypt after you have retrieved the data.
At the current time there are no implementations explicitly for using KMS keys within SQL operations, the only support is for encryption at rest of the file storage.
You options are:
Switch to native MySQL encryption
Switch from encrypted to hash to match (not ideal) for this field.
Perform post processing after narrowing down to decrypt.
I'm trying to encrypt a password to be stored across multiple instances of my program, but it needs to be able to be decrypted by all users on the machine. I have already looked around for options, and had javaDPAPI recommended to me. However, this only seems to encrypt based on the current user context.
Based on microsoft's documentation there seems to be a way to do this, but I am not sure how to do this and I am not very familiar with java and have never done something like this.
You need to set the Boolean in the second constructor of DataProtector mentioned here to true.
This needs some adaptation in the class you linked too:
changing this.protector = new DataProtector() to
this.protector = new DataProtector(true) should do it, I think.
You could even add a project unique "entropy" byte array so that only someone knowing that entropy could decrypt the secret, and other programs cannot. This is also part of the original Microsoft design.
My app needs to encrypt some data (a user session token). Most examples I see around have a method that generates a Key using a passphrase and a salt, like:
public static Key generateKey(char[] passphrase, byte[] salt) {
...
}
My understanding is that we have three options for generating the passphrase:
Have the user enter it every time the app starts (annoying to the user).
Hard-code the passphrase into the app itself. More convenient for the user, but someone can find out what your passphrase is given your app binary.
Randomly generate a passphrase, but then we have to store the generated Key on disk. Now we've just shifted the problem to having to store the key securely on disk, which also seems impossible. If the attacker finds the generated key, big problem.
Option #1 won't work for me. Options #2 and #3 seem inherently flawed, unless I'm grossly misunderstanding how to go about this (hoping that I am). What's the recommended way to do this if we can't go with #1? Do we put in a bunch of obfuscated hoops for an attacker to jump through and hope for the best?
Thanks
"Do we put in a bunch of obfuscated hoops for an attacker to jump through and hope for the best?" Basically yes. The size and number of the hoops being how hard you want to make it.
If you are not using a server, then whatever you do to obsfucate and encrypt your data is reversible. However, you can make it REALLY hard. For example, a technique I used to protect some video assets.
Replaced the first 1024 bytes of the header (it's MP4) with 1024 bytes taken from the middle of one of the apps image assets. I tried several repairers, all of which failed to automagically recover the file - although it can be done manually. Then...
Encrypted the file using a private key which is 256 bytes taken from another image asset.
When the key is extracted, it's hashed through an algorithm which does all kinds of otherwise non-sensical maths to mangle the key.
Used a pre-compile obsfucator.
I've tried myself to reverse engineer this, even knowing how it's done, and it's so hard as to make the effort not worth the result.
There are numerous discussions on SO which summarise as; If you simply want to stop copying, make it difficult (cost vs reward) but otherwise sleep easy because there is ultimately nothing you can do. If the data is commercially sensitive, then a server coupled with system level security (e.g whole device encryption and no root) is required.
You store the salt along with the encrypted data, it is not secret information. You can derive the key on either something the user enters, or some sort of a device property: (hashed) IMEI, MAC address, etc.
Basically, think who are you protecting your data from and why. Since the user needs this, there is not much point trying to protect it from them. If you store this in a private file, other apps cannot read it on a non-rooted phone. If you want to protect it on rooted phones, encryption might help, but as long as the key resides in the app, or is derived based on something on the device, it is only making it harder, not impossible to recover.
Android does have a system-wide keystore service, but it has no public API and is subject to change. You could use that to protect your key(s), if you are willing to take the risk of your app breaking on future versions. Some details here: http://nelenkov.blogspot.com/2012/05/storing-application-secrets-in-androids.html
This question already has answers here:
How do I securely store encryption keys in java? [closed]
(4 answers)
Closed 7 years ago.
I'm working on a software project where the application will end up being run in an untrusted environment. I have a need to perform some ancillary cryptographic signing (meaning this is not the primary means of securing data), but do not wish to leave the key in plain view as such:
private static final String privateKey = "00AABBCC....0123456789";
What method can I use to reasonably secure this? I'm aware that nothing is full proof, but this will add an extra layer in the security wall.
For clarification: I've got what is essentially a String that I don't wish to have easily pulled out in a debugger or via reflection. I'm aware that decompilation of the class file could essentially render this moot, but that's an acceptable risk.
Obviously storing the key offsite would be ideal, but I can't guarantee Internet access.
It's impossible to secure a key in an untrusted environment. You can obfuscate your code, you can create a key from arbitrary variables, whatever. Ultimately, assuming that you use the standard javax.crypto library, you have to call Mac.getInstance(), and sometime later you'll call init() on that instance. Someone who wants your key will get it.
However, I think the solution is that you tie the key to the environment, not the program. A signature is meant to say that the data originated from a known source, and has not been tampered with since that source provided it. Currently, you're trying to say "guarantee that my program produced the data." Instead, change your requirement to "guarantee that a particular user of my program produced the data." The onus is then shifted to that user to take care of his/her key.
Forget about obscuring it in the code. It will only make your software harder to read, debug and maintain. You'll also be nailed if your software has to go through a security audit.
If you can't put the key in secure storage (securely on disk, secure memory or a passphrase in someones head), don't bother with anything else.
If you're in a *nix environment, storing the key on disk with root/root 400 permissions might be "good enough".
On Windows, you could use the DPAPI to store the data in Microsofts secure memory.
You could also use a lightweight PBE to encrypt the sensitive key and have the user enter the passphrase when the application starts up.
Whose private key is that? The private key is supposed to be private, so it is wrong to distribute it.
First off - good on you for thinking about this problem!
Is it possible to instead generate a private key, communicate with your Certificate Authority and have it sign the key (and manage a CRL as well)?
As an alternative, if this is going to be running on Windows, you can use the Crypto API to securely store a private key that is marked as not-exportable. How you distribute that key securely can be another challenge though.
Can you break the private key out into two parts: store one in your program, then interactively request the second half - when your app starts ?