Develop Reference

Best way to encrypt messages in Java where ciphertext is public

I'm writing an app in Java where users must be able to share encrypted messages, and communication happens exclusively via a database where all data is publicly visible.
Requirements are roughly:
Each user may publish some information (e.g. a public key) to the public database but should only do this once
Other users must be able to encrypt a message for a target user and publish it publicly on the same database
The intended recipient must be able to decrypt the message, but no other user should be able to do so
The encryption must be strong enough that no plausible brute force computation should be able to decrypt the message (now or in the future)
Messages are arbitrary length, but usually quite small (think short emails, tweets etc.)
Must be easy to implement in regular Java (libraries like Bouncy Castle are fine if needed)
Users already have Ed25519 key pairs for digital signatures if necessary
What crypto algorithm or combination of algorithms would be best to meet these requirements? I'm assuming some form of asymmetric encryption algorithm with a public/private key pair for each user but alternative ideas welcome. I definitely want to avoid the "roll your own crypto" trap though....

You (almost) always roll your own protocols. Which is usually where the hacks are.
Now, if you were to roll your own algorithm (as in, you avoid, say, AES-256, and write something on your own. We'll just XOR every byte with a repeated application of 'OhWowThisIsTheBestSecurityEvar', it's like a one-time pad right, can't be cracked! - that kinda thinking is what the whole 'dont roll your own crypto' meme is all about. Don't do that.
Even if you use off the shelf 'protocols', it's real easy to mess it up and create holes. The protocol side (HOW you use the crypto algorithms) is by its nature not easily abstracted into a single, hard-to-impossible to abuse prebuilt library.
So, roll your own. Protocol that is.
It seems nearly trivial here - but it's not. The basic job is to use Public/Private key crypto, in the usual fashion, encrypting the message with symmetric crypto (say, AES-256), generating a random IV and random key, and storing both in the DB, but the key is stored encrypted - encrypted using Public/Private key crypto.
That is, at its core, enough to do what your requirements say you want. But, what about replay attacks? Possibly overzealous or implausible, but what if I can write stuff into your DB? I could replay the message: Store the exact same ball of encrypted bytes but with different timestamps, and you'd think it was real.
That's exactly one of those protocol thingies: It helps if you include the timestamp, sender, etc all inside the blob to be encrypted - you want a replay attack to be innocuous, and generally, if the message (including the metadata) is the exact, precise same, it should be. Maybe. Usually. It depends on what your users expect and what they'll be using it for. There is no such thing as perfect crypto, after all. But if even that is not acceptable, there are solutions to that too, though the easy route is simply to ensure nobody has raw SQL-level write access to the DB.
If your system gets 'hacked' it'll almost always be in the second-level 'protocol' bits. Can I just call your help desk and inpersonate a user? Can I just say I lost my password and get a new one mailed to me, and 'just' hack the user's email instead? Can I stick a keylogger in their computer? Maybe litter some USB sticks with RAT worms around the parking lot; a RAT worm specifically written to find their key files, catch them in entering the password for this key file, and send it all to me? No 'off the shelf java library' is ever going to protect you against any of this. You can't do security in a 'I dont really know what I am doing, but I at least I know I don't know so I'll make sure to get a community and expert recommended library and try to follow its manual as well as I can and surely I'll be fine!' - kind of fashion. Not if you take it seriously.
Some of the algorithms you should probably use:
BouncyCastle supports ElGamal, a Public/Private key encryption scheme. The idea is that all users have a public and private key; the public key is known to your server (and all users on demand; your server is a clearinghouse for them and will have to vouch for the truth, that is, if you go: "here is the public key for user Foo", you're vouching that this is true, and presumably vouching that the personal info of user Foo on your site is correct. How - that's on you, and has nothing to do with crypto, but with politics and local processes. Do you call them? Do they identify themselves with a passport? What? The private key is known only to them. You'll need to handroll some sort of invalidation scheme. Perhaps users can anoint a few other users - they then get the right to invalidate their key. The idea being, if a user feels their private key is compromised, they ask one of their buddies to log in and authenticate themselves to the system, and tell the system to mark your public key as no longer valid for any further communications.
Note that you'll have to find a way to use that Ed25519 key pair you have if you want to use that for the vouching system as above.
Use AES-256 along with your plane jane basic new SecureRandom() to generate a key for each and every message you want to store in the system. To store a message, you take the data, generate a random key, encrypt the data using that key and the AES-256 algorithm, store the encrypted data, then you encrypt the key using ElGamal and the user's public key, and store that too. To decrypt this data, the user fetches the (with their public key encrypted) key data (which can be public), and the encrypted data (also public), and can undo the job on their end by first using their private key + ElGamal to derive the randomly generated AES-256 key used, and then use that. You don't encrypt the entire message with ElGamal; that's quite slow and not the common way to do it. AES-256 is blazingly fast. But symmetric. You'll need a 'block mode' and a 'padding mode' for your encryption in addition to an algorithm (Which will be AES-256). block mode should probably be GCM; you may read about CBC; that's outdated (worse, and slower). Definitely don't pick ECB, that's straight up insecure. Padding probably doesn't matter, depends, as usual, on so many factors.
You mentioned nothing about signatures. If user "Foo" wants to send a message to user "Bar" in a way that nobody but Bar can read it, all they have to do is the above. But if they do, "Bar" has absolutely no idea who sent it. GCM has some built in support for MACs, which is what you need to tag a message in a way that senders can prove they were, in fact, the sender, and also to tag date and time, though this is not easy; basically you as a server would tag any message with 'I, server, decree, and you are going to have to trust me, that this was present in the DB at this point in time and appeared recently; to me anyway. Signed, server'.
That'll give you a few terms (MAC, GCM, AES-256, ElGamal, Bouncy Castle, signatures, and a few more) to search the internet for and read up on.

It sounds to me that you are developing a chat system.
The way to fulfil your requirements you need to combine two crypto systems.
The first one is a Diffie-Hellman key exchange - in short: each party generates a private/public key pair. The public key is stored on the server. If I try to send a message to Bob I'm using your database for "Bob" and get his public key. Next I'm generating a "shared secret" with my private key and Bob's public key - this shared secret is usually 32 bytes long.
Now the second phase begins - I'm encrypting my message with an AES algorithm (best one could be "GCM"-mode) and save the encrypted message (encoded as Base64-string) in your public database (of course with any hint that the counterpart is me).
Third phase: Bob is getting the encrypted message with the note it's from Michael. Now Bob is searching in the database for Michael's public key, builds the shared secret with his (Bob's) private key and Michael's public. Some kind of magic will happen - the shared secret is the same key I used for encryption. Now Bob is been able to decrypt my message.
Just a note: beware the risks because if Bob will lose his private key (maybe stored on his stolen smartphone) he will no longer read any messages for him, so a backup of his private key is needed.

How does the hsm intervene in the encryption of a transaction with a termianl?

I'm trying to understand how the hsm and keys are linked to the transaction using a terminal. I'm supposed to develop a authentication server that receives transactions from terminals and parse them to get different data. But what I fail to understand is how the hsm is used to secure the data.

Firstly you must understand card personalization. When banks(issuers) want to issue new cards they get new BINs from Mastercard, Visa. Issuers generate new IMKs ( AC,Mac, Enc, CVC3 etc.) tied to those BINs. During card personalization every kind of IMK diversifed with PAN and loaded to card so every card get its UDK (unique derived key, UDKAC, UDKMAC etc.). Key generation and key diversification process is done with using HSM encryption software.
During emv transaction, card generate its encrypted data (cryptogram) with using its UDK and sent it to authentication server. Authentication server look transaction BIN and get card UDK with deriving IMK keys with the help of HSM. So it knows the UDK keys and calculate same cryptogram. If those cryptograms are same so server decide that card has the right keys and it accept transaction.

Where do we store key/passphrase/salt for encryption?

My app needs to encrypt some data (a user session token). Most examples I see around have a method that generates a Key using a passphrase and a salt, like:
public static Key generateKey(char[] passphrase, byte[] salt) {
...
}
My understanding is that we have three options for generating the passphrase:
Have the user enter it every time the app starts (annoying to the user).
Hard-code the passphrase into the app itself. More convenient for the user, but someone can find out what your passphrase is given your app binary.
Randomly generate a passphrase, but then we have to store the generated Key on disk. Now we've just shifted the problem to having to store the key securely on disk, which also seems impossible. If the attacker finds the generated key, big problem.
Option #1 won't work for me. Options #2 and #3 seem inherently flawed, unless I'm grossly misunderstanding how to go about this (hoping that I am). What's the recommended way to do this if we can't go with #1? Do we put in a bunch of obfuscated hoops for an attacker to jump through and hope for the best?
Thanks

"Do we put in a bunch of obfuscated hoops for an attacker to jump through and hope for the best?" Basically yes. The size and number of the hoops being how hard you want to make it.
If you are not using a server, then whatever you do to obsfucate and encrypt your data is reversible. However, you can make it REALLY hard. For example, a technique I used to protect some video assets.
Replaced the first 1024 bytes of the header (it's MP4) with 1024 bytes taken from the middle of one of the apps image assets. I tried several repairers, all of which failed to automagically recover the file - although it can be done manually. Then...
Encrypted the file using a private key which is 256 bytes taken from another image asset.
When the key is extracted, it's hashed through an algorithm which does all kinds of otherwise non-sensical maths to mangle the key.
Used a pre-compile obsfucator.
I've tried myself to reverse engineer this, even knowing how it's done, and it's so hard as to make the effort not worth the result.
There are numerous discussions on SO which summarise as; If you simply want to stop copying, make it difficult (cost vs reward) but otherwise sleep easy because there is ultimately nothing you can do. If the data is commercially sensitive, then a server coupled with system level security (e.g whole device encryption and no root) is required.

You store the salt along with the encrypted data, it is not secret information. You can derive the key on either something the user enters, or some sort of a device property: (hashed) IMEI, MAC address, etc.
Basically, think who are you protecting your data from and why. Since the user needs this, there is not much point trying to protect it from them. If you store this in a private file, other apps cannot read it on a non-rooted phone. If you want to protect it on rooted phones, encryption might help, but as long as the key resides in the app, or is derived based on something on the device, it is only making it harder, not impossible to recover.
Android does have a system-wide keystore service, but it has no public API and is subject to change. You could use that to protect your key(s), if you are willing to take the risk of your app breaking on future versions. Some details here: http://nelenkov.blogspot.com/2012/05/storing-application-secrets-in-androids.html

Encryption/decryption using shared key in java?

I have customer token which i am sending from one webapplication say app1 to another webapplication say app2. I want to encrypt the customer token
at app1 and decrypt it at app2 using key which is shared at both app1 and app2. i am not getting how to start with this? It would be very
helpful if somebody can point me some sample code or some tutotrial using shared key as there are too much encryption/decryption stuff(like symmetric key, public-private key) on net which has really confused me. Another limitation is that i have really short time for this to go in details. Thanks in advance.
EDIT:- I am looking for simple programme something like given at http://sanjaal.com/java/186/java-encryption/tutorial-java-des-encryption-and-decryption/ but using AES? Not able to find this kind of example in AES using shared key?

I would suggest this :
Have each app of yours assigned a public/private keypair and store the private key securely protected with a password in a secret key. Make sure that this is very secure. Needless to say the public certificate(which contains the public key as well) will be public. Each app will have the public key certificates of all the other app. Now when ever an app wants to communicate with other app ;
First sign(basically encrypting) the token with the sending app's private key. Then encrypt the resulting value with the public key of the app you want to send the data to.
This way the app that receives this value can be assured that no man in the middle will be able to make out what you have sent and also verify that the token has come from a trusted entity.
But if you use a shared key(symmetric key), then if the symmetric key is compromised, then all the apps will be compromised.

Usually you use a asymmetric algorithm (Eg. RSA) to encrypt a symmetric key (Eg. AES) to securely share it with another party and then your subsequent communication is encrypted with the symmetric key. That is VERY basic gist of it but there are a lot more factors to consider.
And I can see yourself getting in to VERY big trouble right now for several reasons.
You don't understand the concepts of cryptography
You are not willing to thoroughly study it
Simply wants to grab some code from the web and use it.
Really short time
I understand that you might not have a choice perhaps your employer/client wants to get this done quickly. But I thoroughly advice you NOT to go ahead unless you know what you are doing. If you are handling sensitive details such as credit card information or other critical customer information, do your self a favor and study cryptograph in depth.
It doesn't matter how strong the cryptographic algorithm is if you use it in an incorrect fashion. So you need to understand HOW to properly use each algorithm and it's advantages/disadvantages.

Complex encryption/decryption model - is this even possible?

Suppose I have a server which is publishing information (e.g. via a message bus) to four parties: A, B, C and D. All traffic can be discovered in encrypted form by any party. In order to make use of the information, obviously it would need to be decrypted:
Party A should be able to read all information (i.e. decrypt information intended for A, B and C)
Party B should be able to read information intended for B and C
Party C should only be able to read information intended for party C
Party D should be able to read information for B and D
Obviously this could be achieved by having completely separate public/private key pairs for each party and then sharing the private keys as per the requirements above. Unfortunately this does not scale nicely to hundreds of parties.
Is there a better way?
EDIT
Basically, what I would like to do is for each person to have their private key and for me to say, when encrypting a message, that it is encrypted with key = A | B | C such that this means that a person with any of key A, B or C can decrypt it. Imagine a trunk which can have n locks fitted to it, any of which can open the trunk.

Imagine a supermarket. Every single shelf is independently stockable. There are a hundred shelf stackers, each of whom stack a number of shelves. These stackers have managers, who can overview the shelves of all of their subordinates. These stock managers have section managers possibly having complex relationships, such as manager A being able to see a subset of manager B's shelves. There are a few store managers who can see everything.
I don't think the scalability problem comes from using public-key crypto. It rather comes from the complexity in your requirement (wanting to have so many configurable groups).
If you are going to send out the same encrypted message to hundreds of parties, and there can be an arbitrary subset of them that are supposed to read it, and you want to be able modify these permissions later, you need to give everyone his own key-pair.
You would then send out the message encrypted symmetrically (with a random session key) along with copies of the session key encrypted for all recipients.
If you see that very often you address the exact same subset, you may extend the validity of those session keys to span multiple messages. Then you do not need to transmit all the keys every time (you should still expire them after some time, though).
None of the shelf stackers, stock managers, section managers or store managers would have any kind of clue about public/private key cryptography
Well, their software/device would handle that.

What you want can be achieved using normal protocols based on public key cryptography. The Bouncy Castle Crypto APIs has support for both OpenPGP and CMS, either of which can work from Scala.
TO set everything up:
Every party is given a public/private key pair.
No one EVER shares the private key. The private key must stay private
Every party knows every other party's public key
The protocols allow encryption using multiple public keys. For example, if you use something like the PGPEncryptedDataGenerator, then you would call the addMethod(PGPPublicKey) method for every recipient you wish to be able to decrypt the message.
There are a lot of nuances to the API's, but the unit tests and examples will really help you navigate them.
Implementation Details
Both protocols work in fundamentally the same way.
Securely generate a random symmetric key
For every recipient, encrypt the symmetric key with the recipient's public key. All of the encrypted public keys are written into a message header.
Encrypt the message using the symmetric key. This becomes the message content.
Recipients reverse the process to decrypt the message
Search message header for key addressed to self.
Use private key to decrypt symmetric key.
Use symmetric key to decrypt the message content.

Here's a way that requires n key exchanges, but keeps the messages themselves as small as for two-party communication.
Assuming that the general form of your problem is:
There are n parties, numbered p_1 through p_n
Messages have a security level m, such that only parties p_1 to p_m can decrypt the message.
You can use this key exchange step:
Generate n AES keys, numbered k_1 to k_n.
Securely give each key k_i to all parties p_1 to p_i.
Then to send a message with security level m, just encrypt it with the key k_m.
One side-effect of this scheme is that party p_b does not have the ability to prevent party p_a from decrypting messages they send if a < b. Hopefully that's okay.

Can you use custom encryption like Blowfish?
In such a case you can share the Blowfish encryption key and store it in a, let's say, a properties file!
When the data arrives, depending on the destination and the key value in properties file you can try decrypting them