I am trying to generate hash key using SHA-256 which is producing 64 length String but i need key of size<=32, what is the best algorithm recommended maintaining uniqueness? Please advice.
As already indicated you loose collision resistance for each bit you drop. Hashes however are considered to be indistinguishable from random. Because of the avalanche effect, each bit of the input is "represented" by each of the bits in the hash. So you can indeed simply use the first 128 bits / 16 bytes of the output of the hash. That would still leave you with some 64 bit of collision resistance. The more or less standard way to do this is to take the leftmost bytes.
Additional hints:
To have some additional security, use the result of a HMAC with a static, randomly generated 128 bit key instead of the output of a hash;
Of course you could also encode the hash in base 64 and use that, if you can store any string instead of only hexadecimals. In that case you can fit 32 * 6 = 192 bits into the value, which would result in higher security than a SHA-1 hash (which is considered insecure nowadays, keep to SHA-2).
I'm learning about RSA cryptography, so I must understand about ke length.
Here, I found explanation about key length means.
There said:
When we say a "1024-bit RSA key", we mean that the modulus has length
1024 bits, i.e. is an integer greater than 2^1023 but lower than 2^1024.
Such an integer could be encoded as a sequence of 1024 bits, i.e. 128
bytes.
What I've got from there, 1024 bit key means the key has 1024 binary number sequence int it.
We all know, 1 byte = 8 bits. So, 1024 bits = 128 bytes. Okay, it's in binary. How about in character?
According to ASCII binary code here, each character has 8 bits binary number. So, in my mind, if key has 1024 bit length, it means the key contains of 1024/8 = 128 characters. So, I created a java program to generate prime number that has 128 numbers length. So far, the program works well.
But again, I rethought the real meaning of RSA 1024-bit. So, I googled and found this. I tested it and I get that the bit length of public key modulus is 1024. But, the public key has 309 numbers length.
Now, I really confuse.
My question: what's the real means of 1024-bit key length in RSA? As I thought or as I found here?
The example saves the key as Object Stream, which is java serialized object stream, it has some metainfo about javatypes and special format to encode binary data. To see genuine size of the number try "pub.getModulus().bitLength()". For me it prints 1024.
The important parameter is "greater than 2^1023". This means, the raw RSA encryption algorithm can represent arbitrary numbers from 0 to 2^2013-1, but not numbers greater or equal to the modulus. By Key Design the Modulus is the product of two different prime numbers and will never meet a power of 2, like 2^1024.
So the maximum number of bits you can represent is key-length-1, which are floor((key length -1)/8) complete bytes; in the best case you loose one bit of the key length.
As the PKCS-standard for mapping text to numbers considers complete bytes, you always need one more byte for representing the equivalent number; for a 1024-bit-key the maximum length of text to be encoded is 127 bytes.
The other number "309 bytes" comes from the X.509-Standard for storing/exchanging keys, a dialect of the ASN.1-Protocol; it includes RSA-type information, the modulus and the public exponent.
I have implemented an AES (CBC) algorithm in Java crypto with PKS5Padding. I noticed that for a input string size of 128 bytes; size of the cyphered text is 144 bytes. Is this normal? For my understanding AES algo should not pad this string since it is a multiplication of 16.
PKCS#7/PKCS#5 padding is always applied. Check the Wikipedia page. If your 16 byte aligned data would end with a byte 01 and you would remove the padding, then you would have one less byte of plaintext. So for AES 1 to 16 bytes are added, and having a multiple of the block size is therefore the worst situation instead of the best.
I am trying to understand the implications of using an MD5 Hash as Cassandra Key, in terms of "memory/storage consumption":
MD5 Hash of my content (in Java) = byte[] is 16 bytes long. (16 bytes is from wikipedia for generic md5, I am not shure if the java implementations also returns 16 bytes)
Hex encode this value, to be able to print it in human readable format => 1byte becomes 2hex values
I have to represent every hex value as a "character" in java => result= "two string character values" (for examle "FF" is a string of length/size = 2.)
Java uses UTF-16 => so every "string character" is encoded with two bytes. "FF" would require 2x2 bytes?
Conclusion => The MD5 Hash in Bytes format is 16 bytes, but represented as a java hex utf16 string consumes 16x2x2 = 64Bytes (in memory)!?!? Is this correct?
What is the storage Consumption in Cassandra, using this as a row-key?
If I had directly used the byte-array from the Hash function i would assume it consumes 16 bytes in Cassandra?
But if I use the hex-String representation (as noted above), can cassandra "compress" it to a 16 byets or will it also take 64bytes in cassandra? I assume 64 bytes in Cassandra, is this correct?
What kind of keys do you use? Do you use directly the outpout of an hash function or do you first encode into a hex string and then use the string?
(In MySQL I always, whenever I used a hash-key, I used the hex-string representation of it...So it is directly readable in the MySQL Tools and in the whole application. But I now realize it wastes storage???)
Maybe my thinking is completely incorrect, then it would be kind to explain where I am wrong.
Thans very much!
jens
Correct on both counts: byte[] would be 16 bytes, utf16-as-hex would be 64.
In 0.8, Cassandra has key metadata so you can tell it "this key is a byte[]" and it will display in hex in the cli.
I've got a bunch of 48-bit (6 byte) values that I need to encrypt symmetrically. The two requirements are:
The resulting encrypted value needs to also be 48-bits (6 bytes) long. They key itself can be (and would preferably be) much longer to guard again brute force attacks.
The resulting encrypted value needs to be deterministic, i.e. value A using key B will always produce encrypted value C (we encrypt on the fly and show the encrypted data to the user so need to always show the same value)
All block ciphers I've found have used a minimum block size of 64 and appear to be fixed (you can't use an arbitrary block size). Should I be thinking about a stream cipher?
I'm doing this in Java.
Note: I've seen this question and associated answers but wasn't clear on whether the suggestions would satisfy my 2nd requirement.
Consider format preserving encryption.
(Sorry, I originally misread the requirements thinking it was the INPUT data that needed to be 6 bytes.)
I don't think you can do exactly what you want with standard cryptographic algorithms:
the problem with stream ciphers is that standard ones effectively work by generating a stream of pseudorandom bits from the key and then XORing these bits with the plaintext; effectively this means that you should never use the same stream of bits twice (e.g. if you do, then XORing two ciphertexts gives you the same result as XORing the corresponding plaintexts; and in any case with 48 bits, there are only 2^48 possible bitstreams, so you can just test them all by brute force);
the problem with block ciphers is that there's no standard one as far as I'm aware that has a block size of 48 bits.
Now, that doesn't mean that a 48-bit block cipher couldn't be developed-- and indeed I dare say there are some out there-- just that none of the bog-standard ciphers that have undergone years of scrutiny from the cryptographic community have that block size.
So I would suggest options are:
relax the requirement of a 48-bit ciphertext; for example, TripleDES has a 64-bit block size and is "fairly" secure (equivalent to 112 bit security)[*];
in principle, you could implement your own block cipher with whatever block size you require, sticking as close as you can to a standard design, e.g. a Feistel network following some generally recommended design principles-- as a starting point, see Schneier, "Applied Cryptography", pp. 346ff, "Theory of Block Cipher Design".
The obvious problem with the latter option is that, whist standard block ciphers are generally based on common general principles, they adopt particular design decisions that have been subject to considerable scrutiny; yours presumably won't be.
I would also recommend standing back a bit from the problem (or perhaps explaining a bit more what you're trying to do), because it seems to be based on requirements that would normally go against good security practice (having the same plaintext always encrypt to the same ciphertext is something one would normally specifically avoid, for example). So you could have the best designed Feistel cipher in the world, but introduce some other vulnerability in how you're using it.
[*] TripleDES is generally not recommended because AES gives better security more efficiently (you might like to see some comparative timings of block ciphers that I took in Java to see just how bad it is). However, this might not matter in your particular application.
No, just "pad" your data out with some bytes you don't care about (but which are always the same if that's your requirement) so that you reach the size of a block. (If you're using an appropriate padding mode, then this will be done for you.)
I believe this is what you are looking for
http://web.cs.ucdavis.edu/~rogaway/papers/shuffle.html
This algorithm lets you construct PRP (i.e. arbitrary length block cipher) from secure PRF (e.g. sha256, blake2)
Block cipher in CTR mode has the same issue as a stream cipher.
Without a proper MAC (which require more bytes added) it will susceptible to bit flipping.
And without unique IV (which also require more bytes added) it will be just a broken implementation.
You can use a stream cipher only, if you have a unique salt for every encryption (don't even think about re-using the same salt, as that would be trivial to break).
When you have such unique values (e. g. a sequence number that's already associated with your values), you can use e.g. the stream cipher RC4-drop.
When you don't have such unique numbers already, you probably can't use a stream cipher, because you only have 48 bits for your result (so no space left for the salt.)
As for a block cipher with 48 bits - sorry, I don't know such a cipher, either. Maybe what you can do, is combining four 48 bit values into a 192 bit value, resulting in three 64 bit blocks, encode them, and then split them into four 48 bit values again. (I have no idea, if that would be possible in your situation or not?)
If you have a unique counter / sequence number associated with each plaintext value, then you can just use any block cipher in CTR (counter) mode.
To encrypt value V that is sequence number N under key K:
Expand N to the size of a block;
Encrypt N with the block cipher and key K;
Take the first 48 bits of the result and XOR them with V.
Decryption is the same. The most important thing to remember with this method is:
Never, ever use the same key and sequence number to encrypt two different values.
Sequence numbers must be unique. If your sequence restarts, you must use a new key.
There is a 48-bit block 80-bit key cipher designed in 2009 - KATAN48 (the family version of KTANTAN48 has some key scheduling issue. So far, it was not broken, and has quite high security margins, so it has passed the test of time.
Here's a proposed solution that meets both of your requirements.
What if you use a cipher with a 32-bit block size (such as Skip32) and run it twice, as described below.
You have a 48-bit value to encode, for example:
f2800af40110
Step 1:
Split this into a 32-bit value and a 16-bit value using some method. Here we'll just grab the left 4 bytes and the right 2 bytes (but in practice you could use a secret bitmask, see below).
32-bit value: f2800af4
16-bit value: 0110
Step 2:
Encrypt the first one with a secret key K1, using Skip32 and let's say we get:
Encrypted 32-bit value: b0daf2b9
Step 3:
Split this into two 16-bit values (again you could use a secret bitmask, but for this example we'll grab the left/right two bytes).
Value 1: b0da
Value 2: f2b9
Step 4:
Combine value 1 with the the 16-bit value from step 1 to get a new 32-bit value:
b0da0110
Step 5:
Encrypt the resulting 32-bit value with secret key K2, again using Skip32:
Encrypted 32-bit value: 6135d8f4
Step 6:
Combine this 32-bit value with value 2 from step 3 to get a 48-bit encrypted result.
6135d8f4f2b9
The result is both deterministic and reversible. No two inputs will produce the same output.
Note on splitting/combining values
Steps 1 and 3 above will split a value in a predictable way. I'm not sure if this introduces any weakness, but one alternative is to use a bitmask. If we want to split a 48-bit input number into 32-bits and 16-bits, we could come up with a bitmask, essentially a number with 16 1's that can dictate how we split bits of the input number to get two output numbers, as follows:
INPUT : 111100101000000000001010111101000000000100010000
BITMASK: 001010111001110000100000010001100000011001100000
| | ||| ||| | | || || ||
VALUE 1: 1 0 101 000 0 1 10 00 00 => a860
VALUE 2: 11 1 0 00 0000 010101 110 000000 10 10000 => e015c050
Similarly for steps 4 and 6 you could combine two values by interleaving bits based on a bitmask.
If you use separate bitmasks for each step and separate keys for each step you end up with 2 keys and 4 bitmasks, all of which would be needed to encrypt/decrypt values.
Quick note on Skip32 and it's use cases
"The Skip32 has the uncommon properties of being fast, creating very dissimilar encrypted values for consecutive input values, and producing output of the same size as the input. These make this cipher particularly useful for obfuscating series of sequential integers (e.g. auto-incremented database ids)."
- https://docs.rs/skip32/1.0.5/skip32/
Any thoughts about this approach from someone more experienced with cryptography than I am?