We are using the keytool bundled with the java installation to generate keys to do an asymmetric RSA encryption. In the light of recent events somebody asked me whats happening under the hood of the java keytool. Especially regarding the randomness of the resulting numbers. (e.g. "huh why isn't there any random user input taken like mouse movements or keyboard input?"
So what are the 'randomness sources' of the java keytool to create its keys?
I did a quick research myself however the only information I found was a post from 2000:
The keytool.exe uses the SecureRandom as basis of its random numbers.
The Sun provider for SecureRandom follows the IEEE P1363 standard,
the Sun SecureRandom provider complies with NIST's FIPS PUB 140-1 section 4.11.
The Sun provider for SecureRandom mixes in other sources of entropy with the results from the thread contention process. Among other
things this includes the current time, the state of the VM's memory
usage, system properties, and file system activity.
The algorithm can perform poorly in the absence of a JIT and so we are considering supplying an alternative provider which will take
advantage of platform specific support for an entropy gathering device
such as /dev/random or the Pentium III thermal-noise RNG.
But this was back in 2K so may be someone of you could shed some light on that and provide an update to the above (different in Java7?). Depending on your answer I would be interessted if you would advise to switch to another provider like bouncycastle...
Update:
I now assume that the keytool is using java.security.SecureRandom (and thus the default provider) as a base for its random numbers. I found another interessting article, which pointed me to the file which controls the configuration of the SecureRandom API JAVA_HOME/lib/security/java.security
In there it states the following:
Select the source of seed data for SecureRandom. By default an attempt
is made to use the entropy gathering device specified by the
securerandom.source property. If an exception occurs when accessing
the URL then the traditional system/thread activity algorithm is used.
On Solaris and Linux systems, if file:/dev/urandom is specified and it
exists, a special SecureRandom implementation is activated by default.
This "NativePRNG" reads random bytes directly from /dev/urandom. On
Windows systems, the URLs file:/dev/random and file:/dev/urandom
enables use of the Microsoft CryptoAPI seed functionality.
securerandom.source=file:/dev/urandom
Since we are on a windows system I assume that the Microsoft CryptoAPI is used. Since Win7 is used it is the CNG (CryptoAPI Next Generation). Does anybody know what 'use of the Microsoft CryptoAPI seed functionality.' means? The most probable method seems to be: CryptGenRandom function
Update: Oracle seem to have improved some issues with Java 8.
I wanted to share my findings here:
keytool.exe uses the SecureRandom as basis of its random numbers, as can be seen in its sourcecode: Keytool and CertAndKeyGen.
Normally, as the SecureRandom API states: "A cryptographically strong random number minimally complies with the statistical random number generator tests specified in FIPS 140-2, Security Requirements for Cryptographic Modules, section 4.9.1." Thus all implementations of SecureRandom should be in compliance with FIPS-140-2.
The configuration of the Security Providers (thus also for SecureRandom) is done in JAVA_HOME/lib/security/java.security. The default is (top of the list): security.provider.1=sun.security.provider.Sun
When not changing 3.) on Linux, the default implementation for SecureRandom is NativePRNG, while on Windows the default is SHA1PRNG. In our case it is a windows pc generating the keys thus it is SHA1PRNG.
When looking at the implementation the following stands out:
"Note that if a seed is not provided, we attempt to provide sufficient
seed bytes to completely randomize the internal state of the generator
(20 bytes). However, our seed generation algorithm has not been
thoroughly studied or widely deployed."
The SeedGenerator (singleton across all SHA1PRNG SecureRandom objects) has the following order for its "seed sources":
NativeSeedGenerator() (/dev/random on linux, on windows MS CryptoAPI is used)
URLSeedGenerator(url) if property “securerandom.source” is something else
ThreadedSeedGenerator fallback
Now the following problems have been found with SHA1PRNG:
Inconsistencies
Statistical bias I (pp.152, french, use translator if n.)
"A black box test with a generated file of 500MB, however, showed the existence of statistical biases of the output by an order of 15."
Statistical bias II (pp.1)
"The experimental results in this paper show that sequences produced by pseudorandom generators SHA1PRNG (in Java) could be distinguished from uniformly chosen sequences with a high probability"
Poor variance (page 12) and limited state size (p.9)
"The random bytes had grave difficulties with the STS tests, failing Monobit, Runs and the first eight Serial tests. This indicates poor variance in single bits and tuples up to eight bits." AND
"In Java adding more entropy (>160bit) to an instance will not enhance security. This limitation is alarming, since it renders the PRNGs useless for key generation > 160 bit (as
e.g., in AES' case)."
The result is:
The default implementation for all java windows installation is SHA1PRNG
SHA1PRNG behaves highly erratic
SHA1PRNG is highly proprietary - it does not comply with FIPS-140-2 (as it should be).
Unfortunately (to quote the implementors) "our seed generation algorithm has not been thoroughly studied". But with the wide distribution of java it is now widely deployed and used.
Thus the Java keygeneration mechanism (at least on windows) can be assumed as broken. As a consequence most of the authors advise to use some piece of hardware like an HSM / TRNG.
It seems that, besides details of the high level API documented here, the implementation is probably platform (Windows/Linux) AND JDK implementation (OpenJDK/OracleJDK) specific. You can see the source code of the OpenJDK implementation of NativePRNG (used on Linux) here.
Perhaps consider using the bouncycastle provider just for improved transparency and consistency across platforms.
There is actually a Paper by Michaelis, Meyer, Schwenk from Ruhr-Uni Bochum which has analyzed the Randomness in Java a bit more closely. It is called "Randomly Failed". It doesnt give Java Implementation the best marks, but it does not seem too alerting, eighter.
http://www.scribd.com/doc/131955288/Randomly-Failed-The-State-of-Randomness-in-Current-Java-Implementations
http://mail.openjdk.java.net/pipermail/security-dev/2013-March/007034.html
Note that with Java 8 some improvements around the JEP123 have been made (however it still misses a RNG which uses a more resilient algorithm (like Fortuna). You can see some discussions around JEP123 on the OpenJDK security-dev list:
http://mail.openjdk.java.net/pipermail/security-dev/2013-January/006319.html
Unless you are using a hardware security module, I really wouldn't worry about it. Subtle concerns about whether your not your random numbers are truly random fade into the background against the main concern that your keys are living on your hard disk.
Referring back to your question - keytool will hand off all cryptographic operations to a JCA/JCE provider. Each provider will have it's own SecureRandom implementation and the details of those implementations can be tricky to find online, except by digging through the source. The thread you linked to contains more information than I've seen before.
Related
I've been told to encrypt some form data (strings) using AES256 and was given a nice library which already does it all. I was just trying to make sure I understand it a bit better and learn a bit about encryption since it isn't something I am ever comfortable with. While doing that I ran a test I saw on some website, it said to call this Cipher.getMaxAllowedKeyLength("AES") which gives you the maximum key length. The result was 128.
Anyways the max allowed key length is 128, does that mean I can not use AES256? or are those unrelated?
EDIT: I should mention that I do know how to get the unlimited policy files to change this, I'm just trying to understand this whole deal better before proceeding.
For US export restriction reasons, Java ships with 128-bit security by default only. You need to download and install the Java Cryptography Extension if you want to work with 256-bit+ security.
The getMaxAllowedKeyLength() has been introduced just for this purpose, otherwise you would have to handle an exception during the Cipher encryption/decryption operations (update and doFinal) to test if the restrictions apply. As the policy files may change in time or for different versions of Java, it is easier to test with a method.
Note that getMaxAllowedKeyLength() should not be used for any other reason than testing for restrictions. Notably, it may well return Integer.MAX_VALUE instead of a valid key size.
And of course, if it returns 128 you cannot use AES with a key size of 256.
To remedy this you need to install the Unlimited Strength Jurisdiction Policy Files for the Oracle JRE / JDK and then copy it into the (jre)/lib/security folder of all the Java installations where you want to use larger key sizes. You can overwrite the files that are already there. You may need local admin rights or similar rights on that folder to do so.
If that is not possible you could use another implementation of AES that doesn't require the Cipher class as this class actually enforces the limitations. There are a few tricks around this issue as well.
I have used sha1prng in both my android program and java program as the pseudo ramdom number generator algorithm. I seeded both of them with the same value.
But the sequesnce generated in android is different from the one generated in java. Why is this happening and what is the solution to this problem?
I think it's because SHA1PRNG implementations on Windows and Android are different. Android uses Crypto as the provider while Windows SDK uses Sun JCE provider as SHA1PRNG implementation. The output sequence with the same seed differes even in different versions of JDK as it's discussed here: http://www.derkeiler.com/Newsgroups/sci.crypt/2006-04/msg00765.html. For different implementations of SHA1PRNG you may want to check this link: http://www.cigital.com/justice-league-blog/2009/08/14/proper-use-of-javas-securerandom/.
I am looking for a good library that can perform pairing based cryptography (PBC).
One I could find is jPBC
What have others used? and their experiences?
The idea is to test the performance of algorithms using standard pairings (Weil, Tate) as well as some of the newer proposals without getting my hands too much dirty in the math.
I do work in this space as well. The best thing we could find in java was jPBC. Its not very good.
Non java alternatives:
MIRACL: I believe this is the current
fastest c implementation.
charm crypto: a python framework for
rapidly prototyping crypto systems. Full disclosure, I am a dev on
it. It has bindings to some subset of MIRACL and Lynn's PBC lib.
These are more than enough to impliment most schemes.These subsets
are expanding and probably can be readily expanded without getting
into the math involved. Given specific requests, we might even be willing to do those
extensions
Furthermore, it already has support for benchmarking that can
give specific time spent in cryptographic operations, the number of operations (e.g.
pairings and exponentiations), and other stats.
An open-source java implementation of Miracl is at
https://dsl-external.bbn.com/tracsvr/openP3S/wiki/jmiracl
Includes benchmarks
MIRACL is the gold standard for Elliptic Curve Cryptography over GF(p) and GF(2m) and additionally supports even more esoteric Elliptic Curves and Lucas function-based schemes. It also includes over twenty protocols based on the new paradigm of Pairing-Based Cryptography. Using MIRACL means that AES encryption, RSA public key cryptography, Diffie-Hellman Key exchange and DSA digital signature are all just a few procedure calls away.
You can read more about MIRACL here and download the SDK: https://github.com/miracl/MIRACL
I want to encrypt/decrypt lots of small (2-10kB) pieces of data. The performance is ok for now: On a Core2Duo, I get about 90 MBytes/s AES256 (when using 2 threads). But I may need to improve that in the future - or at least reduce the impact on the CPU.
Is it possible to use dedicated AES encryption hardware with Java (using JCE, or maybe a different API)?
Would Java take advantage of special CPU features (SSE5?!), if I get a better CPU?
Or are there faster JCE providers? (I tried SunJCE and BouncyCastle - no big difference.)
Other possiblilities?
The JVM will not, by itself, take advantage of special CPU features when executing code which happens to be an AES encryption: recognizing some code as being an implementation of AES is beyond the abilities of the JIT compiler. To use special hardware (e.g. the "Padlock" on VIA processors, or the AES-NI instructions on the newer Intel processors), you must go, at some point, through "native code".
Possibly, a JCE provider could do that for you. I am not aware of any readily available JCE provider which includes optimized native code for AES (there was a project called Apache JuiCE, but it seems to be stalled and I do not know its status). However it is conceivable that SunJCE will do that in a future version (but with Oracle buying Sun and the overfeaturism of OpenJDK 7, it is unclear when the next Java version will be released). Alternatively, bite the bullet and use native code yourself. Native code is invoked through JNI, and for the native AES code, a popular implementation is the one from Brian Gladman. When you get a bigger and newer processor with the AES-NI instruction, replace that native code with some code which knows about these instructions, as Intel describes.
By using AES-128 instead of AES-256 you should get a +40% speed boost. Breaking AES-128 is currently beyond the technological reach of Mankind, and should stay so for the next few decades. Do you really need a 256-bit key for AES ?
Just in case people run into this. JAVA 8 now uses AES-NI. See this: AES-NI intrinsics enabled by default?
You can benefit from improved AES speeds by using SunPKCS11 security provider together with mozilla-nss library.
Setup is described at
Improved Advanced Encryption Standard (AES) Crypto performance on Java with NSS using Intel® AES-NI Instructions
and JavaTM PKCS#11 Reference Guide
A simple google search will identify some JCE providers which claim hardware acceleration Solaris Crypto Framework. I have heard the break-even point is 4K (where under 4k its faster to perform using in JVM java providers).
I might look at using the NSS implementation, it might have some compiler optimizations for your platform (and you can certainly build from source with them enabled); though I have not used it myself. The big benefit with hardware a provider is probably the fact that the keys can be stored in hardware in a way that supports using them without exposing them to the OS.
Update: I should probably mention that the Keyczar source had some helpful insight (somewhere in source or surrounding docs) about reducing the overhead for initializing the Cipher. It also does exactly what you want (see Encrypter), and seems to implement asynchronous encryption (using a thread pool).
I would also suggest using AES-128 rather than 256. If the code is loosely coupled, and is still around in however many years it takes for AES-128 to become archaic, my guess is that it will be much easier to update the encryption at that point (when hardware will be more powerful) than it would be to try to optimize performance via hardware now.
Of course, that is assuming it is loosely coupled :D
Usually the step which consumes more time is Initiating the KeyGenerator.
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256); // This step takes more time
KeyGenerator aesKey = keyGen.generateKey();
The way I solved it is by generating a pool of KeyGenerator instances before the server statup and reusing them for just for Key Generation
I've been working on a system that uses asymmetric encryption in a large number of files. I'm currently using RSA with 4096-bit keys to encrypt a 256-bit randomly generated AES key for each file, but performance is somewhat lacking, as one required operation is to scan through all the files (estimated number when the system is in use is around 10,000) and identify which ones can be decrypted using a specific private key. While I don't expect this operation to be instant, it is taking too long at the moment (~2 files processed per second). I considered reducing the key length, but even taking it down to 2048 bits doesn't provide the level of performance I need. 512 bits would just about cut it, but as such keys can now be cracked trivially that is out of the question.
Can anybody point me in the direction of a system that is faster but of similar cryptographic strength? It would need to be implemented via a Java JCA provider (e.g. something like bouncycastle) in order to plug in to my existing application neatly. I know bouncy castle supports El Gamal, but I can't find any details on how strong this algorithm is, or if it is even likely to be any faster than RSA. I also hear about elliptic curve systems that only need relatively short keys (384 bits or the like), but don't know where to find an implementation of one of these.
For your question as asked, try Diffie-Hellman over elliptic curves, also known as "ECDH". Estimating security is a bit difficult once we deal with sizes that cannot be cracked with current technology, since this depends on how we bet on future technological evolutions. Yet one can say that ECDH over the P-256 curve provides "128 bits" of security, a level which is similar to what you would get from 2048-bit RSA. That level is widely sufficient for all current usages, or, more appropriately said, if P-256 is not enough for you then your problem has very special needs and cryptographic strength is likely to be the least of your worries.
On my PC (a 2.4 Ghz Intel Core2, 64-bit mode, running Linux), OpenSSL claims to crunch out about 900 ECDH instances per second, using a single core.
Edit: for estimation of key security, depending on the length, for several algorithms, see this site.
Why don't you calculate a cryptographically strong hash of each key, and then store that in the clear with each filename? Then, given a key that you need to match against all the files, you can simply hash the key and look it up in the table.
I'd go for an approach that requires less RSA operations. SSL/TLS, although they use RSA etc for encrypting AES etc keys, do not use AES for the data simply because it is a computationally expensive operation at sufficiently large key sizes for security to be done on a per-packet, or in your case, per-file basis.
Another public key system is: http://en.wikipedia.org/wiki/ElGamal_encryption. Security-wise I believe it has yet to be broken but would personally put my trust in RSA for now. I do not know if there are any elliptic curve encryption algorithms currently available - that is to say I know they are being researched but understand they may not be ready for production use and I heard there were patent issues.