I have figured out that Intel Ivy Bridge supplies a True Random Number Generator in hardware.
Now I am mainly programming in Java and wondering what the ways are to access it?
Does java.util.Random try to use an in-built hardware RNG for example if one is present?
Check out this project: https://code.google.com/p/lizalab-rdrand-util/
It extends java.util.Random, so it can be used just as well. Seems to only work in Linux and Mac OSX right now.
Source
Related
We are using the keytool bundled with the java installation to generate keys to do an asymmetric RSA encryption. In the light of recent events somebody asked me whats happening under the hood of the java keytool. Especially regarding the randomness of the resulting numbers. (e.g. "huh why isn't there any random user input taken like mouse movements or keyboard input?"
So what are the 'randomness sources' of the java keytool to create its keys?
I did a quick research myself however the only information I found was a post from 2000:
The keytool.exe uses the SecureRandom as basis of its random numbers.
The Sun provider for SecureRandom follows the IEEE P1363 standard,
the Sun SecureRandom provider complies with NIST's FIPS PUB 140-1 section 4.11.
The Sun provider for SecureRandom mixes in other sources of entropy with the results from the thread contention process. Among other
things this includes the current time, the state of the VM's memory
usage, system properties, and file system activity.
The algorithm can perform poorly in the absence of a JIT and so we are considering supplying an alternative provider which will take
advantage of platform specific support for an entropy gathering device
such as /dev/random or the Pentium III thermal-noise RNG.
But this was back in 2K so may be someone of you could shed some light on that and provide an update to the above (different in Java7?). Depending on your answer I would be interessted if you would advise to switch to another provider like bouncycastle...
Update:
I now assume that the keytool is using java.security.SecureRandom (and thus the default provider) as a base for its random numbers. I found another interessting article, which pointed me to the file which controls the configuration of the SecureRandom API JAVA_HOME/lib/security/java.security
In there it states the following:
Select the source of seed data for SecureRandom. By default an attempt
is made to use the entropy gathering device specified by the
securerandom.source property. If an exception occurs when accessing
the URL then the traditional system/thread activity algorithm is used.
On Solaris and Linux systems, if file:/dev/urandom is specified and it
exists, a special SecureRandom implementation is activated by default.
This "NativePRNG" reads random bytes directly from /dev/urandom. On
Windows systems, the URLs file:/dev/random and file:/dev/urandom
enables use of the Microsoft CryptoAPI seed functionality.
securerandom.source=file:/dev/urandom
Since we are on a windows system I assume that the Microsoft CryptoAPI is used. Since Win7 is used it is the CNG (CryptoAPI Next Generation). Does anybody know what 'use of the Microsoft CryptoAPI seed functionality.' means? The most probable method seems to be: CryptGenRandom function
Update: Oracle seem to have improved some issues with Java 8.
I wanted to share my findings here:
keytool.exe uses the SecureRandom as basis of its random numbers, as can be seen in its sourcecode: Keytool and CertAndKeyGen.
Normally, as the SecureRandom API states: "A cryptographically strong random number minimally complies with the statistical random number generator tests specified in FIPS 140-2, Security Requirements for Cryptographic Modules, section 4.9.1." Thus all implementations of SecureRandom should be in compliance with FIPS-140-2.
The configuration of the Security Providers (thus also for SecureRandom) is done in JAVA_HOME/lib/security/java.security. The default is (top of the list): security.provider.1=sun.security.provider.Sun
When not changing 3.) on Linux, the default implementation for SecureRandom is NativePRNG, while on Windows the default is SHA1PRNG. In our case it is a windows pc generating the keys thus it is SHA1PRNG.
When looking at the implementation the following stands out:
"Note that if a seed is not provided, we attempt to provide sufficient
seed bytes to completely randomize the internal state of the generator
(20 bytes). However, our seed generation algorithm has not been
thoroughly studied or widely deployed."
The SeedGenerator (singleton across all SHA1PRNG SecureRandom objects) has the following order for its "seed sources":
NativeSeedGenerator() (/dev/random on linux, on windows MS CryptoAPI is used)
URLSeedGenerator(url) if property “securerandom.source” is something else
ThreadedSeedGenerator fallback
Now the following problems have been found with SHA1PRNG:
Inconsistencies
Statistical bias I (pp.152, french, use translator if n.)
"A black box test with a generated file of 500MB, however, showed the existence of statistical biases of the output by an order of 15."
Statistical bias II (pp.1)
"The experimental results in this paper show that sequences produced by pseudorandom generators SHA1PRNG (in Java) could be distinguished from uniformly chosen sequences with a high probability"
Poor variance (page 12) and limited state size (p.9)
"The random bytes had grave difficulties with the STS tests, failing Monobit, Runs and the first eight Serial tests. This indicates poor variance in single bits and tuples up to eight bits." AND
"In Java adding more entropy (>160bit) to an instance will not enhance security. This limitation is alarming, since it renders the PRNGs useless for key generation > 160 bit (as
e.g., in AES' case)."
The result is:
The default implementation for all java windows installation is SHA1PRNG
SHA1PRNG behaves highly erratic
SHA1PRNG is highly proprietary - it does not comply with FIPS-140-2 (as it should be).
Unfortunately (to quote the implementors) "our seed generation algorithm has not been thoroughly studied". But with the wide distribution of java it is now widely deployed and used.
Thus the Java keygeneration mechanism (at least on windows) can be assumed as broken. As a consequence most of the authors advise to use some piece of hardware like an HSM / TRNG.
It seems that, besides details of the high level API documented here, the implementation is probably platform (Windows/Linux) AND JDK implementation (OpenJDK/OracleJDK) specific. You can see the source code of the OpenJDK implementation of NativePRNG (used on Linux) here.
Perhaps consider using the bouncycastle provider just for improved transparency and consistency across platforms.
There is actually a Paper by Michaelis, Meyer, Schwenk from Ruhr-Uni Bochum which has analyzed the Randomness in Java a bit more closely. It is called "Randomly Failed". It doesnt give Java Implementation the best marks, but it does not seem too alerting, eighter.
http://www.scribd.com/doc/131955288/Randomly-Failed-The-State-of-Randomness-in-Current-Java-Implementations
http://mail.openjdk.java.net/pipermail/security-dev/2013-March/007034.html
Note that with Java 8 some improvements around the JEP123 have been made (however it still misses a RNG which uses a more resilient algorithm (like Fortuna). You can see some discussions around JEP123 on the OpenJDK security-dev list:
http://mail.openjdk.java.net/pipermail/security-dev/2013-January/006319.html
Unless you are using a hardware security module, I really wouldn't worry about it. Subtle concerns about whether your not your random numbers are truly random fade into the background against the main concern that your keys are living on your hard disk.
Referring back to your question - keytool will hand off all cryptographic operations to a JCA/JCE provider. Each provider will have it's own SecureRandom implementation and the details of those implementations can be tricky to find online, except by digging through the source. The thread you linked to contains more information than I've seen before.
Looking for a way to read the unique ID / serial# of a USB thumb drive;
please note that
- I am looking for the value of the manufacturer, not the one Windows allocates for it.
- I need to support multiple OS (Windows, Unix, Mac), thus needs to be a Java solution
The idea is to be able to distinguish between different USB thumb drives.
RXTX is the way to go. In the world of model trains, JMRI (Java Model Railroad Interface) has become very popular. JMRI runs on all platforms (Windows, Linux and Mac) and communicates with a variety of USB based devices (command stations). RXTX is in fact used by JMRI.
You might give a look at the following projects:
javax-usb and jusb. They seem to support Linux and Windows.
Anyway, since USB access in Java requires the use of native libraries, you might not achieve the required portability.
I've never tried using it (it's been on my todo list for a good few months now), but there is the "marge" project on java.net:
http://marge.java.net/
This should let you connect to bluetooth devices (although I don't think it is 100% feature complete, there is demo code on there), and then the ClientDevice class has a "getBluetoothAddress" method which I believe should be unique to that device
http://marge.java.net/javadoc/v06/marge-core/net/java/dev/marge/entity/ClientDevice.html
As I say though, I've never tried it...
I have never investigated this thoroughly, but from memory the RXTX library implementation of the javax.comm packages are supposedly very good and now have USB support.
I have used sha1prng in both my android program and java program as the pseudo ramdom number generator algorithm. I seeded both of them with the same value.
But the sequesnce generated in android is different from the one generated in java. Why is this happening and what is the solution to this problem?
I think it's because SHA1PRNG implementations on Windows and Android are different. Android uses Crypto as the provider while Windows SDK uses Sun JCE provider as SHA1PRNG implementation. The output sequence with the same seed differes even in different versions of JDK as it's discussed here: http://www.derkeiler.com/Newsgroups/sci.crypt/2006-04/msg00765.html. For different implementations of SHA1PRNG you may want to check this link: http://www.cigital.com/justice-league-blog/2009/08/14/proper-use-of-javas-securerandom/.
I am after a java implementation of ORDPACK (like what SciPy does) or other similar regression algorithm which considers errors in X and Y.
I have looked through the nist javanumerics site for anything, however none of it is relevant.
At the very least I require a library that can do linear regression with errors in X and Y.
Anyone know of any?
Regards,
Inci
The WEKA machine learning package is quite nice and should provide you with everything you need.
Personally I like using R for anything statistics related. You can use JRI to make calls to R in Java or RServe to call R from Java over TCP/IP.
All packages will allow you to calculate linear regressions and much much more.
My problem is in trying to solve a Binary Integer Program through Java. I want to run a series of experiments and an integral component of these experiments is to solve an integer program where the variables are constrained to be between 0 and 1.
In the past I have solved such problems in MatLab, with the function bintprog. In the search for such a function (or class? I'm very new to Java) to use in Java, I have come up empty handed.
Is there a Java library available to solve Integer Programs that has really good documentation?
In my search, I have seen suggestions to use a package called LP_Solve that has had a Java wrapper built around it, and a similar wrapper built for a package called GLPK (wrappers here and here) (which I have used before). The problem with these tools is that they are not strictly designed for Java, and thusly, lack the kind of documentation that I feel I need, and even worse have complicated instructions to even begin using them in my own code. As I am currently learning the Java language I am wondering if there are any really good packages available to solve Binary Integer Programs, Mixed Integer Linear Programs, or just Integer Programs from my own Java code.
As a side note, I really do not want to switch to another language because I am building off of past code and classes that perform the tasks I desire.
How about Java Integer Linear Program Solver (JILPS)?
LP_Solve with the Java wrapper is what I will be using. It is a free Mixed Integer Linear Program solver. LP_Solve for Java is very easy to install following these instructions. Included in the packages you download are files with lots of example code, which I have found useful. The only part of the installation that slowed me down was having to join the Yahoo group to find the files for download.
IBM cplex even though being c library has java wrappers and documentation
if you want values 0 or 1 try the bool datatype...if your looking at probabilities (that lie between 0 to 1)try limiting a float value such that it is >= 0 and <=1;