I'm doing a Secret Sharing algorithm which encrypts a a message. To do that I need a bigger than message prime and some random numbers of aproximately the same size as the message.
I can do the first with BigInteger.probablePrime(MsgSize+8) but I do not know how to do the later.
I was using Random and later SecureRandom but they don't generate numbers of a given length. My solution was to do randomInt ^ randomInt to BigInteger but is obviously a bad solution.
Some ideas?
Is it Shamir's Secret Sharing that you're implementing? If so, note that you don't actually need a prime bigger than the entire message — it's perfectly fine to break the message into chunks of some manageable size and to share each chunk separately using a fixed prime.
Also, Shamir's Secret Sharing doesn't need a prime-sized field; it's possible to use any finite field GF(pn), including in particular the binary fields GF(2n). Such fields are particularly convenient for computer implementation, since the both the secret and share chunks will then be simply n-bit bitstrings.
The only complications are that, in non-prime fields, you'll have to implement finite field arithmetic (or find an existing implementation) and that you'll need to choose a particular reducing polynomial and agree upon it. However, the former isn't really as complicated as it might seem, and the latter isn't really any harder than choosing and agreeing on a prime. (In particular, a reducing polynomial for GF(2n) can be naturally represented as an n-bit bitstring, dropping the high bit which is always 1.)
Have you tried using the same probablePrime method with a smaller size, then using a large random integer as an offset from that number? That might do the trick, just an idea.
I had the same problem (thats why i found this post).
It is a little late but maybe someone else finds this method usefull:
public static BigDecimal getBigRandom(int d)
{
BigDecimal rnd = new BigDecimal(Math.random());
BigDecimal rndtmp;
for(int i=0;i<=d;i++)
{
rndtmp = new BigDecimal(Math.random());
rndtmp = rndtmp.movePointLeft(rnd.precision());
rnd = rnd.add(rndtmp);
}
return rnd;
}
Usage:
BigDecimal x = getBigRandom(y);
every y will give you approximately 50 digits.
if you need more than (2^31-1)*50 digits simply change int to long ;-)
dont know if it is good, but works for me
Related
I have a system which communicate to external system via webservices in which we used to send random nos as msg id and same is getting stored as a primary key of table in our database. Problem here is since we have approx 80-90 k call on daily basis i have seen so many exceptions saying that duplicate primary key. I am generating random nos in java. How can i be sure that whatever random number i will generate will not be duplicated.
below is code for the generating random nos:
private static int getRandomNumberInRange(int min, int max) {
if (min >= max) {
throw new IllegalArgumentException("max must be greater than min");
}
Random r = new Random();
return r.nextInt((max - min) + 1) + min;
}
There's nothing wrong with using a random number as a primary key. You just need to make sure that numbers are chosen from a range large enough to make the chance of picking a number more than once is virtually zero.
If you generate 100k identifiers per day for 30 years, that's about 1 billion identifiers. So, using a 100-bit number will make a collision virtually impossible over that time. 13 bytes, or maybe 12 if you feel lucky.
I define "virtually zero" as 2-40. There's not much point in defining it as less than 2-50, because things like RAM and hard drives are more likely than that to suffer undetected errors. When you have to satisfy a uniqueness constraint, estimates involving a 50% chance of collision are useless.
There is nothing magic about UUIDs. They are just 122-bit numbers with a verbose encoding. They will work, but they are overkill for this application.
You need to use a large random number, and a good source of randomness. int is not large enough, and you're restricting your range to less than that with your min and max.
The rule of thumb is that you should expect a 50% chance of collision for every 2n/2 numbers, where n is the number of bits in your random number.
The Random class in java.util isn't a good source for truely random numbers (among other problems, it uses a 48 bit seed). You should use SecureRandom, and at least a long. You should also construct it outside your method to avoid the overhead of initialisation.
As others have suggested, a UUID would solve your problem.
hi i'm trying to implement a Diffie–Hellman key exchange
public static Integer secret = 100000;
public static BigInteger g = new BigInteger("5");
public static BigInteger p = new BigInteger("315791951375393537137595555337555955191395351995195751755791151795317131135377351919777977373317997317733397199751739199735799971153399111973979977771537137371797357935195531355957399953977139577337393111951779135151171355371173379337573915193973715113971779315731713793579595533511197399993313719939759551175175337795317333957313779755351991151933337157555517575773115995775199513553337335137111");
public static BigInteger public = g.pow(secret).mod(p);
but the calculation for 100000 already takes several seconds. I won't know hoch much time it would take for a 256bit number.
Is it so slow because of the implementation of BigInteger or am i off the track?
the problem is that g.pow(secret) is a really, really, REALLY big number. It is much bigger than p, and has around secret digits. If you increase secret to be in the range of a normal Diffie-Hellman secret exponent (about the same number of digits as p), your computer won't have enough memory to hold it. All the computers on earth combined wouldn't have enough memory to hold it. It's a really big number.
But g.pow(secret).mod(p) -- the final answer you want -- only has about as many digits as p, so it's a tractable number for computer to keep track of. It's just the intermediate value that is too big to deal with.
So you need to take advantage of the distributive rule for mod with integers like this -- (a * b).mod(p) == (a.mod(p) * b.mod(p)).mod(p). With that rule, you can break down the g.pow(secret) computation into lots and lots of multiplies (O(log2secret) multiplies are all that is needed), and apply a .mod(p) at each step to keep the numbers involved from getting too big.
Use the method described here (and in many other locations):
https://math.stackexchange.com/questions/36318/modulo-arithmetic-with-big-numbers
Good luck.
in Java use modPow of BigInteger, it calculates efficiently
Why were 181783497276652981 and 8682522807148012 chosen in Random.java?
Here's the relevant source code from Java SE JDK 1.7:
/**
* Creates a new random number generator. This constructor sets
* the seed of the random number generator to a value very likely
* to be distinct from any other invocation of this constructor.
*/
public Random() {
this(seedUniquifier() ^ System.nanoTime());
}
private static long seedUniquifier() {
// L'Ecuyer, "Tables of Linear Congruential Generators of
// Different Sizes and Good Lattice Structure", 1999
for (;;) {
long current = seedUniquifier.get();
long next = current * 181783497276652981L;
if (seedUniquifier.compareAndSet(current, next))
return next;
}
}
private static final AtomicLong seedUniquifier
= new AtomicLong(8682522807148012L);
So, invoking new Random() without any seed parameter takes the current "seed uniquifier" and XORs it with System.nanoTime(). Then it uses 181783497276652981 to create another seed uniquifier to be stored for the next time new Random() is called.
The literals 181783497276652981L and 8682522807148012L are not placed in constants, but they don't appear anywhere else.
At first the comment gives me an easy lead. Searching online for that article yields the actual article. 8682522807148012 doesn't appear in the paper, but 181783497276652981 does appear -- as a substring of another number, 1181783497276652981, which is 181783497276652981 with a 1 prepended.
The paper claims that 1181783497276652981 is a number that yields good "merit" for a linear congruential generator. Was this number simply mis-copied into Java? Does 181783497276652981 have an acceptable merit?
And why was 8682522807148012 chosen?
Searching online for either number yields no explanation, only this page that also notices the dropped 1 in front of 181783497276652981.
Could other numbers have been chosen that would have worked as well as these two numbers? Why or why not?
Was this number simply mis-copied into Java?
Yes, seems to be a typo.
Does 181783497276652981 have an acceptable merit?
This could be determined using the evaluation algorithm presented in the paper. But the merit of the "original" number is probably higher.
And why was 8682522807148012 chosen?
Seems to be random. It could be the result of System.nanoTime() when the code was written.
Could other numbers have been chosen that would have worked as well as these two numbers?
Not every number would be equally "good". So, no.
Seeding Strategies
There are differences in the default-seeding schema between different versions and implementation of the JRE.
public Random() { this(System.currentTimeMillis()); }
public Random() { this(++seedUniquifier + System.nanoTime()); }
public Random() { this(seedUniquifier() ^ System.nanoTime()); }
The first one is not acceptable if you create multiple RNGs in a row. If their creation times fall in the same millisecond range, they will give completely identical sequences. (same seed => same sequence)
The second one is not thread safe. Multiple threads can get identical RNGs when initializing at the same time. Additionally, seeds of subsequent initializations tend to be correlated. Depending on the actual timer resolution of the system, the seed sequence could be linearly increasing (n, n+1, n+2, ...). As stated in How different do random seeds need to be? and the referenced paper Common defects in initialization of pseudorandom number generators, correlated seeds can generate correlation among the actual sequences of multiple RNGs.
The third approach creates randomly distributed and thus uncorrelated seeds, even across threads and subsequent initializations.
So the current java docs:
This constructor sets the seed of the random number generator to a
value very likely to be distinct from any other invocation of this
constructor.
could be extended by "across threads" and "uncorrelated"
Seed Sequence Quality
But the randomness of the seeding sequence is only as good as the underlying RNG.
The RNG used for the seed sequence in this java implementation uses a multiplicative linear congruential generator (MLCG) with c=0 and m=2^64. (The modulus 2^64 is implicitly given by the overflow of 64bit long integers)
Because of the zero c and the power-of-2-modulus, the "quality" (cycle length, bit-correlation, ...) is limited. As the paper says, besides the overall cycle length, every single bit has an own cycle length, which decreases exponentially for less significant bits. Thus, lower bits have a smaller repetition pattern. (The result of seedUniquifier() should be bit-reversed, before it is truncated to 48-bits in the actual RNG)
But it is fast! And to avoid unnecessary compare-and-set-loops, the loop body should be fast. This probably explains the usage of this specific MLCG, without addition, without xoring, just one multiplication.
And the mentioned paper presents a list of good "multipliers" for c=0 and m=2^64, as 1181783497276652981.
All in all: A for effort # JRE-developers ;) But there is a typo.
(But who knows, unless someone evaluates it, there is the possibility that the missing leading 1 actually improves the seeding RNG.)
But some multipliers are definitely worse:
"1" leads to a constant sequence.
"2" leads to a single-bit-moving sequence (somehow correlated)
...
The inter-sequence-correlation for RNGs is actually relevant for (Monte Carlo) Simulations, where multiple random sequences are instantiated and even parallelized. Thus a good seeding strategy is necessary to get "independent" simulation runs. Therefore the C++11 standard introduces the concept of a Seed Sequence for generating uncorrelated seeds.
If you consider that the equation used for the random number generator is:
Where X(n+1) is the next number, a is the multipler, X(n) is the current number, c is the increment and m is the modulus.
If you look further into Random, a, c and m are defined in the header of the class
private static final long multiplier = 0x5DEECE66DL; //= 25214903917 -- 'a'
private static final long addend = 0xBL; //= 11 -- 'c'
private static final long mask = (1L << 48) - 1; //= 2 ^ 48 - 1 -- 'm'
and looking at the method protected int next(int bits) this is were the equation is implemented
nextseed = (oldseed * multiplier + addend) & mask;
//X(n+1) = (X(n) * a + c ) mod m
This implies that the method seedUniquifier() is actually getting X(n) or in the first case at initialisation X(0) which is actually 8682522807148012 * 181783497276652981, this value is then modified further by the value of System.nanoTime(). This algorithm is consistent with the equation above but with the following X(0) = 8682522807148012, a = 181783497276652981, m = 2 ^ 64 and c = 0. But as the mod m of is preformed by the long overflow the above equation just becomes
Looking at the paper, the value of a = 1181783497276652981 is for m = 2 ^ 64, c = 0. So it appears to just be a typo and the value 8682522807148012 for X(0) which appears to be a seeming randomly chosen number from legacy code for Random. As seen here. But the merit of these chosen numbers could still be valid but as mentioned by Thomas B. probably not as "good" as the one in the paper.
EDIT - Below original thoughts have since been clarified so can be disregarded but leaving it for reference
This leads me the conclusions:
The reference to the paper is not for the value itself but for the methods used to obtain the values due to the different values of a, c and m
It is mere coincidence that the value is otherwise the same other than the leading 1 and the comment is misplaced (still struggling to believe this though)
OR
There has been a serious misunderstanding of the tables in the paper and the developers have just chosen a value at random as by the time it is multiplied out what was the point in using the table value in the first place especially as you can just provide your own seed value any way in which case these values are not even taken into account
So to answer your question
Could other numbers have been chosen that would have worked as well as these two numbers? Why or why not?
Yes, any number could have been used, in fact if you specify a seed value when you Instantiate Random you are using any other value. This value does not have any effect on the performance of the generator, this is determined by the values of a,c and m which are hard coded within the class.
As per the link you provided, they have chosen (after adding the missing 1 :) ) the best yield from 2^64 because long can't have have a number from 2^128
I would like to get clarifications on Pseudo Random Number generation.
My questions are:
Is there any chance for getting repeated numbers in Pseudo Random Number Generation?
When i googled i found true random number generation. Can i get some algorithms for true random number generation, so that i can use it with
SecureRandom.getInstance(String algorithm)
Please give guidance with priority given to security.
1) Yes, you can generally have repeated numbers in a PRNG. Actually, if you apply the pigeon hole principle, the proof is quite straightforward (ie, suppose you have a PRNG on the set of 32-bit unsigned integers; if you generate more than 2^32 pseudo random numbers, you will certainly have at least one number generated at least 2 times; in practice, that would happen way faster; usually the algorithms for PRNGs will cycle through a sequence, and you have a way to calculate or estimate the size of that cycle, at the end of which every single number will start repeating, and the image of the algorithm is usually way, way smaller than the set from which you take your numbers).
If you need non-repeated numbers (since security seems to be a concern for you, note that this is less secure than a sequence of (pseudo) random numbers in which you allow repeated numbers!!!), you can do as follows:
class NonRepeatedPRNG {
private final Random rnd = new Random();
private final Set<Integer> set = new HashSet<>();
public int nextInt() {
for (;;) {
final int r = rnd.nextInt();
if (set.add(r)) return r;
}
}
}
Note that the nextInt method defined above may never return! Use with caution.
2) No, there's no such thing as an "algorithm for true random number generation", since an algorithm is something known, that you control and can predict (ie, just run it and you have the output; you know exactly its output the next time you run it with the same initial conditions), while a true RNG is completely unpredictable by definition.
For most common non security-related applications (ie, scientific calculations, games, etc), a PRNG will suffice. If security is a concern (ie, you want random numbers for crypto), then a CSPRNG (cryptographycally secure PRNG) will suffice.
If you have an application that cannot work without true randomness, I'm really curious to know more about it.
Yes, any random number generator can repeat. There are three general solutions to the non-duplicate random number problem:
If you want a few numbers from a large range then pick one and reject
it if it is a duplicate. If the range is large, then this won't cause
too many repeated attempts.
If you want a lot of numbers from a small range, then set out all the numbers in an
array and shuffle the array. The Fisher-Yates algorithm is standard for array
shuffling. Take the random numbers in sequence from the shuffled array.
If you want a lot of numbers from a large range then use an appropriately sized
encryption algorithm. E.g. for 64 bit numbers use DES and encrypt 0, 1, 2, 3, ...
in sequence. The output is guaranteed unique because encryption is reversible.
Pseudo RNGs can repeat themselves, but True RNGs can also repeat themselves - if they never repeated themselves they wouldn't be random.
A good PRNG once seeded with some (~128 bit) real entropy is practically indistinguishable from a true RNG. You certainly won't get noticeably more collisions or repetitions than with a true RNG.
Therefore you are unlikely to ever need a true random number generator, but if you do check out the HTTP API at random.org. Their API is backed by a true random source. The randomness comes from atmospheric noise.
If a PRNG or RNG never repeated numbers, it would be... really predictable, actually! Imagine a PRNG over the numbers 1 to 8. You see it print out 2, 5, 7, 3, 8, 4, 6. If the PRNG tried its hardest not to repeat itself, now you know the next number is going to be 1 - that's not random at all anymore!
So PRNGs and RNGs produce random output with repetition by default. If you don't want repetition, you should use a shuffling algorithm like the Fisher-Yates Shuffle ( http://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle ) to randomly shuffle an array of the numbers you want, in random order.
Also, if you need a source of random number generation for cryptographic purposes, seek out a provider of cryptographic PRNGs for your language. As long as it's cryptographically strong it should be fine - A true RNG is a lot more expensive (or demands latency, such as using random.org) and not usually needed.
I was wondering what the easiest way to convert a UUID to a unique integer would be? I have tried using the hash code but people tell me that it is not going to always be unique if i use the hash code?
So what is the easiest way? Is the hash code unique?
You are going to have a problem since the UUID is 128 bits and the int is only 32bit. You'll either have to accept the risk of collisions and try to fudge it to a smaller space (hashCode is probably a good way to do that) or find an alternative (use the UUID directly, map to a BigInteger - difficult to tell without knowing why)
Answering the How can I have a unique application wide Integer:
If it needs to be unique even after restarts or if you application is clustered you may use a Database sequence.
If it just needs to be unique during the runtime use a static AtomicInteger.
EDIT (example added):
public class Sequence {
private static final AtomicInteger counter = new AtomicInteger();
public static int nextValue() {
return counter.getAndIncrement();
}
}
Usage:
int nextValue = Sequence.nextValue();
This is thread safe (different threads will always receive distinct values, and no values will be "lost")
A UUID is a 16-Byte number (128 bit). You can't crunch it into an int (32 bit) while preserving it's uniqueness.
Mathematically spoken: 296 UUIDs will share the same Java-int-size hash value (which is ... a lot ;) )
A way out - some real life UUID often have a rather static part. So in isolated scenarios, the real unique portion of UUIDs may be less then 32 bit.
We had a requirement to convert all our UUIDs into serial numbers. Finally, we tested and used the next algorithm:
Get CRC64 of uuid(16 bytes) using the ECMA polynomial
0xC96C5795D7870F42. Do not use ISO polynomial because it could cause
a lot of collisions for some UUID generation algorithms.
Now we have crc64(8 bytes). Take the first N bytes(in our case
5 in yours it will be 4 bytes for int and all bytes for int64)
We tested this method and it works well for several million UUIDs.
Our additional step: convert 5 bytes number into a number with base 36 and finally we have SN: 4YD3SOJB.
No, the hash code is not (and cannot be) unique. The thing with a GUID/UUID is that you need all 128 bits to guarantee uniqueness, therefore scaling it down in any way will give problems, see e.g. GUIDs are globally unique, but substrings of GUIDs aren't.
Honestly, I think you're better off with just using sequential integers and skip the GUID thing altogether. If you need GUIDs for any reason, then use them and don't try generating an integer from them.
You may convert uuid to BigInteger and keep it unique.
BigInteger big = new BigInteger(uuid, 16);