I need to a unique and 64-bit ID Generator?
I know that UUID is 128 bit but If exist any version of id generator in java get it to me.
It's important that the ids are not sequence.
The least significant half of a UUID in Java is very unique, and that would give you a 64bit number.
UUID.randomUUID().getLeastSignificantBits()
Also see: Likelihood of collision using most significant bits of a UUID in Java
As pointed out in the comments, the number is not guaranteed unique, but duplications are unlikely
you could use a Blowfish encryption over a normal counter, or generate a UUID4 and xor the first and last 16 bytes.
Related
What is a good algorithm to generate a unique 64-bit ID starting from multiple numeric 64-bit IDs? Example:
Input: [2, 9875, 0, 223568, ...] a list of random 64-bit IDs
Output: a unique 64-bit numeric ID, that have to be the same for the given input
I'm looking a way to avoid ID collision.
My apologies for the unclear question.
If speed does not matter, what about:
feeding all your ids in the md5-algorithm and than simply use
a) first 64 bits or
b) last 64 bits or
c) first 64 bits xor last 64 bits
If speed matters
What about:
Step 1: reorder the bytes of all 64 bit IDs (in a fixed but different order for each 64 bit ID of your Input.)(This might help a bit if the values are not really randomly distributed)
Step 2: xor all the rearranged 64 bit IDs to get the new 64 bit id.
If you have no extra information about the range of your 64 bit input IDs or the distribution of the values, there is no way to avoid collisions in a 'clever'/'best' way. Because whatever you come up with, you will always find a set of inputs which lead to collisions.
This old question will be useful for you if you want more information, but you can try this:
java.util.UUID.randomUUID().hashCode()
If the previous solution didn't work in your case, try this:
private static final AtomicLong COUNTER = new AtomicLong(System.currentTimeMillis()*100000);
Or this algorithm (GitHub link).
I want to generate time-based UUIDs, but I don't want any part of the UUID value to be based on the machine they are generated on.
Version 1 UUIDs are time-based (with the first 64 bits representing time, and the second 64 bits representing the MAC address of the machine they are generated on).
Version 4 UUIDs are not time-based, but are generated from random numbers.
In my case, I want to be able to generate multiple time-based UUIDs for the same timestamp, on the same machine - and I want those UUIDs to be unique. The UUIDs must still be time-based, so that sorting them lexicographically gives an ordering consistent with the timestamps used to create them.
Is there any good reason not to generate UUIDs with my own scheme - where the first 64 bits would represent the timestamp (like version 1) and the second 64 bits would be randomly generated (like version 4)?
ULIDs meet your requirements pretty much exactly. You may need to generate them application-side since it looks like native database support seems lacking, but it's a drop-in for a 128bit UUID so you simply need to encode it as hex to insert into any UUID type column.
If space isn't a concern you can combine the multithreaded/multi-machine collision resistance of a UUID with the lexical ordering of a timestamp:
System.currentTimeMillis() + "-" + UUID.randomUUID()
I need to generate a hash value used for uniqueness of many billions of records in Java. Trouble is, I only have 16 numeric digits to play with. In researching this, I have found algorithms for 32-bit hash, which return Java integers. But this is too small, as it only has a range of +/ 2 billion, and have will have more records that that. I cannot go to a 64-bit hash, as that will give me numeric values back that are too large (+/ 4 quintillion, or 19 digits). Trouble is, I am dealing with a legacy system that is forcing me into a static key length of 16 digits.
Suggestions? I know no hash function will guarantee uniqueness, but I need a good one that will fit into these restrictions.
Thanks
If your generated hash is too large you can just mod it with your keyspace max to make it fit.
myhash = hash64bitvalue % 10^16
If you are limited to 16 decimal digits, your key space contains 10^16 values.
Even if you find a hash that gives uniform distribution on your data set, due to Birthday Paradox you will have a 50% chance of collision on ~10^8 items of data, which is an order of magnitude less than your billions of records.
This means that you cannot use any kind of hash alone and rely on uniqueness.
A straightforward solution is to use a global counter instead. If global counter is infeasible, counters with preallocated ranges can be used. For example, 6 most significant digits denote fixed data source index, 10 least significant digits contain monotonous counter maintained by that data source.
So your restriction is 53 bit?
For my understanding order number of bit in hashcode doesn't affect its value (order and value of bit are fully independent from each other). So you could get 64-bit hash function and use only last 53 bits from it. And you must use binary operations for this ( hash64 & (1<<54 - 1) ) not arithmetic.
You don't have to store your hashes in a human readable form (hex, as you said). Just store the 64-bit long datatype (generated by a 64-bit hash function) in your database, which is only 8 bytes. And not the 19 bytes of which you were scared off.
If that isn't a solution, improve the legacy system.
Edit: Wait!
64-bit: 264 =
18446744073709551616
16 hex-digits: 1616 =
18446744073709551616
Exact fit! So make a hex representation of your 64-bit hash, and there you are.
If you can save 16 alphanumeric characters then you can use a hexadecimal representation and pack 16^16 bits into 16 chars. 16^16 is 2^64.
I was wondering what the easiest way to convert a UUID to a unique integer would be? I have tried using the hash code but people tell me that it is not going to always be unique if i use the hash code?
So what is the easiest way? Is the hash code unique?
You are going to have a problem since the UUID is 128 bits and the int is only 32bit. You'll either have to accept the risk of collisions and try to fudge it to a smaller space (hashCode is probably a good way to do that) or find an alternative (use the UUID directly, map to a BigInteger - difficult to tell without knowing why)
Answering the How can I have a unique application wide Integer:
If it needs to be unique even after restarts or if you application is clustered you may use a Database sequence.
If it just needs to be unique during the runtime use a static AtomicInteger.
EDIT (example added):
public class Sequence {
private static final AtomicInteger counter = new AtomicInteger();
public static int nextValue() {
return counter.getAndIncrement();
}
}
Usage:
int nextValue = Sequence.nextValue();
This is thread safe (different threads will always receive distinct values, and no values will be "lost")
A UUID is a 16-Byte number (128 bit). You can't crunch it into an int (32 bit) while preserving it's uniqueness.
Mathematically spoken: 296 UUIDs will share the same Java-int-size hash value (which is ... a lot ;) )
A way out - some real life UUID often have a rather static part. So in isolated scenarios, the real unique portion of UUIDs may be less then 32 bit.
We had a requirement to convert all our UUIDs into serial numbers. Finally, we tested and used the next algorithm:
Get CRC64 of uuid(16 bytes) using the ECMA polynomial
0xC96C5795D7870F42. Do not use ISO polynomial because it could cause
a lot of collisions for some UUID generation algorithms.
Now we have crc64(8 bytes). Take the first N bytes(in our case
5 in yours it will be 4 bytes for int and all bytes for int64)
We tested this method and it works well for several million UUIDs.
Our additional step: convert 5 bytes number into a number with base 36 and finally we have SN: 4YD3SOJB.
No, the hash code is not (and cannot be) unique. The thing with a GUID/UUID is that you need all 128 bits to guarantee uniqueness, therefore scaling it down in any way will give problems, see e.g. GUIDs are globally unique, but substrings of GUIDs aren't.
Honestly, I think you're better off with just using sequential integers and skip the GUID thing altogether. If you need GUIDs for any reason, then use them and don't try generating an integer from them.
You may convert uuid to BigInteger and keep it unique.
BigInteger big = new BigInteger(uuid, 16);
Please give me sample code to generate UUID of long type in java without using timestamp.
Thanks
A real UUID is 128 bits. A long is 64 bits.
This is not just pedantry. UUID stands for Universal Unique IDentifier.
The "universal uniqueness" of the established UUID schemes are based on:
encoding a MAC address and a timestamp,
encoding a hash of a DNS name and a timestamp, or
using a 122 bit random number ... which is large enough that the probability of a collision is very very small.
With 64 bits, there are simply not enough bits for "universal uniqueness". For instance, the birthday paradox means that if we had a number of computers generating random 64 bit numbers, the probability of a potentially detectable collision would be large enough to be of concern.
Now if you just want a UID (not a UUID), then any 64-bit sequence generator will do the job, provided that you take steps to guard against the sequence repeating. (If the sequence repeats, then the IDs are not unique in time; i.e. over time a given ID may denote different entities.)
Have you looked at java.util.UUID?
If you just want a simple unique long you can use AtomicLong.incrementAndGet(). This doesn't use a timestamp but does reset to 0 every time you start it and is not unique across JVMs.
What is the requirement not to use timestamps all about? UUID uses a timestamp. (amoungst other things)