Develop Reference

How to produce a True UUID in Java?

I was under the impression that the UUID spec required a guaranteed, true, globally unique result, not unique 99.99999999999% of the time, but truly 100% of the time. From the spec:
A UUID is 128 bits long, and can guarantee
uniqueness across space and time.
It looks like java only support V3 and V4 of the UUID spec. V4 isn't truly unique. With the V3 implementation using nameUUIDFromBytes, the following results in duplicates, because the computer is too fast (edit: looping to 10 and called new Date().getTime() will produce duplicates because the computer loops faster than new Date().getTime() can produce a different value on each iteration):
String seed;
for (int i = 0; i < 10; i++) {
seed = "<hostname>" + new Date().getTime();
System.out.println(java.util.UUID.nameUUIDFromBytes(seed.getBytes()));
}
Am I mistaken in assuming that a UUID is 100% unique, and that it is only practically unique but not perfectly so? Is there anyway to do this in Java?

There are different methods of UUID generation. The kind you're using is behaving exactly as it should. You're using nameUUIDFromBytes, a "Static factory to retrieve a type 3 (name based) UUID based on the specified byte array."
This generates the same UUID if given the same name. As you've discovered, your loop is passing-in the same name every time, so you get the same UUID.
Have a look at Gabe's advice here: Which UUID version to use?
He recommends you use V4, which as others have pointed out is good enough for any realistic use case.

Because your entropy is limited to your memory, you can never ensure a UUID is "guaranteed, true, globally unique result". However, 99.99999999999% is already pretty good.
If you want to ensure unique values in your database, you could use a simple integer that's incremented to be sure it's unique. If you want to use UUIDs and be really sure they're unique, you just have to check that upon creation. If there's a duplicate, just create another one until it's unique.
Duplicates can happen, but IIRC, part of them is created dependent on your current time, so if you're just creating one every 5 minutes, you should be safe.

As others have pointed out, the type-4 UUID returned by UUID.randomUUID() is likely to be unique enough for any practical application. Cases where it's not are likely to be pathological: for example, rolling back a VM to a live snapshot, without restarting the Java process, so that the random-number generator goes back to an exact prior state.
By contrast, a type-3 or type-5 UUID is only as unique as what you put into it.
A type-1 UUID (time-based) should be very slightly "more" unique, under certain constraints. The Java platform does not include support for generating a type-1 UUID, but I've written code (possibly not published) to call a UUID generating library via JNI. It was 18 lines of C and 11 lines of Java.

Why the bson java implementation uses 4 bytes inc field?

In BSON Java implementation, an ObjectId is composed by 3 pieces (source code: http://grepcode.com/file/repo1.maven.org/maven2/org.mongodb/mongo-java-driver/2.9.0/org/bson/types/ObjectId.java#ObjectId.%3Cinit%3E%28int%2Cint%2Cint%29 ):
XXXX XXXX XXXX
-------------------------
time machine&pid inc
(each X represents a byte)
this is a bit different from what's described in document (doc: http://docs.mongodb.org/manual/core/object-id/ )
XXXX XXX XX XXX
--------------------------
time machine pid inc
(each X represents a byte)
Can anyone let me know why the java-driver didn't follow the spec?
Thanks!

I will put this as answer since it is a bit long for a comment.
There are a couple of JIRA links to this:
https://jira.mongodb.org/browse/JAVA-81
https://jira.mongodb.org/browse/JAVA-337
The second acknowledges that the spec is different under Java however makes no reference as to why.
If I were to make a guess it could be due to the way the PID and machine id in Java works, it could be related to: https://jira.mongodb.org/browse/JAVA-586.
You may find your answer better on the Google Group: mongodb-user since the maintainers hang out there.

I expect the original intent of an ObjectID was to generate a reasonably unique primary key, rather than packing fields that drivers would then start parsing as data.
As the MongoDB ecosystem has evolved, some developers have found it useful to interpret the ObjectID from multiple drivers as well as ensure consistency of generated IDs.
If you look at the BSON spec you will see there are a few subtypes for UUID used by older drivers, and various changes for interoperability. For example, there is mention on PYTHON-387 of supporting "legacy" byte orders and endianness for the C# and Java drivers.
As per JAVA-337 in the MongoDB issue tracker, the Java driver's ObjectID inconsistency is planned to be addressed in the 3.0 Java driver release.

I cannot explain why they are different, but I can tell you that the Python driver generates object ids using the same approach that the Java one does:
https://github.com/mongodb/mongo-python-driver/blob/master/bson/objectid.py

SingleColumnValueFilter has no impact on result

hy,
this question is pretty similar to SingleColumnValueFilter not returning proper number of rows .
I use four SingleColumnValueFilter's w/ operator EQUAL and add them to a FilterList with Operator MUST_PASS_ONE. the number of results is the same as w/o setting the FilterList. The value to compare is a byte[] that should be correct as I just store the values from previous results. (it is an IP address that I convert to InetAddress, new InetAddress(value as byte[]), when retrieving the data, and for the query described I just call InetAddress.getAddress which returns a byte[])
Do you have any ideas what might be the problem? Am I using the Filter wrong?
EDIT:
I also used the original values retrieved by the query as value for SingleColumnValueFilter, and there was no difference in the results, thus the byte[] contents can't be the problem.

I think I can give the answer myself, sorry for not debugging and checking all the hbase code before.
I just checked the implementation of the compare algorithm (which is lexicographically), and thus i realized that the length is not taken into account, though I thought it would be filled up w/ zero's; unfortunately it is not.
The only reasonable option would be to create a custom comparator (eg see How do you use a custom comparator with SingleColumnValueFilter on HBase?)

Why does FileChannel.map take up to Integer.MAX_VALUE of data?

I am getting following exception when using FileChannel.map
Exception in thread "main" java.lang.IllegalArgumentException: Size exceeds Integer.MAX_VALUE
at sun.nio.ch.FileChannelImpl.map(Unknown Source)
at niotest.NioTest.readUsingNio(NioTest.java:38)
at niotest.NioTest.main(NioTest.java:64)
Quickly looking into OpenJdk implementation shows that the method map(..) in FileChannelImpl takes size of type long as input. But inside the body, it compares it with Integer.MAX_VALUE and throws error if its greater than that. Why take long size as input but limit it to max integer length?
Anyone knows specific reason behind this implementation?
or is it some kind of bug?
Source URL - http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/sun/nio/ch/FileChannelImpl.java
I am running this program using 64bit JRE on 64bit Windows-2k8

It's not an implementation specific bug. The size is defined in the FileChannel.map as long, but...
size - The size of the region to be mapped; must be non-negative and no greater than Integer.MAX_VALUE
All compliant JVM implementations will be this way. I suspect the reason is a combination of history (who would need to access a file larger than 2GB? ;) and trying to push things forward in later versions of Java (it will be easier to allow values larger than Integer.MAX than it will be to change the data type from int to long.)
A lot of people find this int-based thinking in the Java API regarding anything File very confounding and short sighted. But remember, Java start development in 1995! I'm sure 2GB seemed like a relatively safe value at the time.

ByteBuffer's capacity is limited to Integer.MAX_VALUE, so there is no way to map anything larger than that.
Look at: MappedByteBuffer map(MapMode mode, long position, long size)
position has to be long for obvious reasons.
size is not necessary to be long but in any calculation it has to be promoted - for example position+size has to be a positive long. OS mapping indeed may use long to carry the mapping, map function (mmap) may need to map more than Integer.MAX_VALUE in order to preserve page size but ByteBuffer just can't use that.
Overall int lays very deep in java's design and there is no size_t C alike type, mass utilizing long instead of int will damper the performance. So in the end: if you need greater maps than 2GB, just use more than a single ByteBuffer.

Hibernate - java.lang.OutOfMemoryError: Java heap space

I get this exception:
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuilder.append(StringBuilder.java:119)
at java.util.AbstractMap.toString(AbstractMap.java:493)
at org.hibernate.pretty.Printer.toString(Printer.java:59)
at org.hibernate.pretty.Printer.toString(Printer.java:90)
at org.hibernate.event.def.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:97)
at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:35)
at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:969)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1114)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:79)
At this code:
Query query = null;
Transaction tx= session.beginTransaction();
if (allRadio.isSelected()) {
query = session.createQuery("select d from Document as d, d.msg as m, m.senderreceivers as s where m.isDraft=0 and d.isMain=1 and s.organization.shortName like '" + search + "' and s.role=0");
} else if (periodRadio.isSelected()) {
query = session.createQuery("select d from Document as d, d.msg as m, m.senderreceivers as s where m.isDraft=0 and d.isMain=1 and s.organization.shortName like '" + search + "' and s.role=0 and m.receivingDate between :start and :end");
query.setParameter("start", start);
query.setParameter("end", end);
}
final List<Document> documents = query.list();
query = session.createQuery("select o from Organization as o");
List<Organization> organizations = query.list(); <---AT THIS LINE
tx.commit();
Im making 2 consecutive queries. If i comment out 1 of them the other works fine.
if i remove transaction thingy exception dissappears. What's going on? Is this a memory leak or something? Thanks in advance.

A tip I picked up from many years of pain with this sort of thing: the answer is usually carefully hidden somewhere in first 10 lines of the stack trace. Always read the stack trace several times, and if that doesn't give enough help, read the source code of the methods where the failure happens.
In this case the problem comes from somewhere in Hibernate's pretty printer. This is a logging feature, so the problem is that Hibernate is trying to log some enormous string. Notice how it fails while trying to increase the size of a StringBuilder.
Why is it trying to log an enormous string? I can't say from the information you've given, but I'm guessing you have something very big in your Organization entity (maybe a BLOB?) and Hibernate is trying to log the objects that the query has pulled out of the database. It may also be a mistake in the mapping, whereby eager fetching pulls in many dependent objects - e.g. a child collection that loads the entire table due to a wrong foreign-key definition.
If it's a mistake in the mapping, fixing the mapping will solve the problem. Otherwise, your best bet is probably to turn off this particular logging feature. There's an existing question on a very similar problem, with some useful suggestions in the answers.

While such an error might be an indicator for a memory leak, it could also just result from high memory usage in your program.
You could try to amend it by adding the following parameter to your command line (which will increase the maximum heap size; adapt the 512m according to your needs):
java -Xmx512m yourprog
If it goes away that way, your program probably just needed more than the default size (which depends on the platform); if it comes again (probably a little later in time), you have a memory leak somewhere.

You need to increase the JVM heap size. Start it with -Xmx256m command-line param.