I'm trying to find a simple and accurate reference for the cost in bytes of Java 64 bit Objects. I've not been able to find this. The primitives are clearly specified but there are all these edge cases and exceptions that I am trying to figure out like padding for an Object and cost vrs. space they actually take up on the heap, etc. From the gist of what I'm reading here: http://btoddb-java-sizing.blogspot.com/ that can actually be different?? :-/
If you turn off the TLAB, you will get accurate accounting and you can see exactly how much memory each object allocation uses.
The best way to see where your memory is being used, is via a memory profiler. Worry about bytes here and there is most likely a waste of time. When you have hundreds of MB, then it makes a difference and the best way to see that is in a profiler.
BTW Most systems use 32-bit references, even in 64-bit JVMs. There is no such thing as a 64-bit Object. Apart from the header, the object will uses the same space whether it is a 32-bit JVM or using 32-bit references in a 64-bit JVM.
You are essentially asking for a simple way to get an accurate prediction of object sizes in Java.
Unfortunately ... there isn't one!
The blog posting you found mentions a number of complicating factors. Another one is that the object sizing calculation can potentially change from one Java release to the next, or between different Java implementation vendors.
In practice, you options are:
Estimate the sizes based on what you know, and accept that your estimates may be wrong. (If you take account of enough factors, you should be able to get reasonable ballpark estimates, at least for a particular platform. But accurate predictions are inherently hard work.)
Write micro benchmarks using the TLAB technique to measure the size of the objects.
The other point is that in most cases it doesn't matter if your object size predictions are not entirely accurate. The recommended approach is to implement, measure and then optimize. This does not require accurate size information until you get to the optimization stage, and at that point you can measure the sizes ... if you need the information.
Related
For what design reason is there no sizeof operator in Java? Knowing that it is very useful in C++ and C#, how can you get the size of a certain type if needed?
Because the size of primitive types is explicitly mandated by the Java language. There is no variance between JVM implementations.
Moreover, since allocation is done by the new operator depending on its argument there is no need to specify the amount of memory needed.
It would sure be convenient sometimes to know how much memory an object will take so you could estimate things like max heap size requirements but I suppose the Java Language/Platform designers did not think it was a critical aspect.
In c is useful only because you have to manually allocate and free memory. However, since in java there is automatic garbage collection, this is not necessary.
In java, you don't work directly with memory, so sizeof is usually not needed, if you still want to determine the size of an object, check out this question.
Memory management is done by the VM in Java, perhaps this might help you: http://www.javamex.com/java_equivalents/memory_management.shtml
C needed sizeof because the size of ints and longs varied depending on the OS and compiler. Or at least it used to. :-) In Java all sizes, bit configurations (e.g. IEEE 754) are better defined.
EDIT - I see that #Maerics provided a link to the Java specs in his answer.
Size of operator present in c/c++ and c/c++ is machine dependent langauge so different data types might have different size on different machine so programmes need to know how big those data types while performing operation that are sensitive to size.
Eg:one machine might have 32 bit integer while another machine might have 16 bit integer.
But Java is machine independent langauge and all the data types are the same size on all machine so no need to find size of data types it is pre defined in Java.
I would like to test how many bytes an object reference use in the Java VM that I'm using. Do you guys know how to test this?
Thanks!
Taking the question literally, on most JVMs, all references on 32-bit JVMs take 4 bytes, one 64-bit JVMs, a reference takes 8 bytes unless -XX:+UseCompressedOops has been used, in which case it takes 4-bytes.
I assume you are asking how to tell how much space an Object occupies. You can use Instrumentation (not a simple matter) but this will only give you a shallow depth. Java tends you break into many objects something which is C++ might be a single structure so it is not as useful.
However, ifyou have a memory issue, I suggest you a memory profiler. This will give you the shallow and deep space objects use and give you a picture across the whole system. This is often more useful as you can start with the biggest consumers and optimise those as even if you have been developing Java for ten years+ you will only be guessing where is the best place to optimise unless you have hard data.
Another way to get the object size if you don't want to use a profiler is to allocate a large array and see how much memory is consumed, You have to do this many times to get a good idea what the average size is. I would set the young space very high to avoid GCs confusing your results e.g. -XX:NewSize=1g
It can differ from JVM to JVM but "Sizeof for Java" says
You might recollect "Java Tip 130: Do You Know Your Data Size?" that described a technique based on creating a large number of identical class instances and carefully measuring the resulting increase in the JVM used heap size. When applicable, this idea works very well, and I will in fact use it to bootstrap the alternate approach in this article.
If you need to be fairly accurate, check out the Instrumentation framework.
This one is the one I use. Got to love those 16-byte references !
alphaworks.ibm.heapanalyzer
I have a HashMap that contains 12 million entries. It maps String values to Long values. Each string is about ten characters long. Is it possible to calculate how much memory this map will need in RAM?
You can guess, and you can make an educated guess by looking at the size of each item that goes into the map, but your best educated guess will be wrong.
JVMs have other structure used to track references and hold type information for the classes. That will add a fixed, yet unknown amount of memory to an accurate (if you can come up with an accurate input estimate) estimate.
As only some of the memory is memory directly holding the data, and some of the memory is memory used as overhead to hold the data, you need to profile your memory consumption and based your estimates on projections of the memory "growth" when using smaller maps.
Note that profiling a JVM is a tricky task, as it is optimizing memory usage in a manner that will present varying results depending on how long the JVM is running, the activity of the Map, etc. You need to do statistical sampling of the input in a variety of conditions; but, odds are good you will eventually be able to put your finger on a reasonable number. More importantly, you will also be able to say "Well it might peak up at around this number temporarily, but should settle down to this on average". Temporal changes to memory are often overlooked in static analysis.
The JVM would know because it has to allocate and manage memory but it doesn’t tell you. So, short of going native, no, there is no way to know how much memory is actually used by your objects.
A profiler will tell you how much memory is being used by the program, and which objects are using what. You might be able to find your objects' memory usage.
VisualVM is included in Java6, and will give you this information.
It's worth mentioning, though, this is not necessarily going to give you a memory 'requirement', just a view of how much memory it is using at that point in time.
This question already has answers here:
How to determine the size of an object in Java
(28 answers)
Closed 7 years ago.
I am working on calculaitng the size [memory used] of a java object [hashmap] . It contains elements of different data types [at runtime] so [ no-of-elem * size-of-element] is not that good an approach. The code right now does it by series of
if (x)
do something
else if (primitives)
lookup size and calculate
However this process is a CPU hog and in-efficient.
I am thinking of following 2 approaches instead:
Serialize the object to a buffer and get the size.
Look into java.lang.instrument to get the size
I am looking for anyones experience with these approaches for performance , efficiency, scaling etc OR if you know any better way.
P.S:
This is a background utility that I am building so the size need no be super accurate though it should be about correct. So I am willing to trade accuracy for performance
I am not interested in the deep-size [the size of objects that are refered by this object will not be computed.]
I am looking for a performance comparisons and understanding how getObjectSize() works internally ..so that I do not messup something else to improve the performance
Thanks
Use getObjectSize() method of the Instrumentation package.
Look here for implementation details:
The serialized size is definitely not the way to go, for two reasons:
In the standard java serialization there can be quite a lot of overhead which would add to the size.
It would not be any quicker than using the getObjectSize() method which we can presume will iterate over all the references, and use some kind of lookup to determine the size of the primitive values/references of an object.
If you need better performance then that really will depend on the distribution of your objects. One possiblility would be to do some random sampling of the values in your map, determine an average and calculate an estimate from this value.
For advice on how to look up a random value in a hashmap, see this question.
You may be interested in an article I wrote a while ago on how to calculate the memory usage of a Java object. It is admittedly aimed primarily at 32-bit Hotspot, although much of it applies in essence to other environments.
You can also download a simple agent for measuring Java object size from the same site which will take some of the hard work out of it for you and should work in 64-bit environments.
Note as others have I think mentioned that the serialised form of an object isn't the same as its form in memory, so using serialisation isn't suitable if you want to measure the memory footprint accurately.
Hmmm. Is there a primer anywhere on memory usage in Java? I would have thought Sun or IBM would have had a good article on the subject but I can't find anything that looks really solid. I'm interested in knowing two things:
at runtime, figuring out how much memory the classes in my package are using at a given time
at design time, estimating general memory overhead requirements for various things like:
how much memory overhead is required for an empty object (in addition to the space required by its fields)
how much memory overhead is required when creating closures
how much memory overhead is required for collections like ArrayList
I may have hundreds of thousands of objects created and I want to be a "good neighbor" to not be overly wasteful of RAM. I mean I don't really care whether I'm using 10% more memory than the "optimal case" (whatever that is), but if I'm implementing something that uses 5x as much memory as I could if I made a simple change, I'd want to use less memory (or be able to create more objects for a fixed amount of memory available).
I found a few articles (Java Specialists' Newsletter and something from Javaworld) and one of the builtin classes java.lang.instrument.getObjectSize() which claims to measure an "approximation" (??) of memory use, but these all seem kind of vague...
(and yes I realize that a JVM running on two different OS's may be likely to use different amounts of memory for different objects)
I used JProfiler a number of years ago and it did a good job, and you could break down memory usage to a fairly granular level.
As of Java 5, on Hotspot and other VMs that support it, you can use the Instrumentation interface to ask the VM the memory usage of a given object. It's fiddly but you can do it.
In case you want to try this method, I've added a page to my web site on querying the memory size of a Java object using the Instrumentation framework.
As a rough guide in Hotspot on 32 bit machines:
objects use 8 bytes for
"housekeeping"
fields use what you'd expect them to
use given their bit length (though booleans tend to be allocated an entire byte)
object references use 4 bytes
overall obejct size has a
granularity of 8 bytes (i.e. if you
have an object with 1 boolean field
it will use 16 bytes; if you have an
object with 8 booleans it will also
use 16 bytes)
There's nothing special about collections in terms of how the VM treats them. Their memory usage is the total of their internal fields plus -- if you're counting this -- the usage of each object they contain. You need to factor in things like the default array size of an ArrayList, and the fact that that size increases by 1.5 whenever the list gets full. But either asking the VM or using the above metrics, looking at the source code to the collections and "working it through" will essentially get you to the answer.
If by "closure" you mean something like a Runnable or Callable, well again it's just a boring old object like any other. (N.B. They aren't really closures!!)
You can use JMP, but it's only caught up to Java 1.5.
I've used the profiler that comes with newer versions of Netbeans a couple of times and it works very well, supplying you with a ton of information about memory usage and runtime of your programs. Definitely a good place to start.
If you are using a pre 1.5 VM - You can get the approx size of objects by using serialization. Be warned though.. this can require double the amount of memory for that object.
See if PerfAnal will give you what you are looking for.
This might be not the exact answer you are looking for, but the bosts of the following link will give you very good pointers. Other Question about Memory
I believe the profiler included in Netbeans can moniter memory usage also, you can try that