Say you are working with a simple class and object creation is not heavy:
class Simple {
public final int data1;
public final float data2;
...
}
You have to continuously put simple-objects into a queue:
queue.add(new Simple(123,123f,...));
Is it faster to work with an Object-Pool and change the Simple-Class modifiable class? I hope not.
Generally, no it isn't faster. If you configure the JVM to use a "throughput" GC, you will get better performance by not attempting to recycle objects. It is only worth considering object pools if either you are "memory constrained" or if GC pauses are problematic. (The amortized cost of garbage collecting an object tends towards zero as the proportion of garbage to non-garbage increases, especially if objects "die young", as per the generational hypothesis.)
In fact:
if the application is multi-threaded, the object pool may well be a concurrency bottleneck, and
mutation of the "effectively immutable" objects may well require additional overhead and/or additional synchronization.
If the object creation rate is "about 20 objects per second", then using an object pool is unlikely to make a significant difference to performance, even if it produces an improvement (which I doubt).
Do the math.
If the object size for Simple is N bytes, and you are allocating M per second, you can estimate the number of bytes per second allocated.
If your "young space" is Y Mbytes, it will take T seconds to trigger a GC based on that allocation rate.
If the "young space" GC takes on average G ms for a space of size Y, you can estimate an upper bound on the time that could hypothetically be saved ... assuming that the object pool had zero associated overheads.
The actual saving will be less, because the "zero overheads" assumption is unrealistic.
In fact the time take to do a "young space" collection doesn't just depend on its size. It also depends on amount of non-garbage that needs to be retained. (Less non-garbage retained is better!) That can make the GC overhead hard to estimate. However, if you have already coded your application without an object pool, you can measure the average "young space" collection time for your application with a typical workload, and then plug that into the calculations above.
Object Pool makes sense if you really are creating some heavy objects or you have other requirements that specify the need of Object Pool. Otherwise I'd just go with creating new ones and relying on JVM to getting rid of all that are not needed (just remember to delete/re-assign the reference to not needed objects first)
Related
We all know that every object allocated in Java adds a weight into future garbage collection cycles, and Optional<T> objects are no different. We use these objects frequently to wrap nullable, which leads to safer code, but at what cost?
Does anyone have information on what kind of additional GC pressure optional objects add vs. simply returning nulls and what kind of impact this has on performance in high-throughput systems?
We all know that every object allocated in Java adds a weight into future garbage collection cycles,…
That sounds like a statement nobody could deny, but let’s look at the actual work of a garbage collector, considering common implementations of modern JVMs and the impact of an allocated object on it, especially objects like Optional instances which are typically of a temporary nature.
The first task of the garbage collector is to identify objects which are still alive. The name “garbage collector” puts a focus on identifying garbage, but garbage is defined as unreachable objects and the only way to find out which objects are unreachable, is via the process of elimination. So the first task is solved by traversing and marking all reachable objects. So the costs of this process do not depend on the total amount of allocated objects, but only those, which are still reachable.
The second task is to make the memory of the garbage available to new allocations. Instead of puzzling with the memory gaps between still reachable objects, all modern garbage collectors work by evacuating a complete region, transferring all alive objects withing that memory to a new location and adapting the references to them. After the process, the memory is available to new allocations as a whole block. So this is again a process whose costs do not depend on the total amount of allocated objects, but only (a part of) the still alive objects.
Therefore, an object like a temporary Optional may impose no costs on the actual garbage collection process at all, if it is allocated and abandoned between two garbage collection cycles.
With one catch, of course. Each allocation will reduce the memory available to subsequent allocations until there’s no space left and the garbage collection has to take place. So we could say, each allocation reduces the time between two garbage collection runs by the size of the allocation space divided by the object size. Not only is this a rather tiny fraction, it also only applies to a single threaded scenario.
In implementations like the Hotspot JVM, each thread uses a thread local allocation buffer (TLAB) for new objects. Once its TLAB is full, it will fetch a new one from the allocation space (aka Eden space). If there is none available, a garbage collection will be triggered. Now it’s rather unlikely that all threads hit the end of their TLAB right at the same time. So for the other threads which still have some space in their TLAB left at this time, it would not make any difference if they had allocated some more objects still fitting in that remaining space.
The perhaps surprising conclusion is that not every allocated object has an impact on the garbage collection, i.e. a purely local object allocated by a thread not triggering the next gc, could be entirely free.
Of course, this does not apply to allocating a large amount of objects. Allocating lots of them causes the thread to allocate more TLABs and eventually trigger the garbage collection earlier than without. That’s why we have classes like IntStream allowing to process a large number of elements without allocating objects, as would happen with a Stream<Integer>, while there is no problem in providing the result as a single OptionalInt instance. As we know now, a single temporary object has only a tiny impact on the gc, if any.
This did not even touch the JVM’s optimizer, which may eliminate object allocations in hot spots, if Escape Analysis has proven that the object is purely local.
For example in a service adapter you might:
a. have an input data model and an output data model, maybe even immutable, with different classes and use Object Mappers to transform between classes and create some short-lived objects along the way
b. have a single data model, some of the classes might be mutable, but the same object that was created for the input is also sent as output
There are other use-cases when you'd have to choose between clear code with many objects and less clear code with less objects and I would like to know if Garbage Collection still has a weight in this decision.
I should make this a comment as IMO it does not qualify as an answer, but it will not fit.
Even if the answer(s) are going to most probably be - do whatever makes your code more readable (and to be honest I still follow that all the time); we have faced this issue of GC in our code base.
Suppose that you want to create a graph of users (we had to - around 1/2 million) and load all their properties in memory and do some aggregations on them and filtering, etc. (it was not my decision), because these graph objects where pretty heavy - once loaded even with 16GB of heap the JVM would fail with OOM or GC would take huge pauses. And it's understandable - lots of data requires lots of memory, you can't run away from it. The solution proposed and that actually worked was to model that with simple BitSets - where each bit would be a property and a potential linkage to some other data; this is by far not readable and extremely complicated to maintain to this day. Lots of shifts, lots of intrinsics of the data - you have to know at all time what the 3-bit means for example, there's no getter for usernameIncome let's say - you have to do quite a lot shifts and map that to a search table, etc. But it would keep the GC pretty low, at least in the ranges where we were OK with that.
So unless you can prove that GC is taken your app time so much - you probably are even safer simply adding more RAM and increasing it(unless you have a leak). I would still go for clear code like 99.(99) % of the time.
Newer versions of Java have quite sophisticated mechanisms to handle very short-living objects so it's not as bad as it was in the past. With a modern JVM I'd say that you don't need to worry about garbage collection times if you create many objects, which is a good thing since there are now many more of them being created on the fly that this was the case with older versions of Java.
What's still valid is to keep the number of created objects low if the creation is coming with high costs, e.g. accessing a database to retrieve data from, network operations, etc.
As other people have said I think it's better to write your code to solve the problem in an optimum way for that problem rather than thinking about what the garbage collector (GC) will do.
The key to working with the GC is to look at the lifespan of your objects. The heap is (typically) divided into two main regions called generations to signify how long objects have been alive (thus young and old generations). To minimise the impact of GC you want your objects to become eligible for collection while they are still in the young generation (either in the Eden space or a survivor space, but preferably Eden space). Collection of objects in the Eden space is effectively free, as the GC does nothing with them, it just ignores them and resets the allocation pointer(s) when a minor GC is finished.
Rather than explicitly calling the GC via System.gc() it's much better to tune your heap. For example, you can set the size of the young generation using command line options like -XX:NewRatio=n, where n signifies the ratio of new to old (e.g. setting it to 3 will make the ratio of new:old 1:3 so the young generation will be 1 quarter of the heap). Alternatively, you can set the size explicitly using -XX:NewSize=n and -XX:MaxNewSize=m. The GC may resize the heap during collections so setting these values to be the same will keep it at a fixed size.
You can profile your code to establish the rate of object creation and how long your objects typically live for. This will give you the information to (ideally) configure your heap to minimise the number of objects being promoted into the old generation. What you really don't want is objects being promoted and then becoming garbage shortly thereafter.
Alternatively, you may want to look at the Zing JVM from Azul (full disclosure, I work for them). This uses a different GC algorithm, called C4, which enables compaction of the heap concurrently with application threads and so eliminates most of the impact of the GC on application latency.
I know that creating an object takes time and that's why the flyweight pattern exists.
What I would like to know is what increases the time of creating a single object the most?
I thought it might be the search of the a slightly larger space in the memory, but I guess it is only slightly larger than each of the fields the object has. Then maybe it is the travel to the correct address in the memory while we are looking for a value of a specific field, but then again: the only thing we added is looking for the address of the object.
There are 3 ways object creation is costly:
1) the object allocation. This is actually pretty cheap (like some nanos), however take into account that
many objects have "embedded" objects which are implicitely also allocated and
Additionally often the time of the constructor running (initializing the object) is more costly than the actual allocation.
2) any allocation consumes Eden space, so the higher the allocation rate, the more CPU is consumed by GC (NewGen GC runs more frequent)
3) CPU caches. If you allocate temporary objects (e.g. Integer when putting to HashMap, those temp objects are are put in the L1 cache evicting some other data. If you use it only once, this does not payoff. Therefore high allocation rate (especially temporarys/immutables) lead to cache misses, causing significant slowdown in case (depending on what the app is actually trying to achieve).
Another issue is life cycle. The VM can handle best short lived or very long lived objects. If your application creates a lot of middle-age-dying objects (e.g. cache's), you will get more frequent Full GC's.
Regarding flyweight patterns. It depends. If its a very smallish object, flyweight frequently will not pay off. However if your usage patterns involves many allocations of the flyweight candidate obejct, flyweight'ing will pay off. That's the reason hotspot caches 10.000 Integer objects internally by default
In modern JVMs the object creation is not as costly as it were. It mostly needs to bump the pointer. In fact, in modern JVMs, many objects are actually secretly allocated on the machine stack, and that's basically free- it takes no time at all.
And regarding flyweight pattern: flyweight pattern is not used as the object creation is costly rather it is used to minimize memory use by sharing as much data as possible with other similar objects.
As part of a memory analysis, we've found the following:
percent live alloc'ed stack class
rank self accum bytes objs bytes objs trace name
3 3.98% 19.85% 24259392 808 3849949016 1129587 359697 byte[]
4 3.98% 23.83% 24259392 808 3849949016 1129587 359698 byte[]
You'll notice that many objects are allocated, but few remain live. This is for a simple reason - the two byte arrays are allocated for each instance of a "client" that is generated. Clients are not reusable - each one can only handle one request and is then thrown away. The byte arrays always have the same size (30000).
We're considering moving to a pool (apache's GenericObjectPool) of byte arrays, as normally there are a known number of active clients at any given moment (so the pool size shouldn't fluctuate much). This way, we can save on memory allocation and garbage collection. The question is, would the pool cause a severe CPU hit? Is this idea a good idea at all?
Thanks for your help!
I think there are good gc related reasons to avoid this sort of allocation behaviour. Depending on the size of the heap & the free space in eden at the time of allocation, simply allocating a 30000 element byte[] could be a serious performance hit given that it could easily be bigger than the TLAB (hence allocation is not a bump the pointer event) & there may even not be enough space in eden available hence allocation directly into tenured which in itself likely to cause another hit down the line due to increased full gc activity (particularly if using cms due to fragmentation).
Having said that, the comments from fdreger are completely valid too. A multithreaded object pool is a bit of a grim thing that is likely to cause headaches. You mention they handle a single request only, if this request is serviced by a single thread only then a ThreadLocal byte[] that is wiped at the end of the request could be a good option. If the request is short lived relatively to your typical young gc period then the young->old reference issue may not be a big problem (as the probability of any given request being handled during a gc is small even if you're guaranteed to get this periodically).
Probably pooling will not help you much if at all - possibly it will make things worse, although it depends on a number of factors (what GC are you using, how long the objects live, how much memory is available, etc.):
The time of GC depends mostly on the number of live objects. Collector (I assume you run a vanilla Java JRE) does not visit dead objects and does not deallocate them one by one. It frees whole areas of memory after copying the live objects away (this keeps memory neat and compacted). 100 dead objects can collect as fast as 100000. On the other hand, all the live objects must be copied - so if you, say, have a pool of 100 objects and only 50 are used at a given time, keeping the unused object is going to cost you.
If your arrays currently tend to live shorter than the time needed to get tenured (copied to the old generation space), there is another problem: your pooled arrays will certainly live long enough. This will produce a situation where there is a lot of references from old generation to young - and GCs are optimized with a reverse situation in mind.
Actually it is quite possible that pooling arrays will make your GC SLOWER than creating new ones; this is usually the case with cheap objects.
Another cost of pooling comes from synchronizing objects across threads and cleaning them up after use. Both are trickier than they sound.
Summing up, unless you are well aware of the internals of your GC and understand how it works under the hood, AND have a results from a profiler that show that managing all the arrays is a bottleneck - DO NOT POOL. In most cases it is a bad idea.
If garbage collection in your case is really a performance hit (often cleaning up the eden space does not take much time if not many objects survive), and it is easy to plug in the object pool, try it, and measure it.
This certainly depends on your application's need.
The pool would work out much better as long as you always have a reference to it, this way the garbage collector simply ignores the pool and will only be declared once (you could always declare it static to be on the safe side). Although it would be persistent memory but I doubt that will be a problem for your application.
This question is limited in scope to HotSpot generations. Is there any way to programmatically find out in which generation a particular instance lives. Data such as:
Young or old generation?
If young, which survivor space?
Inside TLAB? Which thread?
Any technique (ex., BTrace, JVMTI) works so long as I can do something like this:
Object x = new Object();
HotSpotGenerationInfo info = HotSpotGenerationUtil.getInfo(x);
Beggars can't be choosers but ideally I could also learn when the instance of interest was being moved from one generation to another at the moment it happens (i.e., event callback based -- not interested in the delay & overhead implicit in polling.)
Not interested in answers that just say "no" without justification :)
As far as I know, you can not directly query which memory pool an object currently lives in. However, objects are promoted to a different memory pool by a garbage collection run, and you can query the number of major/minor gc runs since VM start using JMX. If you additionally take note of these counters when the object is created, you can reconstruct whether there was a GC since and from that which pool the object lives in.
There's an additional complication to the "count the number of GCs since the object was created" approach - it doesn't take into account premature object promotion.
If the survivor spaces are basically too small, and memory pressure from Eden (ie the rate of objects surviving at least once) is high, then objects will be promoted to tenured before they hit the full tenuring threshold.
In real examples, healthy applications will typically have non-zero percentages of premature promotion. In fact, a 0% premature promotion rate is a really bad sign - it says that your survivor spaces are much, much too big and you're wasting a lot of memory.