I'm writing some stuff that uses ByteBuffers. In the docs of the API it says
There is no way to free a buffer explicitly (without JVM specific
reflection). Buffer objects are subject to GC and it usually takes two
GC cycles to free the off-heap memory after the buffer object becomes
unreachable.
However in a SO post's accepted answer I read
BigMemory uses the memory address space of the JVM process, via direct
ByteBuffers that are not subject to GC unlike other native Java
objects.
Now what should I do, shall I free the created buffer? Or do I misunderstand something in the docs or the answer?
It depends how you create the buffer, there are many possible use cases. Regular ByteBuffer.allocate() will be created on the heap and will be collected by the GC. Other options e.g. native memory might not.
Terracotta BigMemory is a type of native off-heap memory which is not governed by the JVM GC. If you allocate a buffer in this type of memory you have to clear it yourself.
It might be a good idea to clear the buffer even if it's allocated in the heap memory. GC will take care of collecting unused buffer it but this will take some time.
As the documentation of the BufferUtils in LWJGL also say: There is no way to explicitly free a ByteBuffer.
The ByteBuffer objects that are allocated with the standard mechanism (namely, by directly or indirectly calling ByteBuffer#allocateDirect) are subject to GC, and will be cleaned up eventually.
The answer that you linked to seems to refer to the BigMemory library in particular. Using JNI, you can create a (direct) ByteBffer that is not handled by the GC, and where it is up to you to actually free the underlying data.
However, a short advice: When dealing with LWJGL and other libraries that rely on (direct) ByteBuffer objects for the data transfer to the native side, you should think about the usage pattern of these buffers. Particularly for OpenGL binding libraries, you'll frequently need a ByteBuffer that only has space for 16 float values, for example (e.g. containing a matrix that is sent to OpenGL). And in many cases, the methods that do the data transfer with these buffers will be called frequently.
In such a case, it is usually not a good idea to allocate these small, short-lived buffers repeatedly:
class Renderer {
void renderMethodThatIsCalledThousandsOfTimesPerSecond() {
ByteBuffer bb = ByteBuffer.allocateDirect(16 * 4);
fill(bb);
passToOpenGL(bb);
}
}
The creation of these buffers and the GC can significantly reduce performance - and distressingly in the form of GC pauses, that could cause lags in a game.
For such cases, it can be beneficial to pull out the allocation, and re-use the buffer:
class Renderer {
private final ByteBuffer MATRIX_BUFFER_4x4 = ByteBuffer.allocateDirect(16 * 4);
void renderMethodThatIsCalledThousandsOfTimesPerSecond() {
fill(MATRIX_BUFFER_4x4);
passToOpenGL(MATRIX_BUFFER_4x4);
}
}
Related
I did search a lot and checked multiple answers.. none of them is clear to me.
Java has ByteBuffer. It has 2 flavors. 1 is direct and the other one is non-direct.
The direct buffer is good for IO.
What is the need for non-direct bytebuffer when we have byte[]? When to use that?
non-direct ByteBuffers are stored in the heap and are backed by an underlying byte array. These are typically used when you need a buffer that is readable and writable by the Java application, but doesn't need the same level of performance as a direct ByteBuffer.
So why not always use direct ByteBuffer?
Garbage Collection: Non-direct ByteBuffers are subject to garbage collection, which can free up memory automatically when it is no longer needed. With direct ByteBuffers, you have to manually free the memory.
Concurrency: Direct ByteBuffers are not thread-safe and require explicit synchronization in order to be safely accessed by multiple threads, which can add complexity and overhead to your code.
Complexity: Direct ByteBuffers often require more manual handling and can involve working with native code, which can make them more complex and harder to work with than non-direct ByteBuffers.
Increased Latency: Direct ByteBuffers can have increased latency compared to non-direct ByteBuffers, as the memory is allocated outside of the Java heap and must be transferred to and from the native heap.
Performance Variation: Performance with direct ByteBuffers can vary depending on the underlying system and hardware, making it harder to predict and guarantee performance.
I always used MemoryUtil to store float buffers, but people seem to use BufferUtils for it:
private IntBuffer convertToIntBuffer(int[] data) {
IntBuffer buffer = BufferUtils.createIntBuffer(data.length);
return buffer.put(data).flip();
}
private FloatBuffer convertToFloatBuffer(float[] data) {
FloatBuffer buffer = MemoryUtil.memAllocFloat(data.length);
return buffer.put(data).flip();
}
LWJGL 3's org.lwjgl.BufferUtils class is only a small facade over Java's java.nio.ByteBuffer.allocateDirect() method, allowing you to use the JVM's memory allocator to allocate off-heap memory and return a NIO ByteBuffer (or typed view thereof), including making sure that the ByteOrder is nativeOrder().
The NIO Buffer allocated by ByteBuffer.allocateDirect() is managed by the JRE internally and the native memory is freed implicitly as part of the garbage collection cycle once it becomes unreachable.
There are a lot of downsides to allocating off-heap memory with this approach, such as (quote from "Memory management in LWJGL 3"):
-begin-of-quote-
It is slow, much slower than the raw malloc() call. A lot of overhead on top of a function that is already slow.
It scales badly under contention.
It arbitrarily limits the amount of allocated memory (-XX:MaxDirectMemorySize).
Like Java arrays, the allocated memory is always zeroed-out. This is not necessarily bad, but having the option would be better.
There's no way to deallocate the allocated memory on demand (without JDK-specific reflection hacks). Instead, a reference queue is used that usually requires two GC cycles to free the native memory. This quite often leads to OOM errors under pressure.
-end of quote-
LWJGL 3's org.lwjgl.system.MemoryUtil class on the other hand allows you to use other native/off-heap memory allocators instead of the JVM's ByteBuffer allocator to allocate off-heap native memory, including the option of just giving you the raw virtual memory address as a long, avoiding the NIO Buffer instance.
LWJGL supports the system allocator of the C standard library (malloc) as well as currently jemalloc and rpmalloc. All of these provide a very much faster alternative to Java's ByteBuffer.allocateDirect(), alleviating the above mentioned drawbacks.
Because the native memory is not managed by the JVM anymore, you have to free the memory yourself, for which there is the org.lwjgl.system.MemoryUtil.memFree() method.
Before you continue, you should however read the mentioned LWJGL blog post in its entirety, as there are more options, such as org.lwjgl.system.MemoryStack, for allocating native off-heap memory in particular situations (such as short-lived memory), which is even faster than all the other alternatives above mentioned.
we meet a problem that about jvm gc. we have a large QPS application that jvm heap memory increased very fast. it will use more than 2g heap memory at a few seconds, then gc triggers that will collected more than 2g memory every time also very frequency。GC collect case like below picture.so this have two problems
gc need some time. what is more, it is frequent.
system will not stabilized.
I abstract the problem like below code. System allocate short-lived object fast.
public static void fun1() {
for(int i = 0; i < 5000; i++) {
Byte[] bs = new Byte[1024 * 1024 * 5];
bs = null;
}
}
so, I have some questions:
many say that set object equals null will let gc thread collect easy。what the mean of that? we all know that minor GC is always triggered when the JVM is unable to allocate space for a new object.Thus, whether a object
is null, gc will triggered only when space is not enough. so set object is null is not meaningful.
how to optimize this if exists large short-lived object? I means how to collect this objects not when young generation is not enough.
Any suggestions will help me.
I would say you need to cache them. Instead of recreating them every time try to find a way to reuse already created ones.
Try to allocate more memory to your app with the option
-Xms8g -Xmx8g
gc is called when there is not enough memory so if you have more gc won't be called so often.
It's hard to suggest something valuable without the huge objects nature. Try to add an example of such an object as well as how to you create/call them.
If you have big byte arrays inside the short lived objects try to place them out of java (e.g. in a file and keep file path only).
many say that set object equals null will let gc thread collect easy
This question is covered by several others on stack overflow so i won't repeat those here. E.g. Is unused object available for garbage collection when it's still visible in stack?
how to optimize this if exists large short-lived object? I means how to collect this objects not when young generation is not enough.
Increase the size of the young generation. So that it can contain the object. Or use G1GC which can selectively collect regions containing such objects instead of the entire old generation.
You could also try using direct byte buffers which will allocate the memory outside the heap thus reducing GC pressure. But the memory will stay allocated until the Buffers pointing to the memory get collected.
Other than that you should redesign your code to avoid such frequent, large allocations. Often one can use object pools or thread-local variables as scratch memory.
I am in a position where I want to pass byte[] to a native method via JNA. All the examples I've found about this sort of thing either use a Memory instance or use a directly-allocated ByteBuffer and then get a Pointer from that.
However, when I read the docs they say that underlying native memory -- which as I understand it is allocated "off the books" outside of the JVM-managed heap -- these Java objects consume only get freed when the objects' finalize() method is called.
But when that finalizer gets called has nothing to do with when the objects go out of scope. They could hang around for a long time before the garbage collector actually finalizes them. So any native memory they've allocated will stay allocated for an arbitrarily-long time after they go out of scope. If they are holding on to a lot of memory and/or if there are lots of the objects it would seem to me you have an effective memory leak. Or least will have a steady-state memory consumption potentially a lot higher than it would seem to need to be. In other words, similar behavior to what's described in JNA/ByteBuffer not getting freed and causing C heap to run out of memory
Is there any way around this problem with JNA? Or will I need to give up on JNA for this and use JNI instead so that I can use JNIEnv::GetByteArrayElements() so there's no need for any "off the books" memory allocations that can persist arbitrarily long? Is it acceptable to subclass Memory in order to get access to the dispose() method and use that to free up the underlying native memory on my timeline instead of the GC's timeline? Or will that cause problems when the finalizer does run?
JNA provides Memory.disposeAll() and Memory.dispose() to explicitly free memory (the latter requires you to subclass Memory), so if you do ever encounter memory pressure for which regular GC is not sufficient, you have some additional control available.
I understand that when a directbytebuffer is allocated, its not subject to garbage collection, but what I'm wondering is if the wrapping object is garbage collected.
For example, if I allocated a new DirectByteBuffer dbb, and then duplicated(shallow copied) it using dbb.duplicate(), I'd have two wrappers around the same chunk of memory.
Are those wrappers subject to garbage collection? If I did
while(true){
DirectByteBuffer dbb2 = dbb.duplicate();
}
Would I eventually OOM myself?
In the Sun JDK, a java.nio.DirectByteBuffer—created by ByteBuffer#allocateDirect(int)—has a field of type sun.misc.Cleaner, which extends java.lang.ref.PhantomReference.
When this Cleaner (remember, a subtype of PhantomReference) gets collected and is about to move into the associated ReferenceQueue, the collection-related thread running through the nested type ReferenceHandler has a special case treatment of Cleaner instances: it downcasts and calls on Cleaner#clean(), which eventually makes its way back to calling on DirectByteBuffer$Deallocator#run(), which in turn calls on Unsafe#freeMemory(long). Wow.
It's rather circuitous, and I was surprised to not see any use of Object#finalize() in play. The Sun developers must have had their reasons for tying this in even closer to the collection and reference management subsystem.
In short, you won't run out of memory by virtue of abandoning references to DirectByteBuffer instances, so long as the garbage collector has a chance to notice the abandonment and its reference handling thread makes progress through the calls described above.
A direct ByteBuffer object is just like any other object: it can be garbage-collected.
The memory used by a direct buffer will be released when the ByteBuffer object is GC'd (this is not explicitly stated for ByteBuffer, but is implied by the documentation of MappedByteBuffer).
Where things get interesting is when you've filled your virtual memory space with direct buffers, but still have lots of room in the heap. It turns out that (at least on the Sun JVM), running out of virtual space when allocating a direct buffer will trigger a GC of the Java heap. Which may collect unreferenced direct buffers and free their virtual memory commitment.
If you're running on a 64-bit machine, you should use -XX:MaxDirectMemorySize, which puts an upper limit on the number of buffers that you can allocate (and also triggers GC when you hit that limit).
Looking at the source code to DirectByteBuffer it just returns a new instance, so no, you won't OOM yourself.
As long as the rest of your code doesn't hold onto a reference to the original dbb then that object will get garbage collected as normal. The extra dbb2 objects will similarly get garbage collected when there is no longer any reference to them(ie, the end of the while loop).
Both the Java object and the native memory are freed at the same time by the garbage collector.
However, note that because the garbage collector doesn’t work well at cleaning up direct buffers, it’s best to allocate and reuse long-lived direct buffers instead of creating and abandoning new ones.
when a directbytebuffer is allocated, its not subject to garbage
collection
Where did you get that idea? It's not correct. Are you getting them mixed up with MappedByteBuffers?