I have been reading about this article:
Creating a memory leak with Java
Since I myself have issues with threads:
Springboot increasing threads even if server.tomcat.max-threads=5
Does this mean that Long running threads in Java will create memory leaks?
What if I need to have a long running thread in the first place.
Isn't most application long running threads also?
Long running thread doesn't create memory leak. It is what you do inside it. Technically memory leaks happens when garbage collector could not collect free space, as the space is marked as being used. i.e. if references are held.
Now in a long running thread, you could have an object reference present for the lifetime of the thread. This object itself could be expensive. This is the case in first link you shared(threadlocal holding transitively all the references)
On your second link, the problem seems to lie somewhere. Here what I generally do if I suspect memory leak
Use jmap to get count of each class instances
Force full GC
Again count the instances of each class, these are the objects GC was not able to clean
Repeat multiple times, you will notice some objects, which should have been cleared. This will give you some idea. Following those references in code you can get some idea.
Related
When I get OOM error, how do I decide weather I should increase heap size or there is memory leak problem with my code?
Also, how do I decide with inital heap size of my application? In my current application, we had started with 512MB but now have increased to 4GB. But this was done by trial and error method. Is there any systematic way to decide required heap size?
If my question is sounding too basic, can anyone please share some references which can help to increase understanding of this?
I don't think Java or the JVM actually define "leak". But your programs are clearly affected by them. I'm sure they define "out of memory".
You have a memory leak, if there is an object that will never be examined again by the application, held by a reference in a scope that is never exited, or held by another object that is examined occasionally. Such a leak may not seriously impact the long term stability of your program.
You have a bad leak if you generate a new object with these properties when some bit of code in the application is run, and that bit of code may run an indefinite number of times. Eventually a program with a bad leak runs out of memory.
It seems to me that 'leak' is a poor term in the GC context. Leaks are exactly what will be garbage-collected.
What won't be GC'd, and what causes OutOfMemoryErrors, is memory that isn't leaked when it should be, i.e. references to objects held beyond their true useful lifetime, typically a reference that is a member variable that should be method-local.
I'm reading up on different JVM implementations, and I'm wondering why a stack-based memory management isn't more widespread (not to be confused with escape analysis). Are any of you familiar with attempts on writing JVMs with stack-based memory management?
This is just not really practical.
As soon as you have multithreading, you have the need to share references to objects between threads. This means that threads need to hold references to other threads stacks, and these get invalidated as soon as the method that initially created the object returns.
The heap is effectively the shared area of memory that all threads in a process can see, so any object that needs to be seen by multiple threads naturally lives there.
Another way to say this is that the stack is private to a thread, whereas the heap is shared between them.
I know that I Java's Runtime object can report the JVM's memory usage. However, I need the memory usage for a certain thread. Any idea how to get this?
I appreciate your answer!
A Thread shares everything except its stack and the CPU cycles with all other threads in the VM. All objects created by the Thread are pooled with all the other objects.
The problem is to define what the memory usage of a Thread is. Is it only those objects it created? What if these objects subsequently are referenced by other threads? Do they only count half, then? What about objects created somewhere else, but are now referenced by this Thread?
I know of no tool trying to measure the memory consumption of separate Threads.
I am working on an application & my code is having out of memory error. I am not able to see memory utilisation of the code.so I am very confused were to see.
Also after my little analysis I came to kow that there is private static object getting creating & in the constructor of that class. some more objects are getting created. & that class is multithreaded..
so I want to know since the static objects does not get garbage collected.. will all the objects related to the constructor will not be garbage collected??
A static reference is only collected when the class is unloaded, and this only happened when the class loader is not used any more. If you haven't got multiple class loaders it is likely this will never be unloaded (until your program stops)
However, just because an object was once referenced statically doesn't change how it is collected. If you had a static reference to an object and no longer have a reference to that object, it will be collected as normal.
Having multiple threads can make finding bugs harder, but it doesn't change how objects are collected either.
You need to take a memory dump of your application and see why memory is building up. It is possible the objects you retaining are all needed. In this case you need to
reduce your memory requirement
increase your maximum memory.
You might not have a memory leak - you might simply surpassed the amount of avaialble RAM your system can provide.
you can add several JVM arguments to limit the size of RAM allocated to your runtime enviorment, and control the garbage collector - the tradeoff is it usually consumes more CPU.
You say you are not capable of seeing the memory utilisation?
Have you tried using JVisualVM (in $JAVA_HOME/bin/jvisualvm)
It should be capable of attaching to local processes and take heap dumps.
Also, Eclipse Memory Analyzer has some good reports for subsequent analysis
I've been having this memory leak issue for days and I think I have some clues now. The memory of my java process keeps growing but yet the heap does not increase. I was told that this is possible if I create many threads, because Java threads uses memory outside of the heap.
My java process is a server type program so there are 1000-2000 threads. Created and deleted ongoing. How do I reclaim the memory used by a java thread? Do I simply erase all references to the thread object and make sure that this is terminated?
Yes. That is the answer. As long as there is an active reference to any Java object, then that object won't be garbage collected when it's done.
If you're creating and destroying threads and not pooling them, I think you have other issues as well.
From the Java API docs threads die when:
All threads that are not daemon threads have died, either by returning from the call to the run method or by throwing an exception that propagates beyond the run method.
Threads die when they return from their run() method. When they die they are candidates for garbage collection. You should make sure that your threads release all references to objects and exit the run() method.
I don't think that nulling references to your threads will really do the trick.
You should also check out the new threading facilities in Java 5 and up. Check the package java.util.concurrent in the API documentation here.
I also recommend you to check the book Concurrency in Practice. It's being priceless for me.
There are two things that will cause a Thread to be not garbage collected.
Any thread that is still alive will not be garbage collected. A thread is alive until the run method called by Thread.start() exits, either normally or by throwing an exception. Once this happens (and the thread's uncaught exception handler has finished), the thread is dead.
Any live reference to the Thread object for a thread will prevent it from being garbage collected. The live reference could be in your code, or if you are using thread pools, they could be part of the pool data structures.
The memory of my java process keeps growing but yet the heap does not increase.
That would be because each thread has a large (e.g. 1Mb) stack segment that is not allocated in the Java heap.
A thread's stack segment is only allocated when the thread is started, and released as soon as the thread terminates. The same also applies (I think) to the thread's thread-local map. A Thread object that is not "alive" doesn't use much memory at all.
So to sum it up. You appear to have lots of live threads. They won't be garbage collected as long as they are alive, and the only way to make them release their memory is to cause them to die ... somehow.
To reduce memory usage, you to need to do one or more of:
Look at the thread code (the run() methods, etc) to figure out why they are still hanging around.
Reduce the size of the thread stacks. (In theory, you can go as low as 64K ...)
Redesign your app so that it doesn't create thousands of threads. (Thread pools and some kind of work queue is one possible approach.)
That is a lot of threads, each of which imposes a memory overhead, and well as other resources for managing them (context switching etc). Use a profiler to view the thread activity - you'll likely find that most of the threads are idle most of the time.
I'd suggest the first step is to look at managing the threads using the thread pools provided by java.util.concurrent. Rather than creating threads, look to create tasks that are handed off to the pools. Tweak the pools until you have a much smaller number of threads that are kept reasonably busy. This may well resolve the memory issue; it will certainly improve performance.