Develop Reference

Java memory leak vs heap dump information

We're running against an issue related with memory.
Our monitor is saying that out service is consuming 800MiB of memory.
We've created a heap dump. However, this heap dump is telling us that total heap size is 41MiB.
Out service is running inside a container. We've read a lot some issue related with java and docker containers. However, we're trying to get some more information.
How could we otain more memory related information about java vm?
Any ideas?

The big discrepancy between the memory usage reported by the heap dump and the usage reported by external monitoring could be due to a few things. Some things to look at include:
A JVM's memory utilization does not just consist of the Java heap. It includes the JVM executable and shared libraries, metaspace, the native (malloc) heap and memory segments that hold thread stacks. It potentially also includes memory mapped files, and memory segments shared with other processes.
The memory used by the Java heap includes objects that are unreachable, and space that isn't currently allocated to objects; i.e. free heap space.
How can you get more information about Java memory utilization?
A primary source is "Java Platform, Standard Edition Troubleshooting Guide":
The Native Memory Tracking section explains how to use jcmd to get an overall picture.
The JFR Memory Tab provides live monitoring of Java Heap usage stats and the GC.
The Jmap Utility can be used to obtain heap and GC configuration information, heap histograms and classloader stats.
GC logging can give useful insights into GC behavior.

Monitoring Java internal objects & memory usage

I have a Java web server running as a Windows service.
I use Tomcat 8 with Java 1.8.*
For a few months now, I've detected that the memory usage is increasing quite rapidly. I cannot make up for sure if it's heap or stack.
The process starts with ~200MB and after a week or so, it can reach up to 2GB.
Shortly after it will generate OutOfMemory exception (the memory usage will be 2GB - 2.5GB).
This has repeated multiple times on multiple environments.
I would like to know if there's a way to monitor the process and view it's internal memory usage, even to the level of viewing which objects are using the most amount of memory.
Can 'Java Native Memory Tracking' be used for this?
This will help me to detect any memory leaks that might cause this.
Thanks in advance.

To monitor the memory usage of a Java process, I'd use a JMX client such as JVisualVM, which is bundled with the Oracle JDK:
https://visualvm.java.net/jmx_connections.html
To identify the cause of a memory leak, I'd instruct the JVM to take a heap dump when it runs out of memory (on the Oracle JVM, this can be accomplished by specifying -XX:-HeapDumpOnOutOfMemoryError when starting your Java program), and then analyze that heap dump using a tool such as Eclipse MAT.

quoting:
the process starts with ~200MB and after a week or so, it can reach up to 2GB. Shortly after it will generate OutOfMemory exception (the memory usage will be 2GB - 2.5GB).
The problem might not be as simple as seeing what java objects you have got in JVisualVM (e.g millions of strings)
What you need to do is identify the code that leaks.
One way you could do that is to force the execution of particular code and then monitor the memory.
The easiest way to force the execution of code inside classes/objects is to use a tool like https://github.com/lorenzoongithub/nudge4j (particularly since you are on java 8)
alternatively you could just wire up nashorn to a command line or run your progam via jjs https://docs.oracle.com/javase/8/docs/technotes/guides/scripting/nashorn/shell.html

How to you check memory usage and force garbage collection for jetty application

I think I may have a memory leak in a servlet application running in production on jetty 8.1.7.
Is there a way of seeing how much heap memory is actually being used at an instance of time, not the max memory allocated with -Xmx, but the actual amount of memory being used.
Can I force a garbage collection to occur for an application running within jetty

yes, both are easily achievable using: VisualVM (see: http://docs.oracle.com/javase/6/docs/technotes/guides/visualvm/monitor_tab.html) This one is shipped with Oracle JDK by default (=> no extra installation required)
However for the memory leak detection, I'd suggest to do memory dump and analyze it later with eclipse MAT ( http://www.eclipse.org/mat/ ) as it has quite nice UI visualizing java memory dumps.
EDIT:
For the ssh only access, yes you can use the mentioned two tools. However you need to run them on the machine with running window manager and remotely connect over ssh to the other machine (you need to have java on both of these machines):
For visualVM: you need to have VisualVM running on one maching and via the ssh connect to remote one, see: VisualVM over ssh
and for the memory dump: use jmap (for sample usage see: http://kadirsert.blogspot.de/2012/01/…) afterwards download the dump file and load if locally to eclipse MAT

enable jmx and connect up to it using jconsole
http://wiki.eclipse.org/Jetty/Tutorial/JMX

You can call System.gc(). That will typically perform a full GC ... but this facility can be disabled. (There is a JVM option to do this with HotSpot JVMs.)
However, if your problem is a memory leak, running the GC won't help. In fact, it is likely to make your server even slower than it currently is.
You can also monitor the memory usage (in a variety of ways - see other Answers) but that only gives you evidence that a memory leak might leak.
What you really need to do is find and fix the cause of the memory leak.
Reference:
How to find a Java Memory Leak

You can use jvisualvm.exe which is under the %JAVA_HOME%\bin folder. By using this application you can monitor memory usage and can force gc.

about out of memory Exception

I have used Thread Pool for New IO server design . I have used newFixedThreadPool as a Executors factory method for thread pool creation. My server is throwing Exception when i execute my server for 20 to 30 minute . how to handle this exception.
java.lang.OutOfMemoryError: Java heap space

Obviously you are using too much memory, so now you need to find out why. Without your source it is very hard to to say what is wrong, but even with source it can be problematic when the program start to become complex.
What I have found helpful is to take memory dumps and look at them in tools such as Memory Analyzer (MAT). It can even compare several dumps to see what kind of objects are allocated. When you get an idea of what objects exists which you don't think should be there you can use the tool to see what roots it has (which objects has a reference to it).
To get a memory dump form a running java program use jmap -dump:format=b,file=heap.bin and to automatically get a memory dump when your program gets and OutOfMemoryError you can run it with java -XX:+HeapDumpOnOutOfMemoryError failing.java.Program

normally its java -Xms5m -Xmx15m MyApp
-Xms set initial Java heap size
-Xmx set maximum Java heap size
and in eclipse under java run configuration as VM argument
-Xms128m
-Xmx512m
-XX:permSize=128M
-XX:MaxPermSize=384M

You can defnitely try increasing the HEAP SIZE and check if you are getting the issue.
However, I would prefer you to try profiling your application to find out why your heap size and where the memory is being consumed.
There are few Profilers available as open source you can try.

General strategy to resolve Java memory leak?

I have a standalone program that I run locally, it is meant to be a server type program running 24/7. Recently I found that it has a memory leak, right now our only solution is to restart it every 4 hours. What is the best way to go about finding this memory leak? Which tool and method should we use?

If you are using Java from Sun and you use at least Java 6 update 10 (i.e. the newest), then try running jvisualvm from the JDK on the same machine as your program is running, and attach to it and enable profiling.
This is most likely the simplest way to get started.

When it comes to hunting memory problems, I use SAP Memory Analyzer Eclipse Memory Analyser (MAT), a Heap Dump analysis tool.
The Memory Analyzer provides a general purpose toolkit to analyze Java heap dumps. Besides heap walking and fast calculation of retained sizes, the Eclipse tool reports leak suspects and memory consumption anti-patterns. The main area of application are Out Of Memory Errors and high memory consumption.
Initiated by SAP, the project has since been open sourced and is now know as Eclipse Memory Analyser. Check out the Getting Started page and especially the Finding Memory Leaks section (I'm pasting it below because I fixed some links):
Start by running the leak report to automatically check for memory leaks.
This blog details How to Find a Leaking Workbench Window.
The Memory Analyzer grew up at SAP. Back then, Krum blogged about Finding Memory Leaks with SAP Memory Analyzer. The content is still relevant!
This is probably the best tool you can get (even for money) for heap dump analysis (and memory leaks).
PS: I do not work for SAP/IBM/Eclipse, I'm just a very happy MAT user with positive feedback.

You need a memory profiler. I recommend trying the Netbeans profiler.

One approach would be to take heap dumps on a regular basis, then trend the instance counts of your classes to try to work out which objects are being consistently created but not collected.
Another would be to switch off parts of your app to try to narrow down where the problem is.
Look at tools like jmap and jhat.

You might look up JMX and the jconsole app that ships with Java. You can get some interesting statistics out-of-the-box, and adding some simple instrumentation to your classes can provide a whole lot more.

As already stated jvisualvm is a great way to get started, but once you know what is leaking you may need to find what is holding references to the objects in question for which I'd recommend jmap and jhat, e.g
jmap -dump:live,file=heap.dump.out,format=b <pid>
and
jhat heap.dump.out
where <pid> is easily found from jvisualvm. Then in a browser navigate to localhost:7000 and begin exploring.

You need to try and capture Java heap dump which is a memory print of the Java process.
It's a critical process for memory consumption optimisation and finding memory leaks.
Java heap dump is an essential object for diagnosing memory-linked issues including java.lang.OutOfMemoryError, Garbage Collection issues, and memory leaks which are all part of Java web development process
For clarity, a Heap dump contains information such as Java classes and objects in a heap during instant of taking the snapshot.
To do it, you need to run jmap -dump:file=myheap.bin <program pid>.
To learn more about how to capture Java heat dumps, check out: https://javatutorial.net/capture-java-heap-dump