I have project written in java that using JNI uses a C++ library.
All the code was written by us, so I have all the source code.
After few hours the machine runs out of memory although my process just iterate over files and all the memory regarding the previous file deleted.
I'm sure that there is a memory leak, usually I use Valgrind, but it seems he can't cope with Java very well and believes that the JVM is leaking, even for "hello world" java project.
I've tested the C++ parts (the major flows) with unit-tests and used valgrind on the unit-tests, but couldn't find any leakage. It doesn't prove anything because there are many potential flows I could've missed.
My major question, how can I find my leak ?
It will be extremely helpful to know who is consuming the memory, the java or the native part ? they are in the same process.
Thanks.
You can use jemalloc to debug native memory leaks. This blog post has a detailed example of using jemalloc to debug a native memory leak in java applications.
From my experience, Valgrind is actually usable with the JVM, and it remains the best tool to hunt leaks in C/C++ code, even with JNI. So your question kinda contains the answer that you need ;)
If you fail to use JNI and Valgrind together, please refer to Valgrind and Java.
Related
We have a Java Application that uses SWT and some other DLL through JNA in Windows 7 environment.
We have seen the JVM process's memory usage increases with time but the JVM heap is relatively steady with normal GC activity.
We suspect there are some memory leak from SWT and/or JNA/DLL, but cannot be sure why or where.
Is there any tools on the Windows platform that can help to detect this kind of problem? Or any facility in Java platform that can help to resolve this problem? Any suggestion is welcome.
In Java 8 there is a command line switch: -XX:NativeMemoryTracking=<value>, where value can be off, summary or detail.
You can set this to detail and then you can query it using jcmd.
There's some official documentation on this here. This feature is fairly limited though, but it may just help find the cause.
Failing that, a crude but effective way of identifying the source of the problem is to replace the dll with a dummy version (or even a non-native dummy) and see if you can reproduce the same problem.
I've always seen memory viewers for native windows application, but Java Applications run in the Heap of the Java Virtual machine.
So how do we look inside the memory of a running Java App, how do we get a look at what goes on the stack.
Is there any application to spy on a Java Application memory
Is it possible to write an application to do so ? in Visual C#.
Thanks for any help.
It is actually possible to look into Java memory via Heap Dumps.
Nearly every JVM you will encounter is capable of doing them, and there are programs available to analyze them to. A good place to start writing a program to process heap dumps would probably be to look at the code of one of these programs...
This blog seems to have some information on the subject:
http://javaworks.wordpress.com/2009/12/03/jvm-heap-dumps-generation-and-analysis/
Here's Oracle's Java Heap Analysis Tool:
http://download.oracle.com/javase/6/docs/technotes/tools/share/jhat.html
Edit: FYI, Not much can be assumed to go on the stack. Primitives are presumably allocated there (considering the lack of new), but objects are always allocated on the heap. An arguable exception would be the Java HotSpot VM, which I have heard does allocate some objects on the stack to improve performance. I think you probably meant heap. ^__^
I am suspecting a native memory leak in my java code. Are there any tools which do native memory profiling? Also, does any tool support native memory analysis of a running java process?
Thanks!!
Edit:
I have already tried Memory Validator and Purify but it seems they support only 32-bit processes. Is there some tool similar to the above ones which can simply attach to a running windows process and give us native memory analysis for that particular process?
The Troubleshooting guide for Java SE 6 with Hotspot VM contains a fairly elaborate section on techniques to aid in detecting native memory leaks. These include:
wrapping all memory allocation and deallocation calls to track the amount of memory used.
relying on platform specific support like the debug support provided by the Microsoft Visual C++ compiler or on mtrace (and MALLOC_TRACE) to debug memory allocations on Linux.
using memory leak analysis tools like Rational Purify.
among others. Notably, the article mentions that no ideal solution exists for all platforms.
Also, consider using the -Xcheck:jni flag that appears to be available in most JVMs. The -X flag itself indicates that the flag is non-standard, but the flag appears to be available in the IBM JDK, Oracle JRockit R28, and even the Oracle/Sun JVM. Enabling the flag switches on a mode where wrappers are added around JNI calls, thereby allowing you to track illegal arguments passed to JVM calls as noted in the JNI programmers' guide and specification. While, it's use in detecting memory leaks is subjective, it would definitely help if you suspect that the leak is being caused due to invalid parameters being issued.
AFAIK you can't do it with Java tools like JProfiler, JVisualVM etc. If you have memory leak in native code use tools for native code. You ie. can run it from C (i.e. loading jvm.dll). You can look at this articles finding memory leaks using Visual Studio or Memory Leak Detection in C++ (Linux)
Note: of course if you leak is connected to heap leak (forgot about deleteglobalref) you can find it with Java tools, but it is very rare in JNI.
I have been working on an open-source project named "MySafe" (https://github.com/serkan-ozal/mysafe) It basicly intercepts and monitors "Unsafe" calls. (In fact, it makes more than). With version 2.0, it can be useful for tracking and detecting "Unsafe" based native memory leaks.
Demo code: https://github.com/serkan-ozal/mysafe/blob/master/src/test/java/tr/com/serkanozal/mysafe/NativeMemoryLeakHuntingDemo.java
Diagram showing source of the leak: https://github.com/serkan-ozal/mysafe/blob/master/src/test/resources/native-memory-leak-hunting.png
To diagnose native memory leak, JIT code symbol mapping and Linux recent profiling tools are required: perf, perf-map-agent and bcc.
Please refer to details in related answer https://stackoverflow.com/a/52767721/737790
Many thanks to Brendan Gregg
These are tools you can use for debugging
libtcmalloc HPROF: For heap profiling
The jcmd Utility, PSS for the process: Can help in confirming a native leak.
Native Memory Tracking: Tracking native memory leak in JVM (only works for allocation inside JVM)
Core dump analysis, pmap and gdb checking for anon blocks and process memory overtime
-Xcheck:jni
Further details can be found out here
https://www.bro-code.in/blog/java-debugging-native-memory-leak
http://www.oracle.com/technetwork/java/javase/memleaks-137499.html#gbyvk
I'm a big fan of JProfiler. That's the best tool for profiling and memory leaks. It's fairly cheap relative to most tools, really easy to learn, and lots of features.
http://www.ej-technologies.com/products/jprofiler/overview.html
I've just upgraded some old Java source which has been running on a Sun Java 1.4.2 VM to Sun Java (JRE) 6 VM. More or less the only thing I had to change was to add explicit datatypes for some abstract objects (Hashmap's, Vector's and so on). The code itself it quite memory intensive, using up to 1G of heap memory (using -Xmx1024m as a parameter to start the VM).
Since I read alot about better performance on newer Java VM's, this was one of the reasons I did this upgrade.
Can anyone think of a reason why the performance is worse in my case now (just in general, of course, since you can't take a look at the code)?
Has anyone some advice for a non Java guru what to look for if I wanted to optimize (speed wise) existing code? Any hints, recommended docs, tools?
Thanks.
Not much information here. But here are a couple of things you might want to explore:
Start the VM with Xmx and Xms as the same value (in your case 1024M)
Ensure that the server jvm dll is being used to start the virtual machine.
Run a profiler to see what objects are hogging memory or what objects are not being garbage collected
Hook up your VM with the jconsole and trace through the objects
If your application nearly runs out of free space, garbage collection time may dominate computation time.
Enable gc debugging to look for this. Or, even better, simply start jconsole and attach it to your program.
Theoretically it could be that you application consumes more memory, because there were changes to the way Strings share their internal char[]. Less sharing is done after 1.4.
Check my old blog at http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/5100 (new blog is here)
I would compare the Garbage Collector logs to see whether memory usage is really the problem.
If that doesn't help, us a profiler such as Yourkit to find the differences.
Definitely use a profiler on the app (YourKit is great)...it's easy to waste a lot of time guessing at the problem when most of the time you'll be able to narrow it down really quickly in the profiler.
The sun JVM spits out a LOT of extra noise when run under valgrind, which makes tracking memory problems in the application very challenging.
I'd like to find either a suppression file, or a VM runtime mode, that will strip out spurious memory errors in order to separate the wheat from the chaff in this situation. Any suggestions?
What about profiling this native code outside of the Java application? Usually the JNI code is a wrapper around some library that is not Java specific. Not sure if that is true for your specific case, but if it is then maybe the memory problems can be isolated by writing a plain C or C++ test framework around that library?
If your framework is in C++ then you might also be able to supply your own new and delete operators and track memory usage yourself. You'll have to collect statistics and process them with some scripts but it can work well.
I can't answer your posted question, but can you elaborate on what problem you're having?
In other words, can you tell us if it is...
In the JNI layer and not a JVM object scope issue?
A use of free'd memory?
A buffer underwrite/overwrite?
Other memory corruption?
I recently had to debug a Java/C that had issues (after 30+ minutes into its run), which it turns out was using memory after it had been free'd. I tried using dmalloc, a custom memory leak library of mine, Valgrind and none worked as I needed.
Eventually I created a simple set of wrappers around free, malloc, calloc, realloc that simply printed memory addresses and sizes to a file. After it aborted (within GDB) I could backtrack in time and figure out when the memory was free'd and where the references were that did not get removed.
IF your issue is in C/C++ and you can trap the error in a debugger this might work for you. Yes, it's tedious, but maybe no worse than sifting through megabytes of Valgrind output.
Hope that helps & good luck.
While not as spiffy as valgrind (based on what I've read), you might try jmap and jhat. These tools allow you to take a snapshot of the running process and see what's going on. I've used this technique for simple memory leaks and it's worked well. However, if the memory problems are caused by non-jvm allocations, this won't help.