Develop Reference

How to detect memory-pressure in a java program?

I have a batch process, written in java, that analyzes extremely long sequences of tokens (maybe billions or even trillions of them!) and observes bi-gram patterns (aka, word-pairs).
In this code, bi-grams are represented as Pairs of Strings, using the ImmutablePair class from Apache commons. I won't know in advance the cardinality of the tokens. They might be very repetitive, or each token might be totally unique.
The more data I can fit into memory, the better the analysis will be!
But I definitely can't process the whole job at once. So I need to load as much data as possible into a buffer, perform a partial analysis, flush my partial results to a file (or to an API, or whatever), then clear my caches and start over.
One way I'm optimizing memory usage is by using Guava interners to de-duplicate my String instances.
Right now, my code looks essentially like this:
int BUFFER_SIZE = 100_000_000;
Map<Pair<String, String>, LongAdder> bigramCounts = new HashMap<>(BUFFER_SIZE);
Interner<String> interner = Interners.newStrongInterner();
String prevToken = null;
Iterator<String> tokens = getTokensFromSomewhere();
while (tokens.hasNest()) {
String token = interner.intern(tokens.next());
if (prevToken != null) {
Pair<String, String> bigram = new ImmutablePair(prevToken, token);
LongAdder bigramCount = bigramCounts.computeIfAbsent(
bigram,
(c) -> new LongAdder()
);
bigramCount.increment();
// If our buffer is full, we need to flush!
boolean tooMuchMemoryPressure = bigramCounts.size() > BUFFER_SIZE;
if (tooMuchMemoryPressure) {
// Analyze the data, and write the partial results somewhere
doSomeFancyAnalysis(bigramCounts);
// Clear the buffer and start over
bigramCounts.clear();
}
}
prevToken = token;
}
The trouble with this code is that this is a very crude way of determining whether there is tooMuchMemoryPressure.
I want to run this job on many different kinds of hardware, with varying amounts of memory. No matter the instance, I want this code to automatically adjust to maximize the memory consumption.
Rather than using some hard-coded constant like BUFFER_SIZE (derived through experimentation, heuristics, guesswork), I actually just want ask the JVM whether the memory is almost full. But that's a very complicated question, considering the complexities of mark/sweep algorithms, and all the different generational collectors.
What would be a good general-purpose approach for accomplishing something like this, assuming this batch-job might run on a variety of different machines, with different amounts of available memory? I don't need this to be extremely precise... I'm just looking for a rough signal to know that I need to flush the buffer soon, based on the state of the actual heap.

The simplest way to get a first glimpse of what is going on with the process' heap space is Runtime.freeMemory() together with .maxMemory and .totalMemory. Yet the first does not factor in garbage and so is an under-estimation at best and may be completely misleading just before the GC kicks in.
Assuming that for your application "memory pressure" basically means "(soon) not enough", the interesting value is free memory right after a GC.
This is available by using GarbageCollectorMXBean
which provides GcInfo with memory usage after the GC.
The bean can be watched exactly after GC since it is a NotificationEmitter, despite this is not being advertised in the Javadoc. Some minimal code, patterned after a longer example is
void registerCallback() {
List<GarbageCollectorMXBean> gcbeans =
java.lang.management.ManagementFactory.getGarbageCollectorMXBeans();
for (GarbageCollectorMXBean gcbean : gcbeans) {
System.out.println(gcbean.getName());
NotificationEmitter emitter = (NotificationEmitter) gcbean;
emitter.addNotificationListener(this::handle, null, null);
}
}
private void handle(Notification notification, Object handback) {
if (!notification.getType()
.equals(GarbageCollectionNotificationInfo.GARBAGE_COLLECTION_NOTIFICATION)) {
return;
}
GarbageCollectionNotificationInfo info = GarbageCollectionNotificationInfo
.from((CompositeData) notification.getUserData());
GcInfo gcInfo = info.getGcInfo();
gcInfo.getMemoryUsageAfterGc().forEach((name, memUsage) -> {
System.err.println(name+ "->" + memUsage);
});
}
There will be several memUsage entries and this will also differ depending on the GC. But from the values provided, used, committed and max we can derive upper limits on free memory which again should give the "rough signal" the OP is asking for.
The doSomeFancyAnalysis will certainly also need its share of fresh memory, so with a very rough estimate how much that will be per bigramm to analyze, this could be the limit to watch for.

Benchmarking JVM memory consumption similarly to how it is done by the Android OS

When trying to benchmark a specific method, in regards to how many objects are created and how many bytes they occupy while that method is running, in Android it is possible to do this:
Debug.resetThreadAllocCount()
Debug.resetThreadAllocSize()
Debug.startAllocCounting()
benchmarkMethod()
Debug.stopAllocCounting()
var memoryAllocCount = Debug.getThreadAllocCount()
var memoryAllocSize = Debug.getThreadAllocSize()
I would now like to benchmark the same method, but on a normal desktop application, where these methods are not available. I have not found anything similar, and any other memory benchmarking code I have tried does not provide consistent results, like the above code does, which gives the exact same result every time when the same benchmark runs.
Any suggestion, preferably just code would be appreciated, however I would be open to try some software as well if it is able to perform the task I am trying to do.

ThreadMXBean.getThreadAllocatedBytes can help:
com.sun.management.ThreadMXBean bean =
(com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean();
long currentThreadId = Thread.currentThread().getId();
long before = bean.getThreadAllocatedBytes(currentThreadId);
allocatingMethod();
long after = bean.getThreadAllocatedBytes(currentThreadId);
System.out.println("Allocated " + (after - before) + " bytes");
The method returns an approximation of the total allocated memory, but this approximation is usually quite precise.
Also, async-profiler has Java API for profiling allocations. It does not only count how many objects are allocated, but also shows the exact allocated objects with the stack traces of the allocation sites.
public static void main(String[] args) throws Exception {
AsyncProfiler profiler = AsyncProfiler.getInstance();
// Dry run to skip allocations caused by AsyncProfiler initialization
profiler.start("alloc", 0);
profiler.stop();
// Real profiling session
profiler.start("alloc", 0);
allocatingMethod();
profiler.stop();
profiler.execute("file=alloc.svg"); // save the output to alloc.svg
}
How to run:
java -Djava.library.path=/path/to/async-profiler -XX:+UseG1GC -XX:-UseTLAB Main
-XX:+UseG1GC -XX:-UseTLAB options are needed to record all allocations. Otherwise, async-profiler will work in sampling mode, recording only a small portion of allocations.
Here is how the output will look like:

Confused on result of the implementation for testing object memory consumption in java

I use the below implemenatation to test the memory consumption of the java objects.
But it prints usage as 104936 B when the i limit is from 384 to 1694 in the calling method's for loop.
It prints usage as 0B for when the above i limit is less than 384.
Why is this?
IDE : eclipse kepler
java : 1.5 (this acts differently when the
java version is also changed)
OS : Ubuntu 12.10
public static void main(String[] args) {
Runtime runtime = Runtime.getRuntime();
long totalStart = runtime.totalMemory();
long start = runtime.freeMemory();
SampleTester sampleTester = new SampleTester();
sampleTester.callingMethod();
long totalEnd = runtime.totalMemory();
long end = runtime.freeMemory();
System.out.println("Usage [(("+totalEnd+"-"+end+") - ("+totalStart+"-"+start+"))] \t: " + ((totalEnd-end) - (totalStart-start)));
}
private void callingMethod(){
for(int i = 0; i < 1694; i++){
ArrayList<String> arrayList = new ArrayList<String>();
}
}

Each ArrayList you create is immediately eligible for GC when the current loop iteration ends.
So theoretically the JVM could allocate each ArrayList in the space formerly occupied by the one from the last iteration and only need one "slot", which gets discarded when the last iteration ends and the method exists.
That explains the "0" value (this or a similar optimization).
In practice, the JVM doesn't always do that, but will leave eligible object uncollected for some time in order to be more performant (because doing GC too often can be just as bad for performance).
When and how the JVM decides which optimization to apply is entirely implementation-defined and knowing the very details is rarely useful information (because you can't depend on it and it can change with the next version).
So if you want to find out how much memory n ArrayList object need, make sure that you each one of those is actually reachable and as such not eligible for garbage collection when you check the current used memory.

Using too much Ram in Java

I'm writing a program in java which has to make use of a large hash-table, the bigger the hash-table can be, the better (It's a chess program :P). Basically as part of my hash table I have an array of "long[]", an array of "short[]", and two arrays of "byte[]". All of them should be the same size. When I set my table size to ten-million however, it crashes and says "java heap out of memory". This makes no sense to me. Here's how I see it:
1 Long + 1 Short + 2 Bytes = 12 bytes
x 10,000,000 = 120,000,000 bytes
/ 1024 = 117187.5 kB
/ 1024 = 114.4 Mb
Now, 114 Mb of RAM doesn't seem like too much to me. In total my CPU has 4Gb of RAM on my mac, and I have an app called FreeMemory which shows how much RAM I have free and it's around 2Gb while running this program. Also, I set the java preferences like -Xmx1024m, so java should be able to use up to a gig of memory. So why won't it let me allocate just 114Mb?

You predicted that it should use 114 MB and if I run this (on a windows box with 4 GB)
public static void main(String... args) {
long used1 = memoryUsed();
int Hash_TABLE_SIZE = 10000000;
long[] pos = new long[Hash_TABLE_SIZE];
short[] vals = new short[Hash_TABLE_SIZE];
byte[] depths = new byte[Hash_TABLE_SIZE];
byte[] flags = new byte[Hash_TABLE_SIZE];
long used2 = memoryUsed() - used1;
System.out.printf("%,d MB used%n", used2 / 1024 / 1024);
}
private static long memoryUsed() {
return Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
}
prints
114 MB used
I suspect you are doing something else which is the cause of your problem.
I am using Oracle HotSpot Java 7 update 10

Has not taken into account that each object is a reference and also use memory, and more "hidden things"... we must also take into account also the alignment... byte is not always a byte ;-)
Java Objects Memory Structure
How much memory is used by Java
To see how much memory is really in use, you can use a profiler:
visualvm
If you are using standard HashMap (or similar from JDK), each "long" (boxing/unboxing) really are more than 8bytes), you can use this as a base... (use less memory)
NativeIntHashMap

From what I have read about BlueJ, and serious technical information is almost impossible to find, BlueJ VM is quite likely not to support primitive types at all; your arrays are actually of boxed primitives. BlueJ uses a subset of all Java features, with emphasis on object orientation.
If that is the case, plus taking into consideration that performance and efficiency are quite low on BlueJ VM's list of priorities, you may actually be using quite a bit more memory than you think: a whole order of magnitude is quite imaginable.

I believe one way it would be to clean the heap memory after each execution, one link is here:
Java heap space out of memory

OutOfMemoryErrors even after using WeakReference's for keys and values

Below is a small test I've coded to educate myself on references API. I thought this would never throw OOME but it is throwing it. I am unable to figure out why. appreciate any help on this.
public static void main(String[] args)
{
Map<WeakReference<Long>, WeakReference<Double>> weak = new HashMap<WeakReference<Long>, WeakReference<Double>>(500000, 1.0f);
ReferenceQueue<Long> keyRefQ = new ReferenceQueue<Long>();
ReferenceQueue<Double> valueRefQ = new ReferenceQueue<Double>();
int totalClearedKeys = 0;
int totalClearedValues = 0;
for (long putCount = 0; putCount <= Long.MAX_VALUE; putCount += 100000)
{
weak(weak, keyRefQ, valueRefQ, 100000);
totalClearedKeys += poll(keyRefQ);
totalClearedValues += poll(valueRefQ);
System.out.println("Total PUTs so far = " + putCount);
System.out.println("Total KEYs CLEARED so far = " + totalClearedKeys);
System.out.println("Total VALUESs CLEARED so far = " + totalClearedValues);
}
}
public static void weak(Map<WeakReference<Long>, WeakReference<Double>> m, ReferenceQueue<Long> keyRefQ,
ReferenceQueue<Double> valueRefQ, long limit)
{
for (long i = 1; i <= limit; i++)
{
m.put(new WeakReference<Long>(new Long(i), keyRefQ), new WeakReference<Double>(new Double(i), valueRefQ));
long heapFreeSize = Runtime.getRuntime().freeMemory();
if (i % 100000 == 0)
{
System.out.println(i);
System.out.println(heapFreeSize / 131072 + "MB");
System.out.println();
}
}
}
private static int poll(ReferenceQueue<?> keyRefQ)
{
Reference<?> poll = keyRefQ.poll();
int i = 0;
while (poll != null)
{
//
poll.clear();
poll = keyRefQ.poll();
i++;
}
return i;
}
}
And below is the log when ran with 64MB of heap
Total PUTs so far = 0
Total KEYs CLEARED so far = 77982
Total VALUESs CLEARED so far = 77980
100000
24MB
Total PUTs so far = 100000
Total KEYs CLEARED so far = 134616
Total VALUESs CLEARED so far = 134614
100000
53MB
Total PUTs so far = 200000
Total KEYs CLEARED so far = 221489
Total VALUESs CLEARED so far = 221488
100000
157MB
Total PUTs so far = 300000
Total KEYs CLEARED so far = 366966
Total VALUESs CLEARED so far = 366966
100000
77MB
Total PUTs so far = 400000
Total KEYs CLEARED so far = 366968
Total VALUESs CLEARED so far = 366967
100000
129MB
Total PUTs so far = 500000
Total KEYs CLEARED so far = 533883
Total VALUESs CLEARED so far = 533881
100000
50MB
Total PUTs so far = 600000
Total KEYs CLEARED so far = 533886
Total VALUESs CLEARED so far = 533883
100000
6MB
Total PUTs so far = 700000
Total KEYs CLEARED so far = 775763
Total VALUESs CLEARED so far = 775762
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at Referencestest.weak(Referencestest.java:38)
at Referencestest.main(Referencestest.java:21)

from http://weblogs.java.net/blog/2006/05/04/understanding-weak-references
I think your use of HashMap is likely to be the issue. You might want to use WeakHashMap
To solve the "widget serial number"
problem above, the easiest thing to do
is use the built-in WeakHashMap class.
WeakHashMap works exactly like
HashMap, except that the keys (not the
values!) are referred to using weak
references. If a WeakHashMap key
becomes garbage, its entry is removed
automatically. This avoids the
pitfalls I described and requires no
changes other than the switch from
HashMap to a WeakHashMap. If you're
following the standard convention of
referring to your maps via the Map
interface, no other code needs to even
be aware of the change.

The heart of the problem is probably that you're filling your heap with WeakReference-objects, the weak references are cleared when you're getting low on memory, but the reference objects themselves are not, so your hashmap is filling up with boat-load if WeakReference objects (not to mention the object array the hashmap uses, which will grow indefinitely), all pointing to null.
The solution, as already pointed out, is a weak hashmap, which will clear out those objects if they're no longer in use (this is done during put).
EDIT:
As Kevin pointed out, you already have your reference-queue logic worked out (I didn't pay close enough attention), a solution using your code is to just clear it out of the map at the point where the key has been collected. This is exactly how weak hash map works (where the poll is simply triggered on insert).

Even when your weak references let go of the things they are referencing, they still do not get recycled themselves.
So eventually your hash will fill up with references to nothing and crash.
What you would need (if you wanted to do it this way) would be to have an event triggered by object deletion that went in and removed the reference from the hash. (which would cause threading issues you need to be aware of as well)

I'm not a java expert at all, but I know in .NET when doing a lot of large object memory allocation you can get heap fragmentation to the point where only small pieces of contiguous memory are available for allocation even though much more memory appears as "free".
A quick google search on "java heap fragmentation" brings up some seemingly relevant result although I haven't taken a good look at them.

Other's have correctly pointed out what the problem is; e.g. #roe, #Bill K.
But another way to solve this kind problem (apart from scratching your head, asking on SO, etc), is to look and see how the Sun recommended approach works. In this case, you can find it in the source code for the WeakHashMap class.
There are a few ways to find Java source code:
If you have a decent Java IDE to running, it should be able to show you the source code of any class in the class library.
Most J2SE JDK downloads include source JAR files for (at least) the public API classes.
You can specifically download full source distributions for the OpenJDK-based releases of Java.
But the ZERO EFFORT approach is to do a Google search, using the fully qualified name of the class with ".java.html" tacked on the end. For example, searching for "java.util.WeakHashMap.java.html" gives this link in the first page of search results.
And the source will tell you that the standard WeakHashMap implementation explicitly polls its reference queue to expunge stale (i.e. broken) weak references from the map's key set. In fact, it does this every time you access or update the map, or even just ask for its size.

An other problem might be that Java for some reason don't always activate its garbadge collecter when running out of memmory, so you might need to insert explicit calls to activate the collector. Try something like
if( (putCount%1000)===0)
Runtime.getRuntime().gc();
in your loop.
Edit: It seems that the new java implementations from sun now does call the garbadge collector before throwing OutOfMemmoryException, but I am pretty sure that the following program would throw OutOfMemmoryException with jre1.3 or 1.4
public class Test {
public static void main(String args[]) {
while(true) {
byte []data=new byte[1000000];
}
}
}