EhCache comes with the ability to choose an eviction policy for when a cache fills up to its maximum size. This eviction policy is used to determine which elements to "evict" from the cache so that it does not overflow.
The three eviction policy options for on-heap memory stores are:
LFU (Least Frequently Used) - the default
LRU (Least Recently Used)
FIFO (First In, First Out)
My question is: how does one determine which one of these policies is most effective for a particular application? Obviously, each will have their own strengths and weaknesses, and different applications will fair better or worse with each of these depending on numerous factors.
Is there a benchmark that can be setup? I'd love to write a performance test, but wouldn't know where to start.
It is better to test with your own code/data, than try do guess without full information.
Write a sample code, that generate data (data should be as close as possible to your real samples, it could be stored in database, or send to your app using messages, depending on it's workflow). After it try to write a simple code, that will use read/write methods, that are used by application and test it with all 3 strategies.
Related
My Spring based web app is deployed to production in a Tomcat cluster (4+ nodes) with sticky sessions. The max number of nodes will not exceed 8-10 in a few years time.
I need to cache some data(mostly configuration), to avoid hitting Oracle. Since the nature of this data is mostly configuration, I would say the ratio of reads to writes is 999999 / 1.
I don't want to use a full-blown caching solution such as Infinispan/Hazelcast/Redis as it adds operation complexity to the product and the requirement is to cache some small, mostly read-only data(let's say a few hundred kilobytes the most)
At first, I wanted to implement a simple replicating map myself, then I saw [JGroups][1] ships with a [ReplicatedHashMap][1]. I think it suits my needs but I'm not sure whether I'm missing something.
What else should I consider?
Has anyone used it in production?
ReplicatedHashMap is one class of 700 lines, so it isn't particularly complex, and uses JGroups, which has been used for decade(s) in production.
If you need something simple, without transactions/overflow-store etc, then it might be right for your job. Note that you could modify it and/or write your own, with RHM as template.
RHM replicates all data to all nodes, so if you have many nodes (you don't), or your data is large, then ReplCache may be the better choice.
I'm playing around with ChronicleSet, which is backed by ChronicleMap. I've done some initial testing, and it's pretty good for our needs. Much more efficient in RAM usage than other solutions, with access time a bit slower, but still pretty fast.
However, one thing I'm testing is setting the maximum number of entries, and it doesn't seem to be working as expected.
I'm using the following code:
ChronicleSetBuilder<Long> postalSetBuilder =
ChronicleSetBuilder.of(Long.class)
.name("long-map")
.entries(2_000_000L)
.maxBloatFactor(1.0);
According to the documentation, this means that the maximum number of entries allowed in this case would be 2M. However, when testing, I can reliably go up to 2x the maximum specified, and a little bit mor,e until I get an exception like this:
Error: ChronicleMap{name=city-postal-codes-map, file=null, identityHashCode=3213500}: Attempt to allocate #129 extra segment tier, 128 is maximum.
Possible reasons include:
- you have forgotten to configure (or configured wrong) builder.entries() number
- same regarding other sizing Chronicle Hash configurations, most likely maxBloatFactor(), averageKeySize(), or averageValueSize()
- keys, inserted into the ChronicleHash, are distributed suspiciously bad. This might be a DOS attack
In this case, the ChronicleMap object call to size() spit out 4,079,238. So I'm wondering how I can set an explicit limit (like the 2M specified above), and have Chronicle reliably reject any requests to add additional elements after that.
It's not possible to configure exact specific limit of entries, because of shared-nothing segmented storage, there is no single counter of entries. To make Chronicle Map to fail close to the configured entries() limit, you should configure allowSegmentTiering(false).
In an application we are using an LRU(Least Recently Used) cache(Concurrent HashMap Implementation) with Max-size constrain. I'm wondered whether i could improve the performance of this cache. Following were few alternatives which i found on the net .
Using Google Gauva pool library.(since my implementation uses LRU , I dont see any benefit from gauva library)
If i wrap the objects as soft-references and store as values in LRU map(with out any size constrain) , can i see any benefit ? (This is not an ideal way of caching. After major gc run , all the soft references will be garbage collected).
How about using a hybrid pool which is a combination of LRU map + a soft reference map.(idea is when ever a object is pruned from LRU map , it is stored in a soft reference map.
By this approach we can have more number of objects in cache. But this approach might be a time consuming one.)
Are there any other methods to improve the performance of cache?
First of all, welcome to the club of cache implementers and improvers!
Don't use LRU. There are a lot of algorithms that are better then LRU, that are now
more then 10 years old meanwhile. As a start read these relevant resarch papers:
Wikipedia: LIRS Caching Algorithm
Wikipedia: Adaptive Replacement Cache
Within these papers you find also more basic papers about the idea of adaptive caching.
(e.g. 2Q, LRFU, LRU-k).
Warpping objects: It depends on what you want to achieve. Actually you have at least three additional object for a cache entry: The hashtable entry, the weakreference object, the cache entry object. With this approach you increase the memory footprint and if you
have a low efficiency, e.g. because of short expiry, you have a lot of GC trashing.
Adapt to available memory: If you want to adapt to the available memory it is better to evict entries if memory becomes lower. This way you evict entries that are used very seldom, instead of a random eviction. However, this approach affords more coding. EHCache with Auto Resource Control has implemented something like this.
The reference wrappers are a nice and easy way if you want to use more memory for the cache but avoid low heap conditions, but it is nothing high performance in terms of over all memory efficiency and access times.
Measure it! It depends heavily on the usage scenario whether you get an "performance improvement" or not. Ideally you need to know and consider the access pattern, the cached object sizes, expiry constraints and the expected parallelism. I put together a benchmark suite that you can find on GitHub cache2k benchmarks.
However, for now, these benchmarks just focus on the replacement policy efficiency and access times. Comparison of memory overhead and possible parallelism is missing. This will be added in somehow half a year scope. The benchmark results are available on the cache2k benchmark results page.
If you are generally interested in the topic and do some research in the field consider contributing to cache2k. I am especially happy about more benchmarking code, or descriptions of usage scenarios and traces of access patterns to optimize and improve the replacement algorithm.
When I read about files in textbooks it seems that some concepts I knew about OS are repeated for files on the application level.
For example the terms block and page are used for logical representation of data in files (so we are not in the HD level organization). But I can not understand what is the idea here. Do we in the application define a block size and a page size and use that when accessing files e.g. using NIO or blocking IO?
How would we define these sizes normally? Arbitrarily? Am I confused here?
UPDATE after request of #RobinGreen
An example of what I am saying is e.g. the slotted-block page structure or the list representation for variable length records described e.g. in the book of Silberschatz for Database System concepts in the section for files
In Linux, the operating system block size is unrelated to the hard drive's reported sector size in bytes (which is also fake these days - for BIOS compatibility reasons!).
At the application level, you can store data in fixed-size or variable-size blocks, which are unrelated to OS level blocks.
So there are many levels, all unrelated to each other!
Of course it is a good idea to read and write data in chunks, nevertheless. Reading and writing data 1 byte at a time would involve too many round-trips to the kernel, for example. But the right size to use is an empirical question: which size is most efficient for your use case?
Page sizes are slightly different. They are defined by the CPU architecture (at least on the x86/x86_64 family) and affect paging/swapping. An application does not directly encounter paging and swapping, but it encounters the effects, in terms of lower performance.
I need to cache objects in Java using a proportion of whatever RAM is available. I'm aware that others have asked this question, but none of the responses meet my requirements.
My requirements are:
Simple and lightweight
Not dramatically slower than a plain HashMap
Use LRU, or some deletion policy that approximates LRU
I tried LinkedHashMap, however it requires you to specify a maximum number of elements, and I don't know how many elements it will take to fill up available RAM (their sizes will vary significantly).
My current approach is to use Google Collection's MapMaker as follows:
Map<String, Object> cache = new MapMaker().softKeys().makeMap();
This seemed attractive as it should automatically delete elements when it needs more RAM, however there is a serious problem: its behavior is to fill up all available RAM, at which point the GC begins to thrash and the whole app's performance deteriorates dramatically.
I've heard of stuff like EHCache, but it seems quite heavy-weight for what I need, and I'm not sure if it is fast enough for my application (remembering that the solution can't be dramatically slower than a HashMap).
I've got similar requirements to you - concurrency (on 2 hexacore CPUs) and LRU or similar - and also tried Guava MapMaker. I found softValues() much slower than weakValues(), but both made my app excruciatingly slow when memory filled up.
I tried WeakHashMap and it was less problematic, oddly even faster than using LinkedHashMap as an LRU cache via its removeEldestEntry() method.
But by the fastest for me is ConcurrentLinkedHashMap which has made my app 3-4 (!!) times faster than any other cache I tried. Joy, after days of frustration! It's apparently been incorporated into Guava's MapMaker, but the LRU feature isn't in Guava r07 at any rate. Hope it works for you.
I've implemented serval caches and it's probably as difficult as implementing a new datasource or threadpool, my recommendation is use jboss-cache or a another well known caching lib.
So you will sleep well without issues
I've heard of stuff like EHCache, but it seems quite heavy-weight for what I need, and I'm not sure if it is fast enough for my application (remembering that the solution can't be dramatically slower than a HashMap).
I really don't know if one can say that EHCache is heavy-weight. At least, I do not consider EHCache as such, especially when using a Memory Store (which is backed by an extended LinkedHashMap and is of course the the fastest caching option). You should give it a try.
I believe MapMaker is going to be the only reasonable way to get what you're asking for. If "the GC begins to thrash and the whole app's performance deteriorates dramatically," you should spend some time properly setting the various tuning parameters. This document may seem a little intimidating at first, but it's actually written very clearly and is a goldmine of helpful information about GC:
https://www.oracle.com/technetwork/java/javase/memorymanagement-whitepaper-150215.pdf
I don't know if this would be a simple solution, especially compared with EHCache or similar, but have you looked at the Javolution library? It is not designed for as such, but in the javolution.context package they have an Allocator pattern which can reuse objects without the need for garbage collection. This way they keep object creation and garbage collection to a minimum, an important feature for real-time programming. Perhaps you should take a look and try to adapt it to your problem.
This seemed attractive as it should
automatically delete elements when it
needs more RAM, however there is a
serious problem: its behavior is to
fill up all available RAM
Using soft keys just allows the garbage collector to remove objects from the cache when no other objects reference them (i.e., when the only thing referring to the cache key is the cache itself). It does not guarantee any other kind of expulsion.
Most solutions you find will be features added on top of the java Map classes, including EhCache.
Have you looked at the commons-collections LRUMap?
Note that there is an open issue against MapMaker to provide LRU/MRU functionality. Perhaps you can voice your opinion there as well
Using your existing cache, store WeakReference rather than normal object refererences.
If GC starts running out of free space, the values held by WeakReferences will be released.
In the past I have used JCS. You can set up the configuration to try and meet you needs. I'm not sure if this will meet all of your requirements/needs but I found it to be pretty powerful when I used it.
You cannot "delete elements" you can only stop to hard reference them and wait for the GC to clean them, so go on with Google Collections...
I'm not aware of an easy way to find out an object's size in Java. Therefore, I don't think you'll find a way to limit a data structure by the amount of RAM it's taking.
Based on this assumption, you're stuck with limiting it by the number of cached objects. I'd suggest running simulations of a few real-life usage scenarios and gathering statistics on the types of objects that go into the cache. Then you can calculate the statistically average size, and the number of objects you can afford to cache. Even though it's only an approximation of the amount of RAM you want to dedicate to the cache, it might be good enough.
As to the cache implementation, in my project (a performance-critical application) we're using EhCache, and personally I don't find it to be heavyweight at all.
In any case, run several tests with several different configurations (regarding size, eviction policy etc.) and find out what works best for you.
Caching something, SoftReference maybe the best way until now I can imagine.
Or you can reinvent an Object-pool. That every object you doesn't use, you don't need to destroy it. But it to save CPU rather than save memory
Assuming you want the cache to be thread-safe, then you should examine the cache example in Brian Goetz's book "Java Concurrency in Practice". I can't recommend this highly enough.