I have used LinkedHashMap with accessOrder true along with allowing a maximum of 500 entries at any time as the LRU cache for data. But due to scalability issues I want to move on to some thread-safe alternative. ConcurrentHashMap seems good in that regard, but lacks the features of accessOrder and removeEldestEntry(Map.Entry e) found in LinkedHashMap. Can anyone point to some link or help me to ease the implementation.
I did something similar recently with ConcurrentHashMap<String,CacheEntry>, where CacheEntry wraps the actual item and adds cache eviction statistics: expiration time, insertion time (for FIFO/LIFO eviction), last used time (for LRU/MRU eviction), number of hits (for LFU/MFU eviction), etc. The actual eviction is synchronized and creates an ArrayList<CacheEntry> and does a Collections.sort() on it using the appropriate Comparator for the eviction strategy. Since this is expensive, each eviction then lops off the bottom 5% of the CacheEntries. I'm sure performance tuning would help though.
In your case, since you're doing FIFO, you could keep a separate ConcurrentLinkedQueue. When you add an object to the ConcurrentHashMap, do a ConcurrentLinkedQueue.add() of that object. When you want to evict an entry, do a ConcurrentLinkedQueue.poll() to remove the oldest object, then remove it from the ConcurrentHashMap as well.
Update: Other possibilities in this area include a Java Collections synchronization wrapper and the Java 1.6 ConcurrentSkipListMap.
Have you tried using one of the many caching solutions like ehcache?
You could try using LinkedHashMap with a ReadWriteLock. This would give you concurrent read access.
This might seem old now, but at least just for my own history tracking, I'm going to add my solution here: I combined ConcurrentHashMap that maps K->subclass of WeakReference, ConcurrentLinkedQueue, and an interface that defines deserialization of the value objects based on K to run LRU caching correctly. The queue holds strong refs, and the GC will evict the values from memory when appropriate. Tracking the queue size involved AtomicInteger, as you can't really inspect the queue to determine when to evict. The cache will handle eviction from/adding to the queue, as well as map management. If the GC evicted the value from memory, the implementation of the deserialization interface will handle retrieving the value back. I also had another implementation that involved spooling to disk/re-reading what was spooled, but that was a lot slower than the solution I posted here, as Ihad to synchronize spooling/reading.
You mention wanting to solve scalability problems with a "thread-safe" alternative. "Thread safety" here means that the structure is tolerant of attempts at concurrent access, in that it won't suffer corruption by concurrent use without external synchronization. However, such tolerance does not necessarily help to improve "scalability". In the simplest -- though usually misguided -- approach, you'll try to synchronize your structure internally and still leave non-atomic check-then-act operations unsafe.
LRU caches require at least some awareness of the total structure. They need something like a count of the members or the size of the members to decide when to evict, and then they need to be able to coordinate the eviction with concurrent attempts to read, add, or remove elements. Trying to reduce the synchronization necessary for concurrent access to the "main" structure fights against your eviction mechanism, and forces your eviction policy to be less precise in its guarantees.
The currently accepted answer mentions "when you want to evict an entry". Therein lies the rub. How do you know when you want to evict an entry? Which other operations do you need to pause in order to make this decision?
The moment you use another data structure along with concurrenthashmap, the atomicity of the operations sucha adding a new item in concurrenthashmap and adding in other data structure cant be guaranteed without additional synchronization such as ReadWriteLock which will degrade performance
Wrap the map in a Collections.synchronizedMap(). If you need to call additional methods, then synchronize on the map that you got back from this call, and invoke the original method on the original map (see the javadocs for an example). The same applies when you iterate over the keys, etc.
Related
I want to use a Comparator based key value Map. This will have reads and a rare write operation (once every 3 months through a scheduler). The initial load of the collection will be done at application startup.
Also note that the write will:
Add a single entry to the Map
Will not modify any existing entry to the Map.
Will ConcurrentSkipListMap be a good candidate for this. Is the get operation on this allows access to multiple threads simultaneously? I'm looking for concurrent non blocking read but atomic write.
ConcurrentHashMap is exactly what you're looking for. From the Javadoc:
Retrieval operations (including get) generally do not block, so may overlap with update operations (including put and remove). Retrievals reflect the results of the most recently completed update operations holding upon their onset. (More formally, an update operation for a given key bears a happens-before relation with any (non-null) retrieval for that key reporting the updated value.)
That sounds like it satisfies your requirement for "concurrent non blocking read but atomic write".
Since you're doing so few writes, you may want to specify a high loadFactor and appropriate initialSize when creating the ConcurrentHashMap, which will prevent table resizing as you're populating the map, though this is a modest benefit at best. (You could also set a concurrencyLevel of 1, though Java 8's Javadoc seems to imply that is no longer used as a sizing hint.)
If you absolutely must have a SortedMap or NavigableMap, then ConcurrentSkipListMap is the out-of-the-box way to go. But I would double-check that you actually need the functionality provided by those interfaces (getting the first/last key, submaps, finding nearby entries, etc.) before using them. You will pay a steep price (log n vs. constant time for most operations).
Since you are looking for concurrent operations you have basically 3 competitors.
Hashtable, ConcurrentHashMap, ConcurrentSkipListMap (or Collections.synchronizedMap() but that's not efficient).
Out of these 3 latter 2 are more suitable for concurrent operation as they just lock the portion of map rather than locking the entire map like Hashtable.
Out of latter 2 SkipListMap uses skip list data structure which ensures average O(log n) performance for fast search and variety of operations.
It also offers number of operations that ConcurrentHashMap can't, i.e. ceilingEntry/Key(), floorEntry/Key(), etc. It also maintains a sort order which would otherwise have to be calculated.
Thus if you had asked only for faster search i'd have suggested ConcurrentHashMap, but since you have also mentioned 'rare write operations' and 'desired sorting' order I think ConcurrentSkipListMap wins the race.
If you are willing to try third party code, you could consider a copy-on-write version of maps, which are ideal for infrequent writes. Here's one that came up via Googling:
https://bitbucket.org/atlassian/atlassian-util-concurrent/wiki/CopyOnWrite%20Maps
Never tried it myself so caveat emptor.
I'm newcomer in concurrency. I read about Guava Cache and MultiMap. I look for something that can combine some possibilities of both:
From Cache I want auto-removal after ACCESS_TIMEOUT and WRITE_TIMEOUT has been expired.
From Multimap I want multiple values associated with one key.
All that must be concurrent.
I has multiple writers and multiple readers. I want to add values with rundom keys and remove them.
Question: Is there map implementation that fits my needs?
UPDATED: Striped<Lock> solution
More I read about Striped<Lock> - more attractive that seems to me. But it arose even more questions in my head:
If I use something like Striped<Lock> with Guava Cache which already uses ConcurrentHashMap I can face the problems with deadlocks or performance decline. Am I wrong?
If I use Striped<Lock> over Cache it still doesn't remove the question linked with multiple values per key.
Does Striped<Lock> eliminate the need of using concurrent map in my case? (I suppose the answer is YES) but in GitHub a saw the contrary.
You could start with a Cache<SomeKey, Collection<SomeValue>> (so you still get the expiration) and use synchronized collections (Collections.synchronized*()) as the values.
But what's really the question here is the type of concurrent access you need on the collections:
Is it enough that the operations are synchronized so the collections don't get corrupted, or do you need higher-level semantics like what ConcurrentMap.putIfAbsent() offers?
Do you need to do multiple operations on the collections of values in an atomic way? Like if you need to do
if (c.contains(v)) {
c.remove(v);
} else {
c.add(v);
}
you usually want to put that into a synchronized(c) { } block.
If so, you'll probably want to wrap the collection inside a class exposing those higher-level semantics and managing the lock around multiple operations to get the atomicity you need, and use that class as the value: Cache<SomeKey, SomeValuesContainer>.
As mentioned in the comments, Striped<Lock> can be used to synchronize the access to multiple Caches/ConcurrentHashMaps without imposing a single lock and its performance impact in case of even moderate contention.
If you need multiple Caches/ConcurrentHashMaps, that is: why don't the Peers (or a wrapper around it) actually contain that information?
1. Deadlocks, performance
Guava's Cache is similar to ConcurrentHashMap, but it doesn't use it. However, both work in the same way by having segments which can be locked independently, thus reducing contention when accessing the map concurrently (especially when updating). Using a Striped<Lock> around the access to either one cannot cause a deadlock, which only happens if you're not locking multiple locks in a consistent order: that can't happen here, as you'll always lock your Lock obtained from Striped<Lock> before calling the Cache or ConcurrentHashMap, which then locks its segment (invisible to you).
As to performance, yes, locking has a cost but it really depends on the level of contention (and that can be tuned with the number of stripes in a Striped<Lock> or the concurrencyLevel in a Cache). However, you need proper concurrency support anyway since without it you can get invalid results (or corrupt your data), so you have to do something (using either locking or a lock-free algorithm).
2. Multiple values per key
My original answer still stands. It's difficult to get an exact idea of what you're exactly trying to do from your multiple questions (it's better if you can provide a complete, consistent context in one question), but I think you don't need more than concurrent modification of the multiple values per key so the synchronized collections should be enough (but you need at least that). You'll have to reason about your access patterns as you add them to make sure they still fit the model, though: make sure your replaceAll*() methods lock what they need, for example.
3. Is ConcurrentMap still needed with Striped<Lock>?
YES! Especially with Striped<Lock> vs a single Lock, because you'll still get concurrent updates for keys which don't use the same stripe (that's the whole point of Striped<Lock>) so you need data structures which support concurrent modification. If you use a simple HashMap, you have every chance of corrupting it under enough load (and cause infinite loops, for example).
I was reading the official Oracle documentation about Concurrency in Java and I was wondering what could be the difference between a Collection returned by
public static <T> Collection<T> synchronizedCollection(Collection<T> c);
and using for example a
ConcurrentHashMap. I'm assuming that I use synchronizedCollection(Collection<T> c) on a HashMap. I know that in general a synchronized collection is essentially just a decorator for my HashMap so it is obvious that a ConcurrentHashMap has something different in its internals. Do you have some information about those implementation details?
Edit: I realized that the source code is publicly available:
ConcurrentHashMap.java
I would read the source of ConcurrentHashMap as it is rather complicated in the detail. In short it has
Multiple partitions which can be locked independently. (16 by default)
Using concurrent Locks operations for thread safety instead of synchronized.
Has thread safe Iterators. synchronizedCollection's iterators are not thread safe.
Does not expose the internal locks. synchronizedCollection does.
The ConcurrentHashMap is very similar to the java.util.HashTable class, except that ConcurrentHashMap offers better concurrency than HashTable or synchronizedMap does. ConcurrentHashMap does not lock the Map while you are reading from it. Additionally,ConcurrentHashMap does not lock the entire Mapwhen writing to it. It only locks the part of the Map that is being written to, internally.
Another difference is that ConcurrentHashMap does not throw ConcurrentModificationException if the ConcurrentHashMap is changed while being iterated. The Iterator is not designed to be used by more than one thread though whereas synchronizedMap may throw ConcurrentModificationException
This is the article that helped me understand it Why ConcurrentHashMap is better than Hashtable and just as good as a HashMap
Hashtable’s offer concurrent access to their entries, with a small caveat, the entire map is locked to perform any sort of operation.
While this overhead is ignorable in a web application under normal
load, under heavy load it can lead to delayed response times and
overtaxing of your server for no good reason.
This is where ConcurrentHashMap’s step in. They offer all the features
of Hashtable with a performance almost as good as a HashMap.
ConcurrentHashMap’s accomplish this by a very simple mechanism.
Instead of a map wide lock, the collection maintains a list of 16
locks by default, each of which is used to guard (or lock on) a single
bucket of the map. This effectively means that 16 threads can modify
the collection at a single time (as long as they’re all working on
different buckets). Infact there is no operation performed by this
collection that locks the entire map. The concurrency level of the
collection, the number of threads that can modify it at the same time
without blocking, can be increased. However a higher number means more
overhead of maintaining this list of locks.
The "scalability issues" for Hashtable are present in exactly the same way in Collections.synchronizedMap(Map) - they use very simple synchronization, which means that only one thread can access the map at the same time.
This is not much of an issue when you have simple inserts and lookups (unless you do it extremely intensively), but becomes a big problem when you need to iterate over the entire Map, which can take a long time for a large Map - while one thread does that, all others have to wait if they want to insert or lookup anything.
The ConcurrentHashMap uses very sophisticated techniques to reduce the need for synchronization and allow parallel read access by multiple threads without synchronization and, more importantly, provides an Iterator that requires no synchronization and even allows the Map to be modified during interation (though it makes no guarantees whether or not elements that were inserted during iteration will be returned).
Returned by synchronizedCollection() is an object all methods of which are synchronized on this, so all concurrent operations on such wrapper are serialized. ConcurrentHashMap is a truly concurrent container with fine grained locking optimized to keep contention as low as possible. Have a look at the source code and you will see what it is inside.
ConcurrentHashMap implements ConcurrentMap which provides the concurrency.
Deep internally its iterators are designed to be used by only one thread at a time which maintains the synchronization.
This map is used widely in concurrency.
I'm looking to implement a simple cache without doing too much work (naturally). It seems to me that one of the standard Java collections ought to suffice, with a little extra work. Specifically, I'm storing responses from a server, and the keys can either be the request URL string or a hash code generated from the URL.
I originally thought I'd be able to use a WeakHashMap, but it looks like that method forces me to manage which objects I want to keep around, and any objects I don't manage with strong references are immediately swept away. Should I try out a ConcurrentHashMap of SoftReference values instead? Or will those be cleaned up pretty aggressively too?
I'm now looking at the LinkedHashMap class. With some modifications it looks promising for an MRU cache. Any other suggestions?
Whichever collection I use, should I attempt to manually prune the LRU values, or can I trust the VM to bias against reclaiming recently accessed objects?
FYI, I'm developing on Android so I'd prefer not to import any third-party libraries. I'm dealing with a very small heap (16 to 24 MB) so the VM is probably pretty eager to reclaim resources. I assume the GC will be aggressive.
If you use SoftReference-based keys, the VM will bias (strongly) against recently accessed objects. However it would be quite difficult to determine the caching semantics - the only guarantee that a SoftReference gives you (over a WeakReference) is that it will be cleared before an OutOfMemoryError is thrown. It would be perfectly legal for a JVM implementation to treat them identically to WeakReferences, at which point you might end up with a cache that doesn't cache anything.
I don't know how things work on Android, but with Sun's recent JVMs one can tweak the SoftReference behaviour with the -XX:SoftRefLRUPolicyMSPerMB command-line option, which determines the number of milliseconds that a softly-reachable object will be retained for, per MB of free memory in the heap. As you can see, this is going to be exceptionally difficult to get any predictable lifespan behaviour out of, with the added pain that this setting is global for all soft references in the VM and can't be tweaked separately for individual classes' use of SoftReferences (chances are each use will want different parameters).
The simplest way to make an LRU cache is by extending LinkedHashMap as described here. Since you need thread-safety, the simplest way to extend this initially is to just use Collections.synchronizedMap on an instance of this custom class to ensure safe concurrent behaviour.
Beware premature optimisation - unless you need very high throughput, the theoretically suboptimal overhead of the coarse synchronization is not likely to be an issue. And the good news - if profiling shows that you are performing too slowly due to heavy lock contention, you'll have enough information available about the runtime use of your cache that you'll be able to come up with a suitable lockless alternative (probably based on ConcurrentHashMap with some manual LRU treatment) rather than having to guess at its load profile.
LinkedHashMap is easy to use for cache. This creates an MRU cache of size 10.
private LinkedHashMap<File, ImageIcon> cache = new LinkedHashMap<File, ImageIcon>(10, 0.7f, true) {
#Override
protected boolean removeEldestEntry(Map.Entry<File, ImageIcon> eldest) {
return size() > 10;
}
};
I guess you can make a class with synchronized delegates to this LinkedHashMap. Forgive me if my understanding of synchronization is wrong.
www.javolution.org has some interestig features - synchronized fast collections.
In your case it worth a try as it offers also some nifty enhancements for small devices as Android ones.
For synchronization, the Collections framework provides a synchronized map:
Map<V,T> myMap = Collections.synchronizedMap(new HashMap<V, T>());
You could then wrap this, or handle the LRU logic in a cache object.
I like Apache Commons Collections LRUMap
How is the performance of ConcurrentHashMap compared to HashMap, especially .get() operation (I'm especially interested for the case of only few items, in the range between maybe 0-5000)?
Is there any reason not to use ConcurrentHashMap instead of HashMap?
(I know that null values aren't allowed)
Update
just to clarify, obviously the performance in case of actual concurrent access will suffer, but how compares the performance in case of no concurrent access?
I was really surprised to find this topic to be so old and yet no one has yet provided any tests regarding the case. Using ScalaMeter I have created tests of add, get and remove for both HashMap and ConcurrentHashMap in two scenarios:
using single thread
using as many threads as I have cores available. Note that because HashMap is not thread-safe, I simply created separate HashMap for each thread, but used one, shared ConcurrentHashMap.
Code is available on my repo.
The results are as follows:
X axis (size) presents number of elements written to the map(s)
Y axis (value) presents time in milliseconds
The summary
If you want to operate on your data as fast as possible, use all the threads available. That seems obvious, each thread has 1/nth of the full work to do.
If you choose a single thread access use HashMap, it is simply faster. For add method it is even as much as 3x more efficient. Only get is faster on ConcurrentHashMap, but not much.
When operating on ConcurrentHashMap with many threads it is similarly effective to operating on separate HashMaps for each thread. So there is no need to partition your data in different structures.
To sum up, the performance for ConcurrentHashMap is worse when you use with single thread, but adding more threads to do the work will definitely speed-up the process.
Testing platform
AMD FX6100, 16GB Ram
Xubuntu 16.04, Oracle JDK 8 update 91, Scala 2.11.8
Thread safety is a complex question. If you want to make an object thread safe, do it consciously, and document that choice. People who use your class will thank you if it is thread safe when it simplifies their usage, but they will curse you if an object that once was thread safe becomes not so in a future version. Thread safety, while really nice, is not just for Christmas!
So now to your question:
ConcurrentHashMap (at least in Sun's current implementation) works by dividing the underlying map into a number of separate buckets. Getting an element does not require any locking per se, but it does use atomic/volatile operations, which implies a memory barrier (potentially very costly, and interfering with other possible optimisations).
Even if all the overhead of atomic operations can be eliminated by the JIT compiler in a single-threaded case, there is still the overhead of deciding which of the buckets to look in - admittedly this is a relatively quick calculation, but nevertheless, it is impossible to eliminate.
As for deciding which implementation to use, the choice is probably simple.
If this is a static field, you almost certainly want to use ConcurrentHashMap, unless testing shows this is a real performance killer. Your class has different thread safety expectations from the instances of that class.
If this is a local variable, then chances are a HashMap is sufficient - unless you know that references to the object can leak out to another thread. By coding to the Map interface, you allow yourself to change it easily later if you discover a problem.
If this is an instance field, and the class hasn't been designed to be thread safe, then document it as not thread safe, and use a HashMap.
If you know that this instance field is the only reason the class isn't thread safe, and are willing to live with the restrictions that promising thread safety implies, then use ConcurrentHashMap, unless testing shows significant performance implications. In that case, you might consider allowing a user of the class to choose a thread safe version of the object somehow, perhaps by using a different factory method.
In either case, document the class as being thread safe (or conditionally thread safe) so people who use your class know they can use objects across multiple threads, and people who edit your class know that they must maintain thread safety in future.
I would recommend you measure it, since (for one reason) there may be some dependence on the hashing distribution of the particular objects you're storing.
The standard hashmap provides no concurrency protection whereas the concurrent hashmap does. Before it was available, you could wrap the hashmap to get thread safe access but this was coarse grain locking and meant all concurrent access got serialised which could really impact performance.
The concurrent hashmap uses lock stripping and only locks items that affected by a particular lock. If you're running on a modern vm such as hotspot, the vm will try and use lock biasing, coarsaning and ellision if possible so you'll only pay the penalty for the locks when you actually need it.
In summary, if your map is going to be accesaed by concurrent threads and you need to guarantee a consistent view of it's state, use the concurrent hashmap.
In the case of a 1000 element hash table using 10 locks for whole table saves close to half the time when 10000 threads are inserting and 10000 threads are deleting from it.
The interesting run time difference is here
Always use Concurrent data structure. except when the downside of striping (mentioned below) becomes a frequent operation. In that case you will have to acquire all the locks? I read that the best ways to do this is by recursion.
Lock striping is useful when there is a way of breaking a high contention lock into multiple locks without compromising data integrity. If this is possible or not should take some thought and is not always the case. The data structure is also the contributing factor to the decision. So if we use a large array for implementing a hash table, using a single lock for the entire hash table for synchronizing it will lead to threads sequentially accessing the data structure. If this is the same location on the hash table then it is necessary but, what if they are accessing the two extremes of the table.
The down side of lock striping is it is difficult to get the state of the data structure that is affected by striping. In the example the size of the table, or trying to list/enumerate the whole table may be cumbersome since we need to acquire all of the striped locks.
What answer are you expecting here?
It is obviously going to depend on the number of reads happening at the same time as writes and how long a normal map must be "locked" on a write operation in your app (and whether you would make use of the putIfAbsent method on ConcurrentMap). Any benchmark is going to be largely meaningless.
It's not clear what your mean. If you need thread safeness, you have almost no choice - only ConcurrentHashMap. And it's definitely have performance/memory penalties in get() call - access to volatile variables and lock if you're unlucky.
Of course a Map without any lock system wins against one with thread-safe behavior which needs more work.
The point of the Concurrent one is to be thread safe without using synchronized so to be faster than HashTable.
Same graphics would would be very interesting for ConcurrentHashMap vs Hashtable (which is synchronized).