I have a cache refresh logic and want to make sure that it's thread-safe and correct way to do it.
public class Test {
Set<Integer> cache = Sets.newConcurrentHashSet();
public boolean contain(int num) {
return cache.contains(num);
}
public void refresh() {
cache.clear();
cache.addAll(getNums());
}
}
So I have a background thread refreshing cache - periodically call refresh. And multiple threads are calling contain at the same time. I was trying to avoid having synchronized in the methods signature because refresh could take some time (imagine that getNum makes network calls and parsing huge data) then contain would be blocked.
I think this code is not good enough because if contain called in between clear and addAll then contain always returns false.
What is the best way to achieve cache refreshing without impacting significant latency to contain call?
Best way would be to use functional programming paradigm whereby you have immutable state (in this case a Set), instead of adding and removing elements to that set you create an entirely new Set every time you want to add or remove elements. This is in Java9.
It can be a bit awkward or infeasible however to achieve this method for legacy code. So instead what you could do is have 2 Sets 1 which has the get method on it which is volatile, and then this is assigned a new instance in the refresh method.
public class Test {
volatile Set<Integer> cache = new HashSet<>();
public boolean contain(int num) {
return cache.contains(num);
}
public void refresh() {
Set<Integer> privateCache = new HashSet<>();
privateCache.addAll(getNums());
cache = privateCache;
}
}
Edit We don't want or need a ConcurrentHashSet, that is if you want to add and remove elements to a collection at the same time, which in my opinion is a pretty useless thing to do. But you want to switch the old Set with a new one, which is why you just need a volatile variable to make sure you can't read and edit the cache at the same time.
But as I mentioned in my answer at the start is that if you never modify collections, but instead make new ones each time you want to update a collection (note that this is a very cheap operation as internally the old set is reused in the operation). This way you never need to worry about concurrency, as there is no shared state between threads.
How would you make sure your cache doesn't contain invalid entries when calling contains?? Furthermore, you'd need to call refresh every time getNums() changes, which is pretty inefficient. It would be best if you make sure you control your changes to getNums() and then update cache accordingly. The cache might look like:
public class MyCache {
final ConcurrentHashMap<Integer, Boolean> cache = new ConcurrentHashMap<>(); //it's a ConcurrentHashMap to be able to use putIfAbsent
public boolean contains(Integer num) {
return cache.contains(num);
}
public void add(Integer nums) {
cache.putIfAbsent(num, true);
}
public clear(){
cache.clear();
}
public remove(Integer num) {
cache.remove(num);
}
}
Update
As #schmosel made me realize, mine was a wasted effort: it is in fact enough to initialize a complete new HashSet<> with your values in the refresh method. Assuming of course that the cache is marked with volatile. In short #Snickers3192's answer, points out what you seek.
Old answer
You can also use a slightly different system.
Keep two Set<Integer>, one of which will always be empty. When you refresh the cache, you can asynchronously re-initialize the second one and then just switch the pointers. Other threads accessing the cache won't see any particular overhead in this.
From an external point of view, they will always be accessing the same cache.
private volatile int currentCache; // 0 or 1
private final Set<Integer> caches[] = new HashSet[2]; // use two caches; either one will always be empty, so not much memory consumed
private volatile Set<Integer> cachePointer = null; // just a pointer to the current cache, must be volatile
// initialize
{
this.caches[0] = new HashSet<>(0);
this.caches[1] = new HashSet<>(0);
this.currentCache = 0;
this.cachePointer = caches[this.currentCache]; // point to cache one from the beginning
}
Your refresh method may look like this:
public void refresh() {
// store current cache pointer
final int previousCache = this.currentCache;
final int nextCache = getNextPointer();
// you can easily compute it asynchronously
// in the meantime, external threads will still access the normal cache
CompletableFuture.runAsync( () -> {
// fill the unused cache
caches[nextCache].addAll(getNums());
// then switch the pointer to the just-filled cache
// from this point on, threads are accessing the new cache
switchCachePointer();
// empty the other cache still on the async thread
caches[previousCache].clear();
});
}
where the utility methods are:
public boolean contains(final int num) {
return this.cachePointer.contains(num);
}
private int getNextPointer() {
return ( this.currentCache + 1 ) % this.caches.length;
}
private void switchCachePointer() {
// make cachePointer point to a new cache
this.currentCache = this.getNextPointer();
this.cachePointer = caches[this.currentCache];
}
I'm new to WeakReferences, so I'm just trying to understand this. I threw together this short snippet of code to test their behavior. I plan to use them in a project I'm working on, because I need to track references to objects on a temporary basis. ie, I want to have a reference to them in a collection so long as they're not being used anywhere else.
So far, though, this short test code sample hasn't been working like I want it to. No matter how long I wait, weakRef.get() always returns the value, it never returns null.
Please help me figure out what I'm doing wrong. Thanks.
public static void main(String[] args) throws Exception{
String s = "Hello World!";
WeakReference<String> weakRef = new WeakReference<>(s);
System.out.println("Original1: " + s);
System.out.println("Weak1: " + weakRef.get());
s = null;
Thread.sleep(10000);
Runtime.getRuntime().gc();
int count = 0;
while(weakRef.get() != null){
System.out.println("Not null " + count);
count++;
Thread.sleep(1000);
}
System.out.println("Null");
}
Edit: Updated my code sample. I even tried manually invoking the garbage collector, and the the final println still outputs "Weak2: Hello World!".
Edit 2: Changed it again. Moved the 10 second wait to before the garbage collection, then included the loop. The loop keeps running so far for 48 seconds (before I decided to close it) after the garbage collection.
In Java, string literals, like your "Hello, World!" are "interned"; that is, they are put in a cache for reuse. As long as a loaded class depends on one of these interned strings, it won't be reclaimed by the garbage collector, and that is what's preventing your WeakReference from being cleared. Try this instead:
WeakReference<String> weakRef = new WeakReference<>(new String(s));
Of course, a string that isn't assigned from a literal value in your source code—for example, a String created from file contents—will be garbage collected at an appropriate time. You just picked a subtly tricky example to test.
Why can't I just make head=null to delete the complete Linked list?
3 = head > 1 > 2 > 4 > null
By making head = null, JVM will take care of it.As head node is not referenced by any variable , it should be garbage collected.
What is wrong with this solution?
Note: I'm aware of the correct solution to delete the complete link list but I'm curious why I can't just make head=null to delete the complete linked list?
Here's the code of java.util.LinkedList.clear(), verbatim:
public void clear() {
// Clearing all of the links between nodes is "unnecessary", but:
// - helps a generational GC if the discarded nodes inhabit
// more than one generation
// - is sure to free memory even if there is a reachable Iterator
for (Node<E> x = first; x != null; ) {
Node<E> next = x.next;
x.item = null;
x.next = null;
x.prev = null;
x = next;
}
first = last = null;
size = 0;
modCount++;
}
The comment answers your question. It's unnecessary. But it can help the GC, and it can make more objects eligible to GC sooner if there is an Iterator that references one of the nodes: the referenced node still won't be eligible to GC, but all the nodes after and before the referenced node will be, since they're not referenced anymore.
Note that the developer chose to make the clear() method slower (O(n) instead of O(1)), to make the GC faster and reduce "memory leaks". You could do the inverse choice.
Also note that you might never call clear(), and just stop referencing an object of type LinkedList, leaving all the nodes linked together. The GC will collect all the nodes if none of them is reachable through a reference chain from a GC root. That's what happens 99% of the times you use a list.
I have a question about how GC works in Java.
Consider the following code:
class C1 {
ArrayList<int> myList = new ArrayList<int>();
public void setList(ArrayList<int> l) {
myList = l;
}
}
func(C1 C) {
ArrayList<int> l1 = new ArrayList<int>();
l1.add(1);
C.setList(l1);
}
main() {
C1 C = new C1();
func(C);
...
}
my question is:
does GC releases 'l1' after func() returns or not?
No, it doesn't, because there's a root reference (stack variable C) which has a strong reference (myList), to the new ArrayList. After main() returns, then the C1 and the ArrayList are collectible, because the root reference disappears.
There is actually an optimization that HotSpot's JIT does, which is detecting the point at which a local var will no longer be accessed and clearing it at that moment. So the full answer to your question is "it might, but there is no guarantee". Recently I played with some code and measured the memory taken by a large array. Until I actually inserted array.hashCode() at te end of the method, I observed it was being released earlier.
Java's WeakHashMap is often cited as being useful for caching. It seems odd though that its weak references are defined in terms of the map's keys, not its values. I mean, it's the values I want to cache, and which I want to get garbage collected once no-one else besides the cache is strongly referencing them, no?
In which way does it help to hold weak references to the keys? If you do a ExpensiveObject o = weakHashMap.get("some_key"), then I want the cache to hold on to 'o' until the caller doesn't hold the strong reference anymore, and I don't care at all about the string object "some_key".
Am I missing something?
WeakHashMap isn't useful as a cache, at least the way most people think of it. As you say, it uses weak keys, not weak values, so it's not designed for what most people want to use it for (and, in fact, I've seen people use it for, incorrectly).
WeakHashMap is mostly useful to keep metadata about objects whose lifecycle you don't control. For example, if you have a bunch of objects passing through your class, and you want to keep track of extra data about them without needing to be notified when they go out of scope, and without your reference to them keeping them alive.
A simple example (and one I've used before) might be something like:
WeakHashMap<Thread, SomeMetaData>
where you might keep track of what various threads in your system are doing; when the thread dies, the entry will be removed silently from your map, and you won't keep the Thread from being garbage collected if you're the last reference to it. You can then iterate over the entries in that map to find out what metadata you have about active threads in your system.
See WeakHashMap in not a cache! for more information.
For the type of cache you're after, either use a dedicated cache system (e.g. EHCache) or look at Guava's MapMaker class; something like
new MapMaker().weakValues().makeMap();
will do what you're after, or if you want to get fancy you can add timed expiration:
new MapMaker().weakValues().expiration(5, TimeUnit.MINUTES).makeMap();
The main use for WeakHashMap is when you have mappings which you want to disappear when their keys disappear. A cache is the reverse---you have mappings which you want to disappear when their values disappear.
For a cache, what you want is a Map<K,SoftReference<V>>. A SoftReference will be garbage-collected when memory gets tight. (Contrast this with a WeakReference, which may be cleared as soon as there is no longer a hard reference to its referent.) You want your references to be soft in a cache (at least in one where key-value mappings don't go stale), since then there is a chance that your values will still be in the cache if you look for them later. If the references were weak instead, your values would be gc'd right away, defeating the purpose of caching.
For convenience, you might want to hide the SoftReference values inside your Map implementation, so that your cache appears to be of type <K,V> instead of <K,SoftReference<V>>. If you want to do that, this question has suggestions for implementations available on the net.
Note also that when you use SoftReference values in a Map, you must do something to manually remove key-value pairs which have had their SoftReferences cleared---otherwise your Map will only grow in size forever, and leak memory.
Another thing to consider is that if you take the Map<K, WeakReference<V>> approach, the value may disappear, but the mapping will not. Depending on usage, you may as a result end up with a Map containing many entries whose Weak References have been GC'd.
You need two maps: one which maps between the cache key and weak referenced values and one in the opposite direction mapping between the weak referenced values and the keys. And you need a reference queue and a cleanup thread.
Weak references have the ability to move the reference into a queue when the referenced object can not accessed any longer. This queue has to be drained by a cleanup thread. And for the cleanup it is necessary to get the key for a reference. This is the reason why the second map is required.
The following example shows how to create a cache with a hash map of weak references. When you run the program you get the following output:
$ javac -Xlint:unchecked Cache.java && java Cache
{even: [2, 4, 6], odd: [1, 3, 5]}
{even: [2, 4, 6]}
The first line shows the contents of the cache before the reference to the odd list has been deleted and the second line after the odds have been deleted.
This is the code:
import java.lang.ref.Reference;
import java.lang.ref.ReferenceQueue;
import java.lang.ref.WeakReference;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
class Cache<K,V>
{
ReferenceQueue<V> queue = null;
Map<K,WeakReference<V>> values = null;
Map<WeakReference<V>,K> keys = null;
Thread cleanup = null;
Cache ()
{
queue = new ReferenceQueue<V>();
keys = Collections.synchronizedMap (new HashMap<WeakReference<V>,K>());
values = Collections.synchronizedMap (new HashMap<K,WeakReference<V>>());
cleanup = new Thread() {
public void run() {
try {
for (;;) {
#SuppressWarnings("unchecked")
WeakReference<V> ref = (WeakReference<V>)queue.remove();
K key = keys.get(ref);
keys.remove(ref);
values.remove(key);
}
}
catch (InterruptedException e) {}
}
};
cleanup.setDaemon (true);
cleanup.start();
}
void stop () {
cleanup.interrupt();
}
V get (K key) {
return values.get(key).get();
}
void put (K key, V value) {
WeakReference<V> ref = new WeakReference<V>(value, queue);
keys.put (ref, key);
values.put (key, ref);
}
public String toString() {
StringBuilder str = new StringBuilder();
str.append ("{");
boolean first = true;
for (Map.Entry<K,WeakReference<V>> entry : values.entrySet()) {
if (first)
first = false;
else
str.append (", ");
str.append (entry.getKey());
str.append (": ");
str.append (entry.getValue().get());
}
str.append ("}");
return str.toString();
}
static void gc (int loop, int delay) throws Exception
{
for (int n = loop; n > 0; n--) {
Thread.sleep(delay);
System.gc(); // <- obstinate donkey
}
}
public static void main (String[] args) throws Exception
{
// Create the cache
Cache<String,List> c = new Cache<String,List>();
// Create some values
List odd = Arrays.asList(new Object[]{1,3,5});
List even = Arrays.asList(new Object[]{2,4,6});
// Save them in the cache
c.put ("odd", odd);
c.put ("even", even);
// Display the cache contents
System.out.println (c);
// Erase one value;
odd = null;
// Force garbage collection
gc (10, 10);
// Display the cache again
System.out.println (c);
// Stop cleanup thread
c.stop();
}
}
If you need weak values it's surprisingly easy:
public final class SimpleCache<K,V> {
private final HashMap<K,Ref<K,V>> map = new HashMap<>();
private final ReferenceQueue<V> queue = new ReferenceQueue<>();
private static final class Ref<K,V> extends WeakReference<V> {
final K key;
Ref(K key, V value, ReferenceQueue<V> queue) {
super(value, queue);
this.key = key;
}
}
private synchronized void gc() {
for (Ref<?,?> ref; (ref = (Ref<?,?>)queue.poll()) != null;)
map.remove(ref.key, ref);
}
public synchronized V getOrCreate(K key, Function<K,V> creator) {
gc();
Ref<K,V> ref = map.get(key);
V v = ref == null ? null : ref.get();
if (v == null) {
v = Objects.requireNonNull(creator.apply(key));
map.put(key, new Ref<>(key, v, queue));
}
return v;
}
public synchronized void remove(K key) {
gc();
map.remove(key);
}
}
No need for multiple threads; stale map entries are removed by polling the reference queue opportunistically when other methods are called. (This is also how WeakHashMap works.)
Example:
static final SimpleCache<File,BigObject> cache = new SimpleCache<>();
...
// if there is already a BigObject generated for this file,
// and it is hasn't been garbage-collected yet, it is returned;
// otherwise, its constructor is called to create one
BigObject bo = cache.getOrCreate(fileName, BigObject::new)
// it will be gc'd after nothing in the program keeps a strong ref any more