Develop Reference

Refreshing cache without impacting latency to access the cache

I have a cache refresh logic and want to make sure that it's thread-safe and correct way to do it.
public class Test {
Set<Integer> cache = Sets.newConcurrentHashSet();
public boolean contain(int num) {
return cache.contains(num);
}
public void refresh() {
cache.clear();
cache.addAll(getNums());
}
}
So I have a background thread refreshing cache - periodically call refresh. And multiple threads are calling contain at the same time. I was trying to avoid having synchronized in the methods signature because refresh could take some time (imagine that getNum makes network calls and parsing huge data) then contain would be blocked.
I think this code is not good enough because if contain called in between clear and addAll then contain always returns false.
What is the best way to achieve cache refreshing without impacting significant latency to contain call?

Best way would be to use functional programming paradigm whereby you have immutable state (in this case a Set), instead of adding and removing elements to that set you create an entirely new Set every time you want to add or remove elements. This is in Java9.
It can be a bit awkward or infeasible however to achieve this method for legacy code. So instead what you could do is have 2 Sets 1 which has the get method on it which is volatile, and then this is assigned a new instance in the refresh method.
public class Test {
volatile Set<Integer> cache = new HashSet<>();
public boolean contain(int num) {
return cache.contains(num);
}
public void refresh() {
Set<Integer> privateCache = new HashSet<>();
privateCache.addAll(getNums());
cache = privateCache;
}
}
Edit We don't want or need a ConcurrentHashSet, that is if you want to add and remove elements to a collection at the same time, which in my opinion is a pretty useless thing to do. But you want to switch the old Set with a new one, which is why you just need a volatile variable to make sure you can't read and edit the cache at the same time.
But as I mentioned in my answer at the start is that if you never modify collections, but instead make new ones each time you want to update a collection (note that this is a very cheap operation as internally the old set is reused in the operation). This way you never need to worry about concurrency, as there is no shared state between threads.

How would you make sure your cache doesn't contain invalid entries when calling contains?? Furthermore, you'd need to call refresh every time getNums() changes, which is pretty inefficient. It would be best if you make sure you control your changes to getNums() and then update cache accordingly. The cache might look like:
public class MyCache {
final ConcurrentHashMap<Integer, Boolean> cache = new ConcurrentHashMap<>(); //it's a ConcurrentHashMap to be able to use putIfAbsent
public boolean contains(Integer num) {
return cache.contains(num);
}
public void add(Integer nums) {
cache.putIfAbsent(num, true);
}
public clear(){
cache.clear();
}
public remove(Integer num) {
cache.remove(num);
}
}

Update
As #schmosel made me realize, mine was a wasted effort: it is in fact enough to initialize a complete new HashSet<> with your values in the refresh method. Assuming of course that the cache is marked with volatile. In short #Snickers3192's answer, points out what you seek.
Old answer
You can also use a slightly different system.
Keep two Set<Integer>, one of which will always be empty. When you refresh the cache, you can asynchronously re-initialize the second one and then just switch the pointers. Other threads accessing the cache won't see any particular overhead in this.
From an external point of view, they will always be accessing the same cache.
private volatile int currentCache; // 0 or 1
private final Set<Integer> caches[] = new HashSet[2]; // use two caches; either one will always be empty, so not much memory consumed
private volatile Set<Integer> cachePointer = null; // just a pointer to the current cache, must be volatile
// initialize
{
this.caches[0] = new HashSet<>(0);
this.caches[1] = new HashSet<>(0);
this.currentCache = 0;
this.cachePointer = caches[this.currentCache]; // point to cache one from the beginning
}
Your refresh method may look like this:
public void refresh() {
// store current cache pointer
final int previousCache = this.currentCache;
final int nextCache = getNextPointer();
// you can easily compute it asynchronously
// in the meantime, external threads will still access the normal cache
CompletableFuture.runAsync( () -> {
// fill the unused cache
caches[nextCache].addAll(getNums());
// then switch the pointer to the just-filled cache
// from this point on, threads are accessing the new cache
switchCachePointer();
// empty the other cache still on the async thread
caches[previousCache].clear();
});
}
where the utility methods are:
public boolean contains(final int num) {
return this.cachePointer.contains(num);
}
private int getNextPointer() {
return ( this.currentCache + 1 ) % this.caches.length;
}
private void switchCachePointer() {
// make cachePointer point to a new cache
this.currentCache = this.getNextPointer();
this.cachePointer = caches[this.currentCache];
}

How can I capture request data in a static list

I am having a URL for e.g:
http://some_url.suffix?data=test1.
I will be having multiple request of the sort:
http://some_url.suffix?data=test1
http://some_url.suffix?data=test2
http://some_url.suffix?data=test3
http://some_url.suffix?data=test4
I want to maintain a static list at server side which will contain data received from requests across all sessions
List<String> data;
List will contain data1,data2,data3,data4. List will be cleared after particular interval and new list will be used for subsequent requests.
What is the best option to achieve this:
1. static List<String> data = new CopyOnWriteArrayList<String>();
2. Singleton wrapper class to perform operation on normal java.util.List
3. using synchronized block

1st solution can cause a performance issues as the array would be copied on every incoming request.
3rd solution is better, just remember that intrinsic lock should also be static object on your controller/service. This also require to remember of synchonized block every time when touching data so probably this isn't best solution neither.
2nd option is the best solution for that case imo. Create singleton that store data internally and provide synchronized method to play with it. The synchronization stuff will be closed inside one class.
My attempt to implementation of that singleton wrapper would looks like:
//Use enum to have singleton provided by jvm
enum DataCache {
INSTANCE;
//Use LinkedList if you expecting many calls. The insertion will be much faster.
private List<String> data = new LinkedList<>();
synchronized void add(String value) {
data.add(value);
}
/*
* Returns defensive copy, so that no one has reference to this.data.
* If data is fetched only on clear you can make this private instead of synchronized
* (or even better get rid of it and create defensive copy inside clear()).
*/
synchronized List<String> get() {
return new ArrayList<>(data);
}
/*
* Returns last snapshot of data to keep consistency.
*/
synchronized List<String> clear() {
List<String> lastSnapshot = get();
data = new LinkedList<>();
return lastSnapshot;
}
}
Then you can use INSTANCE.add() in method that handle request and INSTANCE.clear() in the scheduler.
Note I don't know how do you play with gathered data, but consider other collection than list. If number of particular data occurrences isn't important it would be better to replace List with Set and HashSet implementation. (Like when you received data=test1 twice and you care only that test1 cames no matter how many times). If you care about numbers you could also consider map of values to number of occurences.

I would use a Singleton service with a method
public boolean addKey(String key, String value);
Then in your service I would use a HashMap>
private static Map<String, List<String>> keysMap;
private Map<String, List<String>> getKeysMap(){
synchronized (this){
if(keysMap == null){
keysMap = new HashMap();
}
return keysMap;
}
}
public void addKey(String key, String value){
List<String> keyParams = getKeysMap().get(key);
if(keyParams == null){
keyParams = new ArrayList();
}
//decide here if you want to store repeated values
keyParams.add(param);
getKeysMap().put(key, keyParams);
}

How should I implement Guava cache when I plan to cache multiple values efficiently?

I have a Java class that has a Guava LoadingCache<String, Integer> and in that cache, I'm planning to store two things: the average time active employees have worked for the day and their efficiency. I am caching these values because it would be expensive to compute every time a request comes in. Also, the contents of the cache will be refreshed (refreshAfterWrite) every minute.
I was thinking of using a CacheLoader for this situation, however, its load method only loads one value per key. In my CacheLoader, I was planning to do something like:
private Service service = new Service();
public Integer load(String key) throws Exception {
if (key.equals("employeeAvg"))
return calculateEmployeeAvg(service.getAllEmployees());
if (key.equals("employeeEff"))
return calculateEmployeeEff(service.getAllEmployees());
return -1;
}
For me, I find this very inefficient since in order to load both values, I have to invoke service.getAllEmployees() twice because, correct me if I'm wrong, CacheLoader's should be stateless.
Which made me think to use the LoadingCache.put(key, value) method so I can just create a utility method that invokes service.getAllEmployees() once and calculate the values on the fly. However, if I do use LoadingCache.put(), I won't have the refreshAfterWrite feature since it's dependent on a cache loader.
How do I make this more efficient?

It seems like your problem stems from using strings to represent value types (Effective Java Item 50). Instead, consider defining a proper value type that stores this data, and use a memoizing Supplier to avoid recomputing them.
public static class EmployeeStatistics {
private final int average;
private final int efficiency;
// constructor, getters and setters
}
Supplier<EmployeeStatistics> statistics = Suppliers.memoize(
new Supplier<EmployeeStatistics>() {
#Override
public EmployeeStatistics get() {
List<Employee> employees = new Service().getAllEmployees();
return new EmployeeStatistics(
calculateEmployeeAvg(employees),
calculateEmployeeEff(employees));
}});
You could even move these calculation methods inside EmployeeStatistics and simply pass in all employees to the constructor and let it compute the appropriate data.
If you need to configure your caching behavior more than Suppliers.memoize() or Suppliers.memoizeWithExpiration() can provide, consider this similar pattern, which hides the fact that you're using a Cache inside a Supplier:
Supplier<EmployeeStatistics> statistics = new Supplier<EmployeeStatistics>() {
private final Object key = new Object();
private final LoadingCache<Object, EmployeeStatistics> cache =
CacheBuilder.newBuilder()
// configure your builder
.build(
new CacheLoader<Object, EmployeeStatistics>() {
public EmployeeStatistics load(Object key) {
// same behavior as the Supplier above
}});
#Override
public EmployeeStatistics get() {
return cache.get(key);
}};

However, if I do use LoadingCache.put(), I won't have the refreshAfterWrite feature since it's dependent on a cache loader.
I'm not sure, but you might be able to call it from inside the load method. I mean, compute the requested value as you do and put in the other. However, this feels hacky.
If service.getAllEmployees is expensive, then you could cache it. If both calculateEmployeeAvg and calculateEmployeeEff are cheap, then recompute them when needed. Otherwise, it looks like you could use two caches.
I guess, a method computing both values at once could be a reasonable solution. Create a tiny Pair-like class aggregating them and use it as the cache value. There'll be a single key only.
Concerning your own solution, it could be as trivial as
class EmployeeStatsCache {
private long validUntil;
private List<Employee> employeeList;
private Integer employeeAvg;
private Integer employeeEff;
private boolean isValid() {
return System.currentTimeMillis() <= validUntil;
}
private synchronized List<Employee> getEmployeeList() {
if (!isValid || employeeList==null) {
employeeList = service.getAllEmployees();
validUntil = System.currentTimeMillis() + VALIDITY_MILLIS;
}
return employeeList;
}
public synchronized int getEmployeeAvg() {
if (!isValid || employeeAvg==null) {
employeeAvg = calculateEmployeeAvg(getEmployeeList());
}
return employeeAvg;
}
public synchronized int getEmployeeEff() {
if (!isValid || employeeAvg==null) {
employeeAvg = calculateEmployeeEff(getEmployeeList());
}
return employeeAvg;
}
}
Instead of synchronized methods you may want to synchronize on a private final field. There are other possibilities (e.g. Atomic*), but the basic design is probably simpler than adapting Guava's Cache.
Now, I see that there's Suppliers#memoizeWithExpiration in Guava. That's probably even simpler.

Getting an InputStream to read more than once, regardless of markSupported()

I need to be able to re-use a java.io.InputStream multiple times, and I figured the following code would work, but it only works the first time.
Code
public class Clazz
{
private java.io.InputStream dbInputStream, firstDBInputStream;
private ArrayTable db;
public Clazz(java.io.InputStream defDB)
{
this.firstDBInputStream = defDB;
this.dbInputStream = defDB;
if (db == null)
throw new java.io.FileNotFoundException("Could not find the database at " + db);
if (dbInputStream.markSupported())
dbInputStream.mark(Integer.MAX_VALUE);
loadDatabaseToArrayTable();
}
public final void loadDatabaseToArrayTable() throws java.io.IOException
{
this.dbInputStream = firstDBInputStream;
if (dbInputStream.markSupported())
dbInputStream.reset();
java.util.Scanner fileScanner = new java.util.Scanner(dbInputStream);
String CSV = "";
for (int i = 0; fileScanner.hasNextLine(); i++)
CSV += fileScanner.nextLine() + "\n";
db = ArrayTable.createArrayTableFromCSV(CSV);
}
public void reloadDatabase()//A method called by the UI
{
try
{
loadDatabaseToArrayTable();
}
catch (Throwable t)
{
//Alert the user that an error has occurred
}
}
}
Note that ArrayTable is a class of mine, which uses arrays to give an interface for working with tables.
Question
In this program, the database is shown directly to the user immediately after the reloadDatabase() method is called, and so any solution involving saving the initial read to an object in memory is useless, as that will NOT refresh the data (think of it like a browser; when you press "Refresh", you want it to fetch the information again, not just display the information it fetched the first time). How can I read a java.io.InputStream more than once?

You can't necessarily read an InputStream more than once. Some implementations support it, some don't. What you are doing is checking the markSupported method, which is indeed an indicator if you can read the same stream twice, but then you are ignoring the result. You have to call that method to see if you can read the stream twice, and if you can't, make other arrangements.
Edit (in response to comment): When I wrote my answer, my "other arrangements" was to get a fresh InputStream. However, when I read in your comments to your question about what you want to do, I'm not sure it is possible. For the basics of the operation, you probably want RandomAccessFile (at least that would be my first guess, and if it worked, that would be the easiest) - however you will have file access issues. You have an application actively writing to a file, and another reading that file, you will have problems - exactly which problems will depend on the OS, so whatever solution would require more testing. I suggest a separate question on SO that hits on that point, and someone who has tried that out can perhaps give you more insight.

you never mark the stream to be reset
public Clazz(java.io.InputStream defDB)
{
firstDBInputStream = defDB.markSupported()?defDB:new BufferedInputStream(defDB);
//BufferedInputStream supports marking
firstDBInputStream.mark(500000);//avoid IOException on first reset
}
public final void loadDatabaseToArrayTable() throws java.io.IOException
{
this.dbInputStream = firstDBInputStream;
dbInputStream.reset();
dbInputStream.mark(500000);//or however long the data is
java.util.Scanner fileScanner = new java.util.Scanner(dbInputStream);
StringBuilder CSV = "";//StringBuilder is more efficient in a loop
while(fileScanner.hasNextLine())
CSV.append(fileScanner.nextLine()).append("\n");
db = ArrayTable.createArrayTableFromCSV(CSV.toString());
}
however you could instead keep a copy of the original ArrayTable and copy that when you need to (or even the created string to rebuild it)
this code creates the string and caches it so you can safely discard the inputstreams and just use readCSV to build the ArrayTable
private String readCSV=null;
public final void loadDatabaseToArrayTable() throws java.io.IOException
{
if(readCSV==null){
this.dbInputStream = firstDBInputStream;
java.util.Scanner fileScanner = new java.util.Scanner(dbInputStream);
StringBuilder CSV = "";//StringBuilder is more efficient in a loop
while(fileScanner.hasNextLine())
CSV.append(fileScanner.nextLine()).append("\n");
readCSV=CSV.toString();
fileScanner.close();
}
db = ArrayTable.createArrayTableFromCSV(readCSV);
}
however if you want new information you'll need to create a new stream to read from again

Java's WeakHashMap and caching: Why is it referencing the keys, not the values?

Java's WeakHashMap is often cited as being useful for caching. It seems odd though that its weak references are defined in terms of the map's keys, not its values. I mean, it's the values I want to cache, and which I want to get garbage collected once no-one else besides the cache is strongly referencing them, no?
In which way does it help to hold weak references to the keys? If you do a ExpensiveObject o = weakHashMap.get("some_key"), then I want the cache to hold on to 'o' until the caller doesn't hold the strong reference anymore, and I don't care at all about the string object "some_key".
Am I missing something?

WeakHashMap isn't useful as a cache, at least the way most people think of it. As you say, it uses weak keys, not weak values, so it's not designed for what most people want to use it for (and, in fact, I've seen people use it for, incorrectly).
WeakHashMap is mostly useful to keep metadata about objects whose lifecycle you don't control. For example, if you have a bunch of objects passing through your class, and you want to keep track of extra data about them without needing to be notified when they go out of scope, and without your reference to them keeping them alive.
A simple example (and one I've used before) might be something like:
WeakHashMap<Thread, SomeMetaData>
where you might keep track of what various threads in your system are doing; when the thread dies, the entry will be removed silently from your map, and you won't keep the Thread from being garbage collected if you're the last reference to it. You can then iterate over the entries in that map to find out what metadata you have about active threads in your system.
See WeakHashMap in not a cache! for more information.
For the type of cache you're after, either use a dedicated cache system (e.g. EHCache) or look at Guava's MapMaker class; something like
new MapMaker().weakValues().makeMap();
will do what you're after, or if you want to get fancy you can add timed expiration:
new MapMaker().weakValues().expiration(5, TimeUnit.MINUTES).makeMap();

The main use for WeakHashMap is when you have mappings which you want to disappear when their keys disappear. A cache is the reverse---you have mappings which you want to disappear when their values disappear.
For a cache, what you want is a Map<K,SoftReference<V>>. A SoftReference will be garbage-collected when memory gets tight. (Contrast this with a WeakReference, which may be cleared as soon as there is no longer a hard reference to its referent.) You want your references to be soft in a cache (at least in one where key-value mappings don't go stale), since then there is a chance that your values will still be in the cache if you look for them later. If the references were weak instead, your values would be gc'd right away, defeating the purpose of caching.
For convenience, you might want to hide the SoftReference values inside your Map implementation, so that your cache appears to be of type <K,V> instead of <K,SoftReference<V>>. If you want to do that, this question has suggestions for implementations available on the net.
Note also that when you use SoftReference values in a Map, you must do something to manually remove key-value pairs which have had their SoftReferences cleared---otherwise your Map will only grow in size forever, and leak memory.

Another thing to consider is that if you take the Map<K, WeakReference<V>> approach, the value may disappear, but the mapping will not. Depending on usage, you may as a result end up with a Map containing many entries whose Weak References have been GC'd.

You need two maps: one which maps between the cache key and weak referenced values and one in the opposite direction mapping between the weak referenced values and the keys. And you need a reference queue and a cleanup thread.
Weak references have the ability to move the reference into a queue when the referenced object can not accessed any longer. This queue has to be drained by a cleanup thread. And for the cleanup it is necessary to get the key for a reference. This is the reason why the second map is required.
The following example shows how to create a cache with a hash map of weak references. When you run the program you get the following output:
$ javac -Xlint:unchecked Cache.java && java Cache
{even: [2, 4, 6], odd: [1, 3, 5]}
{even: [2, 4, 6]}
The first line shows the contents of the cache before the reference to the odd list has been deleted and the second line after the odds have been deleted.
This is the code:
import java.lang.ref.Reference;
import java.lang.ref.ReferenceQueue;
import java.lang.ref.WeakReference;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
class Cache<K,V>
{
ReferenceQueue<V> queue = null;
Map<K,WeakReference<V>> values = null;
Map<WeakReference<V>,K> keys = null;
Thread cleanup = null;
Cache ()
{
queue = new ReferenceQueue<V>();
keys = Collections.synchronizedMap (new HashMap<WeakReference<V>,K>());
values = Collections.synchronizedMap (new HashMap<K,WeakReference<V>>());
cleanup = new Thread() {
public void run() {
try {
for (;;) {
#SuppressWarnings("unchecked")
WeakReference<V> ref = (WeakReference<V>)queue.remove();
K key = keys.get(ref);
keys.remove(ref);
values.remove(key);
}
}
catch (InterruptedException e) {}
}
};
cleanup.setDaemon (true);
cleanup.start();
}
void stop () {
cleanup.interrupt();
}
V get (K key) {
return values.get(key).get();
}
void put (K key, V value) {
WeakReference<V> ref = new WeakReference<V>(value, queue);
keys.put (ref, key);
values.put (key, ref);
}
public String toString() {
StringBuilder str = new StringBuilder();
str.append ("{");
boolean first = true;
for (Map.Entry<K,WeakReference<V>> entry : values.entrySet()) {
if (first)
first = false;
else
str.append (", ");
str.append (entry.getKey());
str.append (": ");
str.append (entry.getValue().get());
}
str.append ("}");
return str.toString();
}
static void gc (int loop, int delay) throws Exception
{
for (int n = loop; n > 0; n--) {
Thread.sleep(delay);
System.gc(); // <- obstinate donkey
}
}
public static void main (String[] args) throws Exception
{
// Create the cache
Cache<String,List> c = new Cache<String,List>();
// Create some values
List odd = Arrays.asList(new Object[]{1,3,5});
List even = Arrays.asList(new Object[]{2,4,6});
// Save them in the cache
c.put ("odd", odd);
c.put ("even", even);
// Display the cache contents
System.out.println (c);
// Erase one value;
odd = null;
// Force garbage collection
gc (10, 10);
// Display the cache again
System.out.println (c);
// Stop cleanup thread
c.stop();
}
}

If you need weak values it's surprisingly easy:
public final class SimpleCache<K,V> {
private final HashMap<K,Ref<K,V>> map = new HashMap<>();
private final ReferenceQueue<V> queue = new ReferenceQueue<>();
private static final class Ref<K,V> extends WeakReference<V> {
final K key;
Ref(K key, V value, ReferenceQueue<V> queue) {
super(value, queue);
this.key = key;
}
}
private synchronized void gc() {
for (Ref<?,?> ref; (ref = (Ref<?,?>)queue.poll()) != null;)
map.remove(ref.key, ref);
}
public synchronized V getOrCreate(K key, Function<K,V> creator) {
gc();
Ref<K,V> ref = map.get(key);
V v = ref == null ? null : ref.get();
if (v == null) {
v = Objects.requireNonNull(creator.apply(key));
map.put(key, new Ref<>(key, v, queue));
}
return v;
}
public synchronized void remove(K key) {
gc();
map.remove(key);
}
}
No need for multiple threads; stale map entries are removed by polling the reference queue opportunistically when other methods are called. (This is also how WeakHashMap works.)
Example:
static final SimpleCache<File,BigObject> cache = new SimpleCache<>();
...
// if there is already a BigObject generated for this file,
// and it is hasn't been garbage-collected yet, it is returned;
// otherwise, its constructor is called to create one
BigObject bo = cache.getOrCreate(fileName, BigObject::new)
// it will be gc'd after nothing in the program keeps a strong ref any more