Develop Reference

How to create a write-behind cache from inmemory to database?

My goal is to cache data inmemory for 60s. As soon as the entry is read again from cache, I want to remove it from cache (permit single reads only).
If those 60s expired in the meantime and the entry is still available in cache, I want to writebehind the entry into a database.
Is there any existing technology/spring/apache framework that already offers such a cache?
(sidenote: I don't want to use complex libraries like redis, ehcache etc for such a simple usecase).
If set up manually, I'd be doing as follows. But probably there are better options?
#Service
public class WriteBehindCache {
static class ObjectEntry {
Object data;
LocalDateTime timestamp;
public ObjectEntry(Object data) {
this.data = data;
timestamp = LocalDateTime.now();
}
}
Map<String, ObjectEntry> cache = new ConcurrentHashMap<>();
//batch every minute
#Scheduled(fixedRate = 60000)
public void writeBehind() {
LocalDateTime now = LocalDateTime.now();
List<ObjectEntry> outdated = cache.values()
.filter(entry -> entry.getValue().timestamp.plusSeconds(60).isBefore(now))
.collect(Collectors.toList());
databaseService.persist(outdated);
cache.removeAll(outdated); //pseudocode
}
//always keep most recent entry
public void add(String key, Object data) {
cache.put(key, new ObjectEntry(data));
}
//fallback lookup to database if cache is empty
public Object get(String key) {
ObjectEntry entry = cache.remove(key);
if (entry == null) {
entry = databaseService.query(key);
if (entry != null) databaseService.remove(entry);
}
return entry;
}
}

Your solution has two problems:
You are doing a sequential scan for persisting, which will get costly when there are a lot of entries
The code has race conditions
Due to the race conditions the code does not satisfy your requirements. Its possible to construct a concurrent access sequence where an entry is removed from the cache but as well was written to the database
Is there any existing technology/spring/apache framework that already offers such a cache? (sidenote: I don't want to use complex libraries like redis, ehcache etc for such a simple usecase).
I think you can solve the concurrency issues based on the ConcurrentHashMap. But I don't know an elegant way for the timeout. Still, a possible solution is to use a caching library. I'd like to offer an example based on cache2k which is not heavy (about a 400k jar) and has other nice use cases as well. As an extra there is also good support for the Spring caching abstraction.
public static class WriteBehindCache {
Cache<String, Object> cache = Cache2kBuilder.of(String.class, Object.class)
.addListener((CacheEntryExpiredListener<String, Object>) (cache, entry)
-> persist(entry.getKey(), entry.getValue()))
.expireAfterWrite(60, TimeUnit.SECONDS)
.build();
public void add(String key, Object data) {
cache.put(key, data);
}
public Object get(String key) {
return cache.invoke(key, e -> {
if (e.exists()) {
Object v = e.getValue();
e.remove();
return v;
}
return loadAndRemove(e.getKey());
});
}
// stubs
protected void persist(String key, Object value) {
}
protected Object loadAndRemove(String key) {
return null;
}
}
With this wiring the cache blocks out concurrent operation on one entry, so it is certain that only one database operation runs for one entry at a time.
You can do it in similar ways with other caching libraries. Using the JCache/JSR107 API the code would look almost identical.
A more "lighter" approach is to use jhalterman's expiringmap
Personally, I believe a cache should be in every developers toolbox. However, I am the author of cache2k. Of course, I need to say that.

Is this an actual shortcut for a Java Lazy-Getter [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
The following is less a question than a request for evaluation.
So you will most likely be familiar with the following pattern of a lazy getter.
private Object obj;
public Object getObject() {
if(obj==null) {
obj = new Object();
}
return obj;
}
That code
is long
requires a class variable
requires some logic inside the getter
So recently a colleague and I came up with the following interface (simplified):
public interface LazyGetterSupport {
default Object get(Supplier<Object> impl) {
String key = impl.getClass().getName();
Object retVal;
if ((retVal = getInstanceCache().get(key)) == null) {
retVal = impl.get();
getInstanceCache().put(key, retVal);
}
return retVal;
}
Map<String, Object> getInstanceCache();
}
Sidenote: Not using HashMap#computeIfAbsent bc of Bug-8071667
That Interface is then implemented by the class that you want to use lazy getters in. You need to provide an implementation of getInstanceCache() as follows:
private Map<String, Object> instanceCache;
#Override
public Map<String, Object> getInstanceCache() {
if (instanceCache == null) {
instanceCache = new HashMap<>();
}
return instanceCache;
}
But given that you can start rewriting all other lazy getters in that class (and subclasses) like this:
public Object getObject() {
return get(Objetct::new);
}
This works because the key we get by using impl.getClass().getName(); inside our Interface will actually always be unique for every lambda we use inside implementations of its get-method. Yet it will stay the same for each call-site for at least the life duration of our VM.
Benefits are obvious. We no longer have to create a class-variable for every lazy-getter and the getter themselves become shorter. It may not be a world of a difference, but in our use-case we often have classes with 20+ Lazy Getters for UI Elements. That is where this new technique comes to shine.
I would love to know your thoughts on this approach and whether you have any concerns towards using this in production.

Or you could simply use #Getter(lazy=true) from Project Lombok.

Your cache is per class which doesn't feel future proof to me. If you had some generic Supplier that was used multiple times then using the Class as the cache key wouldn't work. I'd just store a reference on a LazyValue which you can use and cache however you like. Here's a thread safe implementation:
public class LazyValue<T> {
private AtomicReference<T> reference = new AtomicReference<>();
private final Supplier<T> supplier;
public LazyValue(Supplier supplier) { this.supplier = supplier; }
public T get() {
T t = reference.get();
if (t == null) {
synchronized (reference) {
t = reference.get();
if (t == null) {
t = supplier.get();
reference.set(t);
}
}
}
return t;
}
}
public class MyBean {
private LazyValue<Object> lazyObject = new LazyValue(Object::new);
public Object getObject() {
return lazyObject.get();
}
}

Looks interesting, but you replaced a bunch of member variables with a Hashmap. While this works, it increases the memory usage of your class, and adds O(1) overhead to member access. Synchronizing the lazy getter method is not enough, you need to synchronize on the HashMap, adding keys concurrently will break something. In the traditional lazy getter, synchronizing on the one attribute is enough. Here, if you concurrently access different attributes they have to wait on each other.
And in your use case, I really wonder why one would need lazy getters for UI elements.

Object o;
public Object getInstance() {
return o == null ? (o = new Object()) : o;
}
No need for a fancy HashMap, just convert "long" 4 lines of code into 1.

You could also allow the client to actually set a property.
interface LazyGetterSupport<T> {
T get();
static <T> LazyGetterSupport<T> of(T value, Supplier<T> create, Consumer<T> store) {
T result = value;
if (result == null) {
result = create.get();
store.accept(result);
}
return result;
}
}
which you could call using
class Client {
Object field;
void getField() {
LazyGetterSupport.of(field, Object::new, o -> field = o).get();
}
}

Extract list of String from RealmResult [duplicate]

I have the ViewValue class defined as follows:
class ViewValue {
private Long id;
private Integer value;
private String description;
private View view;
private Double defaultFeeRate;
// getters and setters for all properties
}
Somewhere in my code i need to convert a list of ViewValue instances to a list containing values of id fields from corresponding ViewValue.
I do it using foreach loop:
List<Long> toIdsList(List<ViewValue> viewValues) {
List<Long> ids = new ArrayList<Long>();
for (ViewValue viewValue : viewValues) {
ids.add(viewValue.getId());
}
return ids;
}
Is there a better approach to this problem?

We can do it in a single line of code using java 8
List<Long> ids = viewValues.stream().map(ViewValue::getId).collect(Collectors.toList());
For more info : Java 8 - Streams

You could do it in a one-liner using Commons BeanUtils and Collections:
(why write your own code when others have done it for you?)
import org.apache.commons.beanutils.BeanToPropertyValueTransformer;
import org.apache.commons.collections.CollectionUtils;
...
List<Long> ids = (List<Long>) CollectionUtils.collect(viewValues,
new BeanToPropertyValueTransformer("id"));

Use google collections. Example:
Function<ViewValue, Long> transform = new Function<ViewValue, Long>() {
#Override
public Long apply(ViewValue from) {
return from.getId();
}
};
List<ViewValue> list = Lists.newArrayList();
List<Long> idsList = Lists.transform(list, transform);
UPDATE:
On Java 8 you don't need Guava. You can:
import com.example.ViewValue;
import java.util.ArrayList;
import java.util.List;
import java.util.function.Function;
import java.util.stream.Collectors;
Function<ViewValue, Long> transform = ViewValue::getId;
List<ViewValue> source = new ArrayList<>();
List<Long> result = source.stream().map(transform).collect(Collectors.toList());
Or just:
List<ViewValue> source= new ArrayList<>();
List<Long> result = source.stream().map(ViewValue::getId).collect(Collectors.toList());
NEXT UPDATE (The last one after Javaslang to Vavr name change):
Currently it's worth to mention about the solution with Javaslang library(http://www.javaslang.io/) Vavr library (http://www.vavr.io/). Let's assume that we have our list with genuine objects:
List<ViewValue> source = newArrayList(new ViewValue(1), new ViewValue(2), new ViewValue(2));
We could make transformation with List class from Javaslang library (on the long run the collect is not convenient):
List<Long> result = io.vavr.collection.List.ofAll(source).map(ViewValue::getId).toJavaList();
But you will see the power with only the Javaslang lists:
io.vavr.collection.List<ViewValue> source = javaslang.collection.List.of(new ViewValue(1), new ViewValue(2), new ViewValue(3));
io.vavr.collection.List<Long> res = source.map(ViewValue::getId);
I encourage to take a look available collections and new types on that library (I like especially the Try type). You will find the documentation under the following address: http://www.javaslang.io/javaslang-docs/ http://www.vavr.io/vavr-docs/.
PS. Due to the Oracle and the "Java" word within the name they had to change the library name from javaslang to something else. They had decided to Vavr.

EDIT: This answer is based on the idea that you'll need to do similar things for different entities and different properties elsewhere in your code. If you only need to convert the list of ViewValues to a list of Longs by ID, then stick with your original code. If you want a more reusable solution, however, read on...
I would declare an interface for the projection, e.g.
public interface Function<Arg,Result>
{
public Result apply(Arg arg);
}
Then you can write a single generic conversion method:
public <Source, Result> List<Result> convertAll(List<Source> source,
Function<Source, Result> projection)
{
ArrayList<Result> results = new ArrayList<Result>();
for (Source element : source)
{
results.add(projection.apply(element));
}
return results;
}
Then you can define simple projections like this:
private static final Function<ViewValue, Long> ID_PROJECTION =
new Function<ViewValue, Long>()
{
public Long apply(ViewValue x)
{
return x.getId();
}
};
And apply it just like this:
List<Long> ids = convertAll(values, ID_PROJECTION);
(Obviously using K&R bracing and longer lines makes the projection declaration a bit shorter :)
Frankly all of this would be a lot nicer with lambda expressions, but never mind...

I've implemented a small functional library for this usecase. One of the methods has this signature:
<T> List<T> mapToProperty(List<?> objectList, String property, Class<T> returnType)
Which takes the string and uses reflection to create a call to the property then it returns a List backed by the objectList where get and iterator implemented using this property call.
The mapToProperty functions is implemented in terms of a general map function that takes a Function as a mapper though, just as another post described. Very usefull.
I suggest you read up on basic functionl programming and in particular take a look at Functors (objects implementing a map function)
Edit: Reflection really doesn't have to be expensive. The JVM has improved a lot in this area. Just make sure to compile the invocation once and reuse it.
Edit2: Sample code
public class MapExample {
public static interface Function<A,R>
{
public R apply(A b);
}
public static <A,R> Function<A,R> compilePropertyMapper(Class<A> objectType, String property, Class<R> propertyType)
{
try {
final Method m = objectType.getMethod("get" + property.substring(0,1).toUpperCase() + property.substring(1));
if(!propertyType.isAssignableFrom(m.getReturnType()))
throw new IllegalArgumentException(
"Property "+property+" on class "+objectType.getSimpleName()+" is not a "+propertyType.getSimpleName()
);
return new Function<A,R>()
{
#SuppressWarnings("unchecked")
public R apply(A b)
{
try {
return (R)m.invoke(b);
} catch (Exception e) {
throw new RuntimeException(e);
}
};
};
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public static <T1,T2> List<T2> map(final List<T1> list, final Function<T1,T2> mapper)
{
return new AbstractList<T2>()
{
#Override
public T2 get(int index) {
return mapper.apply(list.get(index));
}
#Override
public int size() {
return list.size();
}
};
}
#SuppressWarnings("unchecked")
public static <T1,T2> List<T2> mapToProperty(List<T1> list, String property, Class<T2> propertyType)
{
if(list == null)
return null;
else if(list.isEmpty())
return Collections.emptyList();
return map(list,compilePropertyMapper((Class<T1>)list.get(0).getClass(), property, propertyType));
}
}

You could use a wrapper:
public class IdList impements List<Long>
{
private List<ViewValue> underlying;
pubic IdList(List<ViewValue> underying)
{
this.underlying = underying;
}
public Long get(int index)
{
return underlying.get(index).getId()
}
// other List methods
}
Though that's even more tedious work, it could improve performance.
You could also implement your and my solution generic-ly using reflection, but that would be very bad for performance.
There's no short and easy generic solution in Java, I'm afraid. In Groovy, you would simply use collect(), but I believe that involves reflection as well.

That depends on what you then do with the List<Long>, and the List<ViewValue>
For example you might get sufficient functionality from creating your own List implementation that wraps a List<ViewValue>, implementing iterator() with an iterator implementation that iterates over the ViewValues, returning the id.

You can populate a map from the properties of a list of objects (say id as key and some property as value) as below
Map<String, Integer> mapCount = list.stream().collect(Collectors.toMap(Object::get_id, Object::proprty));

Java: Reflection overuse?

I am wondering whether I am overusing java reflection.
I have a class which is a data holder for a couple of maps. I have public get(...) methods which given a key as input return the value associated with it in the corresponding map.
Since the maps are large I load them only when I actually want to access them. So, in every get(...) methods, I check whether the map is null. If it is, I call the corresponding loadMap(..) method.
Here is a sample code snippet
public getId(String name)
{
try
{
if(nameMap1 == null)
loadNameMap1();
} catch(...) {....}
return nameMap1.getId(name);
}
The problem is that I have multiple maps. So, for loading each map I have a different loadMap(..) method and the try catch block in the get(...) methods. So, instead of that I wrote a method called loadMap(Object map, String methodName) which uses reflection to call the appropriate method, and handles all exceptions.
private synchronized void loadMap(Object map, String methodName)
{
if (map == null)
try
{
Method method = this.getClass().getDeclaredMethod(methodName, new Class[0]);
method.invoke(this, new Object[0]);
}
catch (..)
}
Am I overusing reflection here? Is there a better way to do this? Does this qualify as "limited use of reflection" as written in Effective Java by Joshua Bloch
(Side note: I cannot refactor the class into multiple classes )

// could also be static
private Map<String, Callable<Map>> myLoaders;
private synchronized void loadMap(Object map, String mapName)
{
if (map == null)
try
{
Callable<Map> mapLoader = myLoaders.get(mapName);
map = mapLoader.call();
}
catch (..)
}
// and in the constructor or other init code
myLoaders.put("map1", new Callable<Map>(){
Map call(){
// load map 1
}});
I think, though that if all you are doing is move a common try/catch logic from a couple of methods were it needs to be repeated to a single place, this is the wrong approach. You lose a lot of compiler error checking support this way. Some people would use a tool like Aspect/J for this, but I think you just have to live with the fact that Java has no real facility for this, reduce the clutter to a minimum by using shared private functions, and accept the couple of copy/pasted lines. As long as there is no "real code" in those lines, it is not really harmful code duplication.
So:
public getId(String name){
try{
if (nameMap1 == null)
loadNameMap1();
}
catch (....){
privateHelperFunctionThatCutsThisDownToOneLine(name, "id", "nameMap1");
}
}
// you are left with the above repetitive three (or seven) lines,
// but that is Java for you...
// in return, you get nice, static compile-time error checking
private void privateHelperFunctionThatCutsThisDownToOneLine(){
// all the long repeated code in the exception handler
// goes here.
}

You don't want to load all the maps because they are too large. But using your method you're gonna end up with everything loaded in memory eventually. You may have a look at ehcache which may be configured a a lazy map system with element eviction when no longer needed.

I'd say yes you are overusing reflection.
Perhaps you should take a more OO approach
public interface MapMaker <K,V> {
public Map<K,V> create();
}
public class LazyMap<K,V> implements Map<K,V> {
private MapMaker<K,V> creation;
private Map<K,V> theMap = null;
public LazyMap( MapMaker<K,V> creation) {
this.creation=creation;
}
protected Map<K,V> getMap() {
if( theMap == null) {
synchronized(this) {
if( theMap == null ) {
theMap = creation.create();
}
}
}
return theMap;
}
//Map interface
public V get(Object key) { return getMap().get(key); }
//repeat for all
}

Threadsafe way of exposing keySet()

This must be a fairly common occurrence where I have a map and wish to thread-safely expose its key set:
public MyClass {
Map<String,String> map = // ...
public final Set<String> keys() {
// returns key set
}
}
Now, if my "map" is not thread-safe, this is not safe:
public final Set<String> keys() {
return map.keySet();
}
And neither is:
public final Set<String> keys() {
return Collections.unmodifiableSet(map.keySet());
}
So I need to create a copy, such as:
public final Set<String> keys() {
return new HashSet(map.keySet());
}
However, this doesn't seem safe either because that constructor traverses the elements of the parameter and add()s them. So while this copying is going on, a ConcurrentModificationException can happen.
So then:
public final Set<String> keys() {
synchronized(map) {
return new HashSet(map.keySet());
}
}
seems like the solution. Does this look right?

That solution isn't particularly helpful unless you plan to also synchronize on the map everywhere it is used. Synchronizing on it doesn't stop someone else from invoking methods on it at the same time. It only stops them from also being able to synchronize on it.
The best solution really seems to be just use ConcurrentHashMap in the first place if you know you need concurrent puts and removes while someone may be iterating. If the concurrency behavior that class offers isn't what you need, you'll probably just need to use a fully synchronized Map.

Good question. I would use Google Guava library. More specifically com.google.common.collect.ImmutableSet.copyOf(Collection<? extends E>) method. In documentation it has been said that this method is thread safe.

Another option would be to use ConcurrentHashMap. Its keySet() is thread safe so there might be no need to synchronize or take a copy.

If you are interested on thread-safe iterator with exact snapshot of elements through out the iteration process then go for the below.
public class ThreadSafeIteratorConcurrentMap
{
private ConcurrentMap<String, String> itrSafeMap = null;
public ThreadSafeIteratorConcurrentCollection() {
itrSafeMap = new ConcurrentHashMap<String, String>
}
public void synchronized put(psConference conference, String p_key)
{
itrSafeMap.putIfAbsent(p_key, conference);
}
public psConference getConference(String p_key)
{
return (itrSafeMap.get(p_key));
}
public void synchronized remove(String p_key)
{
itrSafeMap.remove(p_key);
}
public boolean containsKey(String p_key)
{
return itrSafeMap.containsKey(p_key);
}
// Get the size of the itrSafeMap.
public int size()
{
return itrSafeMap.size();
}
public Iterator<String> valueIterator()
{
return (itrSafeMap.values().iterator());
}
public Iterator<String> keyIterator()
{
return (itrSafeMap.keySet().iterator());
}
}
Then where ever you want thread safe iterator with exact snapshot of elements; then use it in synchronized block like below.
synchronized(threadSafeIteratorConcurrentMapObject) {
Iterator<String> keyItr = threadSafeIteratorConcurrentMapObject.keyIterator();
while(keyItr.hasNext()){
// Do whatever
}
}
If you don't mind modification on the collection while iteration; only concentrating on snapshot of elements at the time of iterator creation; then without synchronization block you can use keyItr. Which is already thread safe; it wont through ConcurrentModificationException.

You can create an temporary Map using Collections.UnmodifiableMap, then iterate the keyset .