Develop Reference

Kotlin concurrency for ConcurrentHashMap

I am trying to support concurrency on a hashmap that gets periodically cleared. I have a cache that stores data for a period of time. After every 5 minutes, the data in this cache is sent to the server. Once I flush, I want to clear the cache. The problem is when I am flushing, data could potentially be written to this map while I am doing that with an existing key. How would I go about making this process thread safe?
data class A(val a: AtomicLong, val b: AtomicLong) {
fun changeA() {
a.incrementAndGet()
}
}
class Flusher {
private val cache: Map<String, A> = ConcurrentHashMap()
private val lock = Any()
fun retrieveA(key: String){
synchronized(lock) {
return cache.getOrPut(key) { A(key, 1) }
}
}
fun flush() {
synchronized(lock) {
// send data to network request
cache.clear()
}
}
}
// Existence of multiple classes like CacheChanger
class CacheChanger{
fun incrementData(){
flusher.retrieveA("x").changeA()
}
}
I am worried that the above cache is not properly synchronized. Are there better/right ways to lock this cache so that I don't lose out on data? Should I create a deepcopy of cache and clear it?
Since the above data could be being changed by another changer, could that not lead to problems?

You can get rid of the lock.
In the flush method, instead of reading the entire map (e.g. through an iterator) and then clearing it, remove each element one by one.
I'm not sure if you can use iterator's remove method (I'll check that in a moment), but you can take the keyset iterate over it, and for each key invoke cache.remove() - this will give you the value stored and remove it from the cache atomically.
The tricky part is how to make sure that the object of class A won't be modified just prior sending over network... You can do it as follows:
When you get the some x through retrieveA and modify the object, you need to make sure it is still in the cache. Simply invoke retrieve one more time. If you get exactly the same object it's fine. If it's different, then it means that object was removed and sent over network, but you don't know if the modification was also sent, or the state of the object prior to the modification was sent. Still, I think in your case, you can simply repeat the whole process (apply change and check if objects are the same). But it depends on the specifics of your application.
If you don't want to increment twice, then when sending the data over network, you'll have to read the content of the counter a, store it in some local variable and decrease a by that amount (usually it will get zero). Then in the CacheChanger, when you get a different object from the second retrieve, you can check if the value is zero (your modification was taken into account), or non-zero which means your modification came just a fraction of second to late, and you'll have to repeat the process.
You could also replace incrementAndGet with compareAndSwap, but this could yield slightly worse performance. In this approach, instead of incrementing, you try to swap a value that is greater by one. And before sending over network you try to swap the value to -1 to denote the value as invalid. If the second swap fails it means that someone has changed the value concurrently, you need to check it one more time in order to send the freshest value over network, and you repeat the process in a loop (breaking the loop only once the swap to -1 succeeds). In the case of swap to greater by one, you also repeat the process in a loop until the swap succeeds. If it fails, it either means that somebody else swapped to some greater value, or the Flusher swapped to -1. In the latter case you know that you have to call retrieveA one more time to get a new object.

The easiest solution (but with a worse performance) is to rely completely on locks.
You can change ConcurrentHashMap to a regular HashMap.
Then you have to apply all your changes directly in the function retrieve:
fun retrieveA(key: String, mod: (A) -> Unit): A {
synchronized(lock) {
val obj: A = cache.getOrPut(key) { A(key, 1) }
mod(obj)
cache.put(obj)
return obj
}
}
I hope it compiles (I'm not an expert on Kotlin).
Then you use it like:
class CacheChanger {
fun incrementData() {
flusher.retrieveA("x") { it.changeA() }
}
}
Ok, I admit this code is not really Kotlin ;) you should use a Kotlin lambda instead of the Consumer interface. It's been some time since I played a bit with Kotlin. If someone could fix it I'd be very grateful.

How do I memoize the value emitted by a Single?

Say I have an expensive calculation that creates an object. I want to give the caller some flexibility as to where that happens, with subscribeOn(). But I also don't want to make that calculation more than once, because of side effects (e.g. the object is backed by some external data store).
I can write
MyObject myObject = MyObject.createExpensively(params);
return Single.just(myObject);
but this does the expensive work on the calling thread.
I can write
Callable<MyObject> callable = () -> MyObject.createExpensively(params);
return Single.fromCallable(callable);
but this will invoke createExpensively() (with side effects) once per subscription, which isn't what I want if there are multiple subscribers.
If I want to ensure that createExpensively() is only called once, and its side effects only occur once, what's the pattern I'm looking for here?

You could use Single.cache():
Single.fromCallable(() -> MyObject.createExpensively(params)).cache();

Single.fromCallable(() -> MyObject.createExpensively(params)).cache();
cache() -> Stores the success value or exception from the current Single and replays it to late SingleObservers. Please have a look here for more info.

RxJava emit items manually

I'm a newbie in RxJava world. I started few days ago and I'm trying to do something very concrete.
I would like to use RxJava to communicate some events from one class to other classes. My emitter class do some work and when finish I want notify that event to every subscribed class.
Also, I would like to don't loose any event, so if emitter finish an action and notifies to subscribers but no one consumes that event it should keep it.
I have been looking Subjects because i read was the good approach for emit by demand but I can't find one that solves my problem, may be I could try PublishSubject in the way they say in that link:
Note that a PublishSubject may begin emitting items immediately upon creation (unless you have taken steps to prevent this), and so there is a risk that one or more items may be lost between the time the Subject is created and the observer subscribes to it. If you need to guarantee delivery of all items from the source Observable, you’ll need either to form that Observable with Create so that you can manually reintroduce “cold” Observable behavior (checking to see that all observers have subscribed before beginning to emit items), or switch to using a ReplaySubject instead.
At the moment I have an Observable that emits the complete list of events to every new Subscription:
protected List<T> commands = new ArrayList<>();
Observable<T> observe = Observable.defer(() -> Observable.fromIterable(commands));
Disposable subscribe(Consumer<T> onNext, Consumer<Throwable> onError) {
return observe.subscribe(onNext/*, onError*/);
}
But this is far away from what I want. As I said I'm pretty noob with Rx, if I can't find a way I will end up using ReplaySubject (The complete list of commands would be repeated on every subscription but at least I will never lose one event).
EDIT:
Finally I split my problem in two parts. For emit items manually I use PublishSubject
PublishSubject<T> actionSubject = PublishSubject.create();
actionSubject.subscribe(onceAction);
actionSubject.onNext(action);

You are probably looking for a BehaviorSubject which replays last value emitted for every new subscriber.
Anyhow, you can create ReplaySubject with the factory method called createWithSize(capacity) to define size-bounded buffer, which discards the oldest item on overflow. There is also a method to create time-bounded buffer with similar behavior.

Isnt it what are you looking for?
Observable<Integer> observable = Observable.create((ObservableEmitter<Integer> e) -> {
for (int i = 0; i < 10; i++) {
e.onNext(i);
}
e.onComplete();
});
System.out.println("FIRST");
observable
.doOnComplete(() -> System.out.println("FIRST COMPLETE"))
.subscribe(System.out::println);
System.out.println("SECOND");
observable
.doOnComplete(() -> System.out.println("SECOND COMPLETE"))
.subscribe(System.out::println);
Output:
FIRST
0
1
2
3
4
5
6
7
8
9
FIRST COMPLETE
SECOND
0
1
2
3
4
5
6
7
8
9
SECOND COMPLETE

Get the latest value of an Observable and emit it immeditely

I'm trying to get the latest value of a given Observable and get it to emit
immediately once it's called. Given the code below as an example:
return Observable.just(myObservable.last())
.flatMap(myObservable1 -> {
return myObservable1;
})
.map(o -> o.x) // Here I want to end up with a T object instead of Observable<T> object
This does not work because by doing this the flatMap will emit myObservable1 which in turn will have
to emit to reach the map.
I don't know if doing such a thing is even possible. Does anyone have any clue on how to achieve this goal? Thank you

last() method will not be of any help here as it waits for the Observable to terminate to give you the last item emitted.
Assuming that you do not have the control over the emitting observable you could simply create a BehaviorSubject and subscribe it to the observable that emits the data that you want to listen and then subscribe to the created subject. Since Subject is both Observable and Subscriber you will get what you want.
I think (do not have the time to check it now) you may have to manually unsubscribe from the original observable as the BehaviorSubject once all of his subscribers unsubscribe will not unsubscribe automatically.
Something like this:
BehaviorSubject subject = new BehaviorSubject();
hotObservable.subscribe(subject);
subject.subscribe(thing -> {
// Here just after subscribing
// you will receive the last emitted item, if there was any.
// You can also always supply the first item to the behavior subject
});
http://reactivex.io/RxJava/javadoc/rx/subjects/BehaviorSubject.html

In RxJava, subscriber.onXXX is called asynchronous.It means that if your Observable emit items in new thread, you can never get the last item before return, except you block the thread and wait for the item.But if the Observable emit item synchronously and you dont' change it's thread by subscribeOn and observOn,
such as the code:
Observable.just(1,2,3).subscribe();
In this case, you can get the last item by doing like this:
Integer getLast(Observable<Integer> o){
final int[] ret = new int[1];
Observable.last().subscribe(i -> ret[0] = i);
return ret[0];
}
It's a bad idea doing like this.RxJava prefer you to do asynchronous work by it.

What you actually want to achieve here is to take an asynchronous task and transform it to a synchronous one.
There are several ways to achieve it, each one with it's pros and cons:
Use toBlocking() - it means that this thread will be BLOCKED, until the stream is finish, in order to get only one item simply use first() as it will complete once an item is delivered.
let's say your entire stream is Observable<T> getData();
then a method that will get the last value immediately will look like this:
public T getLastItem(){
return getData().toBlocking().first();
}
please don't use last() as it will wait for the stream to complete and only then will emit the last item.
If your stream is a network request and it didn't get any item yet this will block your thread!, so only use it when you are sure that there is an item available immediately (or if you really want a block...)
another option is to simply cache the last result, something like this:
getData().subscribe(t-> cachedT = t;) //somewhere in the code and it will keep saving the last item delivered
public T getLastItem(){
return cachedT;
}
if there wasn't any item sent by the time you request it you will get null or whatever initial value you have set.
the problem with this approch is that the subscribe phase might happen after the get and might make a race condition if used in 2 different threads.

Supply value to Observable

I am new to rxjava and I have the following problem:
Objects are irregularly dropped into a FIFO queue by an outside system. I need an Observable which runs every second, takes an item from the queue (if there is one) and emits it to subscribers.
Two problems:
The queue items are produced while the Observable is alive, it is not possible to provide all items upfront. The queue may run empty, in which case the Observable must standby and not emit anything. (It would be nice if the Observable would jump-start immediately when an item becomes available in the queue after a pause, but then the queue would probably need to be an Observable as well if we don't want to poll more frequently, no idea how.)
It must be possible for the outside system to complete the Observable. I could set a variable and read it from within the Observable, but I'd like to know if there is a more elegant way to do that.
LinkedList<Layer> queue = new LinkedList<Layer>(); // the queue
boolean stopObservable = false; // the variable to stop the observable
Observable.create(new Observable.OnSubscribe<Layer>() {
#Override public void call(Subscriber<? super Layer> subscriber) {
try {
if (!queue.isEmpty()) {
Layer layer = queue.poll();
subscriber.onNext(layer);
} else {
if (stopObservable) { subscriber.onCompleted(); }
}
} catch (Exception e) {
subscriber.onError(e);
}
}
}).somethingThatCreatesTheInterval().subscribeOnEtc.
For the interval, I cannot use .sample(), because it drops items, and it is important that all items are emitted.
.throttleWithTimeout() looks better, but it also seems to drop items.
rx is very cool, but tough to get into. Any input appreciated.

I did something similar when I needed to poll external web services at a regular time interval.
For the time interval you could proceed with a timer ; upon each tick with a granularity of 1s the observable chain will poll and maybe pick one layer, if that layer is null then nothing is emitted
Observable.timer(0, 1, TimeUnit.SECOND)
.flatMap(tick -> Observable.just(queue.poll()).filter(layer -> layer != null))
.subscribe(layer -> System.out.format("The layer is : %s", layer));
Now if you want to abort the whole the chain you may add takeUntil. So when your external system want to stop it will submit something in stopObservable which will stop subsequent subscription :
// somewhere before
PublishSubject stopNotifier = PublishSubject.create();
// somewhere process the queue
Observable.timer(0, 1, TimeUnit.SECOND)
.takeUntil(stopNotifier)
.flatMap(tick -> Observable.just(queue.poll()))
.subscribe(layer -> System.out.format("The layer is : %s", layer));
// when not anymore interested (calling onComplete works too)
stopNotifier.onNext("cancel everything about the queue");
I'm writing this response from a tablet so you may assume I may have misspell some words or made naive programming errors ;)

If possible, you should use a PublishSubject<Layer> instead of a LinkedList<Layer>. Then, the outside system can provide new items by calling publishSubject.onNext, and since PublishSubject is a subclass of Observable, your system can treat it as an Observable, and, depending on what semantics with regard to timing you want, apply one of these operators to it:
sample
debounce
throttleFirst/throttleLast/throttleWithTimeout
.zipWith(Observable.timer(1, TimeUnit.SECONDS), (value, tick) -> value) (might do a lot of buffering!)
no timing modification at all (consider this as well)