Develop Reference

Advices on concurrency JAVA

I am kind of new at concurrency (I believe i am doing good with Monitors, Semaphores and reentrantLocks) so i can't use them.
I have to use any class or interface from concurrent package.
Point 1.- Few threads must access an array and make a sumatory of its elements (I used Atomic Variables here, there is an atomic variable used as a pointer for the array).
Point 2.- Here, a number of Threads (cars), in a shore have, to cross a river. There is a Thread that simulates a boat. When 10 cars are in the boat it sails to the other shore. This secuance is repeated until all cars crosses the river. I don't really know which ones to use here.
Point 3.- Here, some Threads have to read some information and other Threads have to modify that information (any number of times). I believe here I have to use ReadWriteLock.
Point 4.- A producer / consumer like problem. Here I used BlockingQueue.
Point 5.- Made up an exchanger like problem and solve it (done, really simple one... 2 threads exchanging a String variable).
If you have any recomendation on which method to use in a certain points (like "No, use atomic variables in point 3 and cyclicBarrier in point 1"), will help me a lot!
Thanks!

Atomic Variables
Atomic Variables are proceed with CAS algorithm and they have can be consumed by multiple threads safely :
CAS :
A typical CAS operation works on three operands:
The memory location on which to operate (M) The existing expected
value (A) of the variable The new value (B) which needs to be set The
CAS operation updates atomically the value in M to B, but only if the
existing value in M matches A, otherwise no action is taken.
You can use Exchanger which could be used to swap , exchange some
information between two threads
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Exchanger.html
Blocking queue is consumer producer model a you mentioned , So for
using it you need to create two threads one will be Producer it will
put to the Blocking queue , consumer will consume (read) from it .
There are many implementations of BlockingQueue
Here is more
detailed information:
http://tutorials.jenkov.com/java-util-concurrent/blockingqueue.html
So from you list is all on their places.
Also you can reed something here about java.util.concurrent
I think this will be helpful :
http://tutorials.jenkov.com/java-util-concurrent/index.html

Point 2: if the thread is a boat, it should take cars on one bank of the river and unload them on the other bank. Looks like most natural representation of cars on a bank is a BlockingQueue and ArrayList for cars on the boat. Generally, always consider BlockingQueue first.
Point 3: this is usually done by synchronized methods, or ReentrantLocks.

Parallel lock-free ascending id generation

I have a map which should associate Strings with an id. There must not be gaps between ids and they must be unique Integers from 0 to N.
Request always comes with two Strings of which one, both or none may have been already indexed.
The map is built in parallel from the ForkJoin pool and ideally i would like to avoid explicit synchronized blocks. I am looking for an optimal way to maximize throughput with or without locking.
I don't see how to use AtomicInteger without creating gaps in sequence for the keys which were already present in the map.
public class Foo {
private final Map<String, Integer> idGenerator = new ConcurrentHashMap<>();
// invoked from multiple threads
public void update(String key1, String key2) {
idGenerator.dosomething(key, ?) // should save the key and unique id
idGenerator.dosomething(key2, ?) // should save the key2 and its unique id
Bar bar = new Bar(idGenerator.get(key), idGenerator.get(key2));
// ... do something with bar
}
}
I think that size() method combined with merge() might solve the problem but i cannot quite convince myself of that. Could anyone suggest an approach for this problem?
EDIT
Regarding duplicate flag, this cannot be solved with AtomicInteger.incrementAndGet() as suggested in the linked answer. If i did this blindly for every String there would be gaps in sequences. There is a need for compound operation which checks if the key exists and only then generates id.
I was looking for a way to implement such compound operation via Map API.
The second provided answer goes against requirements i have specifically laid out in the question.

There is not a way to do it exactly the way you want it -- ConcurrentHashMap is not in and of itself, lock-free. However, you can do it atomically without having to do any explicit lock management by using the java.util.Map.computeIfAbsent function.
Here's a code sample in the style of what you provided that should get you going.
ConcurrentHashMap<String, Integer> keyMap = new ConcurrentHashMap<>();
AtomicInteger sequence = new AtomicInteger();
public void update(String key1, String key2) {
Integer id1 = keyMap.computeIfAbsent(key1, s -> sequence.getAndIncrement());
Integer id2 = keyMap.computeIfAbsent(key2, s -> sequence.getAndIncrement());
Bar bar = new Bar(id1, id2);
// ... do something with bar
}

I'm not sure you can do exactly what you want. You can batch some updates, though, or do the checking separately from the enumerating / adding.
A lot of this answer is assuming that order isn't important: you need all the strings given a number, but reordering even within a pair is ok, right? Concurrency could already cause reordering of pairs, or for members of a pair not to get contiguous numbers, but reordering could lead to the first of a pair getting a higher number.
latency is not that important. This application should chew large amount of data and eventually produce output. Most of the time there should be a search hit in a map
If most searches hit, then we mostly need read throughput on the map.
A single writer thread might be sufficient.
So instead of adding directly to the main map, concurrent readers can check their inputs, and if not present, add them to a queue to be enumerated and added to the main ConcurrentHashMap. The queue could be a simple lockless queue, or could be another ConCurrentHashMap to also filter duplicates out of not-yet-added candidates. But probably a lockless queue is good.
Then you don't need an atomic counter, or have any problems with 2 threads incrementing the counter twice when they see the same string before either of them can add it to the map. (Because otherwise that's a big problem.)
If there's a way for a writer to lock the ConcurrentHashMap to make a batch of updates more efficient, that could be good. But if the hit rate is expected to be quite high, you really want other reader threads to keep filtering duplicates as much as possible while we're growing it instead of pausing that.
To reduce contention between the main front-end threads, you could have multiple queues, like maybe each thread has a single-producer / single-consumer queue, or a group of 4 threads running on a pair of physical cores shares one queue.
The enumerating thread reads from all of them.
In a queue where readers don't contend with writers, the enumerating thread has no contention. But multiple queues reduce contention between writers. (The threads writing these queues are the threads that access the main ConcurrentHashMap read-only, where most CPU time will be spent if hit-rates are high.)
Some kind of read-copy-update (RCU) data structure might be good, if Java has that. It would let readers keep filtering out duplicates at full speed, while the enumerating thread constructs a new table with with a batch of insertions done, with zero contention while it's building the new table.
With a 90% hit rate, one writer thread could maybe keep up with 10 or so reader threads that filter new keys against the main table.
You might want to set some queue-size limit to allow for back-pressure from the single writer thread. Or if you have many more cores / threads than a single writer can keep up with, when maybe some kind of concurrent set to let the multiple threads eliminate duplicates before numbering is helpful.
Or really, if you can just wait until the end to number everything, that would be a lot simpler, I think.
I thought about maybe trying to number with room for error on race conditions, and then going back to fix things up, but that probably isn't better.

Design AppServer Interview Discussion

I encountered the following question in a recent System Design Interview:
Design an AppServer that interfaces with a Cache and a DB.
I came up with this:
public class AppServer{
public Database DB;
public Cache cache;
public Value get(Key k){
Value res = cache.get(k);
if(res == null){
res = DB.get(k);
cache.set(k, res);
}
}
public void set(Key k, Value v){
cache.set(k, v);
DB.set(k, v);
}
}
This code is fine and works correctly, but follow ups to the question are:
What if there are multiple threads?
What if there are multiple instances of the AppServer?
Suddenly AppServer performance degrades a ton, we find out this is because our cache is consistently missing. Cache size is fixed (already largest that it can be). How can we prevent this?
Response:
I answered that we can use Locks or Conditional Variables. In Java, we can add Synchronized to each method to allow for mutual exclusion, but the interviewer mentioned that this isn't too efficient and wanted only critical parts synchronized.
I thought that we only need to synchronize the 2 set lines in void set(Key k, Value v) and 1 set method in Value get(Key k), however the interviewer pushed for also synchronizing res = DB.get(k);. I agreed with him at the end, but don't fully understand. Don't threads have independent stacks and shared heaps? So when a thread executes get, it stores res in local variable on stack frame, even if another thread executes get sequentially, the former thread retains its get value. Then each thread sets their respective fetched values.
How can we handle multiple instances of the AppServer?
I came up with a Distributed Queue Solution like Kafka, every time we perform a set / get command we queue that command, but he also mentioned that set is ok because the action sets a value in the cache / db, but how would you return the correct value for get? Can someone explain this?
Also there are possible solutions with a versioning system and event system?
Possible solutions:
L1, L2, L3 caches - layers and more caches
Regional / Segmentation caches - use different cache for user groups.
Any other ideas?
Will upvote all insightful responses :)

1
Although JDBC is "supposed" to be thread safe, some drivers aren't and I'm going to assume that Cache isn't thread safe either (although most caches should be thread safe) so in that case, you would need to make the following changes to your code:
Make both fields final
Synchronize the ENTIRE get(...)method
Synchronize the ENTIRE set(...)method
Assuming there is no other way to access the said fields, the behavior of your get(...) method depends on 2 things: first, that updates from the set(...) method can be seen, and secondly, that a cache miss is then stored only by a single thread. You need to synchronize because the idea is to only have one thread perform an expensive DB query in the case that there is a cache miss. If you do not synchronize the entire get(...) method, or you split the synchronized statement, it is possible for another thread to also see a cache miss between the lookup and insertion.
The way I would answer this question is honestly just to toss the entire thing. I would look at how JCIP wrote the cache and base my answer on that.
2
I think your queue solution is fine.
I believe your interviewer means that if another instance of AppServer did not have cached what was already set(...) by another instance of AppServer, then it would lookup and find the correct value in the DB. This solution would be incorrect if you are using multiple threads because it is possible for 2 threads to be set(...)ing conflicting values, then the caches would have 2 different values while depending on the thread safety of your DB, it might not even have the value at all.
Ideally, you'd never create more than a single instance of your AppServer.
3
I don't have enough experience to evaluate this question specifically, but perhaps an LRU cache would improve performance somewhat, or using a hash ring buffer. It might be a stretch but if you wanted to throw out there, perhaps even using ML to determine the best values to either preload to retain at certain times of the day, for example, could also work.
If you are always missing values from your cache, there is no way to improve your code. Performance would be dependent on your database.

Writing with a single thread LMAX

I've got introduced to LMAX and this wonderful concept called RingBuffer.
So guys tell that when writing to the ringbuffer with only one thread performance is way better than with multiple producers...
However i dont really find it possible for tipical application to use only one thread for writes on ringbuffer... i dont really understand how lmax is doing that (if they do). For example N number of different traders put orders on exchange, those are all asynchronious requests that are getting transformed to orders and put into ringbuffer, how can they possibly write those using one thread?
Question 1. I might missing something or misunderstanding some aspect, but if you have N concurrent producers how is it possible to merge them into 1 and not lock each other?
Question 2. I recall rxJava observables, where you could take N observables and merge them into 1 using Observable.merge i wonder if it is blocking or maintaining any lock in any way?

The impact on a RingBuffer of multi-treaded writing is slight but under very heavy loads can be significant.
A RingBuffer implementation holds a next node where the next addition will be made. If only one thread is writing to the ring the process will always complete in the minimum time, i.e. buffer[head++] = newData.
To handle multi-threading while avoiding locks you would generally do something like while ( !buffer[head++].compareAndSet(null,newValue)){}. This tight loop would continue to execute while other threads were interfering with the storing of the data, thus slowing town the throughput.
Note that I have used pseudo-code above, have a look at getFree in my implementation here for a real example.
// Find the next free element and mark it not free.
private Node<T> getFree() {
Node<T> freeNode = head.get();
int skipped = 0;
// Stop when we hit the end of the list
// ... or we successfully transit a node from free to not-free.
// This is the loop that could cause delays under hight thread activity.
while (skipped < capacity && !freeNode.free.compareAndSet(true, false)) {
skipped += 1;
freeNode = freeNode.next;
}
// ...
}

Internally, RxJava's merge uses a serialization construct I call emitter-loop which uses synchronized and is blocking.
Our 'clients' use merge mostly in throughput- and latency insensitive cases or completely single-threaded and blocking isn't really an issue there.
It is possible to write a non-blocking serializer I call queue-drain but merge can't be configured to use that instead.
You can also take a look at JCTools' MpscArrayQueue directly if you are willing to handle the producer and consumer threads manually.

Separate hashset to run the list at several threads

I googled and search here for this question and did not find anything similar to what I´m looking for.
I populated a HashSet with few objects called Person, I need to set four or five threads to search these Person in a huge text, thread seems to be the best solution for the better usage from the hardware.
The doubt is, how can I separate this HashSet and start 4 threads? I tried to create a new HashSet list and start a new thread with this new hashset divided in 4.
It seems to be a good solution but, is there a better way to do it? How can I separate the hashset and send at pieces to 4 or 5 new threads?

Access to a HashSet is O(1) so if you split it across multiple threads, it won't go any faster. You are better off attempting to split the file of searching is expensive. However if its efficient enough, one thread will be optimal.
It is worth remembering that using all the cores on your machine can mean your program is slower. If you just want to use up all the CPU on you machine, you can create a thread pool which does nothing but use up all the CPU on your machine.

You can implement a producer-consumer scheme: have a single thread read the values from the hash set one by one and put them in a queue which is then processesed by several worker threads. You can use the ExecutorService class to manage the workers.
Edit: Here's what you can do:
Define your worker class:
public class Worker implements Runnable {
private Person p;
public Worker(Person p) {
this.p = p;
}
public void run() {
// search for p
}
}
In the main thread:
ExecutorService s = Executors.newCachedThreadPool();
for(Person p: hashSet) {
s.submit(new Worker(p));
}

A couple of things to consider:
1) You could use the same HashSet, but you will need to synchronize it (wrap the calls to it with a synchronized block. But if all you are doing is looking up things in the hash, being multi-threaded will not buy you much.
2) If you want to split the HashSet, then you can consider a split on key ranges. So for example if you are searching for a name, names that start with A-F go in HashSet1, G-L HashSet2, etc. This way your searches can be completely parallel.

You cane iterate through the hash set using Iterator. & while iterating fetch the value and create a thread and fire it.
Else
you can use ExecutorService API where simultaneous tasks can be run in parallel.