Develop Reference

How to fix race conditions without using synchronized (Lock free sequence counter implementation)?

Have a scenario where multiple threads have race condition on comparison code.
private int volatile maxValue;
private AtomicInteger currentValue;
public void constructor() {
this.current = new AtomicInteger(getNewValue());
}
public getNextValue() {
while(true) {
int latestValue = this.currentValue.get();
int nextValue = latestValue + 1;
if(latestValue == maxValue) {//Race condition 1
latestValue = getNewValue();
}
if(currentValue.compareAndSet(latestValue, nextValue) {//Race condition 2
return latestValue;
}
}
}
private int getNewValue() {
int newValue = getFromDb(); //not idempotent
maxValue = newValue + 10;
return newValue;
}
Questions :
The obvious way to fix this would be add synchronized block/method around the if condition. What are other performant way to fix this using concurrent api without using any kind of locks ?
How to get rid of the while loop so we can get the next value with no or less thread contention ?
Constraints :
The next db sequences will be in increasing order not necessarily evenly distributed. So it could be 1, 11, 31 where 21 may be have asked by other node. The requested next value will always be unique. Also need to make sure all the sequences are used and once we reach the max for previous range then only request to db for another starting sequence and so on.
Example :
for db next sequences 1,11,31 with 10 increment, the output next sequence should be 1-10, 11-20, 31-40 for 30 requests.

First of all: I would recommend thinking one more time about using synchronized, because:
look at how simple such code is:
private int maxValue;
private int currentValue;
public constructor() {
requestNextValue();
}
public synchronized int getNextValue() {
currentValue += 1;
if (currentValue == maxValue) {
requestNextValue();
}
return currentValue;
}
private void requestNextValue() {
currentValue = getFromDb(); //not idempotent
maxValue = currentValue + 10;
}
locks in java actually are pretty intelligent and have pretty good performance.
you talk to DB in your code — the performance cost of that alone can be orders of magnitude higher than the performance cost of locks.
But in general, your race conditions happen because you update maxValue and currentValue independently.
You can combine these 2 values into a single immutable object and then work with the object atomically:
private final AtomicReference<State> stateHolder = new AtomicReference<>(newStateFromDb());
public int getNextValue() {
while (true) {
State oldState = stateHolder.get();
State newState = (oldState.currentValue == oldState.maxValue)
? newStateFromDb()
: new State(oldState.currentValue + 1, oldState.maxValue);
if (stateHolder.compareAndSet(oldState, newState)) {
return newState.currentValue;
}
}
}
private static State newStateFromDb() {
int newValue = getFromDb(); // not idempotent
return new State(newValue, newValue + 10);
}
private static class State {
final int currentValue;
final int maxValue;
State(int currentValue, int maxValue) {
this.currentValue = currentValue;
this.maxValue = maxValue;
}
}
After fixing that you will probably have to solve the following problems next:
how to prevent multiple parallel getFromDb(); (especially after taking into account that the method is idempotent)
when one thread performs getFromDb();, how to prevent other threads from busy spinning inside while(true) loop and consuming all available cpu time
more similar problems
Solving each of these problems will probably make your code more and more complicated.
So, IMHO it is almost never worth it — locks work fine and keep the code simple.

You cannot completely avoid locking with the given constraints: since (1) every value returned by getFromDb() must be used and (2) calling getFromDb() is only allowed once maxValue has been reached, you need to ensure mutual exclusion for calls to getFromDb().
Without either of the constraints (1) or (2) you could resort to optimistic locking though:
Without (1) you could allow multiple threads calling getFromDb() concurrently and choose one of the results dropping all others.
Without (2) you could allow multiple threads calling getFromDb() concurrently and choose one of the results. The other results would be "saved for later".

The obvious way to fix this would be add synchronized block around the if condition
That is not going to work. Let me try and explain.
When you hit the condition: if(latestValue == maxValue) { ... }, you want to update both maxValue and currentValue atomically. Something like this:
latestValue = getNewValue();
currentValue.set(latestValue);
getNewValue will get your next starting value from the DB and update maxValue, but at the same time, you want to set currentValue to that new starting one now. Suppose the case:
you first read 1 from the DB. As such maxValue = 11, currentValue = 1.
when you reach the condition if(latestValue == maxValue), you want to go to the DB to get the new starting position (let's say 21), but at the same time you want every thread to now start from 21. So you must also set currentValue.
Now the problem is that if you write to currentValue under a synchronized block, for example:
if(latestValue == maxValue) {
synchronized (lock) {
latestValue = getNewValue();
currentValue.set(latestValue);
}
}
you also need to read under the same lock, otherwise you have race. Initially I thought I can be a bit smarter and do something like:
if(latestValue == maxValue) {
synchronized (lock) {
if(latestValue == maxValue) {
latestValue = getNewValue();
currentValue.set(latestValue);
} else {
continue;
}
}
}
So that all threads that wait on a lock do not override the previously written value to maxValue when the lock is released. But that still is a race and will cause problems elsewhere, in a different case, rather trivially. For example:
ThreadA does latestValue = getNewValue();, thus maxValue == 21. Before it does currentValue.set(latestValue);
ThreadB reads int latestValue = this.currentValue.get();, sees 11 and of course this will be false : if(latestValue == maxValue) {, so it can write 12 (nextValue) to currentValue. Which breaks the entire algorithm.
I do not see any other way then to make getNextValue synchronized or somehow else protected by a mutex/spin-lock.

I don't really see a way around synchonizing the DB call - unless calling the DB multiple times is not an issue (i.e. retrieving several "new values").
To remove the need to synchronize the getNextValue method, you could use a BlockingQueue which will remove the need to atomically update 2 variables. And if you really don't want to use the synchronize keyword, you can use a flag to only let one thread call the DB.
It could look like this (looks ok, but not tested):
private final BlockingQueue<Integer> nextValues = new ArrayBlockingQueue<>(10);
private final AtomicBoolean updating = new AtomicBoolean();
public int getNextValue() {
while (true) {
Integer nextValue = nextValues.poll();
if (nextValue != null) return nextValue;
else getNewValues();
}
}
private void getNewValues() {
if (updating.compareAndSet(false, true)) {
//we hold the "lock" to run the update
if (!nextValues.isEmpty()) {
updating.set(false);
throw new IllegalStateException("nextValues should be empty here");
}
try {
int newValue = getFromDb(); //not idempotent
for (int i = 0; i < 10; i++) {
nextValues.add(newValue + i);
}
} finally {
updating.set(false);
}
}
}
But as mentioned in other comments, there is a high chance that the most costly operation here is the DB call, which remains synchronized, so you may as well synchronize everything and keep it simple, with very little difference performance wise.

As getFromDb hits the database you really want some locking - the other threads should block not also go for the database or spin. Really, if you are doing that every 10 iterations, you can probably synchronize the lot. However, that is no fun.
Any reasonable, non-microcontroller platform should support AtomicLong as lock-free. So we can conveniently pack the two ints into one atomic.
private final AtomicLong combinedValue;
public getNextValue() {
for (;;) {
long combined = combinedValue.get();
int latestValue = (int)combined;
int maxValue = (int)(combined>>32);
int nextValue = latestValue + 1;
long nextCombined = (newValue&0xffffffff) | (maxValue<<32)
if (latestValue == maxValue) {
nextValue();
} else if (currentValue.compareAndSet(combined, nextCombined)) {
return latestValue;
}
}
}
private synchronized void nextValue() {
// Yup, we need to double check with this locking.
long combined = combinedValue.get();
int latestValue = (int)combined;
int maxValue = (int)(combined>>32);
if (latestValue == maxValue) {
int newValue = getFromDb(); //not idempotent
int maxValue = newValue + 10;
long nextCombined = (newValue&0xffffffff) | (maxValue<<32)
combinedValue.set(nextCombined);
}
}
An alternative with memory allocation would be to lump both values into one object and use AtomicReference. However, we can observe that the value changes more frequently than the maximum, so we can use a slow changing object and a fast offset.
private static record Segment(
int maxValue, AtomicInteger currentValue
) {
}
private volatile Segment segment;
public getNextValue() {
for (;;) {
Segment segment = this.segment;
int latestValue = segment.currentValue().get();
int nextValue = latestValue + 1;
if (latestValue == segment.maxValue()) {
nextValue();
} else if (segment.currentValue().compareAndSet(
latestValue, nextValue
)) {
return latestValue;
}
}
}
private synchronized void nextValue() {
// Yup, we need to double check with this locking.
Segment segment = this.segment;
int latestValue = segment.currentValue().get();
if (latestValue == segment.maxValue()) {
int newValue = getFromDb(); //not idempotent
int maxValue = newValue + 10;
segment = new Segment(maxValue, new AtomicInteger(newValue));
}
}
(Standard disclaimer: Code not so much as compiled, tested or thought about much. records require a quite new at time of writing JDK. Constructors elided.)

What an interesting question. As others have said you get round with your problem by using synchronized keyword.
public synchronized int getNextValue() { ... }
But because you didn't want to use that keyword and at the same time want to avoid race condition, this probably helps. No guarantee though. And please don't ask for explanations, I'll throw you with OutOfBrainException.
private volatile int maxValue;
private volatile boolean locked = false; //For clarity.
private AtomicInteger currentValue;
public int getNextValue() {
int latestValue = this.currentValue.get();
int nextValue = latestValue + 1;
if(!locked && latestValue == maxValue) {
locked = true; //Only one thread per time.
latestValue = getNewValue();
currentValue.set(latestValue);
locked = false;
}
while(locked) { latestValue = 0; } //If a thread running in the previous if statement, we need this to buy some time.
//We also need to reset "latestValue" so that when this thread runs the next loop,
//it will guarantee to call AtomicInteger.get() for the updated value.
while(!currentValue.compareAndSet(latestValue, nextValue)) {
latestValue = this.currentValue.get();
nextValue = latestValue + 1;
}
return nextValue;
}
Or you can use Atomic to fight Atomic.
private AtomicBoolean locked = new AtomicBoolean(false);
public int getNextValue() {
...
if(locked.compareAndSet(false, true)) { //Only one thread per time.
if(latestValue == maxValue) {
latestValue = getNewValue();
currentValue.set(latestValue);
}
locked.set(false);
}
...

I can't think of a way to remove all locking since the underlying problem is accessing a mutable value from several threads. However there several improvements that can be done to the code you provided, basically taking advantage of the fact that when data is read by multiple threads, there is no need to lock the reads unless a write has to be done, so using Read/Write locks will reduce the contention. Only 1/10 times there will be a "full" write lock
So the code could be rewritten like this (leaving bugs aside):
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.ReentrantReadWriteLock;
public class Counter {
private final ReentrantReadWriteLock reentrantLock = new ReentrantReadWriteLock(true);
private final ReentrantReadWriteLock.ReadLock readLock = reentrantLock.readLock();
private final ReentrantReadWriteLock.WriteLock writeLock = reentrantLock.writeLock();
private AtomicInteger currentValue;
private AtomicInteger maxValue;
public Counter() {
int initialValue = getFromDb();
this.currentValue = new AtomicInteger(initialValue);
this.maxValue = new AtomicInteger(initialValue + 10);
}
public int getNextValue() {
readLock.lock();
while (true){
int nextValue = currentValue.getAndIncrement();
if(nextValue<maxValue.get()){
readLock.unlock();
return nextValue;
}
else {
readLock.unlock();
writeLock.lock();
reload();
readLock.lock();
writeLock.unlock();
}
}
}
private void reload(){
int newValue = getFromDb();
if(newValue>maxValue.get()) {
this.currentValue.set(newValue);
this.maxValue.set(newValue + 10);
}
}
private int getFromDb(){
// your implementation
}
}

What is the business use case you are trying to solve?
Can the next scenario work for you:
Create SQL sequence (based your database) with counter requirements in the database;
Fetch counters from the database as a batch like 50-100 ids
Once 50-100 are used on the app level, fetch 100 values more from db ...
?

Slightly modified version of user15102975's answer with no while-loop and getFromDb() mock impl.
/**
* Lock free sequence counter implementation
*/
public class LockFreeSequenceCounter {
private static final int BATCH_SIZE = 10;
private final AtomicReference<Sequence> currentSequence;
private final ConcurrentLinkedQueue<Integer> databaseSequenceQueue;
public LockFreeSequenceCounter() {
this.currentSequence = new AtomicReference<>(new Sequence(0,0));
this.databaseSequenceQueue = new ConcurrentLinkedQueue<>();
}
/**
* Get next unique id (threadsafe)
*/
public int getNextValue() {
return currentSequence.updateAndGet((old) -> old.next(this)).currentValue;
}
/**
* Immutable class to handle current and max value
*/
private static final class Sequence {
private final int currentValue;
private final int maxValue;
public Sequence(int currentValue, int maxValue) {
this.currentValue = currentValue;
this.maxValue = maxValue;
}
public Sequence next(LockFreeSequenceCounter counter){
return isMaxReached() ? fetchDB(counter) : inc();
}
private boolean isMaxReached(){
return currentValue == maxValue;
}
private Sequence inc(){
return new Sequence(this.currentValue + 1, this.maxValue);
}
private Sequence fetchDB(LockFreeSequenceCounter counter){
counter.databaseSequenceQueue.add(counter.getFromDb());
int newValue = counter.databaseSequenceQueue.poll();
int maxValue = newValue + BATCH_SIZE -1;
return new Sequence(newValue, maxValue);
}
}
/**
* Get unique id from db (mocked)
* return on call #1: 1
* return on call #2: 11
* return on call #3: 31
* Note: this function is not idempotent
*/
private int getFromDb() {
if (dbSequencer.get() == 21){
return dbSequencer.addAndGet(BATCH_SIZE);
} else{
return dbSequencer.getAndAdd(BATCH_SIZE);
}
}
private final AtomicInteger dbSequencer = new AtomicInteger(1);
}

Slightly modified version of Tom Hawtin - tackline's answer and also the suggestion by codeflush.dev in the comments of the question
Code
I have added a working version of code and simulated a basic multithreaded environment.
Disclaimer: Use with your own discretion
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Random;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
class Seed {
private static final int MSB = 32;
private final int start;
private final int end;
private final long window;
public Seed(int start, int end) {
this.start = start;
this.end = end;
this.window = (((long) end) << MSB) | start;
}
public Seed(long window) {
this.start = (int) window;
this.end = (int) (window >> MSB);
this.window = window;
}
public int getStart() {
return start;
}
public int getEnd() {
return end;
}
public long getWindow() {
return window;
}
// this will not update the state, will only return the computed value
public long computeNextInWindow() {
return window + 1;
}
}
// a mock external seed service to abstract the seed generation and window logic
class SeedService {
private static final int SEED_INIT = 1;
private static final AtomicInteger SEED = new AtomicInteger(SEED_INIT);
private static final int SEQ_LENGTH = 10;
private static final int JITTER_FACTOR = 5;
private final boolean canAddRandomJitterToSeed;
private final Random random;
public SeedService(boolean canJitterSeed) {
this.canAddRandomJitterToSeed = canJitterSeed;
this.random = new Random();
}
public int getSeqLengthForTest() {
return SEQ_LENGTH;
}
public Seed getDefaultWindow() {
return new Seed(1, 1);
}
public Seed getNextWindow() {
int offset = SEQ_LENGTH;
// trying to simulate multiple machines with interleaved start seed
if (canAddRandomJitterToSeed) {
offset += random.nextInt(JITTER_FACTOR) * SEQ_LENGTH;
}
final int start = SEED.getAndAdd(offset);
return new Seed(start, start + SEQ_LENGTH);
}
// helper to validate generated ids
public boolean validate(List<Integer> ids) {
Collections.sort(ids);
// unique check
if (ids.size() != new HashSet<>(ids).size()) {
return false;
}
for (int startIndex = 0; startIndex < ids.size(); startIndex += SEQ_LENGTH) {
if (!checkSequence(ids, startIndex)) {
return false;
}
}
return true;
}
// checks a sequence
// relies on 'main' methods usage of SEQ_LENGTH
protected boolean checkSequence(List<Integer> ids, int startIndex) {
final int startRange = ids.get(startIndex);
return IntStream.range(startRange, startRange + SEQ_LENGTH).boxed()
.collect(Collectors.toList())
.containsAll(ids.subList(startIndex, startIndex + SEQ_LENGTH));
}
public void shutdown() {
SEED.set(SEED_INIT);
System.out.println("See you soon!!!");
}
}
class SequenceGenerator {
private final SeedService seedService;
private final AtomicLong currentWindow;
public SequenceGenerator(SeedService seedService) {
this.seedService = seedService;
// initialize currentWindow using seedService
// best to initialize to an old window so that every instance of SequenceGenerator
// will lazy load from seedService during the first getNext() call
currentWindow = new AtomicLong(seedService.getDefaultWindow().getWindow());
}
public synchronized boolean requestSeed() {
Seed seed = new Seed(currentWindow.get());
if (seed.getStart() == seed.getEnd()) {
final Seed nextSeed = seedService.getNextWindow();
currentWindow.set(nextSeed.getWindow());
return true;
}
return false;
}
public int getNext() {
while (true) {
// get current window
Seed seed = new Seed(currentWindow.get());
// exhausted and need to seed again
if (seed.getStart() == seed.getEnd()) {
// this will loop at least one more time to return value
requestSeed();
} else if (currentWindow.compareAndSet(seed.getWindow(), seed.computeNextInWindow())) {
// successfully incremented value for next call. so return current value
return seed.getStart();
}
}
}
}
public class SequenceGeneratorTest {
public static void test(boolean canJitterSeed) throws Exception {
// just some random multithreaded invocation
final int EXECUTOR_THREAD_COUNT = 10;
final Random random = new Random();
final int INSTANCES = 500;
final SeedService seedService = new SeedService(canJitterSeed);
final int randomRps = 500;
final int seqLength = seedService.getSeqLengthForTest();
ExecutorService executorService = Executors.newFixedThreadPool(EXECUTOR_THREAD_COUNT);
Callable<List<Integer>> callable = () -> {
final SequenceGenerator generator = new SequenceGenerator(seedService);
int rps = (1 + random.nextInt(randomRps)) * seqLength;
return IntStream.range(0, rps).parallel().mapToObj(i -> generator.getNext())
.collect(Collectors.toList());
};
List<Future<List<Integer>>> futures = IntStream.range(0, INSTANCES).parallel()
.mapToObj(i -> executorService.submit(callable))
.collect(Collectors.toList());
List<Integer> ids = new ArrayList<>();
for (Future<List<Integer>> f : futures) {
ids.addAll(f.get());
}
executorService.shutdown();
// validate generated ids for correctness
if (!seedService.validate(ids)) {
throw new IllegalStateException();
}
seedService.shutdown();
// summary
System.out.println("count: " + ids.size() + ", unique count: " + new HashSet<>(ids).size());
Collections.sort(ids);
System.out.println("min id: " + ids.get(0) + ", max id: " + ids.get(ids.size() - 1));
}
public static void main(String[] args) throws Exception {
test(true);
System.out.println("Note: ids can be interleaved. if continuous sequence is needed, initialize SeedService with canJitterSeed=false");
final String ruler = Collections.nCopies( 50, "-" ).stream().collect( Collectors.joining());
System.out.println(ruler);
test(false);
System.out.println("Thank you!!!");
System.out.println(ruler);
}
}

Thread Pool per key in Java

Suppose that you have a grid G of n x m cells, where n and m are huge.
Further, suppose that we have numerous tasks, where each task belong to a single cell in G, and should be executed in parallel (in a thread pool or other resource pool).
However, task belonging to the same cell must be done serially, that is, it should wait that previous task in the same cell to be done.
How can I solve this issue?
I've search and used several thread pools (Executors, Thread), but no luck.
Minimum Working Example
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class MWE {
public static void main(String[] args) {
ExecutorService threadPool = Executors.newFixedThreadPool(16);
Random r = new Random();
for (int i = 0; i < 10000; i++) {
int nx = r.nextInt(10);
int ny = r.nextInt(10);
Runnable task = new Runnable() {
public void run() {
try {
System.out.println("Task is running");
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
threadPool.submit(new Thread(task)); // Should use nx,ny here somehow
}
}
}

You can create a list of n Executors.newFixedThreadPool(1).
Then submit to the corresponding thread by using a hash function.
Ex. threadPool[key%n].submit(new Thread(task)).

A callback mechanism with a synchronized block could work efficiently here.
I have previously answered a similar question here.
There are some limitations (see the linked answer), but it is simple enough to keep track of what is going on (good maintainability).
I have adapted the source code and made it more efficient for your case where most tasks will be executed in parallel
(since n and m are huge), but on occasion must be serial (when a task is for the same point in the grid G).
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.locks.ReentrantLock;
// Adapted from https://stackoverflow.com/a/33113200/3080094
public class GridTaskExecutor {
public static void main(String[] args) {
final int maxTasks = 10_000;
final CountDownLatch tasksDone = new CountDownLatch(maxTasks);
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(16);
try {
GridTaskExecutor gte = new GridTaskExecutor(executor);
Random r = new Random();
for (int i = 0; i < maxTasks; i++) {
final int nx = r.nextInt(10);
final int ny = r.nextInt(10);
Runnable task = new Runnable() {
public void run() {
try {
// System.out.println("Task " + nx + " / " + ny + " is running");
Thread.sleep(1);
} catch (Exception e) {
e.printStackTrace();
} finally {
tasksDone.countDown();
}
}
};
gte.addTask(task, nx, ny);
}
tasksDone.await();
System.out.println("All tasks done, task points remaining: " + gte.size());
} catch (Exception e) {
e.printStackTrace();
} finally {
executor.shutdownNow();
}
}
private final Executor executor;
private final Map<Long, List<CallbackPointTask>> tasksWaiting = new HashMap<>();
// make lock fair so that adding and removing tasks is balanced.
private final ReentrantLock lock = new ReentrantLock(true);
public GridTaskExecutor(Executor executor) {
this.executor = executor;
}
public void addTask(Runnable r, int x, int y) {
Long point = toPoint(x, y);
CallbackPointTask pr = new CallbackPointTask(point, r);
boolean runNow = false;
lock.lock();
try {
List<CallbackPointTask> pointTasks = tasksWaiting.get(point);
if (pointTasks == null) {
if (tasksWaiting.containsKey(point)) {
pointTasks = new LinkedList<CallbackPointTask>();
pointTasks.add(pr);
tasksWaiting.put(point, pointTasks);
} else {
tasksWaiting.put(point, null);
runNow = true;
}
} else {
pointTasks.add(pr);
}
} finally {
lock.unlock();
}
if (runNow) {
executor.execute(pr);
}
}
private void taskCompleted(Long point) {
lock.lock();
try {
List<CallbackPointTask> pointTasks = tasksWaiting.get(point);
if (pointTasks == null || pointTasks.isEmpty()) {
tasksWaiting.remove(point);
} else {
System.out.println(Arrays.toString(fromPoint(point)) + " executing task " + pointTasks.size());
executor.execute(pointTasks.remove(0));
}
} finally {
lock.unlock();
}
}
// for a general callback-task, see https://stackoverflow.com/a/826283/3080094
private class CallbackPointTask implements Runnable {
final Long point;
final Runnable original;
CallbackPointTask(Long point, Runnable original) {
this.point = point;
this.original = original;
}
#Override
public void run() {
try {
original.run();
} finally {
taskCompleted(point);
}
}
}
/** Amount of points with tasks. */
public int size() {
int l = 0;
lock.lock();
try {
l = tasksWaiting.size();
} finally {
lock.unlock();
}
return l;
}
// https://stackoverflow.com/a/12772968/3080094
public static long toPoint(int x, int y) {
return (((long)x) << 32) | (y & 0xffffffffL);
}
public static int[] fromPoint(long p) {
return new int[] {(int)(p >> 32), (int)p };
}
}

This is were systems like Akka in java world make sense.If both X and Y are large, you may want to look at processing them using a message passing mechanism rather than bunch them up in a huge chain of callbacks and futures. One actor has the list of tasks to be done and is handed a cell and the actor would eventually compute the result and persist it. If something fails in the intermediate step, it's not end of world.

If I get you right, you want to execute X tasks (X is very big) in Y queues (Y is much smaller than X).
Java 8 has CompletableFuture class, which represents an (asynchronous) computation. Basically, it's Java's implementation of Promise. Here is how you can organize a chain of computations (generic types omitted):
// start the queue with a "completed" task
CompletableFuture queue = CompletableFuture.completedFuture(null);
// append a first task to the queue
queue = queue.thenRunAsync(() -> System.out.println("first task running"));
// append a second task to the queue
queue = queue.thenRunAsync(() -> System.out.println("second task running"));
// ... and so on
When you use thenRunAsync(Runnable), tasks will be executed using a thread pool (there are other possibilites - see API docs). You can also supply your own thread pool as well.
You can create Y of such chains (possibly keeping references to them in some table).

This library should do the job: https://github.com/jano7/executor
int maxTasks = 16;
ExecutorService threadPool = Executors.newFixedThreadPool(maxTasks);
KeySequentialBoundedExecutor executor = new KeySequentialBoundedExecutor(maxTasks, threadPool);
Random r = new Random();
for (int i = 0; i < 10000; i++) {
int nx = r.nextInt(10);
int ny = r.nextInt(10);
Runnable task = new Runnable() {
public void run() {
try {
System.out.println("Task is running");
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
executor.execute(new KeyRunnable<>((ny * 10) + nx, task));
}

The Scala example given below demonstrates how keys in a map can be executed in parallel and values of a key are executed in serial. Change it to Java syntax if you want to try it in Java (Scala uses JVM libraries). Basically chain the tasks future to have them execute sequentially.
import java.util.concurrent.{CompletableFuture, ExecutorService, Executors, Future, TimeUnit}
import scala.collection.concurrent.TrieMap
import scala.collection.mutable.ListBuffer
import scala.util.Random
/**
* For a given Key-Value pair with tasks as values, demonstrates sequential execution of tasks
* within a key and parallel execution across keys.
*/
object AsyncThreads {
val cachedPool: ExecutorService = Executors.newCachedThreadPool
var initialData: Map[String, ListBuffer[Int]] = Map()
var processedData: TrieMap[String, ListBuffer[Int]] = TrieMap()
var runningTasks: TrieMap[String, CompletableFuture[Void]] = TrieMap()
/**
* synchronous execution across keys and values
*/
def processSync(key: String, value: Int, initialSleep: Long) = {
Thread.sleep(initialSleep)
if (key.equals("key_0")) {
println(s"${Thread.currentThread().getName} -> sleep: $initialSleep. Inserting key_0 -> $value")
}
processedData.getOrElseUpdate(key, new ListBuffer[Int]).addOne(value)
}
/**
* parallel execution across keys
*/
def processASync(key: String, value: Int, initialSleep: Long) = {
val task: Runnable = () => {
processSync(key, value, initialSleep)
}
// 1. Chain the futures for sequential execution within a key
val prevFuture = runningTasks.getOrElseUpdate(key, CompletableFuture.completedFuture(null))
runningTasks.put(key, prevFuture.thenRunAsync(task, cachedPool))
// 2. Parallel execution across keys and values
// cachedPool.submit(task)
}
def process(key: String, value: Int, initialSleep: Int): Unit = {
//processSync(key, value, initialSleep) // synchronous execution across keys and values
processASync(key, value, initialSleep) // parallel execution across keys
}
def main(args: Array[String]): Unit = {
checkDiff()
0.to(9).map(kIndex => {
var key = "key_" + kIndex
var values = ListBuffer[Int]()
initialData += (key -> values)
1.to(10).map(vIndex => {
values += kIndex * 10 + vIndex
})
})
println(s"before data:$initialData")
initialData.foreach(entry => {
entry._2.foreach(value => {
process(entry._1, value, Random.between(0, 100))
})
})
cachedPool.awaitTermination(5, TimeUnit.SECONDS)
println(s"after data:$processedData")
println("diff: " + (initialData.toSet diff processedData.toSet).toMap)
cachedPool.shutdown()
}
def checkDiff(): Unit = {
var a1: TrieMap[String, List[Int]] = new TrieMap()
a1.put("one", List(1, 2, 3, 4, 5))
a1.put("two", List(11, 12, 13, 14, 15))
var a2: TrieMap[String, List[Int]] = new TrieMap()
a2.put("one", List(2, 1, 3, 4, 5))
a2.put("two", List(11, 12, 13, 14, 15))
println("a1: " + a1)
println("a2: " + a2)
println("check.diff: " + (a1.toSet diff a2.toSet).toMap)
}
}

Threads not running concurrently

I hope this is not a repeat question, but I have looked at all the answers in other questions and none have satisfied my problem.
I have a program that has solves the Dining Philosopher's problem, and when I run the program, the threads wait until the next one is done before running another. This causes the thread's output to look like:
Philosopher 1 is EATING.
Philosopher 1 is THINKING.
Philosopher 5 is EATING.
Philosopher 5 is THINKING.
Philosopher 3 is EATING.
Philosopher 3 is THINKING.
... and so on. The expected output doesn't have an order. The threads should run concurrently. Here is my code, all of it is in here, with the interface just specifying the size of DINERS (5) and the State._______ is an enumeration with 3 states: State.HUNGRY, State.THINKING, and State.EATING.
import java.lang.Runnable;
import java.util.concurrent.locks.*;
import java.util.Random;
import java.lang.Thread;
import java.util.concurrent.TimeUnit;
/**
* This class handles the Philosophers, I hope they are hungry.
*
* #version 4-20-15
*/
public class Diner implements Runnable, PhilosopherInterface {
/** The lock used to control Thread access */
private final ReentrantLock lock;
/** The state that the Philosopher is in (ex: Eating, Thinking etc.) */
private State current;
/** The random number used to generate time sleeping */
private Random timeGenerator;
/** The maximum time a thread can sleep */
private final int maxTimeToSleep = 5000;
/** The minimum time a thread can sleep (1ms) */
private final int minTimeToSleep = 1;
private int philNum;
private int philIndex;
private Condition[] condition;
private State[] states;
public Diner(ReentrantLock lock, int philNumber, Condition[] condition, State[] states)
philNum = philNumber;
philIndex = philNum - 1;
current = states[philNumber-1];
timeGenerator = new Random();
this.lock = lock;
this.condition = condition;
this.condition[philIndex] = lock.newCondition();
this.states = states;
states[philIndex] = State.THINKING;
}
#Override
public void takeChopsticks() {
states[philIndex] = State.HUNGRY;
lock.lock();
try{
int left = philIndex-1;
int right = philIndex+1;
if(philNum == DINERS) right = 0;
if(philNum == 1) left = DINERS - 1;
test(left, philIndex, right);
if(states[philIndex] != State.EATING) {
condition[philIndex].await();
}
}catch(InterruptedException e){}
}
#Override
public void replaceChopsticks() {
try{
states[philIndex] = State.THINKING;
int left = philIndex-1;
int right = philIndex+1;
if(philNum == DINERS) right = 0;
if(philNum == 1) left = DINERS - 1;
int leftOfLeft = left-1;
int rightOfRight = right+1;
if(left == 0) leftOfLeft = DINERS-1;
test(leftOfLeft, left, philIndex);
if(right == DINERS-1) rightOfRight = 0;
test(philIndex, right, rightOfRight);
}finally{ lock.unlock(); }
//states[philIndex] = State.THINKING;
//condition[left].signal();
//condition[right].signal();
}
public void think() {
System.out.println("Philosopher " + philNum + " is " + State.THINKING + ".");
int timeToSleep = timeGenerator.nextInt(maxTimeToSleep) + minTimeToSleep;
try {
Thread.sleep(500);
}catch(InterruptedException e) {}
}
public void eat() {
System.out.println("Philosopher " + philNum + " is " + State.EATING + ".");
int timeToSleep = timeGenerator.nextInt(maxTimeToSleep) + minTimeToSleep;
try {
Thread.sleep(500);
}catch(InterruptedException e){}
}
#Override
public void run() {
while(true) {
think();
takeChopsticks();
eat();
replaceChopsticks();
}
}
public State getState() {
return current;
}
private void test(int left, int current, int right) {
if(states[left] != State.EATING && states[current] == State.HUNGRY
&& states[right] != State.EATING) {
states[current] = State.EATING;
condition[current].signal();
}
}
}
Why are the treads not running concurrently? Thanks for the help!
EDIT: To run it, there is a driver that is this:
public class Lunch {
public static void main(String[] args) {
ReentrantLock lock = new ReentrantLock();
Thread[] diners = new Thread[PhilosopherInterface.DINERS];
Condition[] table = new Condition[PhilosopherInterface.DINERS];
State[] states = new State[PhilosopherInterface.DINERS];
for(int i=0; i<PhilosopherInterface.DINERS; i++) {
states[i] = State.THINKING;
}
for(int i=0; i<PhilosopherInterface.DINERS; i++) {
Diner diner = new Diner(lock, i+1, table, states);
diners[i] = new Thread(diner);
diners[i].start();
}
}
}
EDIT2: Figured out the problem, Answer below.

Telling your threads to wait is not forcing them to work concurrently. If a thread needs to follow several steps before another one activates, then these methods(steps) should be synchronized.

I only locked once at the beginning of takeChopsticks() and unlocked at the end of replaceChopsticks(), forcing the thread to do everything before unlocking.
I used the lock() and unlock() methods at the start and finish of both takeChopsticks() and replaceChopsticks(), allowing it to run concurrently.

Try using an ExecutorService. Use ExecutorService.submit(Runnable)
and ExecutorService.shutdown() which will wait until all the Runnables have terminated and shutdown the ExecutorService.

Is my code in a state of deadlock?

On compiling my code below it seems to be in a state of deadlock, and i don't know how i can fix it. I am attempting to write a pipeline as a sequence of threads linked together as a buffer, and each thread can read the preceding node in the pipeline, and consequentially write to the next one. The overall goal is to spilt a randomly generated arraylist of data over 10 threads and sort it.
class Buffer{
// x is the current node
private int x;
private boolean item;
private Lock lock = new ReentrantLock();
private Condition full = lock.newCondition();
private Condition empty = lock.newCondition();
public Buffer(){item = false;}
public int read(){
lock.lock();
try{
while(!item)
try{full.await();}
catch(InterruptedException e){}
item = false;
empty.signal();
return x;
}finally{lock.unlock();}
}
public void write(int k){
lock.lock();
try{
while(item)
try{empty.await();}
catch(InterruptedException e){}
x = k; item = true;
full.signal();
}finally{lock.unlock();}
}
}
class Pipeline extends Thread {
private Buffer b;
//private Sorted s;
private ArrayList<Integer> pipe; // array pipeline
private int ub; // upper bounds
private int lb; // lower bounds
public Pipeline(Buffer bf, ArrayList<Integer> p, int u, int l) {
pipe = p;ub = u;lb = l;b = bf;//s = ss;
}
public void run() {
while(lb < ub) {
if(b.read() > pipe.get(lb+1)) {
b.write(pipe.get(lb+1));
}
lb++;
}
if(lb == ub) {
// store sorted array segment
Collections.sort(pipe);
new Sorted(pipe, this.lb, this.ub);
}
}
}
class Sorted {
private volatile ArrayList<Integer> shared;
private int ub;
private int lb;
public Sorted(ArrayList<Integer> s, int u, int l) {
ub = u;lb = l;shared = s;
// merge data to array from given bounds
}
}
class Test1 {
public static void main(String[] args) {
int N = 1000000;
ArrayList<Integer> list = new ArrayList<Integer>();
for(int i=0;i<N;i++) {
int k = (int)(Math.random()*N);
list.add(k);
}
// write to buffer
Buffer b = new Buffer();
b.write(list.get(0));
//Sorted s = new Sorted();
int maxBuffer = 10;
int index[] = new int[maxBuffer+1];
Thread workers[] = new Pipeline[maxBuffer];
// Distribute data evenly over threads
for(int i=0;i<maxBuffer;i++)
index[i] = (i*N) / maxBuffer;
for(int i=0;i<maxBuffer;i++) {
// create instacen of pipeline
workers[i] = new Pipeline(b,list,index[i],index[i+1]);
workers[i].start();
}
// join threads
try {
for(int i=0;i<maxBuffer;i++) {
workers[i].join();
}
} catch(InterruptedException e) {}
boolean sorted = true;
System.out.println();
for(int i=0;i<list.size()-1;i++) {
if(list.get(i) > list.get(i+1)) {
sorted = false;
}
}
System.out.println(sorted);
}
}

When you start the run methods, all threads will block until the first thread hits full.await(). then one after the other, all threads will end up hitting full.await(). they will wait for this signal.
However the only place where full.signal occurs is after one of the read methods finishes.
As this code is never reached (because the signal is never fired) you end up with all threads waiting.
in short, only after 1 read finishes, will the writes trigger.
if you reverse the logic, you start empty, you write to the buffer (with signal, etc, etc) and then the threads try to read, I expect it will work.
generally speaking you want to write to a pipeline before reading from it. (or there's nothing to read).
I hope i'm not misreading your code but that's what I see on first scan.

Your Buffer class it flipping between read and write mode. Each read must be followed by a write, that by a read and so on.
You write the buffer initially in your main method.
Now one of your threads reaches if(b.read() > pipe.get(lb+1)) { in Pipeline#run. If that condition evaluates to false, then nothing gets written. And since every other thread must still be the very same if(b.read(), you end up with all reading threads that can't progress. You will either have to write in the else branch or allow multiple reads.

Semaphores: Critical Section with priorities

I'm writing a program in Java that deals with Semaphores for an assignment. I'm still new to the idea of Semaphores and concurrency.
The description of the problem is as follows:
A vector V[] of booleans. V[i] is "True"if Pi needs to use the critical section.
A vector of binary semaphores B[] to block processes from entering their critical section: B[i] will be the semaphore blocking process Pi.
A special scheduler process SCHED is used whenever a blocked process needs to be awakened to use the critical section.
SCHED is blocked by waiting on a special semaphore S
When a process Pi needs to enter the critical section, it sets V[i] to "True", signals the semaphore S and then waits on the semaphore B[i].
Whenever SCHED is unblocked, it selects the process Pi with the smallest index i for which V[i] is "True". Process Pi is then awakened by signaling B[i] and SCHED goes back to sleep by blocking on semaphore S.
When a process Pi leaves the critical section, it signals S.
This is my code:
import java.util.concurrent.Semaphore;
public class Process extends Thread {
static boolean V[];
int i;
static Semaphore B[]; //blocking semaphore
static Semaphore S;
private static int id;
static int N;
static int insist = 0;
public static void process (int i, int n) {
id = i;
N = n;
V = new boolean[N];
}
private void delay () {
try {
sleep (random(500));
}
catch (InterruptedException p) {
}
}
private static int random(int n) {
return (int) Math.round(n * Math.random() - 0.5);
}
private void entryprotocol(int i) {
V[Process.id] = true;
int turn = N;
while (V[Process.id] == true && turn == N) {
System.out.println("P" + Process.id + " is requesting critical section");
signal(S);
}
critical(Process.id);
wait(B[Process.id]);
V[Process.id] = false;
}
private void wait(Semaphore S) {
if (Process.id > 0) {
Process.id--;
} else {
//add Process.id to id.queue and block
wait(B[Process.id]);
}
}
private void signal(Semaphore S) {
if (B[Process.id] != null) {
Sched(Process.id);
} else {
Process.id++; //remove process from queue
critical(Process.id); //wakes up current process
}
}
private void critical(int i) {
System.out.println("P" + Process.id + " is in the critical section");
delay();
exitprotocol(i);
}
private void exitprotocol(int i) {
System.out.println("P" + Process.id + " is leaving the critical section");
V[id] = false;
signal(S);
}
public void Sched(int i) {
if (B[Process.id] == null) {
signal(B[Process.id]);
}
wait(S);
}
public void run() {
for (int i = 0; i < 5; i++) {
Sched(i);
entryprotocol(Process.id);
try {
wait(Process.id);
}
catch (InterruptedException p) {
}
signal(S);
}
}
public static void main (String[] args) {
int N = 5;
Process p[] = new Process[N];
for (int i = 0; i < N; i++) {
p[i] = new Process();
p[i].start();
}
}
}
I believe my logic here is correct but I'm getting a lot of errors (such as Exception in thread "Thread-1" java.lang.NullPointerException). Can any shed some light on what I'm doing wrong & provide me with some help. It's greatly appreciated!

Your NPE is probably due to the fact that you never initialize your Semaphore array - but its hard to say without a proper stack trace.
Two pieces of advice:
1) You might want to give your class variables more meaningful names than :
B
N
S
V.
Imagine walking away from this project and revisiting it in 4 months and had to read through that.
2) Figure out your class model on on a white board before writing any code. You have methods that take semaphores with the same name as some of your static fields. What are the relationships of the objects in your program? If you don't know, odds are your program doesn't know either.