Develop Reference

executor not executing threads from within a main thread

I have a Thread which runs always with while(true) loop and basically all it does is to add Runnable objects to an executor.
OrderExecutionThread:
public class OrderExecutionThread extends Thread implements Runnable {
final private static int ORDER_EXEC_THREADS_NUMBER = 10;
private boolean running = true;
private boolean flag = true;
private List<Order> firstSellsList = new ArrayList<>();
private List<Order> secondSellsList = new ArrayList<>();
private ManagedDataSource managedDataSource;
private ExecutorService executorService;
public OrderExecutionThread(ManagedDataSource managedDataSource) {
this.managedDataSource = managedDataSource;
this.executorService = Executors.newFixedThreadPool(ORDER_EXEC_THREADS_NUMBER);
}
#Override
public void run() {
while (running) {
if (!firstSellsList.isEmpty() && !firstBuysList.isEmpty()) {
initAndRunExecution(firstBuysList.get(0), firstSellsList.get(0));
}
}
private void initAndRunExecution(Order buy, Order sell) {
executorService.submit(new OrderExecution(buy, sell, managedDataSource));
}
}
I'm running this thread By doing this in my main class:
new Thread(orderExecutionThread).start();
The executor suppose to execute the OrderExecution runnable object which does this:
#Override
public void run() {
try {
connection = managedDataSource.getConnection();
makeExecution(sell, buy);
} catch (SQLException e) {
e.printStackTrace();
} finally {
try {
if (!connection.isClosed())
connection.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
I know for sure that both lists are not empty and the initAndRunExecution is being called, however the order execution run method is not being called....

I know for sure that both lists are not empty and the initAndRunExecution is being called, however the order execution run method is not being called....
I suspect that this is a problem because your firstSellsList and firstBuysList are not synchronized collections. I suspect that other threads are adding to those lists but your OrderExecutionThread never sees the memory updates so just spins forever seeing empty lists. Whenever you share data between threads you need to worry about how the updates will be published and how the thread cache memory will be updated.
As #Fildor mentions in the comments, one solution would be to use a BlockingQueues instead of your Lists. The BlockQueue (for example LinkedBlockingQueue) is a synchronized class so this takes care of the memory sharing. An alternative benefit is that you don't have to do a spin-loop to watch for entries.
For example, your OrderExecutionThread might do something like:
private final BlockingQueue<Order> firstBuys = new LinkedBlockingQueue<>();
private final BlockingQueue<Order> firstSells = new LinkedBlockingQueue<>();
while (!Thread.currentThread().isInterrupted()) {
// wait until we get a buy
Order buy = firstBuys.take();
// wait until we get a sell
Order sell = firstSells.take();
initAndRunExecution(buy, sell);
}
This will wait until the lists get entries before running the orders.

Eliminating excess synchronization and improving error handling when updating a shared variable

I have a shared object that caches the results of database queries whose interface is "get cached results" and "invalidate cached results." It is acceptable to return slightly stale data.
My current solution is pasted at the bottom of this question. Each cache's get and clear method is accessible via a public method in CacheService. Within Cache, lastUpdated contains the most recent query results; isValid indicates whether the results should be updated; updateGuard is used to ensure that only one thread updates the results; and updateWait lets threads wait for another thread to update the results. To ensure progress and because it is acceptable to return slightly stale data, after lastUpdated is updated I immediately return its results from the updating threads and all threads waiting on the update - I do not check to see if isValid has been set to false again.
Major concern: if lastUpdated = getUpdate() throws an exception (likely the result of a network failure when trying to talk to the database) then presently I'm simply returning lastUpdated - it is acceptable to return slightly stale data, but repeated transient faults during getUpdate() could result in extremely stale data. I want to include some logic along the lines of
final int maxRetries = 5;
...
try {
updateWait.drainPermits();
int retryCount = 0;
while(true) {
try {
lastUpdated = getUpdate();
break;
} catch(Exception e) {
retryCount++;
if(retryCount == maxRetries) {
throw Exception e in all threads waiting on semaphore
}
}
}
isValid = true;
}
However I'm not sure of a good way to implement "throw Exception e in all threads waiting on semaphore" or if there's a better alternative. One option I've considered is to use a Scala Try, i.e. Try<ImmutableList<T>> lastUpdated, but I'm trying not to mix Scala and Java objects where possible in order to make code maintenance easier.
Less Major Concern: Right now I've got three synchronization variables (isValid, updateGuard, updateWait) which seems excessive - I'm looking for a way to safely eliminate one or two of these.
public class CacheService {
private final Cache<Foo> fooCache;
private final Cache<Bar> barCache;
// and so on
private abstract class Cache<T> {
private final AtomicBoolean updateGuard = new AtomicBoolean(false);
private final Semaphore updateWait = new Semaphore(Integer.MAX_VALUE);
private volatile boolean isValid = true;
private volatile ImmutableList<T> lastUpdated = getUpdate();
protected abstract ImmutableList<T> getUpdate();
public void clear() {
isValid = false;
}
public ImmutableList<T> get() {
if(isValid) {
return lastUpdated;
} else {
if(updateGuard.compareAndSet(false, true)) {
try {
updateWait.drainPermits();
lastUpdated = getUpdate();
isValid = true;
} finally {
updateGuard.set(false);
updateWait.release(Integer.MAX_VALUE);
}
} else {
while(updateGuard.get()) {
try {
updateWait.acquire();
} catch(InterruptedException e) {
break;
}
}
}
return lastUpdated;
}
}
}
public CacheService() {
fooCache = new Cache<Foo>() {
#Override
protected ImmutableList<Foo> getUpdate() {
return // database query
}
};
// Likewise when initializing barCache etc
}
}

One way to do this is with a CompletableFuture and completeExceptionally
private abstract static class Cache<T> {
private final AtomicReference<CompletableFuture<ImmutableList<T>>> value =
new AtomicReference<>();
private static final int MAX_TRIES = 5;
protected abstract ImmutableList<T> getUpdate();
public void clear() {
value.getAndUpdate(f -> f != null && f.isDone() ? null : f);
// or value.set(null); if you want the cache to be invalidated while it is being updated.
}
public ImmutableList<T> get() {
CompletableFuture<ImmutableList<T>> f = value.get();
if (f != null) {
try {
return f.get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
}
f = new CompletableFuture<>();
if (!value.compareAndSet(null, f)) {
return get();
}
for(int tries = 0; ; ){
try {
ImmutableList<T> update = getUpdate();
f.complete(update);
return update;
} catch (Exception e){
if(++tries == MAX_TRIES){
f.completeExceptionally(e);
throw new RuntimeException(e);
}
}
}
}
}
You may want to handle the exceptions differently, and you will need to clear it after an exception is thrown if you want to try to get the update again.

Your implementation has a problem. When 100 threads stall on the updateGuard lock, all the threads will execute the getUpdate() path. So, once you have the lock, you need to recheck isValid.
I am not the expert for the Semphore class, but I think combining the updateGuard and the updateWait should be feasible.
Here is just the stripped down version of your get method body:
while (!isValid) {
if (updateWait.tryAcquire()) {
if (!isValid) {
lastUpdate = getUpdate();
isValid = true;
}
} else {
updateWait.acquire();
}
updateWait.release();
}
return lastUpdate;
This should have all the semantics from your code, plus rechecking isValid.
Exception: Within the Java Caching library cache2k we implemented Exception caching. I wrote a blog entry on this, see: About caching exception. This may address some of your issues.
At the bottom line, this is my summary on it:
Fail-fast and always propagate an exception if you cannot do anything useful.
Fail-fast means no retry whatsoever to get rid of blocked resources as soon as possible. The user will retry in any case: on failure or when waiting time gets too long.
When you propagate the exception don't log it as warning additionally.
If you rethrow one exception from a data source to multiple consumers, make sure you explicitly make clear that these exceptions are duplicates
As soon as you return outdated data, because the recent request returns Exceptions, make sure to have a warning mechanism. In cache2k we probably will implement two metrics which say: how many seconds are overdue and how many entries are affected

Producer consumer in batches; second batch shouldn't come until the previous batch is complete

I'm trying to implement a mechanism where the runnables are both producer and consumer;
Situation is-
I need to read records from the DB in batches, and process the same. I'm trying this using producer consumer pattern. I get a batch, I process. Get a batch, process. This gets a batch whenever it sees queue is empty. One of the thread goes and fetches things. But the problem is that I can't mark the records that get fetched for processing, and that's my limitation. So, if we fetch the next batch before entirely committing the previous, I might fetch the same records again. Therefore, I need to be able to submit the previous one entirely before pulling the other one. I'm getting confused as to what should I do here. I've tried keeping the count of the fetched one, and then holding my get until that count is reached too.
What's the best way of handling this situation? Processing records from DB in chunks- the biggest limitation I've here is that I can't mark the records which have been picked up. So, I want batches to go through sequentially. But a batch should use multithreading internally.
public class DealStoreEnricher extends AsyncExecutionSupport {
private static final int BATCH_SIZE = 5000;
private static final Log log = LogFactory.getLog(DealStoreEnricher.class);
private final DealEnricher dealEnricher;
private int concurrency = 10;
private final BlockingQueue<QueryDealRecord> dealsToBeEnrichedQueue;
private final BlockingQueue<QueryDealRecord> dealsEnrichedQueue;
private DealStore dealStore;
private ExtractorProcess extractorProcess;
ExecutorService executor;
public DealStoreEnricher(DealEnricher dealEnricher, DealStore dealStore, ExtractorProcess extractorProcess) {
this.dealEnricher = dealEnricher;
this.dealStore = dealStore;
this.extractorProcess = extractorProcess;
dealsToBeEnrichedQueue = new LinkedBlockingQueue<QueryDealRecord>();
dealsEnrichedQueue = new LinkedBlockingQueue<QueryDealRecord>(BATCH_SIZE * 3);
}
public ExtractorProcess getExtractorProcess() {
return extractorProcess;
}
public DealEnricher getDealEnricher() {
return dealEnricher;
}
public int getConcurrency() {
return concurrency;
}
public void setConcurrency(int concurrency) {
this.concurrency = concurrency;
}
public DealStore getDealStore() {
return dealStore;
}
public DealStoreEnricher withConcurrency(int concurrency) {
setConcurrency(concurrency);
return this;
}
#Override
public void start() {
super.start();
executor = Executors.newFixedThreadPool(getConcurrency());
for (int i = 0; i < getConcurrency(); i++)
executor.submit(new Runnable() {
public void run() {
try {
QueryDealRecord record = null;
while ((record = get()) != null && !isCancelled()) {
try {
update(getDealEnricher().enrich(record));
processed.incrementAndGet();
} catch (Exception e) {
failures.incrementAndGet();
log.error("Failed to process deal: " + record.getTradeId(), e);
}
}
} catch (InterruptedException e) {
setCancelled();
}
}
});
executor.shutdown();
}
protected void update(QueryDealRecord enrichedRecord) {
dealsEnrichedQueue.add(enrichedRecord);
if (batchComplete()) {
List<QueryDealRecord> enrichedRecordsBatch = new ArrayList<QueryDealRecord>();
synchronized (this) {
dealsEnrichedQueue.drainTo(enrichedRecordsBatch);
}
if (!enrichedRecordsBatch.isEmpty())
updateTheDatabase(enrichedRecordsBatch);
}
}
private void updateTheDatabase(List<QueryDealRecord> enrichedRecordsBatch) {
getDealStore().insertEnrichedData(enrichedRecordsBatch, getExtractorProcess());
}
/**
* #return true if processed records have reached the batch size or there's
* nothing to be processed now.
*/
private boolean batchComplete() {
return dealsEnrichedQueue.size() >= BATCH_SIZE || dealsToBeEnrichedQueue.isEmpty();
}
/**
* Gets an item from the queue of things to be enriched
*
* #return {#linkplain QueryDealRecord} to be enriched
* #throws InterruptedException
*/
protected synchronized QueryDealRecord get() throws InterruptedException {
try {
if (!dealsToBeEnrichedQueue.isEmpty()) {
return dealsToBeEnrichedQueue.take();
} else {
List<QueryDealRecord> records = getNextBatchToBeProcessed();
if (!records.isEmpty()) {
dealsToBeEnrichedQueue.addAll(records);
return dealsToBeEnrichedQueue.take();
}
}
} catch (InterruptedException ie) {
throw new UnRecoverableException("Unable to retrieve QueryDealRecord", ie);
}
return null;
}
private List<QueryDealRecord> getNextBatchToBeProcessed() {
List<QueryDealRecord> recordsThatNeedEnriching = getDealStore().getTheRecordsThatNeedEnriching(getExtractorProcess());
return recordsThatNeedEnriching;
}
#Override
public void stop() {
super.stop();
if (executor != null)
executor.shutdownNow();
}
#Override
public boolean await() throws InterruptedException {
return executor.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS) && !isCancelled() && complete();
}
#Override
public boolean await(long timeout, TimeUnit unit) throws InterruptedException {
return executor.awaitTermination(timeout, unit) && !isCancelled() && complete();
}
private boolean complete() {
setCompleted();
return true;
}
}

You're already using a BlockingQueue - it does all that work for you.
However, you're using the wrong method addAll() to add new elements to the queue. That method will throw an exception if the queue is not able to accept elements. Rather you should use put() because that's the blocking method corresponding to take(), which you are using correctly.
Regarding your statement in the post title:
second batch shouldn't come until the previous batch is complete
You need not be concerned about the timing of the incoming versus outgoing batches if you use BlockingQueue correctly.

It looks like a Semaphore will work perfectly for you. Have the producing thread acquire the semaphore while the consuming thread releases the semaphore when it completes the batch.
BlockingQueue blockingQueue = ...;
Semapore semaphore = new Semaphore(1);
Producing-Thread
Batch batch = db.getBatch();
semaphore.acquire(); // wait until previous batch completes
blockingQueue.add(batch);
Consuming Thread
for(;;){
Batch batch = blockingQueue.take();
doBatchUpdate(batch);
semaphore.release(); // tell next batch to run
}

How to use Semaphores for Signaling?

Now I investigate semaphores. I googled following link about this theme:
link
Author of this link wrote about using semaphores for signaling. To show how it works he wrote custom semaphore.
custom semaphore code:
public class Semaphore {
private boolean signal = false;
public synchronized void take() {
this.signal = true;
this.notify();
}
public synchronized void release() throws InterruptedException{
while(!this.signal) wait();
this.signal = false;
}
}
about how use it in code he wrote following:
public class SendingThread {
Semaphore semaphore = null;
public SendingThread(Semaphore semaphore){
this.semaphore = semaphore;
}
public void run(){
while(true){
//do something, then signal
this.semaphore.take();
}
}
}
public class RecevingThread {
Semaphore semaphore = null;
public ReceivingThread(Semaphore semaphore){
this.semaphore = semaphore;
}
public void run(){
while(true){
this.semaphore.release();
//receive signal, then do something...
}
}
}
main:
Semaphore semaphore = new Semaphore();
SendingThread sender = new SendingThread(semaphore);
ReceivingThread receiver = new ReceivingThread(semaphore);
receiver.start();
sender.start();
As I understood order of execution should be following
send - receive
send - receive
send - receive
...
I tryed to write own code using this bluerprint
public class SendReceiveWithCustomSemaphore {
public static void main(String[] args) {
MySemaphore mySemaphore = new MySemaphore();
new Send(mySemaphore).start();
new Receive(mySemaphore).start();
}
}
class MySemaphore {
boolean flag = false;
public synchronized void take() throws InterruptedException {
flag = true;
notify();
}
public synchronized void release() throws InterruptedException {
while (!flag) {
wait();
}
flag = false;
}
}
class Send extends Thread {
MySemaphore mySemaphore;
public Send(MySemaphore semaphore) {
this.mySemaphore = semaphore;
}
#Override
public void run() {
int i = 0;
while (i++ < 10) {
System.out.println("send");
try {
mySemaphore.take();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
class Receive extends Thread {
MySemaphore mySemaphore;
public Receive(MySemaphore semaphore) {
this.mySemaphore = semaphore;
}
#Override
public void run() {
while (true) {
try {
mySemaphore.release();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("receive");
}
}
}
output:
send
send
send
send
send
send
send
send
send
send
receive
Thus it is not expected behaviour for me.
I made a mistake then I wrote code or I didn't understand concept ?
What did author want to say?

Find a better tutorial.
The output that you see is about what I would expect. The "sender" thread never blocks, so it will go on printing "send", "send", "send" forever. Meanwhile, over in the "receiver" thread, each time it calls the semaphore.release() method, it will be blocked until the next time the sender gets to run.
I would expect to see lots of "send" messsages, with occasional "receive" messages mixed in---more or less what you describe seeing.
I don't know what that example is supposed to prove, but for me, it creates the impression that the author does not know how programmers expect Semaphores to behave.
Some authors provide examples of what not to do, or examples containing a deliberate mistake that will be "fixed" in a later example. Are you sure you are not following an example of that kind?
Edit: I followed the link, and it looks like the main problem is that the names were swapped in the definitions of the take() and release() methods. If you just switch the names, it makes more sense.

By the time ReceiveSemafore is started SendSemafore has already executed 10 times.
Consider using a CountDownLatch to start the two threads at the same time. Although as pointed out by Fuhrmanator this will not produce the alternating output that you are looking for.
For this i would use a bounded semaphore with one signal.

producer - consume; how does the consumer stop?

So I have simulated my producer consumer problem and I have the code below. My question is this: how does the consumer stops if he's in constant while(true).
In the code below, I've added
if (queue.peek()==null)
Thread.currentThread().interrupt();
which works nicely in this example. But in my real world design, this doesn't work (sometimes it takes longer time to the producer to 'put' the data so the exception thrown in the consumer is incorrect. In general, I know I can put a 'poison' data such as Object is XYZ and I can check it in the consumer. But this poison makes the code really look bad. Wonder if anyone has a different approach.
public class ConsumerThread implements Runnable
{
private BlockingQueue<Integer> queue;
private String name;
private boolean isFirstTimeConsuming = true;
public ConsumerThread(String name, BlockingQueue<Integer> queue)
{
this.queue=queue;
this.name=name;
}
#Override
public void run()
{
try
{
while (true)
{
if (isFirstTimeConsuming)
{
System.out.println(name+" is initilizing...");
Thread.sleep(4000);
isFirstTimeConsuming=false;
}
try{
if (queue.peek()==null)
Thread.currentThread().interrupt();
Integer data = queue.take();
System.out.println(name+" consumed ------->"+data);
Thread.sleep(70);
}catch(InterruptedException ie)
{
System.out.println("InterruptedException!!!!");
break;
}
}
System.out.println("Comsumer " + this.name + " finished its job; terminating.");
}catch (InterruptedException e)
{
e.printStackTrace();
}
}
}

A: There is simply no guarantee that just because peek returns null, the producer has stopped producing. What if the producer simply got slowed down? Now, the consumer quits, and the producer keeps producing. So the 'peek' -> 'break' idea basically fails.
B: Setting a 'done/run' flag from consumer and reading it in producer also fails, if:
consumer checks the flag, finds it should keep running, then does a 'take'
in meanwhile, producer was setting the flag to 'dont run'
Now consumer blocks forever waiting for a ghost packet
The opposite can also happen, and one packet gets left out un-consumed.
Then to get around this, you will want to do additional synchronization with mutexes over and above the 'BlockingQueue'.
C:
I find 'Rosetta Code' to be fine source of deciding what is good practice, in situations like this:
http://rosettacode.org/wiki/Synchronous_concurrency#Java
The producer and consumer must agree upon an object (or an attribute in the object) that represents end of input. Then the producer sets that attribute in the last packet, and the consumer stops consuming it. i.e. what you referred to in your question as 'poison'.
In the Rosetta Code example above, this 'object' is simply an empty String called 'EOF':
final String EOF = new String();
// Producer
while ((line = br.readLine()) != null)
queue.put(line);
br.close();
// signal end of input
queue.put(EOF);
// Consumer
while (true)
{
try
{
String line = queue.take();
// Reference equality
if (line == EOF)
break;
System.out.println(line);
linesWrote++;
}
catch (InterruptedException ie)
{
}
}

Do NOT use interrupt on Thread, but rather break the loop when not needed anymore :
if (queue.peek()==null)
break;
Or you can also using a variable to mark closing operation pending and then break the loop and close the loop after :
if (queue.peek()==null)
closing = true;
//Do further operations ...
if(closing)
break;

In the real world, most messaging comes with a header of some sort that defines a message type / sub-type or perhaps different objects.
You can create a command and control object or message type that tells the thread to do something when it gets the message (like shutdown, reload a table, add a new listener, etc.).
This way, you can have say a command and control thread just send messages into the normal message flow. You can have the CNC thread talking to an operational terminal in a large scale system, etc.

If your queue can empty before you'd like your consumer to terminate then you'll need a flag to tell the thread when to stop. Add a setter method so the producer can tell the consumer to shutdown. Then modify your code so that instead of:
if (queue.isEmpty())
break;
have your code check
if (!run)
{
break;
}
else if (queue.isEmpty())
{
Thread.sleep(200);
continue;
}

You can use this typesafe pattern with poison pills:
public sealed interface BaseMessage {
final class ValidMessage<T> implements BaseMessage {
#Nonnull
private final T value;
public ValidMessage(#Nonnull T value) {
this.value = value;
}
#Nonnull
public T getValue() {
return value;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ValidMessage<?> that = (ValidMessage<?>) o;
return value.equals(that.value);
}
#Override
public int hashCode() {
return Objects.hash(value);
}
#Override
public String toString() {
return "ValidMessage{value=%s}".formatted(value);
}
}
final class PoisonedMessage implements BaseMessage {
public static final PoisonedMessage INSTANCE = new PoisonedMessage();
private PoisonedMessage() {
}
#Override
public String toString() {
return "PoisonedMessage{}";
}
}
}
public class Producer implements Callable<Void> {
#Nonnull
private final BlockingQueue<BaseMessage> messages;
Producer(#Nonnull BlockingQueue<BaseMessage> messages) {
this.messages = messages;
}
#Override
public Void call() throws Exception {
messages.put(new BaseMessage.ValidMessage<>(1));
messages.put(new BaseMessage.ValidMessage<>(2));
messages.put(new BaseMessage.ValidMessage<>(3));
messages.put(BaseMessage.PoisonedMessage.INSTANCE);
return null;
}
}
public class Consumer implements Callable<Void> {
#Nonnull
private final BlockingQueue<BaseMessage> messages;
private final int maxPoisons;
public Consumer(#Nonnull BlockingQueue<BaseMessage> messages, int maxPoisons) {
this.messages = messages;
this.maxPoisons = maxPoisons;
}
#Override
public Void call() throws Exception {
int poisonsReceived = 0;
while (poisonsReceived < maxPoisons && !Thread.currentThread().isInterrupted()) {
BaseMessage message = messages.take();
if (message instanceof BaseMessage.ValidMessage<?> vm) {
Integer value = (Integer) vm.getValue();
System.out.println(value);
} else if (message instanceof BaseMessage.PoisonedMessage) {
++poisonsReceived;
} else {
throw new IllegalArgumentException("Invalid BaseMessage type: " + message);
}
}
return null;
}
}