I have a Producer Which produces a POJO with a property, type. There can only be two types, "A" and "B". I have a thread pool for Consumer. Whenever I receive a message of type "B" from the Producer, Before I can proceed for execution, I need to make sure that all the other threads in the pool have completed execution(For now a default Thread.sleep). And then a consumer thread should pickup the message of type "B" and run it. Till this thread is running no message can be popped from the Queue.
Example:
class POJO_Message{
String type; //This will contain the type of message "A" or "B"
}
You can use LinkedBlockingDeque. An example:
public class ProducerConsumer {
public static void main(String[] args) {
final LinkedBlockingDeque<Message> queue = new LinkedBlockingDeque<>(10);
final AtomicLong id = new AtomicLong(0);
final Timer producer = new Timer(true);
producer.scheduleAtFixedRate(new TimerTask() {
#Override
public void run() {
queue.add(new Message( String.format("msg: %s" , id.incrementAndGet() ) ) );
}
}, 10, 10);
// consume
for(;;) {
try {
Message msg = queue.take();
System.out.println( msg );
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
private static class Message {
private final String id;
public Message(String id) {
this.id = id;
}
public String getId() {
return id;
}
#Override
public String toString() {
return String.format("Message [id=%s]", id);
}
}
}
You can use ReadWriteLock to do you work.when the message type is 'B',try acquire write lockļ¼other type message acquire read lock.one simple code like this.
public class ConsumerProducerQueue {
ExecutorService executor = Executors.newFixedThreadPool(10);
private ReadWriteLock lock = new ReentrantReadWriteLock();
public void addMessage(Message message) {
if ("B".equals(message.getType())) {
lock.writeLock().lock();
Future<?> result = executor.submit(new Task(message));
try {
result.get();
} catch (Exception e) {
} finally {
lock.writeLock().unlock();
}
} else {
lock.readLock().lock();
Future<?> result = executor.submit(new Task(message));
try {
result.get();
} catch (Exception e) {
} finally {
lock.readLock().unlock();
}
}
}
}
the performance of this method is not good.
Related
I am trying to write a simple function that long-polls multiple messages tothe downstream dependency without exhausting it and only exist when all messages succeeded.
I came up with a way to wrap each message polling into a callable and use a ExecutorService to submit a list of callables.
public void poll(final List<Long> messageIdList) {
ExecutorService executorService = Executors.newFixedThreadPool(messageIdList.size());
List<MessageStatusCallable> callables = messageIdList.stream()
.map(messageId -> new MessageStatusCallable(messageId)).collect(Collectors.toList());
boolean allSuccess = false;
try {
allSuccess = executorService.invokeAll(callables).stream().allMatch(success -> {
try {
return success.get().equals(Boolean.TRUE);
} catch (InterruptedException e) {
e.printStackTrace();
return false;
} catch (ExecutionException e) {
e.printStackTrace();
return false;
}
});
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private class MessageStatusCallable implements Callable<Boolean> {
private Long messageId;
public MessageStatusCallable(Long messageId) {
this.messageId = messageId;
}
/**
* Computes a result, or throws an exception if unable to do so.
*
* #return computed result
* #throws Exception if unable to compute a result
*/
#Override
public Boolean call() throws Exception {
String messageStatus = downstreamService.getMessageStatus(messageId);
while(messageStatus == null || !messageStatus.equals( STATUS_VALUE_SUCCEEDED) {
messageStatus = messageLogToControlServer.getMessageStatus(messageId);
Thread.sleep(TimeUnit.MICROSECONDS.toMillis(100));
}
LOG.info("Message: " + messageId + " Succeded");
return true;
}
}
I wonder if there is a better way to achieve this since Thread.sleep is blocking and ugly.
I'm not sure this is the best solution but it occurred to me you could use a CountDownLatch and ScheduledExecutorService.
public void poll(final List<Long> messageIdList) throws InterruptedException {
CountDownLatch latch = new CountDownLatch(messageIdList.size());
ScheduledExecutorService executorService = Executors.newScheduledThreadPool(POOL_SIZE);
try {
for (Long messageId : messageIdList) {
MessageStatusCallable callable = new MessageStatusCallable(messageId, latch);
executorService.scheduleWithFixedDelay(
() -> {
String messageStatus = downstreamService.getMessageStatus(messageId);
if (STATUS_VALUE_SUCCEEDED.equals(messageStatus)) {
latch.countDown();
throw new CompletionException("Success - killing the task", null);
}
},
0, 100, TimeUnit.MILLISECONDS);
}
latch.await();
} finally {
executorService.shutdown();
}
}
I probably also wouldn't have the Runnable as a lambda other than for brevity in the answer.
I have an object A on which I'm updating some data every second and other objects B and C which want to use the data only once per update.
Every object work in parallel.
How can I make B and C wait for the update in A ?
I've seen some similar questions but their responses didn't help me.
I've seen that I could use a "synchronized" bloc on an object D, but they just put the bloc without telling how to instanciate or share that object.
The following code is what I use for my tests. I managed to get them working in parallel but I'm stuck with the suspending part.
This is the class for A
public class Master{
public static void main(String[] args) throws Exception {
Worker B = new Worker("B");
B.start();
Worker C = new Worker("C");
C.start();
while(true)
{
Thread.sleep(1000);
// update data
// notify every thread waiting that they can resume
}
}
}
This is the class used for B and C
public class Worker extends Thread
{
Worker(String name)
{
super("Worker " + name);
}
public void run()
{
int i = 0;
while(!this.isInterrupted())
{
// wait for A to update data
System.out.println(i);
i++;
}
System.out.println("thread interrupted");
}
}
From there, what do I need to add for the purpose I'm looking for ?
To do it very low level, only using the lang APIs, you should use wait/notifyAll.
Not that I used Main.class as an arbitrary object to synchronize
public class Main {
public static void main(String[] args) {
SharedData sharedData = new SharedData();
Worker w1 = new Worker("Worker 1", sharedData);
Worker w2 = new Worker("Worker 2", sharedData);
w1.start();
w2.start();
while (true) {
try {
Thread.sleep(1000);
sharedData.increase();;
System.out.println("Master: " + sharedData.value());
synchronized (Main.class) {
Main.class.notifyAll();
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
class SharedData {
private int data = 0;
public void increase () {
data++;
}
public int value() {
return data;
}
}
class Worker extends Thread {
private String workerName;
private SharedData sharedData;
public Worker(String workerName, SharedData sharedData) {
super();
this.workerName = workerName;
this.sharedData = sharedData;
}
#Override
public void run() {
while (true) {
try {
synchronized (Main.class) {
Main.class.wait();
}
System.out.println(workerName + ": " + sharedData.value());
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
Not sure if I understand you correctly, but this might be worth checking out for you:
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CountDownLatch.html
Why use threads at all? Why not just do this?
public class Master {
public static void main(String[] args) {
Worker B = new Worker("B");
Worker C = new Worker("C");
while(true) {
Thread.sleep(1000);
updateData();
B.doWork();
C.doWork();
}
}
}
public class Worker
{
public void doWork() {
System.out.println(i);
i++;
}
private int i = 0;
}
I'm currently working on java application which has a scenario of multiple producers adding tasks to a queue and whenever queue is not empty tasks should be executed at predefined rate. (using multiple threads to maintain execution rate) After executing the available tasks executor has to wait till tasks available in the queue again.
I know blockingQueue can be used to triggering part in here and ScheduledExecutorService for execute tasks at fixed rate. But I could not find a way to link ability of both of this for my need. So I would be very thankful if you could give me any suggestion to make this happen.
You need the task queue to be accessible by both the producer and consumer threads. I've written a basic program to demonstrate this, but I'll let you play around with the BlockingQueue API and the ScheduledExecutor as per your needs:
import java.util.concurrent.*;
public class ProducerConsumer {
private static final BlockingQueue<Integer> taskQueue = new LinkedBlockingQueue<>();
public static void main(String[] args) {
ExecutorService consumers = Executors.newFixedThreadPool(3);
consumers.submit(new Consumer());
consumers.submit(new Consumer());
consumers.submit(new Consumer());
ExecutorService producers = Executors.newFixedThreadPool(2);
producers.submit(new Producer(1));
producers.submit(new Producer(2));
}
private static class Producer implements Runnable {
private final int task;
Producer(int task) {
this.task = task;
}
#Override
public void run() {
System.out.println("Adding task: " + task);
taskQueue.add(task); // put is better, since it will block if queue is full
}
}
private static class Consumer implements Runnable {
#Override
public void run() {
try {
Integer task = taskQueue.take(); // block if there is no task available
System.out.println("Executing task: " + task);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
This is the way I could come up with as a solution. It looks little bit rusty but I have tested this and the code is working.
package test;
import java.util.concurrent.*;
public class FixedRateConsumer {
private BlockingQueue<String> queue = new ArrayBlockingQueue<>(20);
private ScheduledExecutorService executorService = new ScheduledThreadPoolExecutor(5);
private boolean continueRunning = true;
public void executeInBackGraound() throws InterruptedException, ExecutionException {
while (continueRunning) {
String s = queue.take();
Worker w = new Worker(s);
ScheduledFuture future = executorService.scheduleAtFixedRate(w, 0, 1, TimeUnit.SECONDS);
w.future = future;
try {
if (!future.isDone()) {
future.get();
}
} catch (CancellationException e) {
// Skipping
}
}
}
public void setContinueRunning(boolean state) {
continueRunning = state;
}
public void addConsumableObject(String s) throws InterruptedException {
queue.put(s);
}
private void consumeString(String s) {
System.out.println("Consumed -> " + s + ", ... # -> " + System.currentTimeMillis() + " ms");
}
private class Worker implements Runnable {
String consumableObject;
ScheduledFuture future;
public Worker(String initialConsumableObject) {
this.consumableObject = initialConsumableObject;
}
#Override
public void run() {
try {
if (consumableObject == null) {
consumableObject = queue.take();
}
consumeString(consumableObject);
consumableObject = null;
if (queue.isEmpty()) {
if (future == null) {
while (future == null) {
Thread.sleep(50);
}
}
future.cancel(false);
}
} catch (Exception e) {
System.out.println("Exception : " + e);
}
}
}
}
I am trying to implement a small producer-consumer example in Java using ExecutorService.
Here is my main class
class Example {
public static void main(String args[]) {
BlockingQueue<String> queue = new ArrayBlockingQueue<>(1000);
Producer producer = new Producer(queue);
Consumer consumer = new Consumer(queue);
ExecutorService executor = Executors.newCachedThreadPool();
// executor.execute(consumer);
Future producerFuture = executor.submit(producer);
Future consumerFuture = executor.submit(consumer);
try {
producerFuture.get();
consumerFuture.get();
} catch (InterruptedException e) {
LOG.error("Failed");
}
executor.shutdown();
executor.awaitTermination(10, TimeUnit.MILLISECONDS);
}
}
Producer Class
public class Producer implements Runnable {
private BlockingQueue<String> queue;
public Producer(BlockingQueue<String> queue) {
this.queue = queue;
}
#Override
public void run() {
for (int i = 0; i < 10; i++) {
try {
queue.put(i + "HELPPPPP");
} catch (InterruptedException ex) {
Logger.getLogger(MigrationToolProducer.class.getName()).log(Level.SEVERE, null, ex);
}
}
Consumer Class
public class Consumer implements Runnable {
private final BlockingQueue<String> queue;
private volatile boolean keepRunning = true;
public Consumer(BlockingQueue<String> queue) {
this.queue = queue;
}
#Override
public void run() {
while (keepRunning) {
String value;
try {
value = queue.take();
} catch(InterruptedException e) {
throw new RuntimeException(e);
}
System.out.println(value);
}
}
}
EDIT The execution is stuck at queue.take() in Consumer Class.
Can anyone please help me fix this problem ? Why is the execution stuck in the consumer ?
One possible solution:
1) On Producer side, put a "END" signal after original 10 puts:
queue.put("END");
2) On Consumer side, once detect "END" signal, break the loop:
public void run() {
while (keepRunning) {
String value;
try {
value = queue.take();
if(value.equals("END")) {
System.out.println("Get END signal. All done!");
break;
}
} catch(InterruptedException e) {
throw new RuntimeException(e);
}
System.out.println(value);
}
}
I have started threads in sequence but i don't know how to stop them in reverse sequence.
For example:
they are starting like this: A->B->C->D
and I want them to stop: D->C->B->A
I don't know how to stop threads at all and not even in this order.
I appreciate any help or advice.
import java.util.*;
class Service extends Thread
{
private RobotController controller;
private String robotID;
private byte[] lock;
public Service(RobotController cntrl, String id)
{
controller = cntrl;
robotID = id;
}
public byte[] getLock() { return lock;}
public void run()
{
lock = new byte[0];
synchronized(lock)
{
byte[] data;
while ((data = controller.getData()) == null)
{
try {
lock.wait();
} catch (InterruptedException ie) {}
}
System.out.println("Robot " + robotID + " Working");
}
}
}
class RobotController
{
private byte[] robotData;
private Vector threadList = new Vector();
private Service thread_A;
private Service thread_B;
private Service thread_C;
private Service thread_D;
private volatile boolean done;
public void setup(){
thread_A = new Service(this, "A");
thread_B = new Service(this, "B");
thread_C = new Service(this, "C");
thread_D = new Service(this, "D");
threadList.addElement(thread_A);
threadList.addElement(thread_B);
threadList.addElement(thread_C);
threadList.addElement(thread_D);
thread_A.start();
thread_B.start();
thread_C.start();
thread_D.start();
start();
stop();
}
public void start()
{
System.out.println("Thread starts");
{
for (int i=0; i <= 3; i++)
{
try {
Thread.sleep(500);
}catch (InterruptedException ie){}
putData(new byte[10]);
Service rbot = (Service)threadList.elementAt(i);
byte[] robotLock = rbot.getLock();
synchronized(robotLock) {
robotLock.notify();
}
}
}
}
public void stop()
{
{
}
}
public synchronized byte[] getData()
{
if (robotData != null)
{
byte[] d = new byte[robotData.length];
System.arraycopy(robotData, 0, d, 0, robotData.length);
robotData = null;
return d;
}
return null;
}
public void putData(byte[] d) { robotData = d;}
public static void main(String args[])
{
RobotController controller = new RobotController();
controller.setup();
}
}
I'll usually include something like a cancel() method in my threads if I want to explicitly terminate them.
class Service extends Thread {
private volatile boolean cancel = false;
public void cancel() {
cancel = true;
}
public void run() {
...
while (!cancel && (data = controller.getData()) == null) {
...
}
}
}
Keep your threads in a stack as mre suggests, then pop through the stack and call cancel and then interrupt on each thread.
I have started threads in sequence but i don't know how to stop them in reverse sequence.
This is difficult to do. There are ways you can stop a thread either by setting a volatile shutdown boolean or interrupting them, but none of these mechanisms are guaranteed to stop a thread immediately.
You certainly can keep a List<Thread> when you build them, call Collections.reverse(threadList) and then call thread.interrupt() on each one in turn. If you must have them finish in order then you should interrupt() them and then join them. Something like:
Collections.reverse(threadList);
for (Thread thread : threadList) {
thread.interrupt();
thread.join();
}
Then each thread should be doing something like:
while (!Thread.currentThread().isInterrupted()) {
...
}
Note that if you are running Thread.sleep(...) or other methods that throw InterruptedException, you'll need to re-enable the interrupt flag:
try {
Thread.sleep(...);
} catch (InterruptedException e) {
// by convention if InterruptedException thrown, interrupt flag is cleared
Thread.currentThread().interrupt();
...
}
Have each thread keep a reference to the next thread to be started. Then each thread can periodically check to see if the thread is still alive. If not, that thread should terminate. When it does, the previous thread will notice and terminate, and so on up the chain.
abstract class ChainThread extends Thread {
private final Thread next;
ChainThread(Thread next) { this.next = next; }
#Override
public final void run() {
next.start();
while (!Thread.currentThread().isInterrupted() && next.isAlive()) {
do();
}
}
abstract void do();
}
If I read the Service code correctly, it waits until there's data to execute on, then finishes. So you don't really need an explicit stop or cancel type signal, the threads will terminate themselves after they do work.
To enforce ordering of shutdown, you could make each Service aware of the previous Service, and then call previousService.join(). Assuming no InterruptedExceptions are thrown, they will then shutdown in order after seeing that the controller has data.
Create the Services this way:
Service serviceA = new Service(controller, "A", null);
Service serviceB = new Service(controller, "B", serviceA);
Service serviceC = new Service(controller, "C", serviceB);
Service serviceD = new Service(controller, "D", serviceC);
and the implementation is edited to exit only after dependent Services are complete:
private final RobotController controller;
private final String robotID;
private byte[] lock;
private final Service dependentService;
public Service(RobotController cntrl, String id, Service dependentService) {
controller = cntrl;
robotID = id;
this.dependentService = dependentService;
}
public byte[] getLock() {
return lock;
}
#Override
public void run() {
lock = new byte[0];
synchronized (lock) {
byte[] data;
while ((data = controller.getData()) == null) {
try {
lock.wait();
}
catch (InterruptedException ie) {
}
}
System.out.println("Robot " + robotID + " Working");
}
if (dependentService != null) {
try {
dependentService.join();
}
catch (InterruptedException e) {
this.interrupt();
}
}
}