I have a flow of units of work, lets call them "Work Items" that are processed sequentially (for now). I'd like to speed up processing by doing the work multithreaded.
Constraint: Those work items come in a specific order, during processing the order is not relevant - but once processing is finished the order must be restored.
Something like this:
|.|
|.|
|4|
|3|
|2| <- incoming queue
|1|
/ | \
2 1 3 <- worker threads
\ | /
|3|
|2| <- outgoing queue
|1|
I would like to solve this problem in Java, preferably without Executor Services, Futures, etc., but with basic concurrency methods like wait(), notify(), etc.
Reason is: My Work Items are very small and fine grained, they finish processing in about 0.2 milliseconds each. So I fear using stuff from java.util.concurrent.* might introduce way to much overhead and slow my code down.
The examples I found so far all preserve the order during processing (which is irrelevant in my case) and didn't care about order after processing (which is crucial in my case).
This is how I solved your problem in a previous project (but with java.util.concurrent):
(1) WorkItem class does the actual work/processing:
public class WorkItem implements Callable<WorkItem> {
Object content;
public WorkItem(Object content) {
super();
this.content = content;
}
public WorkItem call() throws Exception {
// getContent() + do your processing
return this;
}
}
(2) This class puts Work Items in a queue and initiates processing:
public class Producer {
...
public Producer() {
super();
workerQueue = new ArrayBlockingQueue<Future<WorkItem>>(THREADS_TO_USE);
completionService = new ExecutorCompletionService<WorkItem>(Executors.newFixedThreadPool(THREADS_TO_USE));
workerThread = new Thread(new Worker(workerQueue));
workerThread.start();
}
public void send(Object o) throws Exception {
WorkItem workItem = new WorkItem(o);
Future<WorkItem> future = completionService.submit(workItem);
workerQueue.put(future);
}
}
(3) Once processing is finished the Work Items are dequeued here:
public class Worker implements Runnable {
private ArrayBlockingQueue<Future<WorkItem>> workerQueue = null;
public Worker(ArrayBlockingQueue<Future<WorkItem>> workerQueue) {
super();
this.workerQueue = workerQueue;
}
public void run() {
while (true) {
Future<WorkItem> fwi = workerQueue.take(); // deqeueue it
fwi.get(); // wait for it till it has finished processing
}
}
}
(4) This is how you would use the stuff in your code and submit new work:
public class MainApp {
public static void main(String[] args) throws Exception {
Producer p = new Producer();
for (int i = 0; i < 10000; i++)
p.send(i);
}
}
If you allow BlockingQueue, why would you ignore the rest of the concurrency utils in java?
You could use e.g. Stream (if you have java 1.8) for the above:
List<Type> data = ...;
List<Other> out = data.parallelStream()
.map(t -> doSomeWork(t))
.collect(Collectors.toList());
Because you started from an ordered Collection (List), and collect also to a List, you will have results in the same order as the input.
Just ID each of the objects for processing, create a proxy which would accept done work and allow to return it only when the ID pushed was sequential. A sample code below. Note how simple it is, utilizing an unsynchronized auto-sorting collection and just 2 simple methods as API.
public class SequentialPushingProxy {
static class OrderedJob implements Comparable<OrderedJob>{
static AtomicInteger idSource = new AtomicInteger();
int id;
public OrderedJob() {
id = idSource.incrementAndGet();
}
public int getId() {
return id;
}
#Override
public int compareTo(OrderedJob o) {
return Integer.compare(id, o.getId());
}
}
int lastId = OrderedJob.idSource.get();
public Queue<OrderedJob> queue;
public SequentialPushingProxy() {
queue = new PriorityQueue<OrderedJob>();
}
public synchronized void pushResult(OrderedJob job) {
queue.add(job);
}
List<OrderedJob> jobsToReturn = new ArrayList<OrderedJob>();
public synchronized List<OrderedJob> getFinishedJobs() {
while (queue.peek() != null) {
// only one consumer at a time, will be safe
if (queue.peek().getId() == lastId+1) {
jobsToReturn.add(queue.poll());
lastId++;
} else {
break;
}
}
if (jobsToReturn.size() != 0) {
List<OrderedJob> toRet = jobsToReturn;
jobsToReturn = new ArrayList<OrderedJob>();
return toRet;
}
return Collections.emptyList();
}
public static void main(String[] args) {
final SequentialPushingProxy proxy = new SequentialPushingProxy();
int numProducerThreads = 5;
for (int i=0; i<numProducerThreads; i++) {
new Thread(new Runnable() {
#Override
public void run() {
while(true) {
proxy.pushResult(new OrderedJob());
}
}
}).start();
}
int numConsumerThreads = 1;
for (int i=0; i<numConsumerThreads; i++) {
new Thread(new Runnable() {
#Override
public void run() {
while(true) {
List<OrderedJob> ret = proxy.getFinishedJobs();
System.out.println("got "+ret.size()+" finished jobs");
try {
Thread.sleep(200);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}).start();
}
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.exit(0);
}
}
This code could be easily improved to
allow pushing more than one job result at once, to reduce the synchronization costs
introduce a limit to returned collection to get done jobs in smaller chunks
extract an interface for those 2 public methods and switch implementations to perform tests
You could have 3 input and 3 output queues - one of each type for each worker thread.
Now when you want to insert something into the input queue you put it into only one of the 3 input queues. You change the input queues in a round robin fashion. The same applies to the output, when you want to take something from the output you choose the first of the output queues and once you get your element you switch to the next queue.
All the queues need to be blocking.
Pump all your Futures through a BlockingQueue. Here's all the code you need:
public class SequentialProcessor implements Consumer<Task> {
private final ExecutorService executor = Executors.newCachedThreadPool();
private final BlockingDeque<Future<Result>> queue = new LinkedBlockingDeque<>();
public SequentialProcessor(Consumer<Result> listener) {
new Thread(() -> {
while (true) {
try {
listener.accept(queue.take().get());
} catch (InterruptedException | ExecutionException e) {
// handle the exception however you want, perhaps just logging it
}
}
}).start();
}
public void accept(Task task) {
queue.add(executor.submit(callableFromTask(task)));
}
private Callable<Result> callableFromTask(Task task) {
return <how to create a Result from a Task>; // implement this however
}
}
Then to use, create a SequentialProcessor (once):
SequentialProcessor processor = new SequentialProcessor(whatToDoWithResults);
and pump tasks to it:
Stream<Task> tasks; // given this
tasks.forEach(processor); // simply this
I created the callableFromTask() method for illustration, but you can dispense with it if getting a Result from a Task is simple by using a lambda instead or method reference instead.
For example, if Task had a getResult() method, do this:
queue.add(executor.submit(task::getResult));
or if you need an expression (lambda):
queue.add(executor.submit(() -> task.getValue() + "foo")); // or whatever
Reactive programming could help. During my brief experience with RxJava I found it to be intuitive and easy to work with than core language features like Future etc. Your mileage may vary. Here are some helpful starting points https://www.youtube.com/watch?v=_t06LRX0DV0
The attached example also shows how this could be done. In the example below we have Packet's which need to be processed. They are taken through a simple trasnformation and fnally merged into one list. The output appended to this message shows that the Packets are received and transformed at different points in time but in the end they are output in the order they have been received
import static java.time.Instant.now;
import static rx.schedulers.Schedulers.io;
import java.time.Instant;
import java.util.List;
import java.util.Random;
import rx.Observable;
import rx.Subscriber;
public class RxApp {
public static void main(String... args) throws InterruptedException {
List<ProcessedPacket> processedPackets = Observable.range(0, 10) //
.flatMap(i -> {
return getPacket(i).subscribeOn(io());
}) //
.map(Packet::transform) //
.toSortedList() //
.toBlocking() //
.single();
System.out.println("===== RESULTS =====");
processedPackets.stream().forEach(System.out::println);
}
static Observable<Packet> getPacket(Integer i) {
return Observable.create((Subscriber<? super Packet> s) -> {
// simulate latency
try {
Thread.sleep(new Random().nextInt(5000));
} catch (Exception e) {
e.printStackTrace();
}
System.out.println("packet requested for " + i);
s.onNext(new Packet(i.toString(), now()));
s.onCompleted();
});
}
}
class Packet {
String aString;
Instant createdOn;
public Packet(String aString, Instant time) {
this.aString = aString;
this.createdOn = time;
}
public ProcessedPacket transform() {
System.out.println(" Packet being transformed " + aString);
try {
Thread.sleep(new Random().nextInt(5000));
} catch (Exception e) {
e.printStackTrace();
}
ProcessedPacket newPacket = new ProcessedPacket(this, now());
return newPacket;
}
#Override
public String toString() {
return "Packet [aString=" + aString + ", createdOn=" + createdOn + "]";
}
}
class ProcessedPacket implements Comparable<ProcessedPacket> {
Packet p;
Instant processedOn;
public ProcessedPacket(Packet p, Instant now) {
this.p = p;
this.processedOn = now;
}
#Override
public int compareTo(ProcessedPacket o) {
return p.createdOn.compareTo(o.p.createdOn);
}
#Override
public String toString() {
return "ProcessedPacket [p=" + p + ", processedOn=" + processedOn + "]";
}
}
Deconstruction
Observable.range(0, 10) //
.flatMap(i -> {
return getPacket(i).subscribeOn(io());
}) // source the input as observables on multiple threads
.map(Packet::transform) // processing the input data
.toSortedList() // sorting to sequence the processed inputs;
.toBlocking() //
.single();
On one particular run Packets were received in the order 2,6,0,1,8,7,5,9,4,3 and processed in order 2,6,0,1,3,4,5,7,8,9 on different threads
packet requested for 2
Packet being transformed 2
packet requested for 6
Packet being transformed 6
packet requested for 0
packet requested for 1
Packet being transformed 0
packet requested for 8
packet requested for 7
packet requested for 5
packet requested for 9
Packet being transformed 1
packet requested for 4
packet requested for 3
Packet being transformed 3
Packet being transformed 4
Packet being transformed 5
Packet being transformed 7
Packet being transformed 8
Packet being transformed 9
===== RESULTS =====
ProcessedPacket [p=Packet [aString=2, createdOn=2016-04-14T13:48:52.060Z], processedOn=2016-04-14T13:48:53.247Z]
ProcessedPacket [p=Packet [aString=6, createdOn=2016-04-14T13:48:52.130Z], processedOn=2016-04-14T13:48:54.208Z]
ProcessedPacket [p=Packet [aString=0, createdOn=2016-04-14T13:48:53.989Z], processedOn=2016-04-14T13:48:55.786Z]
ProcessedPacket [p=Packet [aString=1, createdOn=2016-04-14T13:48:54.109Z], processedOn=2016-04-14T13:48:57.877Z]
ProcessedPacket [p=Packet [aString=8, createdOn=2016-04-14T13:48:54.418Z], processedOn=2016-04-14T13:49:14.108Z]
ProcessedPacket [p=Packet [aString=7, createdOn=2016-04-14T13:48:54.600Z], processedOn=2016-04-14T13:49:11.338Z]
ProcessedPacket [p=Packet [aString=5, createdOn=2016-04-14T13:48:54.705Z], processedOn=2016-04-14T13:49:06.711Z]
ProcessedPacket [p=Packet [aString=9, createdOn=2016-04-14T13:48:55.227Z], processedOn=2016-04-14T13:49:16.927Z]
ProcessedPacket [p=Packet [aString=4, createdOn=2016-04-14T13:48:56.381Z], processedOn=2016-04-14T13:49:02.161Z]
ProcessedPacket [p=Packet [aString=3, createdOn=2016-04-14T13:48:56.566Z], processedOn=2016-04-14T13:49:00.557Z]
You could launch a DoTask thread for every WorkItem. This thread processes the work.
When the work is done, you try to post the item, synchronized on the controlling object, in which you check if it's the right ID and wait if not.
The post implementation can be something like:
synchronized(controllingObject) {
try {
while(workItem.id != nextId) controllingObject.wait();
} catch (Exception e) {}
//Post the workItem
nextId++;
object.notifyAll();
}
I think that you need an extra queue to hold the incoming order.
IncomingOrderQueue.
When you consume the objects you put them in some storage, for example Map and then from another thread which consumes from the IncomingOrderQueue you pick the ids(hashes) of the objects and then you collect them from this HashMap.
This solution can easily be implemented without execution service.
Preprocess: add an order value to each item, prepare an array if it is not allocated.
Input: queue (concurrent sampling with order values 1,2,3,4 but doesnt matter which tread gets which sample)
Output: array (writing to indexed elements, using a synch point to wait for all threads in the end, doesn't need collision checks since it writes different positions for every thread)
Postprocess: convert array to a queue.
Needs n element-array for n-threads. Or some multiple of n to do postprocessing only once.
I’m new to multithreading.
I’m having difficulties understanding what is wrong with my implemantation, and why every implementation I see is using synchronized blocks and notifys.
The running seems ok so i can’t point to what exactly is not good but I assume there are some multithreading principles I'm not following.
This is the code:
public class Threads {
static Queue<MyThread> queue = new LinkedList<>();
static Thread[] threadsArr = new Thread[10];
public static void main(String[] args) throws InterruptedException {
Threads t = new Threads();
t.startArr();
t.startProcess();
}
void startArr(){
for (int i=0;i<10;i++){
threadsArr[i] = new Thread(new MyThread(i));
}
}
void startProcess(){
for (int i=0;i<100;i++){
queue.add(new MyThread(i));
}
for (int i=0;i<100;i++){
int insertPlace = 0;
boolean isFull = true;
while (isFull){
for (int j=0;j<10;j++){
if (!threadsArr[j].isAlive()){
insertPlace = j;
isFull = false;
}
}
}
threadsArr[insertPlace] = new Thread(new MyThread(i));
threadsArr[insertPlace].start();
}
}
}
and the MyThread class:
public class MyThread implements Runnable {
int threadNumber;
public MyThread(int threadNumber){
this.threadNumber = threadNumber;
}
#Override
public void run() {
System.out.println(threadNumber + " started.");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println(threadNumber + " finished.");
}
}
thanks.
I think the main problem is that you missed the role of a "ThreadPoolExecutor". Basically, the user that uses your class wants to be able to call an "execute(Runnable run)" method, knowing that your Threads class will handle the execution in the process it is allowed to create.
You should rework the API of your class and provide this kind of method (see real ThreadPoolExecutor for instance http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html).
Second, if you are training for a job interview, try to write cleaner and more flexible code by not having constant like "10" all around the class. This should be an attribute provided by the user of your class (in the constructor), specifying how many threads at once he wants to allow (again see the real ThreadPoolExecutor).
Finally I am no expert in implementing ThreadPoolExecutors, but you could consider using a BlockingQueue (http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/BlockingQueue.html) which support, i quote, "operations that wait for the queue to become non-empty when retrieving an element". This could be useful for waiting for a Thread to be available, rather to do it yourself. But I guess there are better answers.
Good luck,
Mathias
I have created a class that attempts to process some messages in multiple threads, where each message belongs to a particular group. Each message is being added to the ConcurrentHashMap which has a key as the group number and is filling up while these threads are "processing". I have noticed that they sometimes run in parallel and sometimes not. And to make matters worse when there are more than 2 process threads running always they deadlock entirely.
[EDIT]
Iteration of the ConcurrentHashMap seemed at the time to be a good way of going through all the elements as the numbered message groups (keys) are not known and it could change over time. The task specified that all messages be grouped together for processing but when when there is only one message in a group it should still process. So I thought this was a way to sort the elements as they arrive without knowing at the beginning which groups exist.
[\EDIT]
public class GroupPriorityProcess implements Runnable {
private static final Object lock = new Object();
private static final Object counterLock = new Object();
private static int threadCounter = 0;
private final int currentThreadNumber;
private static Iterator<Integer> groupIterator;
private ConcurrentHashMap<Integer, LinkedBlockingQueue<Message>> groupMsgQueues;
public GroupPriorityProcess(ConcurrentHashMap<Integer, LinkedBlockingQueue<Message>> groupedMsgQueues) {
groupMsgQueues = groupedMsgQueues;
synchronized(lock){
if (groupIterator == null)
groupIterator = groupedMsgQueues.keySet().iterator();
}
synchronized (counterLock) {
currentThreadNumber = (threadCounter++);
}
}
// Main while loop for threads to process messages
public void run() {
while (true) {
LinkedBlockingQueue<Message> queue = chooseGroup();
synchronized (queue) {
process(queue);
}
}
}
// Loops till finds a message group available for processing.
private LinkedBlockingQueue<Message> chooseGroup() {
synchronized (lock) {
while (!groupIterator.hasNext()) {
groupIterator = groupMsgQueues.keySet().iterator();
}
LinkedBlockingQueue<Message> queue = groupMsgQueues.get(groupIterator.next());
return queue;
}
}
// takes messages from the a particular message group queue to completes the
// send process
private void process(LinkedBlockingQueue<Message> queue) {
try {
while (!queue.isEmpty()) {
Message msg = queue.take();
msg.appendMessage("Thread: " + currentThreadNumber);
msg.completed();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
[EDIT]
The messages are added here in another class GatewayImp.
public void send(Message msg) {
int groupID = msg.getGroupID();
if (groupedMsgQueues.containsKey(groupID)) {
LinkedBlockingQueue<Message> queue = groupedMsgQueues.get(groupID);
queue.add(msg);
} else {
LinkedBlockingQueue<Message> queue = new LinkedBlockingQueue<Message>();
try {
queue.put(msg);
} catch (InterruptedException e) {
e.printStackTrace();
}
groupedMsgQueues.put(groupID, queue);
}
}
}
I have a few times when I use 'check then act' which I thought needs to be in a synchronized block to make atomic but I would like to know there is some better way. Please any help with this is greatly appreciated as I am only just starting to learn about concurrency and I'm finding it hard to get my head around locking in particular.
My first guess would be that you should not synchronize on the queue in your run method.
It can collide with the internal synchronization of LinkedBlockingQueue, when you call queue.take() later in the process method (while still holding the mutex of queue).
To help you debug your code, it is often useful to add verbose logging (e.g., some System.out.println statements). The good news is that you seem to be handle to reproduce the deadlock. Often, this is easier said than done...
I have a Thread that only has to work when a certain circumstance comes in. Otherwise it just iterates over an empty infinite loop:
public void run() {
while(true) {
if(ball != null) {
// do some Calculations
}
}
}
Does it affect the performance when the loop actually does nothing but it has to check if it has to do the calculation every iteration?
Only creating a this Thread when needed is not an option for me, because my class which implements Runnable is a visual object which has be shown all the time.
edit: so is the following a good solution? Or is it better to use a different method (concerning performance)?
private final Object standBy = new Object();
public void run() {
while(true) {
synchronized (standBy) {
while(ball != null) // should I use while or if here?
try{ standBy.wait() }
catch (InterruptedException ie) {}
}
if(ball != null) {
// do some Calculations
}
}
public void handleCollision(Ball b) {
// some more code..
ball = b;
synchronized (standBy) {
standBy.notify();
}
}
You might want to consider putting the thread to sleep and only waking it up only when your 'ball' variable becomes true. There are multiple ways of doing this, from using the very low level, wait and notify statements to using the java.util.concurrent classes which provide a less error prone way of doing this. Have a look at the documentation for the condition interface. A data structure like a BlockingQueue would also be a solution.
Yes it does. This is the most simple implementation of busy waiting, and should be avoided whenever possible. Use wait/notify or java.util.concurrent mechanisms. Maybe you should be more specific about what exactly you want to achieve to get more useful responses.
Yes, it will certainly affect performance. To increase performance, you can consider putting in a bit of a time delay (say 500ms or 1000ms or even higher) in your code depending how crucial timing is to you.
Share a BlockingQueue between your threads.
class Producer implements Runnable {
private final BlockingQueue queue;
Producer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { queue.put(produce()); }
} catch (InterruptedException ex) { ... handle ...}
}
Object produce() { ... }
}
class Consumer implements Runnable {
private final BlockingQueue queue;
Consumer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { consume(queue.take()); }
} catch (InterruptedException ex) { ... handle ...}
}
void consume(Object x) { ... }
}
I found the following interesting thing. In task manager, running that infinite loop like that, would consume 17% of my CPU. Now, if I added a simple
Thread.sleep(1)
inside the loop, which is only one milisecond, the CPU use dropped to almost zero as if I was not using the program, and the response time of the program was still pretty good on average (in my case it needed to reply things fast)
I have a project for my "Operating Systems". I need to write 2 programs with java...
write a program that produce Water with 2 method Oxygen and Hydrogen.
method Oxygen produce one Oxygen and method Hydrogen produce one hydrogen. when 2 Hydrogen and one Oxygen was existed H2O created. I must write this with with Semaphores and threads.
Write the above problem with Monitors and Sychronize.
I've writed some code for this but it gives illegal monitor exeption...
please help me to correct it...
This is my code:
// class for implement Thread for oxygen
public class Thread_O implements Runnable {
public void run() {
thread t = new thread();
try {
t.oxygen();
} catch (InterruptedException ex) {
Logger logger = Logger.getLogger(Thread_O.class.getName());
logger.log(Level.SEVERE, null, ex);
}
}
}
// class for implement Thread for Hydrogen
public class Thread_H implements Runnable {
public void run() {
thread t = new thread();
try {
t.Hydrogen();
} catch (InterruptedException ex) {
Logger logger = Logger.getLogger(Thread_H.class.getName());
logger.log(Level.SEVERE, null, ex);
}
}
}
//class for method Oxygen and Hydrogen
public class thread {
Semaphore O = new Semaphore(0, true);
Semaphore H = new Semaphore(0, true);
Semaphore H2O = new Semaphore(0, true);
Semaphore safe = new Semaphore(1, true);
public void oxygen() throws InterruptedException {
safe.wait();
H.wait();
H.wait();
H2O.release();
H2O.release();
Safe.release();
// System.out.println("O2...!");
}
public void Hydrogen() throws InterruptedException {
H.release();
H2O.wait();
// System.out.println("H2...!");
}
}
and in action of Oxygen Button:
Thread th = new Thread(new Thread_O());
th.start();
I'm not going to decode your homework for you, but an IllegalMonitorException is thrown when you're trying to wait() on an object without being synchronized. So to wait for an object called list:
synchronized (list) {
try {
list.wait();
} catch(Throwable t) {
t.printStackTrace();
}
}
You have to understand how the producer/consumer mechanism work.
Here you'll have one consumer thread and two producers.
First you'll have one thread producing oxygen, and other producing hydrogen.
Then, those molecules should be places "somewhere" ok? That "something" is the thing that has to be monitored and synchronized.
So it should go something like this:
class Water {
char [] waterMolecule = new char[3]; // <-- synchronize access to this
char hydrogen(){
return 'H';
}
char oxygen() {
return 'O';
}
void produce() {
Thread t = new Thread( new Runnable() {
synchronize( waterMolecule ) {
waterMolecule[0] = hydrogen();
}
}):
.... produce the others
}
void consume() {
synchronize watermolecule
if waterMolecule is complete
create water and clean out the molecule.
}
}
That's the basic idea.
Just bear in mind that you won't be able to produce another particle of oxigen until the previous one has been consumed.
Also you must always call wait in a while loop
Here's how that wait/synchronize should be coded.
Here's a number of producer/consumer samples.
Although your homework is already due, I'd like to propose CyclicBarrier as the best solution for this scenario.
It allows some kind of rendezvous for the different threads (here: your molecule producers) and triggers the execution of an additional runnable on completition (here: creation of h20).