I have to make operations on a cache using many threads and thus have created 3 classes that extends from Thread (I realize now they can be Runnables also, or should be). Careful to note, these I'm running more than 1 instance of one of these threads but not at the same time. As in:
public class Operation1 extends Thread
public class Operation2 extends Thread
public class Operation3 extends Thread
Thread[] operation1Threads = new Thread[5];
So after I finish with this one, I create new threads for the second operation (at random).
The following is the way my run method is created.
I make use of cyclic barriers to wait for the threads to work at the same time.
public volatile boolean running = true;
public void run() {
while (running) {
cyclicbarrier1.await();
//some operation here
cyclicbarrier2.await();
}
}
The main thread is focusing on how long to run these for and at the end of that time, I try to stop the threads before working on the second operation. But it seems that my threads don't actually stop.
EDIT: I should note that I've tried interrupting the thread directly. And tried to reset the barrier both of which seemed to be bad practices from answers found on other threads.
for (int i=0; i < numthreads; i++) {
ops[i].interrupt();
}
I use an ExecutorService to manage these operations sequentially.
ExecutorService es = Executors.newSingleThreadExecutor();
es.submit(new Operation1Runnable());
es.submit(new Operation2Runnable());
But the executorservice doesn't go to the next one.
Related
first what i am trying to do:
During the main thread execution i want to pause the main thread and start two parallel threads. As soon as both this parallel threads terminate, i'd like to start again with the main thread.
What i tried:
...
...
main thread is executing
...
...
CyclicBarrier barrier = new CyclicBarrier(2);
Thread child1 = new Thread(new ThreadBuilderTask(barrier,0));
Thread child2 = new Thread(new ThreadBuilderTask(barrier,1));
child1.start();
child2.start();
/* Now i'm expecting that child1 and child2 are running in parallel calling their fooFunction */
child1.join();
child2.join();
/*Now i'm expecting that main thread will wait for child1and also for child2 (that are running in parallel).*/
... main thread starts again after both child1 and child2 finished (reached the await of the barrier)
... (break point set here, never reached)
...
Thread builder custom class
public class ThreadBuilderTask implements Runnable{
private CyclicBarrier barrier;
private int index;
...setters and getters..
#Override
public void run() {
fooFunction(this.getIndex());
try {
this.getBarrier().await();
} catch (InterruptedException | BrokenBarrierException e) {
return;
}
}
public ThreadBuilderTask(CyclicBarrier barrier,int index){
this.barrier = barrier;
this.index = index;
}
public fooFunction(int index){
//Something taking some seconds to execute
}
It's not clear what is happening here but it is definetely not working. As soon as i call join everything stops and the main thread never restart. (I put a breakpoint after the joins to see when the main thread restarts).
Maybe there is a bit of confusion with these concepts and also i'm not sure if i need to use both the barrier and the joins or simply one of those techniques.
Thanks
Davide
As mentioned in the comments I'd also suggest to use CompletableFuture. A very basic example of your described requirements could look like this:
final Runnable runnable1 = ...;
final Runnable runnable2 = ...;
CompletableFuture<Void> future1 = CompletableFuture.runAsync(runnable1);
CompletableFuture<Void> future2 = CompletableFuture.runAsync(runnable2);
CompletableFuture.allOf(future1, future2).get(); // waits for both runnables to finish
You might want to add more/some exception handling to this example. But it should give an idea how this might work.
You may consider to use Java CompletableFuture to achieve the objective.
Using its functions like supplyAsync or runAsync you may start child threads and join their respective result in the end. Or you can simply let the main thread wait until the subsequent threads completes.
Recently I managed to implement a sample scatter-gather function using the same class.
Check Java Doc for more offerings and to find best available function: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html
I am trying to assign a number for each of MyRunnable submitted to ThreadPoolExecutor but I did not succeed.
My code snippet:
import java.util.concurrent.*;
class SimpleThreadFactory implements ThreadFactory {
String name;
static int threadNo = 0;
public SimpleThreadFactory (String name){
this.name = name;
}
public Thread newThread(Runnable r) {
++threadNo;
System.out.println("thread no:"+threadNo);
return new Thread(r,name+":"+threadNo );
}
public static void main(String args[]){
SimpleThreadFactory factory = new SimpleThreadFactory("Ravindra");
ThreadPoolExecutor executor = new ThreadPoolExecutor(1,5,10,TimeUnit.SECONDS,new ArrayBlockingQueue(100),factory);
for ( int i=0; i < 10; i++){
executor.submit(new MyRunnable());
}
executor.shutdown();
}
}
class MyRunnable implements Runnable {
public void run(){
System.out.println("Runnable:"+Thread.currentThread().getName());
}
}
My expectation:
executor.submit(new MyRunnable()); should call newThread in ThreadFactory for each submit on executor. But in reality, it happened only once.
Output:
thread no:1
Runnable:Ravindra:1
Runnable:Ravindra:1
Runnable:Ravindra:1
Runnable:Ravindra:1
Runnable:Ravindra:1
Runnable:Ravindra:1
Runnable:Ravindra:1
Runnable:Ravindra:1
Runnable:Ravindra:1
Runnable:Ravindra:1
Why submit() is not creating new thread for each submitted Runnable task?
How can I assign a sequence number to each of MyRunnable submitted to executor?
Thanks in advance
The problem is the interaction between the CorePoolSize and the queue.
From the Javadoc
"If corePoolSize or more threads are running, the Executor always
prefers queuing a request rather than adding a new thread."
and
"If there are more than corePoolSize but less than maximumPoolSize
threads running, a new thread will be created only if the queue is
full."
So currently, your tasks get queued until there is space in the CorePoolSize (i.e. when your currently executing task finishes), so you never use more than 1 thread, currently.
See ThreadPoolExecutor JavaDoc under Core and maximum pool sizes
When a new task is submitted in method execute(java.lang.Runnable), .... If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full.
To get your code to make more threads I changed this bit:
ThreadPoolExecutor executor = new ThreadPoolExecutor(
// Core pool size
5,
// Max pool size
5,
// Resize time
1,
// Resize time units
TimeUnit.MILLISECONDS,
// Queue of runnables - I CHANGED THIS TO 10
new ArrayBlockingQueue(10),
// Factory to use for threads.
factory);
for (int i = 0; i < 100; i++) {
executor.submit(new MyRunnable());
}
Other answers explain why there have not been more than one thread. My proposal on how to achieve what you actually wanted is this:
For a start, count the instances of the runnable, not the threads they are run on: For example like this ->
class MyRunnable implements Runnable{
private static long _mySequenceCounter = 0; // Maybe use an AtomicLong?
private final long mySeqNo;
MyRunnable(){ mySeqNo = ++_mySequenceCounter; }
// Your other stuff here
}
Inside run-method, you could rename the current thread if that is sufficient for your requirement. Or you could just output the runnable id and leave the Threads name. That has the advantage that you will know, which thread was reused for which task ... if that has any value to you.
Remark: Above snippet is only to lay out the idea of how to meet your requirement to identify the task. Of course you could improve it if you need thread-safety (snippet could be problematic if MyRunnables are created on more than 1 thread).
long should give you quite a lot of sequence numbers, but mind that even long will roll over at some point. So you might want to address this if your application runs very long and has a high frequency of new MyRunnables.
Try to add a sleep to your "run" method : here the execution time is probably too short to require many thread...
I'm trying to use a thread pool to execute some code, however I'm having some trouble getting it to run without errors.
Here is my current structure:
while (!(queue.IsEmpty()))
{
currentItem= queue.GetNextItem();
for (int i = 0; i < currentItem.destinations.GetNoOfItems(); i++) //for each neighbor of currentItem
{
threadPool.submit(new NeighbourThread(currentItem, allVertices, routetype, pqOpen, i, endLocation));
}
//threadPool.shutdown();
}
NeighbourThread class:
public class NeighbourThread implements Runnable {
Vertex tempVertex, endLocation;
VertexHashMap allVertices;
int routetype, i;
PriorityQueue pqOpen;
public NeighbourThread(Vertex tempVertex, VertexHashMap allVertices, int routetype, PriorityQueue pqOpen, int i, Vertex endLocation)
{
...variables
}
#Override
public void run() {
...execution code
}
}
My idea is that it will create the amount of threads required based on currentItem.destinations.GetNoOfItems()(as it reuses threads, I'm assuming if it reaches the limit on thread creation it will wait for a thread to finish execution and reuse it).
Once the threads have been allocated, it will submit each runnable to the thread and start it.
However I need my program to wait for all threads to finish execution before it loops back to the while loop.
After reading the documentation on .shutdown(), I think that stops any future use of the threadpool, which I'm guessing is why I get this error:
Exception in thread "main" java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask#3d4eac69 rejected from java.util.concurrent.ThreadPoolExecutor#42a57993[Shutting down, pool size = 3, active threads = 1, queued tasks = 0, completed tasks = 3]
I'm trying to improve execution time on my program and as I'm currently doing over 1.5 million invocations of what will be in the run() method, I feel this will help.
So is there anyway to get the program to wait until the threads have finished before continuing with the while loop?
The easiest solution is to use the Futures to notify you when they have completed. Unfortunately, Java does not support listenable Futures out of the box, but you can use the Guava library to supplement you here.
Guava adds the ListeneableFuture, which you can make using the Futures utility class:
ListeningExecutorService executor = MoreExecutors.listeningDecorator(threadPool);
// Collect the futures as you add them to the threadpool
List<ListenableFuture<?>> futures = new ArrayList<>();
while (! queue.IsEmpty())
{
currentItem = queue.GetNextItem();
for (int i = 0; i < currentItem.destinations.GetNoOfItems(); i++)
{
// NeighbourThread should be a Runnable and not a Thread!
futures.add(executor.submit(new NeighbourThread(currentItem, allVertices, routetype, pqOpen, i, endLocation)));
}
}
// Get notified when they're all done (doesn't imply success!)
Futures.allAsList(futures)).addListener(new Runnable() {
// When this callback is executed, then everything has finished
}, MoreExecutors.directExecutor());
Alternatively, you could do this with a CountdownLatch if you know how many items you need to run upfront.
I want to run several threads and join them at the end of my main method, so I can know when they have finished and process some info.
I don't want to put my threads in an array and do a join() one by one as join is a blocking method and I stay would waiting in the main thread for some threads still running, while other threads may have already finished, without having a possibility of knowing.
I have thought on the possibility of implementing an observer pattern for my threads: An interface with a update() method, an abstract class extending from thread (or implementing runnable) with set and get methods for the listeners and a class starting all my threads and waiting them to finish.
If my understanding is right, an observer would not block in a specific join() for a thread. Instead it will wait somehow until an update() method is called by a thread to perform an action. In this case, the update() should be called right after the thread finishes.
I'm clueless on how to implement this. I've tried with similar models, but I don't know how to use the observer/listener to wake/block my main thread. I've used this old post as a template: How to know if other threads have finished? but I can't find a way to wake my main method once a thread calls the update() method. There will be only one observer object instantiated for all threads.
Could you think of a way to use an observer pattern to wait for all threads to finish without blocking main with one by one join() calls? Any other suggestion to solve this problem would be greatly appreciated. Thanks in advance.
Java already has an API to do that: a CompletionService.
A service that decouples the production of new asynchronous tasks from the consumption of the results of completed tasks. Producers submit tasks for execution. Consumers take completed tasks and process their results in the order they complete.
I think you don't need an observer pattern. Thread waiting for any results will have to block, otherwise it will finish or loop in infinity. You can use some kind of BlockingQueue - producers will add result of computation to the blocking queue (then finish) and main thread will just receive these results blocking when there's not any result yet..
Good news for you, it's already implemented :) Great mechanism of CompletionService and Executors framework. Try this:
private static final int NTHREADS = 5;
private static final int NTASKS = 100;
private static final ExecutorService exec = Executors.newFixedThreadPool(NTHREADS);
public static void main(String[] args) throws InterruptedException {
final CompletionService<Long> ecs = new ExecutorCompletionService<Long>(exec);
for (final int i = 0; i < NTASKS ; ++i) {
Callable<Long> task = new Callable<Long>() {
#Override
public Long call() throws Exception {
return i;
}
};
ecs.submit(task);
}
for (int i = 0; i < NTASKS; ++i) {
try {
long l = ecs.take().get();
System.out.print(l);
} catch (ExecutionException e) {
e.getCause().printStackTrace();
}
}
exec.shutdownNow();
exec.awaitTermination(50, TimeUnit.MILLISECONDS);
}
Sounds to me like you are looking for something like the Counting Completion Service recently discussed by Dr. Heinz M. Kabutz.
So say that I have 10 things to run, but I can only have 3 threads running at a time.
ArrayList<NewThread> threads = new ArrayList<NewThread>();
for(int i = 1; i < args.length; i++) {
NewThread t = new NewThread(args[i]);
threads.add(newThread);
if( (i%3) == 0) {
for (NewThread nt : threads) {
nt.join();
}
threads.clear();
}
}
The class NewThreads implements Runnable. I thought the join() method would work to make it wait for the threads to finish before looping around again and kicking off the next batch of threads, but instead I get a stack overflow exception. I think I am implementing join() incorrectly, but I am unsure how to do it. I currently am doing it as
public void join() {
this.join();
}
in my NewThread class. Any suggestions on how to get this working or a better way to go about it?
You are implementins or overriding join to call itself endlessly
public void join() {
this.join(); // call myself until I blow up.
}
The simplest solution is to use Thread.join() already there, but a better solution is to use a fixed size thread pool so you don't have to start and stop threads which can waste a lot of time and code.
You can use an ExecutorService
ExecutorService es = Executors.newFixedThreadPool(3);
for(int i=0;i<10;i++)
es.submit(new Task(i));
This is just a simple mistake.
Remove the method
public void join() {
this.join();
}
This method calls itself again and again.
NewThread should extend Thread.
Or 2nd way:
keep the method and call
Thread.currentThread.join();
The rest looks fine.