I am creating program to run a logic in parallel with help of thread. The logic has to be done in a loop of a list and that list may contain 1000 rows to process , so i started each thread in a loop and once thread size reaches 5, i will call thread.join for all the 5 started thread.
So i think that this will start the 2nd set of 5 threads only after the first set of 5 threads are completed? is my understanding correct? i am kind of new to threads.
So my need is to start the 6th thread when any one of the previous set of 5 threads in completed. or start 6th and 7th when any 2 in the previous set of threads are completed.
My code
public static void executeTest(){
for (int i = 0; i < rows1; i++) {
RunnableInterface task = new New RunnableInterface(params);
Thread thread = new Thread(task);
thread.start();
threads.add(thread);
if ((threads.size() % 5 == 0) || ((i == rows1-1) && (threads.size() < 5))) {
waitForThreads(threads);
}
}
}
private static void waitForThreads(List<Thread> threads) {
for (Thread thread : threads) {
try {
thread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
threads.clear();
}
What kind of modification do i need to do to achieve above mentioned results
You can use an ExecutorService with unbounded input queue and fixed number of threads like this:
ExecutorService exec = Executors.newFixedThreadPool(5);
// As many times as needed:
for(int i = 0; i< 100_000; i++) {
Runnable myrunnable = () -> {}; // new RunnableInterface(params)
exec.submit(myrunnable);
}
exec.shutdown();
exec.awaitTermination(1, TimeUnit.DAYS);
In JDK19 the executor service is AutoClosable, so you can simplify to:
try (ExecutorService exec = Executors.newFixedThreadPool(5)) {
...
}
When in doubt, break it down into smaller, simpler functions:
public static void executeTest(){
for (int i = 0; i < rows1; i++) {
startFiveThreads(i);
awaitFiveThreads(i);
}
}
private static void startFiveThreads(int i) {
...
}
private static void awaitFiveThreads(int i) {
...
}
But seriously? I'm giving you a literal answer to your question—a better way to implement your solution. #DuncG showed you a better solution for your problem. Definitely go with what DuncG said, but maybe remember what I said for next time you are trying to implement some tricky/complex algorithm.
I am fetching thousands of URLs and parsing tables from it using parseDocument(doc). It would take hours to parse all tables, so I wanted to use threads to parse a lot of them at the same time, but don't know to do it.
The code below is a for loop that I need to use threads on:
for(int i = 0; i < urlList.size(); i++) {
Document doc = Jsoup.connect(urlList.get(i)).get();
reader.add(parseDocument(doc));
}
for(int i = 0; i < urlList.size(); i++) {
Thread t = new Thread(new Runnable() {
public void run() {
try {
//parsedocument( urlList.get(i))
} catch (Exception e) { //Catching exeptions
e.printStackTrace();
}
}
});
t.start();
}
We create a thread variable called 't' that we could use. For example, can wait for it to finish using .Join(). A thread requires a run void where the actual operations will execute, this is because it uses something called "runnable" which is an interface that needs that Run method. Here you would parse the document. Finally, we start the thread by call t.start().
So let's say I'm creating and starting a bunch of threads in a for loop, that is being executed in the run method of a launcher thread. Let's also say that I want to be able to interrupt the launcher thread and all threads that the thread has created, and I do this through a button.
So something like this -
try{
for(int i = 0; i < n;i++){
Worker currThread = new Worker(someArgs);
workerThreads.add(currThread);
currThread.start();
}
} catch (InterruptedException e){
e.printStackTrace();
}
BUTTON-
public void actionPerformed(ActionEvent arg0) {
List<Worker> threads = launchThread.getWorkerThreads();
for(int i = 0; i < threads.size();i++){
threads.get(i).interrupt();
}
launchThread.interrupt();
}
Now, let's say that I want to make it so that the interrupts cannot occur at the same time as thread creation. I think a way to do this would be to construct a dummy object and put both pieces of code inside a lock
synchronized(dummyObject){
//thread creation or interruption code here (shown above)
}
Will this way work? I ask because I'm not sure how to test to see if it will.
Start the threads separately from creating them.
for(int i = 0; i < n; i++) {
Worker currThread = new Worker(someArgs);
workerThreads.add(currThread);
}
// later
for (Worker w : workerThreads) {
w.start();
}
If that's still not enough, your dummyObject synchronization should work just fine.
// You probably need to make this a (private final) field
Object lock = new Object();
// later
synchronized (lock) {
for(int i = 0; i < n; i++) {
Worker currThread = new Worker(someArgs);
workerThreads.add(currThread);
w.start();
}
}
// later still
public void actionPerformed(ActionEvent arg0) {
synchronized (lock) {
// interruption code here
}
}
The concept of synchronization remains the same however complicated are the underlying operations to be executed.
As you specified, there are two types of mutually exclusive tasks (thread creation and interruption). So locking is pretty much the canonical tool for the job.
I'm writing a Java app which writes to excel sheet bunch of data, and it takes a while to do so.
I'd like to create something like writing out dots to screen like on Linux when you're installing something.
Is that possible in java?printing dots, while other thread actually does the writing to excel, then after its finished the one displaying dots also quits?
I'd like to print dots to console.
A variation to #John V. answer would be to use a ScheduledExecutorService:
// SETUP
Runnable notifier = new Runnable() {
public void run() {
System.out.print(".");
}
};
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
// IN YOUR WORK THREAD
scheduler.scheduleAtFixedRate(notifier, 1, 1, TimeUnit.SECONDS);
// DO YOUR WORK
schedule.shutdownNow();
Modify the notifier object to suit your individual needs.
Its very possible. Use a newSingleThreadExecutor to print the dots while the other thread does the parsing. For example
ExecutorService e = Executors.newSingleThreadExecutor();
Future f = e.submit(new Runnable(){
public void run(){
while(!Thread.currentThread().isInterrupted()){
Thread.sleep(1000); //exclude try/catch for brevity
System.out.print(".");
}
}
});
//do excel work
f.cancel(true);
e.shutdownNow();
Yes, it is possible, you will want to have your working thread set a variable to indicate that it is working and when it is finished. Then create a thread by either extending the Thread class or implementing the Runnable interface. This thread should just infinitely loop and inside this loop it should do whatever printing you want it to do, and then check the variable to see if the work is done. When the variable value changes, break the loop and end the thread.
One note. Watch your processing speed. Use Thread.sleep() inside your loop if your processor usage goes way high. This thread should not be labour intensive. System.gc() is another popular way to make threads wait.
Not an elegant solution, but get's the job done. It prints 1, 2, 3, 1, 2... dots in a loop and terminates everything after 5 seconds.
public class Busy {
public Busy() {
Indicator i = new Indicator();
ExecutorService ex = Executors.newSingleThreadExecutor();
ex.submit(i);
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
i.finished = true;
ex.shutdown();
}
public static void main(String[] args) {
new Busy();
}
private class Indicator implements Runnable {
private static final int DOTS_NO = 3;
private volatile boolean finished = false;
#Override
public void run() {
for (int i = 0; !finished; i = (i + 1) % (DOTS_NO + 1)) {
for (int j = 0; j < i; j++) {
System.out.print('.');
}
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
for (int j = 0; j < i; j++) {
System.out.print("\b \b");
}
}
}
}
}
What is a way to simply wait for all threaded process to finish? For example, let's say I have:
public class DoSomethingInAThread implements Runnable{
public static void main(String[] args) {
for (int n=0; n<1000; n++) {
Thread t = new Thread(new DoSomethingInAThread());
t.start();
}
// wait for all threads' run() methods to complete before continuing
}
public void run() {
// do something here
}
}
How do I alter this so the main() method pauses at the comment until all threads' run() methods exit? Thanks!
You put all threads in an array, start them all, and then have a loop
for(i = 0; i < threads.length; i++)
threads[i].join();
Each join will block until the respective thread has completed. Threads may complete in a different order than you joining them, but that's not a problem: when the loop exits, all threads are completed.
One way would be to make a List of Threads, create and launch each thread, while adding it to the list. Once everything is launched, loop back through the list and call join() on each one. It doesn't matter what order the threads finish executing in, all you need to know is that by the time that second loop finishes executing, every thread will have completed.
A better approach is to use an ExecutorService and its associated methods:
List<Callable> callables = ... // assemble list of Callables here
// Like Runnable but can return a value
ExecutorService execSvc = Executors.newCachedThreadPool();
List<Future<?>> results = execSvc.invokeAll(callables);
// Note: You may not care about the return values, in which case don't
// bother saving them
Using an ExecutorService (and all of the new stuff from Java 5's concurrency utilities) is incredibly flexible, and the above example barely even scratches the surface.
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class DoSomethingInAThread implements Runnable
{
public static void main(String[] args) throws ExecutionException, InterruptedException
{
//limit the number of actual threads
int poolSize = 10;
ExecutorService service = Executors.newFixedThreadPool(poolSize);
List<Future<Runnable>> futures = new ArrayList<Future<Runnable>>();
for (int n = 0; n < 1000; n++)
{
Future f = service.submit(new DoSomethingInAThread());
futures.add(f);
}
// wait for all tasks to complete before continuing
for (Future<Runnable> f : futures)
{
f.get();
}
//shut down the executor service so that this thread can exit
service.shutdownNow();
}
public void run()
{
// do something here
}
}
instead of join(), which is an old API, you can use CountDownLatch. I have modified your code as below to fulfil your requirement.
import java.util.concurrent.*;
class DoSomethingInAThread implements Runnable{
CountDownLatch latch;
public DoSomethingInAThread(CountDownLatch latch){
this.latch = latch;
}
public void run() {
try{
System.out.println("Do some thing");
latch.countDown();
}catch(Exception err){
err.printStackTrace();
}
}
}
public class CountDownLatchDemo {
public static void main(String[] args) {
try{
CountDownLatch latch = new CountDownLatch(1000);
for (int n=0; n<1000; n++) {
Thread t = new Thread(new DoSomethingInAThread(latch));
t.start();
}
latch.await();
System.out.println("In Main thread after completion of 1000 threads");
}catch(Exception err){
err.printStackTrace();
}
}
}
Explanation:
CountDownLatch has been initialized with given count 1000 as per your requirement.
Each worker thread DoSomethingInAThread will decrement the CountDownLatch, which has been passed in constructor.
Main thread CountDownLatchDemo await() till the count has become zero. Once the count has become zero, you will get below line in output.
In Main thread after completion of 1000 threads
More info from oracle documentation page
public void await()
throws InterruptedException
Causes the current thread to wait until the latch has counted down to zero, unless the thread is interrupted.
Refer to related SE question for other options:
wait until all threads finish their work in java
Avoid the Thread class altogether and instead use the higher abstractions provided in java.util.concurrent
The ExecutorService class provides the method invokeAll that seems to do just what you want.
Consider using java.util.concurrent.CountDownLatch. Examples in javadocs
Depending on your needs, you may also want to check out the classes CountDownLatch and CyclicBarrier in the java.util.concurrent package. They can be useful if you want your threads to wait for each other, or if you want more fine-grained control over the way your threads execute (e.g., waiting in their internal execution for another thread to set some state). You could also use a CountDownLatch to signal all of your threads to start at the same time, instead of starting them one by one as you iterate through your loop. The standard API docs have an example of this, plus using another CountDownLatch to wait for all threads to complete their execution.
As Martin K suggested java.util.concurrent.CountDownLatch seems to be a better solution for this. Just adding an example for the same
public class CountDownLatchDemo
{
public static void main (String[] args)
{
int noOfThreads = 5;
// Declare the count down latch based on the number of threads you need
// to wait on
final CountDownLatch executionCompleted = new CountDownLatch(noOfThreads);
for (int i = 0; i < noOfThreads; i++)
{
new Thread()
{
#Override
public void run ()
{
System.out.println("I am executed by :" + Thread.currentThread().getName());
try
{
// Dummy sleep
Thread.sleep(3000);
// One thread has completed its job
executionCompleted.countDown();
}
catch (InterruptedException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}.start();
}
try
{
// Wait till the count down latch opens.In the given case till five
// times countDown method is invoked
executionCompleted.await();
System.out.println("All over");
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
If you make a list of the threads, you can loop through them and .join() against each, and your loop will finish when all the threads have. I haven't tried it though.
http://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#join()
Create the thread object inside the first for loop.
for (int i = 0; i < threads.length; i++) {
threads[i] = new Thread(new Runnable() {
public void run() {
// some code to run in parallel
}
});
threads[i].start();
}
And then so what everyone here is saying.
for(i = 0; i < threads.length; i++)
threads[i].join();
You can do it with the Object "ThreadGroup" and its parameter activeCount:
As an alternative to CountDownLatch you can also use CyclicBarrier e.g.
public class ThreadWaitEx {
static CyclicBarrier barrier = new CyclicBarrier(100, new Runnable(){
public void run(){
System.out.println("clean up job after all tasks are done.");
}
});
public static void main(String[] args) {
for (int i = 0; i < 100; i++) {
Thread t = new Thread(new MyCallable(barrier));
t.start();
}
}
}
class MyCallable implements Runnable{
private CyclicBarrier b = null;
public MyCallable(CyclicBarrier b){
this.b = b;
}
#Override
public void run(){
try {
//do something
System.out.println(Thread.currentThread().getName()+" is waiting for barrier after completing his job.");
b.await();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}
}
To use CyclicBarrier in this case barrier.await() should be the last statement i.e. when your thread is done with its job. CyclicBarrier can be used again with its reset() method. To quote javadocs:
A CyclicBarrier supports an optional Runnable command that is run once per barrier point, after the last thread in the party arrives, but before any threads are released. This barrier action is useful for updating shared-state before any of the parties continue.
The join() was not helpful to me. see this sample in Kotlin:
val timeInMillis = System.currentTimeMillis()
ThreadUtils.startNewThread(Runnable {
for (i in 1..5) {
val t = Thread(Runnable {
Thread.sleep(50)
var a = i
kotlin.io.println(Thread.currentThread().name + "|" + "a=$a")
Thread.sleep(200)
for (j in 1..5) {
a *= j
Thread.sleep(100)
kotlin.io.println(Thread.currentThread().name + "|" + "$a*$j=$a")
}
kotlin.io.println(Thread.currentThread().name + "|TaskDurationInMillis = " + (System.currentTimeMillis() - timeInMillis))
})
t.start()
}
})
The result:
Thread-5|a=5
Thread-1|a=1
Thread-3|a=3
Thread-2|a=2
Thread-4|a=4
Thread-2|2*1=2
Thread-3|3*1=3
Thread-1|1*1=1
Thread-5|5*1=5
Thread-4|4*1=4
Thread-1|2*2=2
Thread-5|10*2=10
Thread-3|6*2=6
Thread-4|8*2=8
Thread-2|4*2=4
Thread-3|18*3=18
Thread-1|6*3=6
Thread-5|30*3=30
Thread-2|12*3=12
Thread-4|24*3=24
Thread-4|96*4=96
Thread-2|48*4=48
Thread-5|120*4=120
Thread-1|24*4=24
Thread-3|72*4=72
Thread-5|600*5=600
Thread-4|480*5=480
Thread-3|360*5=360
Thread-1|120*5=120
Thread-2|240*5=240
Thread-1|TaskDurationInMillis = 765
Thread-3|TaskDurationInMillis = 765
Thread-4|TaskDurationInMillis = 765
Thread-5|TaskDurationInMillis = 765
Thread-2|TaskDurationInMillis = 765
Now let me use the join() for threads:
val timeInMillis = System.currentTimeMillis()
ThreadUtils.startNewThread(Runnable {
for (i in 1..5) {
val t = Thread(Runnable {
Thread.sleep(50)
var a = i
kotlin.io.println(Thread.currentThread().name + "|" + "a=$a")
Thread.sleep(200)
for (j in 1..5) {
a *= j
Thread.sleep(100)
kotlin.io.println(Thread.currentThread().name + "|" + "$a*$j=$a")
}
kotlin.io.println(Thread.currentThread().name + "|TaskDurationInMillis = " + (System.currentTimeMillis() - timeInMillis))
})
t.start()
t.join()
}
})
And the result:
Thread-1|a=1
Thread-1|1*1=1
Thread-1|2*2=2
Thread-1|6*3=6
Thread-1|24*4=24
Thread-1|120*5=120
Thread-1|TaskDurationInMillis = 815
Thread-2|a=2
Thread-2|2*1=2
Thread-2|4*2=4
Thread-2|12*3=12
Thread-2|48*4=48
Thread-2|240*5=240
Thread-2|TaskDurationInMillis = 1568
Thread-3|a=3
Thread-3|3*1=3
Thread-3|6*2=6
Thread-3|18*3=18
Thread-3|72*4=72
Thread-3|360*5=360
Thread-3|TaskDurationInMillis = 2323
Thread-4|a=4
Thread-4|4*1=4
Thread-4|8*2=8
Thread-4|24*3=24
Thread-4|96*4=96
Thread-4|480*5=480
Thread-4|TaskDurationInMillis = 3078
Thread-5|a=5
Thread-5|5*1=5
Thread-5|10*2=10
Thread-5|30*3=30
Thread-5|120*4=120
Thread-5|600*5=600
Thread-5|TaskDurationInMillis = 3833
As it's clear when we use the join:
The threads are running sequentially.
The first sample takes 765 Milliseconds while the second sample takes 3833 Milliseconds.
Our solution to prevent blocking other threads was creating an ArrayList:
val threads = ArrayList<Thread>()
Now when we want to start a new thread we most add it to the ArrayList:
addThreadToArray(
ThreadUtils.startNewThread(Runnable {
...
})
)
The addThreadToArray function:
#Synchronized
fun addThreadToArray(th: Thread) {
threads.add(th)
}
The startNewThread funstion:
fun startNewThread(runnable: Runnable) : Thread {
val th = Thread(runnable)
th.isDaemon = false
th.priority = Thread.MAX_PRIORITY
th.start()
return th
}
Check the completion of the threads as below everywhere it's needed:
val notAliveThreads = ArrayList<Thread>()
for (t in threads)
if (!t.isAlive)
notAliveThreads.add(t)
threads.removeAll(notAliveThreads)
if (threads.size == 0){
// The size is 0 -> there is no alive threads.
}
The problem with:
for(i = 0; i < threads.length; i++)
threads[i].join();
...is, that threads[i + 1] never can join before threads[i].
Except the "latch"ed ones, all solutions have this lack.
No one here (yet) mentioned ExecutorCompletionService, it allows to join threads/tasks according to their completion order:
public class ExecutorCompletionService<V>
extends Object
implements CompletionService<V>
A CompletionService that uses a supplied Executor to execute tasks. This class arranges that submitted tasks are, upon completion, placed on a queue accessible using take. The class is lightweight enough to be suitable for transient use when processing groups of tasks.
Usage Examples.
Suppose you have a set of solvers for a certain problem, each returning a value of some type Result, and would like to run them concurrently, processing the results of each of them that return a non-null value, in some method use(Result r). You could write this as:
void solve(Executor e, Collection<Callable<Result>> solvers) throws InterruptedException, ExecutionException {
CompletionService<Result> cs = new ExecutorCompletionService<>(e);
solvers.forEach(cs::submit);
for (int i = solvers.size(); i > 0; i--) {
Result r = cs.take().get();
if (r != null)
use(r);
}
}
Suppose instead that you would like to use the first non-null result of the set of tasks, ignoring any that encounter exceptions, and cancelling all other tasks when the first one is ready:
void solve(Executor e, Collection<Callable<Result>> solvers) throws InterruptedException {
CompletionService<Result> cs = new ExecutorCompletionService<>(e);
int n = solvers.size();
List<Future<Result>> futures = new ArrayList<>(n);
Result result = null;
try {
solvers.forEach(solver -> futures.add(cs.submit(solver)));
for (int i = n; i > 0; i--) {
try {
Result r = cs.take().get();
if (r != null) {
result = r;
break;
}
} catch (ExecutionException ignore) {}
}
} finally {
futures.forEach(future -> future.cancel(true));
}
if (result != null)
use(result);
}
Since: 1.5 (!)
Assuming use(r) (of Example 1) also asynchronous, we had a big advantage. #