I have a thread with the following form:
each execution of each thread is supposed to run a function in the class. That function is completely safe to run by itself. The function returns a value, say an int.
After all threads have been executed, the function values need to be accumulated.
So, it goes (in pseudo-code) something like that:
a = 0
for each i between 1 to N
spawn a thread independently and call the command v = f(i)
when thread finishes, do safely: a = a + v
end
I am not sure how to use Java in that case.
The problem is not creating the thread, I know this can be done using
new Thread() {
public void run() {
...
}
}
the problem is accumulating all the answers.
Thanks for any info.
I would probably do something like:
public class Main {
int a = 0;
int[] values;
int[] results;
public Main() {
// Init values array
results = new int[N];
}
public int doStuff() {
LinkedList<Thread> threads = new LinkedList<Thread>();
for (final int i : values) {
Thread t = new Thread() {
public void run() {
accumulate(foo(i));
}
};
threads.add(t);
t.start();
}
for (Thread t : threads) {
try {
t.join();
} catch (InterruptedException e) {
// Act accordingly, maybe ignore?
}
}
return a;
}
synchronized void accumulate(int v) {
// Synchronized because a += v is actually
// tmp = a + v;
// a = tmp;
// which can cause a race condition AFAIK
a += v;
}
}
Use an ExecutorCompletionService, Executor, and Callable.:
Start with a Callable that calls your int function:
public class MyCallable implements Callable<Integer> {
private final int i;
public MyCallable(int i) {
this.i = i;
}
public Integer call() {
return Integer.valueOf(myFunction(i));
}
}
Create an Executor:
private final Executor executor = Executors.newFixedThreadPool(10);
10 is the maximum number of threads to execute at once.
Then wrap it in an ExecutorCompletionService and submit your jobs:
CompletionService<Integer> compService = new ExecutionCompletionService<Integer>(executor);
// Make sure to track the number of jobs you submit
int jobCount;
for (int i = 0; i < n; i++) {
compService.submit(new MyCallable(i));
jobCount++;
}
// Get the results
int a = 0;
for (int i = 0; i < jobCount; i++) {
a += compService.take().get().intValue();
}
ExecutorCompletionService allows you to pull tasks off of a queue as they complete. This is a little different from joining threads. Although the overall outcome is the same, if you want to update a UI as the threads complete, you won't know what order the threads are going to complete using a join. That last for loop could be like this:
for (int i = 0; i < jobCount; i++) {
a += compService.take().get().intValue();
updateUi(a);
}
And this will update the UI as tasks complete. Using a Thread.join won't necessarily do this since you'll be getting the results in the order that you call the joins, not the order that the threads complete.
Through the use of the executor, this will also allow you to limit the number of simultaneous jobs you're running at a given time so you don't accidentally thread-bomb your system.
Related
I need to execute a single task by multiple threads, such that when the first thread finishes and before any other thread finishes, all the threads are stopped and start the same task all over again. This should be performed n times.
My attempt is using Callable<V> and the method invokeAny() (that is why I use the set) but not sure how to accomplish the goal.
ExecutorService executor = Executors.newFixedThreadPool(10);
Callable<String> task = () -> {
someTask();
return "";
};
Set<Callable<String>> tasks = new HashSet<>();
IntStream.range(0, n).forEach(i -> {
tasks.add(task);
executor.submit(task);
});
How to finish this? or any better solution?
Here's one suggestion:
class Task implements Callable<Integer> {
private final static Random RND = new Random();
#Override
public Integer call() throws Exception {
try {
// Work on task for a random duration
Thread.sleep(RND.nextInt(5000));
} catch (InterruptedException e) {
System.err.println("I was interrupted."
+ "Someone else probably solved the task before me.");
return -1;
}
// Return some dummy value
return RND.nextInt();
}
}
class Scratch {
public static void main(String[] args) throws InterruptedException {
final int numWorkers = 3; // number of tasks to run in parallel
ExecutorService executor = Executors.newFixedThreadPool(numWorkers);
// Solve task 5 times. (Change it to while (true) { ...} if you like.)
for (int i = 0; i < 5; i++) {
CompletionService<Integer> completionService =
new ExecutorCompletionService<>(executor);
Future<?>[] futures = new Future<?>[numWorkers];
for (int j = 0; j < numWorkers; j++) {
futures[j] = completionService.submit(new Task());
}
Future<Integer> firstToComplete = completionService.take();
try {
Integer result = firstToComplete.get();
System.err.println("We got a result: " + result);
} catch (ExecutionException e) {
// Should not happen. Future has completed.
}
// Cancel all futures (it doesn't matter that we're cancelling
// the one that has already completed).
for (int j = 0; j < numWorkers; j++) {
futures[j].cancel(true);
}
}
executor.shutdown();
}
}
If the task you're solving does not respond to interrupts, passing true to cancel(...) won't help. In that case I'd suggest you do the following changes:
Create an AtomicBoolean done variable in the outer for loop.
Pass this to the constructor to Task and save it in a field in Task.
In the task solving process, check done flag ever so often, and cancel the attempt if done is true.
Instead of calling cancel on the tasks after the first result is in, set done to true and wait for the other threads to return.
I wrote the below code trying to run two threads for calling a function in a for loop, but the results have the same time as if I ran it sequentially without multiple threads. Any thoughts why the multithreading here is not working? Is there a better way to do it? Like for example if I wanted to have 10 threads, using my code this will mean I have to create 10 duplicate run() functions when creating the thread, I wonder if there is an easier way to set the number of threads? Also is it possible to create a number of threads depending on the loop counter so that each loop a thread is created to finish it so if I had 10 loops then 10 threads will run concurrently to finish the processing very fast?
private Thread t1 = new Thread(){
public void run(){
for (int i = 0; i < 2; i++)
{
try {
myfn(i);
} catch (IOException e) {
e.printStackTrace();
}
}
}
};
private Thread t2 = new Thread(){
public void run(){
for (int i = 2; i < 4; i++)
{
try {
myfn(i);
} catch (IOException e) {
e.printStackTrace();
}
}
}
};
public Results getResults() throws IOException, SocketTimeoutException {
t1.start();
t2.start();
try {
t1.join(0);
} catch (InterruptedException e) {
e.printStackTrace();
}
try {
t2.join(0);
} catch (InterruptedException e) {
e.printStackTrace();
}
For running the same task across multiple threads, you're probably looking for a thread pool. Java provides a ThreadPoolExecutor for this.
Here is an introduction to Java concurrency with the following example:
ExecutorService executor = Executors.newFixedThreadPool(1);
Future<Integer> future = executor.submit(() -> {
try {
TimeUnit.SECONDS.sleep(2);
return 123;
}
catch (InterruptedException e) {
throw new IllegalStateException("task interrupted", e);
}
});
future.get(1, TimeUnit.SECONDS);
That example specifically creates a pool with only a single thread, but the parameter to Executors.newFixedThreadPool controls how many threads will be used.
I'm not sure from your original question why you think two threads aren't being utilized.
public class MyThead extend Thread{
private int initValue = 0;
private int upperBound = 0;
public MyThread(int init, int ub){
this.initValue = init;
this.upperBound = ub;
}
public void run(){
for(int i = init; i < upperBound; i++){
myfn(i);
}
}
}
Create threads and start them:
List<Thread> threads = new ArrayList<>();
threads.add(new MyThread(0,2));
threads.add(new MyThread(2,4));
for(Thread t: threads){
t.start()
}
for(Thread t: threads){
t.join();
}
I wrote the below code trying to run two threads for calling a function in a for loop, but the results have the same time as if I ran it sequentially without multiple threads.
There are many reasons why that can happen although it's hard to know what is going on without seeing the myfn(...) code. Here are some possible reasons:
It could be that myfn runs so quickly that running it in different threads isn't going to be any faster.
It could be that myfn is waiting on some other resource in which case the threads can't really run concurrently.
It could be that myfn is blocking on IO (network or disk) and even though you are doing 2 (or more) of them at a time, the disk or the remote server can't handle the increased requests any faster.
Is there a better way to do it? Like for example if I wanted to have 10 threads, using my code this will mean I have to create 10 duplicate run() functions...
The right thing to do here is to create your own class which takes the lower and upper bounds. The right way to do this is to implement Runnable, not extend Thread. Something like:
public class MyRunnable implements Runnable {
private final int start;
private final int end;
public MyRunnable(int start, int end) {
this.start = start;
this.end = end;
}
public void run() {
for (int i = start; i < end; i++) {
myfn(i);
}
}
}
You can then either start the threads by hand or use an ExecutorService which makes the thread maintenance a lot easier:
// this will start a new thread for every job
ExecutorService threadPool = Executors.newCachedThreadPool();
threadPool.submit(new MyRunnable(0, 2));
threadPool.submit(new MyRunnable(2, 4));
// once you've submitted your last task, you shutdown the pool
threadPool.shutdown();
// then we wait until all of the tasks have run
threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
You don't need to copy your threads / loop 10 times, just take the logic and use it appropriately.
public class ExampleThread extends Thread {
private final int start, iterations;
public ExampleThread(int start, int iterations) {
this.start = start;
this.iterations = iterations;
}
#Override public void run() {
for (int i = 0; i < iterations; i++) {
myfn(start + i);
}
}
}
int iterations = 2;
List<Thread> threads = new ArrayList<>();
for (int threadId = 0; threadId < 10; threadId++) {
threads.add(new ExampleThread(threadId * iterations, iterations));
}
threads.forEach(Thread::start);
threads.forEach(t -> {
try {
t.join(0);
} catch (Exception e) {
e.printStackTrace(System.err);
}
});
I have a loop for example for (int i=1;i<=10;i++) and I want to create 10 threads inside it, each one to perform the same task on a set of data and return the result. Then process this result inside the loop. Any ideas how this can be done?
for (int i=1;i<=10;i++) {
Work w = new Work();
Thread t = new Thread(w);
w.getResultFromThread();
//process w
}
class Work implements Runnable {
public void run() {
//perform tasks
}
public int getResultFromThread() {
return result;
}
}
I want each thread to work in parallel but when I recieve the result to be one by one.
If you do not wish to use executors, the you can do it in the following way:
int size = 10;
Thread[] threads = new Thread[size];
Work[] works = new Work[size];
for (int i = 1; i <= size; i++) {
Work w = new Work();
works[i - 1] = w;
Thread t = new Thread(w);
threads[i - 1] = t;
// Start the thread
t.start();
}
// now you have started all the threads
for (int i = 0; i < size; i++) {
// wait for each thread to complete execution, before extracting the result
// [here i am assuming that getResultFromThread() does not block
// till we get the result, if it blocks, then no need to join]
threads[i].join();
int result = works[i].getResultFromThread();
// do something with the result
}
ArrayList<Work> listOfJobs = new ArrayList<Work>();
ArrayList<Thread> threadList = new ArrayList<Thread>();
for(int i = 0 ; i < 10; i++) {
Work w = new Work();
listOfJobs.add(w);
Thread t = new Thread(w);
threadList.add(t);
t.start();
}
for(Thread t : listOfJobs) {
t.join();
w.getResultsFromThread();
}
This way you don't need executors for whatever reason you might have not to use them.
First, you create all the threads and start them(1st loop), then you call join on them, to make sure thread you want to get results from is done(2nd loop).
You can always pass your listOfJobs to some other method to process your results.
I am new to Java and trying to write a method that finds the maximum value in a 2D array of longs.
The method searches through each row in a separate thread, and the threads maintain a shared current maximal value. Whenever a thread finds a value larger than its own local maximum, it compares this value with the shared local maximum and updates its current local maximum and possibly the shared maximum as appropriate. I need to make sure that appropriate synchronization is implemented so that the result is correct regardless of how to computations interleave.
My code is verbose and messy, but for starters, I have this function:
static long sharedMaxOf2DArray(long[][] arr, int r){
MyRunnableShared[] myRunnables = new MyRunnableShared[r];
for(int row = 0; row < r; row++){
MyRunnableShared rr = new MyRunnableShared(arr, row, r);
Thread t = new Thread(rr);
t.start();
myRunnables[row] = rr;
}
return myRunnables[0].sharedMax; //should be the same as any other one (?)
}
For the adapted runnable, I have this:
public static class MyRunnableShared implements Runnable{
long[][] theArray;
private int row;
private long rowMax;
public long localMax;
public long sharedMax;
private static Lock sharedMaxLock = new ReentrantLock();
MyRunnableShared(long[][] a, int r, int rm){
theArray = a;
row = r;
rowMax = rm;
}
public void run(){
localMax = 0;
for(int i = 0; i < rowMax; i++){
if(theArray[row][i] > localMax){
localMax = theArray[row][i];
sharedMaxLock.lock();
try{
if(localMax > sharedMax)
sharedMax = localMax;
}
finally{
sharedMaxLock.unlock();
}
}
}
}
}
I thought this use of a lock would be a safe way to prevent multiple threads from messing with the sharedMax at a time, but upon testing/comparing with a non-concurrent maximum-finding function on the same input, I found the results to be incorrect. I'm thinking the problem might come from the fact that I just say
...
t.start();
myRunnables[row] = rr;
...
in the sharedMaxOf2DArray function. Perhaps a given thread needs to finish before I put it in the array of myRunnables; otherwise, I will have "captured" the wrong sharedMax? Or is it something else? I'm not sure on the timing of things..
I'm not sure if this is a typo or not, but your Runnable implementation declares sharedMax as an instance variable:
public long sharedMax;
rather than a shared one:
public static long sharedMax;
In the former case, each Runnable gets its own copy and will not "see" the values of others. Changing it to the latter should help. Or, change it to:
public long[] sharedMax; // array of size 1 shared across all threads
and you can now create an array of size one outside the loop and pass it in to each Runnable to use as shared storage.
As an aside: please note that there will be tremendous lock contention since every thread checks the common sharedMax value by holding a lock for every iteration of its loop. This will likely lead to poor performance. You'd have to measure, but I'd surmise that letting each thread find the row maximum and then running a final pass to find the "max of maxes" might actually be comparable or quicker.
From JavaDocs:
public interface Callable
A task that returns a result and may
throw an exception. Implementors define a single method with no
arguments called call.
The Callable interface is similar to Runnable, in that both are
designed for classes whose instances are potentially executed by
another thread. A Runnable, however, does not return a result and
cannot throw a checked exception.
Well, you can use Callable to calculate your result from one 1darray and wait with an ExecutorService for the end. You can now compare each result of the Callable to fetch the maximum. The code may look like this:
Random random = new Random(System.nanoTime());
long[][] myArray = new long[5][5];
for (int i = 0; i < 5; i++) {
myArray[i] = new long[5];
for (int j = 0; j < 5; j++) {
myArray[i][j] = random.nextLong();
}
}
ExecutorService executor = Executors.newFixedThreadPool(myArray.length);
List<Future<Long>> myResults = new ArrayList<>();
// create a callable for each 1d array in the 2d array
for (int i = 0; i < myArray.length; i++) {
Callable<Long> callable = new SearchCallable(myArray[i]);
Future<Long> callResult = executor.submit(callable);
myResults.add(callResult);
}
// This will make the executor accept no new threads
// and finish all existing threads in the queue
executor.shutdown();
// Wait until all threads are finish
while (!executor.isTerminated()) {
}
// now compare the results and fetch the biggest one
long max = 0;
for (Future<Long> future : myResults) {
try {
max = Math.max(max, future.get());
} catch (InterruptedException | ExecutionException e) {
// something bad happend...!
e.printStackTrace();
}
}
System.out.println("The result is " + max);
And your Callable:
public class SearchCallable implements Callable<Long> {
private final long[] mArray;
public SearchCallable(final long[] pArray) {
mArray = pArray;
}
#Override
public Long call() throws Exception {
long max = 0;
for (int i = 0; i < mArray.length; i++) {
max = Math.max(max, mArray[i]);
}
System.out.println("I've got the maximum " + max + ", and you guys?");
return max;
}
}
Your code has serious lock contention and thread safety issues. Even worse, it doesn't actually wait for any of the threads to finish before the return myRunnables[0].sharedMax which is a really bad race condition. Also, using explicit locking via ReentrantLock or even synchronized blocks is usually the wrong way of doing things unless you're implementing something low level (eg your own/new concurrent data structure)
Here's a version that uses the Future concurrent primitive and an ExecutorService to handle the thread creation. The general idea is:
Submit a number of concurrent jobs to your ExecutorService
Add the Future returned backed from submit(...) to a List
Loop through the list calling get() on each Future and aggregating the result
This version has the added benefit that there is no lock contention (or locking in general) between the worker threads as each just returns back the max for its slice of the array.
import java.util.concurrent.*;
import java.util.*;
public class PMax {
public static long pmax(final long[][] arr, int numThreads) {
ExecutorService pool = Executors.newFixedThreadPool(numThreads);
try {
List<Future<Long>> list = new ArrayList<Future<Long>>();
for(int i=0;i<arr.length;i++) {
// put sub-array in a final so the inner class can see it:
final long[] subArr = arr[i];
list.add(pool.submit(new Callable<Long>() {
public Long call() {
long max = Long.MIN_VALUE;
for(int j=0;j<subArr.length;j++) {
if( subArr[j] > max ) {
max = subArr[j];
}
}
return max;
}
}));
}
// find the max of each slice's max:
long max = Long.MIN_VALUE;
for(Future<Long> future : list) {
long threadMax = future.get();
System.out.println("threadMax: " + threadMax);
if( threadMax > max ) {
max = threadMax;
}
}
return max;
} catch( RuntimeException e ) {
throw e;
} catch( Exception e ) {
throw new RuntimeException(e);
} finally {
pool.shutdown();
}
}
public static void main(String args[]) {
int x = 1000;
int y = 1000;
long max = Long.MIN_VALUE;
long[][] foo = new long[x][y];
for(int i=0;i<x;i++) {
for(int j=0;j<y;j++) {
long r = (long)(Math.random() * 100000000);
if( r > max ) {
// save this to compare against pmax:
max = r;
}
foo[i][j] = r;
}
}
int numThreads = 32;
long pmax = pmax(foo, numThreads);
System.out.println("max: " + max);
System.out.println("pmax: " + pmax);
}
}
Bonus: If you're calling this method repeatedly then it would probably make sense to pull the ExecutorService creation out of the method and have it be reused across calls.
Well, that definetly is an issue - but without more code it is hard to understand if it is the only thing.
There is basically a race condition between the access of thread[0] (and this read of sharedMax) and the modification of the sharedMax in other threads.
Think what happens if the scheduler decides to let no let any thread run for now - so when you are done creating the threads, you will return the answer without modifying it even once! (of course there are other possible scenarios...)
You can overcome it by join()ing all threads before returning an answer.
I have very similar multithreaded code elsewhere in my codebase that works fine, but I can't see quite what's going wrong here.
This is a simple multi-threaded process to generate some result XML for a search query. The output of running this method is:
Returning from threads
The line System.out.println("Finished multithreading loop");" is never reached.
Modifying the number of threads doesn't help.
private void fillAllResults() {
int threads = 2;
final FutureTask[] tasks = new FutureTask[threads];
final ExecutorService executor = Executors.newCachedThreadPool();
for (int i = 0; i < allResults.size(); i++) {
tasks[i] = new FutureTask<Integer>(new Callable<Integer>() {
public Integer call() throws Exception {
int index;
while ((index = getResultsIndex()) < allResults.size()) {
System.out.println("Processing result " + index);
Result result = allResults.get(index);
fillResultXML(result);
}
System.out.println("Returning from threads");
return 1;
}
});
executor.execute(tasks[i]);
}
for (int i = 0; i < threads; i++) {
try {
tasks[i].get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
executor.shutdown();
System.out.println("Finished multithreading loop");
}
Edit, thanks all for the quick replies! Here's the answers:
It shows 'processing result' as many times as I have results. If allResults.size() is 25, it shows processing result 1, processing result 2 ... processing result 24.
Here's the extra code that's missing:
private List<Result> allResults = new ArrayList<Result>();
private int resultsIndex = 0;
private synchronized int getResultsIndex() {
return resultsIndex++;
}
And in case anyone's wondering, I can guarantee that none of the code within the loop increases the size of allResults.
I suppose it is related to the fact, that your array tasks has a length of threads (i.e. two in your case) but you assign more values to it within the lines
for (int i = 0; i < allResults.size(); i++) {
tasks[i] = ...
....
}
If your list allResults has more than two entries your thread will be stopped by an ArrayIndexOutOfBoundsException. Maybe you catch this one but do not handle it properly outside the code you presented.
It looks like getResultsIndex() isn't updating after every loop resulting in an infinite loop.
It's not clear from your code what allResults and getResultsIndex are, but you never seem to update whatever getResultsIndex returns.