I am new to Java concurrency, and I met a very strange problem:
I read from a large file and used several worker threads to work on the input (some complicated string matching tasks). I used a LinkedBlockingQueue to transmit the data to the worker threads, and a volatile boolean flag in the worker class to respond to the signal when the end-of-file is reached.
However, I cannot get the worker thread to stop properly. The CPU usage by this program is almost zero in the end, but the program won't terminate normally.
The simplified code is below. I have removed the real code and replaced them with a simple word counter. But the effect is the same. The worker thread won't terminate after the whole file is processed, and the boolean flag is set to true in the main thread.
The class with main
public class MultiThreadTestEntry
{
private static String inputFileLocation = "someFile";
private static int numbOfThread = 3;
public static void main(String[] args)
{
int i = 0;
Worker[] workers = new Worker[numbOfThread];
Scanner input = GetIO.getTextInput(inputFileLocation);
String temp = null;
ExecutorService es = Executors.newFixedThreadPool(numbOfThread);
LinkedBlockingQueue<String> dataQueue = new LinkedBlockingQueue<String>(1024);
for(i = 0 ; i < numbOfThread ; i ++)
{
workers[i] = new Worker(dataQueue);
workers[i].setIsDone(false);
es.execute(workers[i]);
}
try
{
while(input.hasNext())
{
temp = input.nextLine().trim();
dataQueue.put(temp);
}
}
catch (InterruptedException e)
{
Thread.currentThread().interrupt();
}
input.close();
for(i = 0 ; i < numbOfThread ; i ++)
{
workers[i].setIsDone(true);
}
es.shutdown();
try
{
es.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e)
{
Thread.currentThread().interrupt();
}
}
}
The worker class
public class Worker implements Runnable
{
private LinkedBlockingQueue<String> dataQueue = null;
private volatile boolean isDone = false;
public Worker(LinkedBlockingQueue<String> dataQueue)
{
this.dataQueue = dataQueue;
}
#Override
public void run()
{
String temp = null;
long count = 0;
System.out.println(Thread.currentThread().getName() + " running...");
try
{
while(!isDone || !dataQueue.isEmpty())
{
temp = dataQueue.take();
count = temp.length() + count;
if(count%1000 == 0)
{
System.out.println(Thread.currentThread().getName() + " : " + count);
}
}
System.out.println("Final result: " + Thread.currentThread().getName() + " : " + count);
}
catch (InterruptedException e)
{
}
}
public void setIsDone(boolean isDone)
{
this.isDone = isDone;
}
}
Any suggestions to why this happens?
Thank you very much.
As Dan Getz already said your worker take() waits until an element becomes available but the Queue may be empty.
In your code you check if the Queue is empty but nothing prevents the other Workers to read and remove an element from the element after the check.
If the Queue contains only one element and t1 and t2 are two Threads
the following could happen:
t2.isEmpty(); // -> false
t1.isEmpty(); // -> false
t2.take(); // now the queue is empty
t1.take(); // wait forever
in this case t1 would wait "forever".
You can avoid this by using pollinstead of take and check if the result is null
public void run()
{
String temp = null;
long count = 0;
System.out.println(Thread.currentThread().getName() + " running...");
try
{
while(!isDone || !dataQueue.isEmpty())
{
temp = dataQueue.poll(2, TimeUnit.SECONDS);
if (temp == null)
// re-check if this was really the last element
continue;
count = temp.length() + count;
if(count%1000 == 0)
{
System.out.println(Thread.currentThread().getName() + " : " + count);
}
}
System.out.println("Final result: " + Thread.currentThread().getName() + " : " + count);
}
catch (InterruptedException e)
{
// here it is important to restore the interrupted flag!
Thread.currentThread().interrupt();
}
}
Related
my exercise is composed of an SharedResource with an array, a NumberGenerator Class and a SumClass.
2 Threads of NumberGenerator and 2 Threads of SumClass.
I have to insert numbers in the array from the SharedResource with both threads of NumberGenerator Class.
This part is done correctly.
My problem is in the run method of the NumberGenerator Class.
These Threads must read alternatively the index of the array, the first thread, lets call it H1, read the index 0-2-4-6... even index, the another one, H2, odd index... But when 1 thread is running the other must wait.
So the output should be something like this:
Array:[10,35,24,18]
Thread name: H1 - Number:10
Thread name: H2 - Number:35
Thread name: H1 - Number:24
Thread name: H2 - Number:18
My problem resides in the notify and wait methods. When I call the wait method, it automatically stop both Threads even when a notifyAll is present.
I need to just use 1 method, I cant use a evenPrint method and an OddPrint method for example.
Code of my class:
public class SumatoriaThread extends Thread{
String name;
NumerosCompartidos compartidos; // Object that contains the array.
double sumatoria; //Parameter used to sum all the numbers
static int pos = 0; // pos of the object
static int threadOrder = 0; //just a way to set the order of the Threads.
int priority; //Value of the priority
public SumatoriaThread(NumerosCompartidos compartidos, String name){
this.name = name;
this.compartidos = compartidos;
sumatoria = 0;
priority = threadOrder;
threadOrder++;
System.out.println("Hilo " + this.name + " creado.");
}
public void run() {
//Array Length
int length = this.compartidos.length();
int i = 0;
while (pos < length) {
//Don't actually know if this is correct.
synchronized (this) {
//Just to be sure that the first Thread run before the second one.
if (priority == 1) {
try {
priority = 0;
//Call a wait method until the next notify runs.
this.wait();
} catch (InterruptedException ex) {
Logger.getLogger(SumatoriaThread.class.getName()).log(Level.SEVERE, null, ex);
}
} else {
//Even numbers
if (pos % 2 == 0) {
System.out.println("Nombre Thead: " + Thread.currentThread().getName() + " valor numero: " + this.compartidos.getValor(pos));
pos++;
//To activate the Thread which is waiting.
this.notifyAll();
try {
//Then I set to wait this one.
this.wait();
} catch (Exception ex) {
ex.printStackTrace();
}
} else {
System.out.println("Nombre Thead: " + Thread.currentThread().getName() + " valor numero: " + this.compartidos.getValor(pos + 1));
pos++;
this.notifyAll();
try {
this.wait();
} catch (InterruptedException ex) {
Logger.getLogger(SumatoriaThread.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
}
}
}
I want fibonacci series to be printed by threads and the 1st number of the series should be printed by 1st thread then 2nd number by 2nd thread then 3rd by 1st thread and 4th by 2nd and so on.
I tried this code by using arrays like printing the array elements using thread but I am not able to switch between the threads.
class Fibonacci{
void printFibonacci() {
int fibArray[] = new int[10];
int a = 0;
int b = 1;
fibArray[0] = a;
fibArray[1] = b;
int c;
for(int i=2;i<10;i++) {
c = a+b;
fibArray[i] = c;
a = b;
b = c;
}
for(int i=0;i<10;i++) {
if(Integer.parseInt(Thread.currentThread().getName())%2==0 && (i%2==0))
{
System.out.println("Thread " +Thread.currentThread().getName()+" "+fibArray[i]);
try{
wait();
}catch(Exception e) {}
}
else if(Integer.parseInt(Thread.currentThread().getName())%2!=0 && (i%2!=0))
{
System.out.println("Thread " +Thread.currentThread().getName()+" "+fibArray[i]);
}
}
}
}
public class FibonacciUsingThread {
public static void main(String[] args) throws Exception {
Fibonacci f = new Fibonacci();
Thread t1 = new Thread(()->
{
f.printFibonacci();
});
Thread t2 = new Thread(()->
{
f.printFibonacci();
});
t1.setName("0");
t2.setName("1");
t1.start();
t1.join();
t2.start();
}
}
The following line in your code is causing t1 to finish before t2 can start.
t1.join();
Apart from this, you need to synchronize on the method, printFibonacci.
You can do it as follows:
class Fibonacci {
synchronized void printFibonacci() throws InterruptedException {
int fibArray[] = new int[10];
int a = 0;
int b = 1;
fibArray[0] = a;
fibArray[1] = b;
int c;
for (int i = 2; i < 10; i++) {
c = a + b;
fibArray[i] = c;
a = b;
b = c;
}
for (int i = 0; i < 10; i++) {
String currentThreadName = Thread.currentThread().getName();
if (currentThreadName.equals("1")) {
if (i % 2 == 0) {
System.out.println("Thread " + Thread.currentThread().getName() + " " + fibArray[i]);
notify();
} else {
wait();
}
} else if (currentThreadName.equals("0")) {
if (i % 2 == 1) {
System.out.println("Thread " + Thread.currentThread().getName() + " " + fibArray[i]);
notify();
} else {
wait();
}
}
}
}
}
public class Main {
public static void main(String[] args) {
Fibonacci f = new Fibonacci();
Thread t1 = new Thread(() -> {
try {
f.printFibonacci();
} catch (InterruptedException e) {
e.printStackTrace();
}
});
Thread t2 = new Thread(() -> {
try {
f.printFibonacci();
} catch (InterruptedException e) {
e.printStackTrace();
}
});
t1.setName("0");
t2.setName("1");
t1.start();
t2.start();
}
}
Output:
Thread 1 0
Thread 0 1
Thread 1 1
Thread 0 2
Thread 1 3
Thread 0 5
Thread 1 8
Thread 0 13
Thread 1 21
Thread 0 34
Apart from all being said and already answered, I would just like to add one alternative approach to Fibonacci sequence implemetation, without arrays and in-advance dimensioning:
public class Fibonacci {
private int index = -1;
private int previous = 0;
private int last = 1;
synchronized public int getNext() {
index++;
if( index == 0 ) return previous;
if( index == 1 ) return last;
int next = last + previous;
if( next < 0 ) throw new ArithmeticException( "integer overflow" );
previous = last;
last = next;
return next;
}
}
Limited only by overflow of numeric data type, in this case integer.
As "#Live and Let Live" pointed out, correctness-wise the main issues with your code is the missing synchronized clause and calling join of the first thread before starting the second thread.
IMO you could clean the code a bit by first separating a bite the concerns, namely, the class Fibonacci would only responsible for calculation the Fibonacci of a given array:
class Fibonacci{
void getFibonacci(int[] fibArray) {
int a = 0;
int b = 1;
fibArray[0] = a;
fibArray[1] = b;
int c;
for(int i=2;i<fibArray.length;i++) {
c = a+b;
fibArray[i] = c;
a = b;
b = c;
}
}
}
In this way, you keep your Fibonacci class concise without any thread-related code. Moreover, the getFibonacci is now more abstract; you can calculate the fib of more than just 10 elements like you had before.
Then on the class FibonacciUsingThread:
public class FibonacciUsingThread {
public static void main(String[] args) throws Exception {
int [] array_fib = new int[10];
Fibonacci f = new Fibonacci();
f.getFibonacci(array_fib);
Thread t1 = new Thread(()->
{
for(int i = 0; i < array_fib.length; i+=2)
System.out.println("Thread 1:" + array_fib[i]);
});
Thread t2 = new Thread(()->
{
for(int i = 1; i < array_fib.length; i+=2)
System.out.println("Thread 2:" + array_fib[i]);
});
t1.start();
t2.start();
t1.join();
t2.join();
}
}
First, you calculate the Fibonaccis using the main thread, there is no point in having all the threads calculate the same thing. Afterward, you specified that Thread 1 and Thread 2 will print the even and odd positions, respectively.
Unless this is just an exercise to play around with threads and synchronization there is not much sense in using threads to do this kind of work. In your code, the part worth parallelizing is the calculation of the Fibonacci numbers themselves, not the printing part.
The code previously shown will not print the Fibonacci numbers in order, for that you need to ensure that the threads wait for one another after iterating through each element of the array. Hence, you need to adapt the code that will be executed by the threads, namely:
Thread t1 = new Thread(()->
{
synchronized (array_fib){
for(int i = 0; i < array_fib.length; i++)
if(i % 2 == 0) {
System.out.println("Thread 1:" + array_fib[i]);
try {
array_fib.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
else
array_fib.notify();
}
});
Thread t2 = new Thread(()->
{
synchronized (array_fib){
for(int i = 0; i < array_fib.length; i++)
if(i % 2 != 0) {
System.out.println("Thread 2:" + array_fib[i]);
try {
array_fib.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
else
array_fib.notify();
}
});
We can remove the code redundancy by extracting a method with the work that will be assigned to the Threads. For instance:
private static void printFib(String threadName, int[] array_fib, Predicate<Integer> predicate) {
for (int i = 0; i < array_fib.length; i++)
if (predicate.test(i)) {
System.out.println(threadName + " : " + array_fib[i]);
try {
array_fib.wait();
} catch (InterruptedException e) {
// do something about it
}
} else
array_fib.notify();
}
and the main code:
public static void main(String[] args) throws Exception{
int [] array_fib = new int[10];
Fibonacci f = new Fibonacci();
f.getFibonacci(array_fib);
Thread t1 = new Thread(()-> {
synchronized (array_fib){
printFib("Thread 1:", array_fib, i1 -> i1 % 2 == 0);
}
});
Thread t2 = new Thread(()-> {
synchronized (array_fib){
printFib("Thread 2:", array_fib, i1 -> i1 % 2 != 0);
}
});
t1.start();
t2.start();
t1.join();
t2.join();
}
As an alternative, you can use a fair Semaphore to alternate between threads, and an AtomicReference to keep the shared status. Here's an example:
import java.util.concurrent.Semaphore;
import java.util.concurrent.atomic.AtomicReference;
public class FibonacciConcurrent {
public static void main(String[] args) throws InterruptedException {
// needs to be fair to alternate between threads
Semaphore semaphore = new Semaphore(1, true);
// set previous to 1 so that 2nd fibonacci number is correctly calculated to be 0+1=1
Status initialStatus = new Status(1, 0, 1);
AtomicReference<Status> statusRef = new AtomicReference<>(initialStatus);
Fibonacci fibonacci = new Fibonacci(20, semaphore, statusRef);
Thread thread1 = new Thread(fibonacci);
Thread thread2 = new Thread(fibonacci);
thread1.start();
thread2.start();
thread1.join();
thread2.join();
}
private static final class Status {
private final long previous;
private final long current;
private final int currentIndex;
private Status(long previous, long current, int currentIndex) {
this.previous = previous;
this.current = current;
this.currentIndex = currentIndex;
}
}
private static final class Fibonacci implements Runnable {
private final int target;
private final Semaphore semaphore;
private final AtomicReference<Status> statusRef;
private Fibonacci(int target, Semaphore semaphore, AtomicReference<Status> statusRef) {
this.target = target;
this.semaphore = semaphore;
this.statusRef = statusRef;
}
#Override
public void run() {
try {
process();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException("Interrupted", e);
}
}
private void process() throws InterruptedException {
while (!Thread.currentThread().isInterrupted()) {
try {
semaphore.acquire();
Status status = statusRef.get();
String threadName = Thread.currentThread().getName();
if (status.currentIndex > target) return;
System.out.println(
threadName + ": fibonacci number #" + status.currentIndex + " - " + status.current);
long next = status.previous + status.current;
Status newStatus = new Status(status.current, next, status.currentIndex + 1);
statusRef.set(newStatus);
} finally {
semaphore.release();
}
}
}
}
}
Will print:
Thread-0: fibonacci number #1 - 0
Thread-1: fibonacci number #2 - 1
Thread-0: fibonacci number #3 - 1
Thread-1: fibonacci number #4 - 2
Thread-0: fibonacci number #5 - 3
Note that this solution does not only print on the threads - it does the actual calculation on the threads as well - e.g. when it's Thread A's turn, it uses the previous status that was calculated by Thread B to calculate the next fibonacci number.
Sometimes the varible total will equal something else when run instead of 50005000 it's allways short like 50005001 when sometimes run why is this happening shouldn't synchronized(this) create a lock which can only be updated once the lock is released by the thread?
import java.util.concurrent.atomic.AtomicLong;
public class CurrentThread {
public static AtomicLong c = new AtomicLong(0L);
public static AtomicLong total = new AtomicLong(0L);
public static void main(String[] args) {
Thread t = Thread.currentThread();
System.out.println(t);
t.setName("My Thread");
System.out.println(t);
for (int x = 0; x < 10; x++) {
System.out.println("Instance " + x);
new Thread(new Updater(x, "Thread: " + String.valueOf(x))).start();
}
try {
Thread.sleep(1000);
}
catch (InterruptedException e) {
}
}
}
class Updater implements Runnable {
public int na;
private String threadName;
public Updater(int n, String threadName) {
this.na = n;
this.threadName = threadName;
}
#Override
public void run() {
this.updateCount();
if(CurrentThread.total.get() == 50005000) {
System.out.println("Passed");
}
else {
System.out.println("Failed");
}
}
public void updateCount() {
while (CurrentThread.c.get() < 10000) {
synchronized(this) {
CurrentThread.c.getAndIncrement();
CurrentThread.total.addAndGet(CurrentThread.c.get());
System.out.println(this.threadName + " " + String.valueOf(CurrentThread.c.get()) + " " + CurrentThread.total.get() + " " + System.nanoTime());
}
}
}
}
You're synchronizing on this, which is effectively not synchronizing at all, since each thread has a different Runnable instance.
Synchronize on something shared between all Updater instances instead, e.g. Updater.class.
Note, however, that synchronizing on AtomicLong is a bit of a code smell - it's meant to do things atomically already.
You can use compareAndSet instead, and avoid synchronizing entirely, e.g.:
while (CurrentThread.c.get() < 10000) {
while (true) {
long currValue = CurrentThread.c.get();
if (currValue >= 10000) break;
long newValue = currValue + 1;
// Only sets c to newValue if its value is still currValue.
if (CurrentThread.c.compareAndSet(currValue, newValue)) {
long total = CurrentThread.total.addAndGet(newValue);
System.out.println(
this.threadName + " " + newValue + " " + total + " " + System.nanoTime());
break;
}
}
}
Note that this makes use of "known" values, like newValue and total, rather than getting them again from the AtomicLong.
How to create a cyclic exchange of three threads? That is: first thread must send data to second, second to third and third thread must send data to first.
I wrote some code, but threads exchange in random oder.
class DataClass {
int value;
String message;
DataClass(int v, String s) {
value = v;
message = s;
}
int getValue() {
return (value);
}
String getMassage() {
return (message);
}
}
class Loop implements Runnable {
int counter;
String name;
Exchanger<DataClass> exchanger;
Loop(int startValue, String id, Exchanger<DataClass> ex) {
counter = startValue;
name = id;
exchanger = ex;
System.out.println(name + ": created");
}
public void run() {
System.out.println(name + ": started");
DataClass data = new DataClass(counter, name);
for (int i = 0; i < 3; ++i) {
try {
DataClass newData = exchanger.exchange(data);
counter += newData.getValue();
System.out.println(name + ": from "
+ newData.getMassage() + ": data: "
+ newData.getValue() + ": state = " + counter);
} catch (InterruptedException e) {
System.err.println(e.toString());
}
}
System.out.println(name + ": ended");
}
}
public class ExchangerDemo {
public static void main(String args[]) {
System.out.println("Main process started");
Exchanger<DataClass> exchanger = new Exchanger<DataClass>();
Loop loop1 = new Loop(1, "First", exchanger);
Loop loop2 = new Loop(2, "Second", exchanger);
Loop loop3 = new Loop(3, "Third", exchanger);
new Thread(loop1).start();
new Thread(loop2).start();
new Thread(loop3).start();
System.out.println("Main process ended");
}
}
For your dependency you should make three classes, and have three distinct Exchange objects (one in each). So thread1 would be between 1 and 2 (output of 1 to 2), thread 2's would be between 2 and 3 and thread 3's exhanger would be between itself and 1. Remember the exchanger's would guard only until it had its input from its feeder, to till it passes to its receiver.
Also synchronized is not as bad as the books make out. use it. Watch http://www.youtube.com/watch?v=WTVooKLLVT8
Also for reference Best way of running two threads alternatively?
Also why do you need three threads? Can you use a thread pool and have each task to the 3 things ?
I have a standard producer consumer problem. Producer puts data into the stack(buffer) consumers take it.
I would like to have many producers and consumers.
the problem is I would like to make only the last living producer to be able to call b.stop()
for(int i = 0; i < 10; i++){
try{
// sleep((int)(Math.random() * 1));
}catch(Exception e){e.printStackTrace();}
b.put((int) (Math.random()* 10));
System.out.println("i = " + i);
}
b.stop();
so then I call b.stop() which changes running field in Buffer to false and notifiesAll()
End then I get:
i = 9 // number of iteration this is 10th iteration
Consumer 2.: no data to take. I wait. Memory: 0
Consumer 1.: no data to take. I wait. Memory: 0
Consumer 3.: no data to take. I wait. Memory: 0
they should die then, so I made method stop() but it did not work.
Code is running please check it
import java.util.Stack;
public class Buffer {
private static int SIZE = 4;
private int i;//number of elements in buffer
public Stack<Integer> stack;
private volatile boolean running;
public Buffer() {
stack = new Stack<>();
running = true;
i = 0;
}
synchronized public void put(int val){
while (i >= SIZE) {
try {
System.out.println("Buffer full, producer waits");
wait();
} catch (InterruptedException exc) {
exc.printStackTrace();
}
}
stack.push(val);//txt = s;
i++;
System.out.println("Producer inserted " + val + " memory: " + i);
if(i - 1 == 0)
notifyAll();
System.out.println(stack);
}
public synchronized Integer get(Consumer c) {
while (i == 0) {
try {
System.out.println(c + ": no data to take. I wait. Memory: " + i);
wait();
} catch (InterruptedException exc) {
exc.printStackTrace();
}
}
if(running){
int data = stack.pop();
i--;
System.out.println(c+ ": I took: " + data +" memory: " + i);
System.out.println(stack);
if(i + 1 == SIZE){//if the buffer was full so the producer is waiting
notifyAll();
System.out.println(c + "I notified producer about it");
}
return data;}
else
return null;
}
public boolean isEmpty(){
return i == 0;
}
public synchronized void stop(){//I THOUGH THIS WOULD FIX IT~!!!!!!!!!!!!!!
running = false;
notifyAll();
}
public boolean isRunning(){
return running;
}
}
public class Producer extends Thread {
private Buffer b;
public Producer(Buffer b) {
this.b = b;
}
public void run(){
for(int i = 0; i < 10; i++){
try{
// sleep((int)(Math.random() * 1));
}catch(Exception e){e.printStackTrace();}
b.put((int) (Math.random()* 10));
System.out.println("i = " + i);
}
b.stop();
}
}
public class Consumer extends Thread {
Buffer b;
int nr;
static int NR = 0;
public Consumer(Buffer b) {
this.b = b;
nr = ++NR;
}
public void run() {
Integer i = b.get(this);
while (i != null) {
System.out.println(nr + " I received : " + i);
i = b.get(this);
}
System.out.println("Consumer " + nr + " is dead");
}
public String toString() {
return "Consumer " + nr + ".";
}
}
public class Main {
public static void main(String[] args) {
Buffer b = new Buffer();
Producer p = new Producer(b);
Consumer c1 = new Consumer(b);
Consumer c2 = new Consumer(b);
Consumer c3 = new Consumer(b);
p.start();
c1.start();c2.start();c3.start();
}
}
What you have to realise is that your threads could be waiting in either of two locations:
In the wait loop with i == 0 - in which case notifyall will kick all of them out. However, if i is still 0 they will go straight back to waiting again.
Waiting for exclusive access to the object (i.e. waiting on a synchronized method) - in which case (if you fix issue 1 above and the lock will be released) they will go straight into a while (i == 0) loop.
I would suggest you change your while ( i == 0 ) loop to while ( running && i == 0 ). This should fix your problem. Since your running flag is (correctly) volatile all should tidily exit.
In your stop method, you set running to false, but your while loop is running as long as i == 0. Set i to something different than zero and it should fix it.
BTW, I don't understand why you have a running variable and a separate i variable, which is actually the variable keeping a thread running.
I would rethink your design. Classes should have a coherent set of responsibilities; making a class responsible for both consuming objects off the queue, while also being responsible for shutting down other consumers, seems to be something you'd want to seperate.
In answer to the to make only the last living producer to be able to call b.stop().
You should add an AtomicInteger to your Buffer containing the number of producers and make each producer call b.start() (which increments it) in its constructor.
That way you can decrement it in b.stop() and only when it has gone to zero should running be set to false.