Develop Reference

Confusion about Blocking Queue and Multi-thread in Java

I was learning Producer-Consumer pattern with multiple threads.
The basic idea is one producer thread will put data into the Blocking Queue, and then many consumer threads will take data from the queue.
The consumer's "run()" method is shown below.
#Override
public void run() {
List<String> record;
while(true) {
if(this.channel.getState()) {
break;
}
try {
// System.out.println(this.channel.getSharedQueue().size());
record = this.channel.getSharedQueue().take();
...
} catch (InterruptedException e) {
e.printStackTrace();
}
}
record = null;
}
When I comment out the "System.out.println(...)", the whole program will get stuck.
However, if "System.out.println(...)" is added, the program will run smoothly.
Can someone help explain the magic behind "System.out.println(...)" ?
Here is the producer's "run()" method.
public void run() {
...
while ((line = br.readLine()) != null && line.length() > 1) {
List<String> record = parseRow(line);
if(record != null)
this.channel.getSharedQueue().put(record);
}
...
this.channel.setState(true);
}
And the main method.
ExecutorService producerExecutor = Executors.newSingleThreadExecutor();
Runnable producer = new Producer(channel, ...);
producerExecutor.execute(producer);
ExecutorService consumerExecutor = Executors.newFixedThreadPool(30);
for(int i = 0; i < 30; i++) {
Runnable consumer = new Consumer(channel);
consumerExecutor.execute(consumer);
}
consumerExecutor.shutdown();
producerExecutor.shutdown();
while (!consumerExecutor.isTerminated() || !producerExecutor.isTerminated());
The Channel class is like
public class Channel {
private volatile boolean state;
...
public boolean getState() {
return this.state;
}
public void setState(boolean state) {
this.state = state;
}
...
}
Thank you for your help!

Java multiple thread join issue

So i need to process a couple data files using threads (already splitted), and i'm having issues on how to stop the main thread till all the subthreads finish.
i looked around and tried to use join() but this causes an issue:
If i join the main thread with the last thread then since the other threads run at the same time, the last thread is not always the last one to finish
If i join the main thread with all the other threads then they don't run at the same time, the second needs the first to finish first.
also tried wait() and notify() but had even more issues. here's a part of my code
public class Matrix extends MapReduce {
ArrayList<String> VecteurLines = new ArrayList<String>();
protected int[] nbrLnCol = {0,0};
protected static double[] res;
public Matrix(String n) {
super(n);
}
public Matrix(String n,String m){
super(n,m);
}
public void Reduce() throws IOException, InterruptedException, MatrixException {
for (int i = 1; i <= Chunks; i++) {
Thread t=new Thread(new RunThread(VecteurLines,i,this));
t.start();
}
}
And here's the class that handles the threads
public class RunThread extends Matrix implements Runnable {
Matrix ma;
ArrayList<String> vec;
int threadNbr;
public RunThread(ArrayList<String> vec, int threadNbr,Matrix ma) {
super("","");
this.vec=vec;this.threadNbr=threadNbr;this.ma=ma; }
#Override
public void run() {
FileInputStream fin = null;
try {
fin = new FileInputStream(ma.getNom()+threadNbr+".txt");
} catch (FileNotFoundException e) {
e.printStackTrace();
}
Scanner sc = new Scanner(fin);
while (sc.hasNext()) {
String nextString = sc.next();
ma.nbrLnCol[0]++;
String [] arr = nextString.split(",");
ma.nbrLnCol[1]=arr.length;
double c=0;
for(int j=0;j<arr.length;j++)
{
c+=(Double.parseDouble(arr[j])*Double.parseDouble(vec.get(j)));
}
res[threadNbr-1]=c;
}
sc.close();
try {
fin.close();
} catch (IOException e) {
e.printStackTrace();
}
File file = new File(ma.getNom()+threadNbr+".txt");
file.delete();
}

Try like this:
private List<Thread> threadList = new ArrayList<>();
public void Reduce() {
threadList.clear();
for (int i = 1; i <= Chunks; i++) {
Thread t =new Thread(new RunThread(VecteurLines,i,this));
threadList.add(t);
}
// start all worker threads
for(int i=0; i<threadList.size(); i++){
threadList.get(i).start();
}
// wait until all worker threads is finished
while (true) {
int threadIsNotLive = 0;
for (int i = 0; i < threadList.size(); i++) {
Thread t = threadList.get(i);
if (!t.isAlive() || t == null) {
++threadIsNotLive;
}
}
if(threadIsNotLive>0 && (threadList.size() == threadIsNotLive)){
break;
// all worker threads is finished
}
else {
Thread.sleep(50);
// wait until all worker threads is finished
}
}
}
OR
public void Reduce() {
List<Thread> threadList = new ArrayList<>();
for (int i = 1; i <= Chunks; i++) {
Thread t =new Thread(new RunThread(VecteurLines,i,this));
threadList.add(t);
}
// start all worker threads
for(int i=0; i<threadList.size(); i++){
threadList.get(i).start();
threadList.get(i).join();
}
}

I believe you need two points in your code:
Your main thread has to end last after all the thread's executed because you said
"how to stop the main thread till all the subthreads finish"
.
Second ,the thread should finish one after another that is the 2nd thread should finish after 1st thread as you said
"the second needs the first to finish first."
Here is my code to do it with join .
public class Matrix extends MapReduce {
ArrayList<String> VecteurLines = new ArrayList<String>();
protected int[] nbrLnCol = {0,0};
protected static double[] res;
public Matrix(String n) {
super(n);
}
public Matrix(String n,String m){
super(n,m);
}
public void Reduce() throws IOException, InterruptedException, MatrixException {
Thread t = null;
for (int i = 1; i <= Chunks; i++) {
Thread t=new Thread(new RunThread(t,VecteurLines,i,this));
t.start();
}
t.join(); // finally main thread joining with the last thread.
}
and
public class RunThread extends Matrix implements Runnable {
Matrix ma;
ArrayList<String> vec;
int threadNbr;
Thread t;
public RunThread(t,ArrayList<String> vec, int threadNbr,Matrix ma) {
this.t = t;
super("","");
this.vec=vec;this.threadNbr=threadNbr;this.ma=ma; }
#Override
public void run() {
FileInputStream fin = null;
try {
fin = new FileInputStream(ma.getNom()+threadNbr+".txt");
} catch (FileNotFoundException e) {
e.printStackTrace();
}
Scanner sc = new Scanner(fin);
while (sc.hasNext()) {
String nextString = sc.next();
ma.nbrLnCol[0]++;
String [] arr = nextString.split(",");
ma.nbrLnCol[1]=arr.length;
double c=0;
for(int j=0;j<arr.length;j++)
{
c+=(Double.parseDouble(arr[j])*Double.parseDouble(vec.get(j)));
}
res[threadNbr-1]=c;
}
sc.close();
try {
fin.close();
} catch (IOException e) {
e.printStackTrace();
}
File file = new File(ma.getNom()+threadNbr+".txt");
file.delete();
if(t!=null){
t.join(); //join with the previous thread eg. thread2 joining with thread1
}
}

Java - How to detect deadlocks and recovery from this?

Right now I write a Java program that has as purpose detect deadlocks and recovery from this situation. The program input is two numbers, N = Number of types of resources and M = Number of process.
I wanted to do something like this:
private static void test2() {
final ReentrantLock lock1 = new ReentrantLock();
final ReentrantLock lock2 = new ReentrantLock();
Thread thread1 = new Thread(new Runnable() {
#Override public void run() {
try {
lock1.lock();
System.out.println("Thread1 acquired lock1");
try {
TimeUnit.MILLISECONDS.sleep(50);
} catch (InterruptedException ignore) {}
lock2.lock();
System.out.println("Thread1 acquired lock2");
}
finally {
lock2.unlock();
lock1.unlock();
}
}
});
thread1.start();
Thread thread2 = new Thread(new Runnable() {
#Override public void run() {
try {
lock2.lock();
System.out.println("Thread2 acquired lock2");
try {
TimeUnit.MILLISECONDS.sleep(50);
} catch (InterruptedException ignore) {}
lock1.lock();
System.out.println("Thread2 acquired lock1");
}
finally {
lock1.unlock();
lock2.unlock();
}
}
});
thread2.start();
// Wait a little for threads to deadlock.
try {
TimeUnit.MILLISECONDS.sleep(100);
} catch (InterruptedException ignore) {}
detectDeadlock();
}
But instead of 2, N locks and I have several problems for doing this. Here my code with my attempt:
class Main {
private static int MAX_AVAILABLE = 10;
private static int IDLE = 1000;
public static void main(String[] args) throws java.lang.Exception{
int n, m; //number of resources and process, respectively
ReentrantLock[] resources; // Locks for resources
int[] available; // Number of instances per resource
Process[] processes; // Processes array
DeadlockDetector supervisor; // Deadlock detaction class
n = Integer.valueOf(args[0]);
m = Integer.valueOf(args[1]);
resources = new ReentrantLock[n];
available = new int[n];
processes = new Process[m];
supervisor = new DeadlockDetector();
// Create resources array
for(int i=0; i<n; ++i){
available[i] = (int)(Math.floor(Math.random()*MAX_AVAILABLE + 1));
resources[i] = new ReentrantLock();
System.out.println("R"+String.valueOf(i+1)+"-> instances: "+String.valueOf(available[i]));
}
// Creating processes
for(int i=0; i<m; ++i){
processes[i] = new Process(i, resources, available, n);
System.out.println("P"+String.valueOf(i+1)+"-> requested "+Arrays.toString(processes[i].requested));
processes[i].start();
}
//Run deadlock detection
try {
TimeUnit.MILLISECONDS.sleep(IDLE);
}catch (InterruptedException ignore){}
supervisor.start();
}
}
class Process extends Thread{
public int id;
public int total; // Total of resources instances needed for finished the process
public ReentrantLock[] resources;
public int[] requested; // Number of instances needed per resource type
public boolean[] needed; // Boolean indicating whether the process needs at least one instance of the resource i
public int n;
private static int MIN_TIME = 1000;
private static int MAX_TIME = 3000;
public Process(int index, ReentrantLock[] res, int[] available, int n_resources){
id = index;
n = n_resources;
resources = res;
total = 0;
requested = new int[n];
needed = new boolean[n];
for(int i=0; i<n; ++i){
requested[i] = (int)(Math.floor(Math.random()*available[i]));
needed[i] = requested[i] > 0;
total += requested[i];
}
}
#Override
public void run(){
int resourceT = 0;
int timeToSleep;
System.out.println("P"+String.valueOf(id+1)+" begin running");
try{
while(total > 0){
resourceT = (int)(Math.floor(Math.random()*n));
if(requested[resourceT] < 1){
System.out.println("P"+String.valueOf(id+1)+"-> I do not need more R"+String.valueOf(resourceT+1));
continue;
}
System.out.println("P"+String.valueOf(id+1)+"-> I'll take R"+String.valueOf(resourceT+1));
resources[resourceT].lock();
timeToSleep = (int)(Math.floor(Math.random()*(MAX_TIME - MIN_TIME)) + MIN_TIME);
try{
TimeUnit.MILLISECONDS.sleep(timeToSleep);
}catch (InterruptedException ignore){}
--total;
--requested[resourceT];
}
}finally{
for(int i=0; i<n; ++i){
if(needed[i] && resources[i].isHeldByCurrentThread())
resources[i].unlock();
}
}
System.out.println("P"+String.valueOf(id+1)+"-> Im finished");
}
}
class DeadlockDetector extends Thread{
public ThreadMXBean threadBean;
public long[] threadIds;
public DeadlockDetector(){
}
#Override
public void run(){
Boolean good;
this.threadBean = ManagementFactory.getThreadMXBean();
threadIds = threadBean.findDeadlockedThreads();
int deadlockedThreads = threadIds != null? threadIds.length : 0;
if(deadlockedThreads>1){
good = false;
System.out.println("Number of deadlocked threads: " + deadlockedThreads);
//recoverDeadlock();
//break;
}
}
public void recoverDeadlock(){
}
}
Please, could anyone help me fix this detail? Thanks!

Why do I get a deadlock?

I have a multithreading program, which sorts threads in order strs times. Every thread has its own monitor. One monitor of this thread (lock) and another monitor of the following thread (unlock) are passed to the constructor of each thread. First, when each thread starts, it must stop when array[0] != this, but if in I write this in line 13, the deadlock appears. So I use Threads.count, which is incremented every iterations. This way the program works. Could you tell me why this happens?
class Foo extends Thread
{
private Object lock, unlock;
Foo(Object lock, Object unlock)
{
this.lock = lock;
this.unlock = unlock;
}
public void run()
{
synchronized(lock)
{
if(Threads.array[Threads.count] != this) // line 13!!!
{
waiter();
}
for(int i = 0; i < Threads.strs; ++i)
{
if(Threads.array[0] == this)
{
System.out.println(i+1);
}
System.out.print(getName() + ' ');
++Threads.count;
if(Threads.array[Threads.thrs-1] == this)
{
System.out.println();
}
if(unlock != lock)
{
synchronized(unlock)
{
unlock.notify();
}
waiter();
}
}
}
}
void waiter()
{
try
{
lock.wait();
}
catch(InterruptedException e)
{
e.printStackTrace();
System.exit(1);
}
}
}
public class Threads
{
public static Thread array[];
public static Object lock[];
public static int count, strs, thrs;
public static void main(String args[])
{
thrs = 0;
strs = 0;
count = 0;
try
{
assert(args.length == 2);
thrs = Integer.parseInt(args[0]);
strs = Integer.parseInt(args[1]);
assert((thrs > 0) && (strs > 0));
}
catch(NumberFormatException | AssertionError e)
{
System.out.println("Uncorrect enter!");
System.exit(1);
}
lock = new Object[thrs];
array = new Thread[thrs];
for(int i = 0; i < thrs; ++i)
{
lock[i] = new Object();
}
for(int i = 0; i < thrs; ++i)
{
if(i != thrs-1)
{
array[i] = new Foo(lock[i],lock[i+1]);
}else
{
array[i] = new Foo(lock[i],lock[0]);
}
array[i].start();
}
}
}

Line 13 basically says "wait to get notified by a preceding thread, unless I am the first thread". Which makes sense: from what I can tell from the code, you want the threads to do their tasks one by one in the order that you have created the threads (which kind of defeats the purpose of using threads, but that is another story).
Also note that the program will not exit since all threads call waiter() at the end the loop.
So the solution is kind of straightforward: have all threads wait at the beginning of the loop, but after creating all threads, trigger the first thread to start running (which in turn will trigger the other threads to start running). Below a slightly adjusted copy of your code with the two changes I mentioned:
class ThreadsInSequence extends Thread
{
private Object lock, unlock;
ThreadsInSequence(Object lock, Object unlock)
{
this.lock = lock;
this.unlock = unlock;
}
public void run()
{
synchronized(lock)
{
for(int i = 0; i < strs; ++i)
{
waiter();
if(array[0] == this)
{
System.out.println(i+1);
}
System.out.print(getName() + ' ');
++count;
if(array[thrs-1] == this)
{
System.out.println();
}
if(unlock != lock)
{
synchronized(unlock)
{
unlock.notify();
}
}
}
}
}
void waiter()
{
try
{
lock.wait();
}
catch(InterruptedException e)
{
e.printStackTrace();
System.exit(1);
}
}
public static Thread array[];
public static Object locks[];
public static int count, strs, thrs;
public static void main(String args[])
{
thrs = 3;
strs = 6;
count = 0;
locks = new Object[thrs];
array = new Thread[thrs];
for(int i = 0; i < thrs; ++i)
{
locks[i] = new Object();
}
for(int i = 0; i < thrs; ++i)
{
if(i != thrs-1)
{
array[i] = new ThreadsInSequence(locks[i],locks[i+1]);
}else
{
array[i] = new ThreadsInSequence(locks[i],locks[0]);
}
array[i].start();
}
synchronized(locks[0]) {
locks[0].notify();
}
}
}

Updating UI-thread with handler

I have scoured the web to find definite examples of this but so far couldn't find one which I could have applied to my project.
I'm trying to create a worker-thread which is run every 100ms. It then should update UI with results. After some research I decided that I probably should use Handlers to manage the UI-updating. I came to this solution:
My activity's Handler:
private final Handler handler = new Handler() {
public void handleMessage(Message msg) {
String aResponse = msg.getData().getString("message");
if ((null != aResponse)) {
// ALERT MESSAGE
Log.v("udppacket", "UDP message!");
if (msg.obj != null)
{
ManagerUdpPacket p = (ManagerUdpPacket) msg.obj;
operatorListFragment.updateContent((int) p.getOperationTime());
}
}
else
{
}
}
};
My other class which has the worker-thread:
public class ManagerUdpReceiver
{
private int minPort = 1234;
private int maxPort = 1240;
private ArrayList<PortListener> portList;
private Handler handler;
private Thread portThread;
private int queryInterval = 100;
private boolean stop = false;
public ManagerUdpReceiver(int minport, int maxport, Handler handler, int queryInterval)
{
minPort = minport;
maxPort = maxport;
this.handler = handler;
this.queryInterval = queryInterval;
//create port listeners from given range and start their threads
start();
}
private void start()
{
stop = false;
// Create Inner Thread Class
portThread = new Thread(new Runnable()
{
// After call for background.start this run method call
public void run()
{
if (portList == null)
{
portList = new ArrayList<PortListener>();
for (int i = minPort; i < maxPort; i++)
{
portList.add(new PortListener(i));
}
}
if (!stop)
{
ManagerUdpPacket p = portList.get(0).receive();
threadMsg("moi", p);
//mHandler.postDelayed(this, queryInterval);
}
else
{
//stop execution and close ports
for (int i = 0; i < portList.size(); i++)
{
portList.get(i).close();
}
}
}
//send message to the handler
private void threadMsg(String msg, ManagerUdpPacket p)
{
if (!msg.equals(null) && !msg.equals(""))
{
Message msgObj = handler.obtainMessage();
//msgObj.obj = p;
Bundle b = new Bundle();
b.putString("message", msg);
msgObj.setData(b);
handler.sendMessage(msgObj);
}
}
});
// Start Thread
portThread.start();
}
public void close()
{
stop = true;
}
}
When I run the program I get exception about running networking code in UI-thread. Now, the worker-thread should receive and process UDP-packets. However, the code for that is inside of the portThread thread! I suppose that handler.postDelayed(this, queryInterval); which I use to loop the thread in every 100ms somehow causes the next cycle to be run in UI-thread instead of my worker-thread.
So my question is what I'm doing wrong here and how to fix it? Or alternatively, how to get the looping work correctly in every 100ms? I'm also not sure where to place the Handler, since I have seen examples where it is inside Activity and inside the worker-thread.

Ok, I think I got it working though I'm not satisfied with it and so leaving this unchecked.
Basically I ended up using TimerTask to run my code every 100ms and notifying UI-thread via Handler. I'm not really sure if this is best choice (I have heard that Timers aren't that great) but seems to work:
dataStreamTimer = new Timer();
dataStreamTask = new TimerTask()
{
public void run()
{
if (portList == null)
{
portList = new ArrayList<PortListener>();
for (int i = minPort; i < maxPort; i++)
{
portList.add(new PortListener(i));
}
}
if (!stop)
{
ManagerUdpPacket p = portList.get(0).receive();
threadMsg("moi", p);
//handler.postDelayed(this, queryInterval);
//stop thread until next query
try {
synchronized(this){
this.wait(queryInterval);
}
} catch (InterruptedException e) {
Log.e("ERR", "InterruptedException in TimerTask.run");
}
}
else
{
//execution has been stopped, clear data:
//stop execution and close ports
for (int i = 0; i < portList.size(); i++)
{
portList.get(i).close();
}
}
}

dont really understand purpose of handlers. Why you dont just prepare data on backround thread and than use myActivity.runOnUIThread() to run your updateContent() method? Maybe p.getOperationTime() is considered network operation, try to save this value to some variable in background thread and than publish it by UI thread.