Develop Reference

Semaphore in Java. producer-consumer problem

I am testing the use of semaphores with the typical producer-consumer problem where I only have one producer and one consumer. The producer adds products one at a time and the consumer can withdraw several simultaneously.
To perform the test, the producer and the consumer store and remove numbers from a array of 10 elements where 0 represents that there are no products and any other number represents a product. Access to store and retrieve items is centralized in a class called Data. I use a mutex to make an orderly use of the vector in case we have more than one thread working simultaneously.
When executing it, I observe that the number of permissions is not correct according to the operations performed by the threads. The application shows an error because the semaphore of the producer says that it has permission, but the data vector is full.
package producer.consumer;
import java.io.IOException;
public class ProducerConsumer {
public static void main(String[] args) throws IOException {
final int MAX = 10;
Data data = new Data(MAX);
Consumer consumer = new Consumer(data);
Producer producer = new Producer(data);
consumer.start();
producer.start();
}
}
package producer.consumer;
public class Producer extends Thread{
private final Data data;
public Producer(Data data) {
this.data = data;
}
#Override
public void run() {
while (true) {
try {
data.add((int) (Math.random() * data.getLength()) + 1);
} catch (InterruptedException ex) {
System.out.println(ex.getMessage());
}
}
}
}
package producer.consumer;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Consumer extends Thread{
private final Data data;
public Consumer(Data data) {
this.data = data;
}
#Override
public void run() {
while (true) {
try {
data.remove((int) (Math.random() * data.getLength()) + 1);
} catch (InterruptedException ex) {
Logger.getLogger(Consumer.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
package producer.consumer;
import java.io.IOException;
import java.util.Arrays;
import java.util.concurrent.Semaphore;
public class Data {
private final int[] data;
private final Semaphore mutex = new Semaphore(1);
private final Semaphore semProducer, semConsumer;
public Data(int MAX) throws IOException {
data = new int[MAX];
semProducer = new Semaphore(MAX);
semConsumer = new Semaphore(0);
}
public int getLength() {
return data.length;
}
public void add(int number) throws InterruptedException {
semProducer.acquire();
mutex.acquire();
System.out.println("trying to add a product");
int i = 0;
while (data[i] != 0) {
i++;
}
data[i] = number;
int permits = semConsumer.availablePermits() + 1;
System.out.println("data added in " + i + " " + Arrays.toString(data)
+ " Resources consumer " + permits
+ " Resources producer " + semProducer.availablePermits());
mutex.release();
semConsumer.release();
}
public void remove(int numberElements) throws InterruptedException {
semConsumer.acquire(numberElements);
mutex.acquire();
System.out.println("trying to withdraw " + numberElements);
for (int i = 0; i < numberElements; i++) {
if (data[i] != 0) {
data[i] = 0;
}
}
int permisos = semProducer.availablePermits() + 1;
System.out.println(" Retired " + numberElements + " " + Arrays.toString(data)
+ " Resources consumer " + semConsumer.availablePermits()
+ " Resources producer " + permisos);
mutex.release();
semProducer.release(numberElements);
}
}
Thank you very much for the help.

Your consumer does not always consume what it claims to consume.
for (int i = 0; i < numberElements; i++) {
if (data[i] != 0) {
data[i] = 0;
}
}
Suppose numberElements is 3, and that we have exactly 3 available elements in data[7], data[8], data[9].
The loop terminates with i == 3, nothing has been removed, but the producer semaphore will still be 'upped' by 3.
In the consumer, if you use i as the array index, it needs to cover the whole array, and you need a separate counter for 'elements removed'.
It is not the case that available elements will always be in the lowest-numbered data slots even though the producer fills those in first. Consider the time sequence that the producer manages to produce at least 5 elements, then the consumer runs to consume 2, and then immediately runs again to consume 3, before any more have been produced. data[0] and data[1] will be empty on the second run of the consumer and we run into the scenario I describe.

EDIT Acquiring and releasing permits seems correct; but you need to make sure that the consumer will actually clear the correct number of elements.
In example, edit the Data class with
public void remove(int numberElements) throws InterruptedException {
semConsumer.acquire(numberElements);
mutex.acquire();
System.out.println("remove: num-elem=" + numberElements);
int consumed=0;
for (int i = 0; consumed<numberElements; i++) {
if (data[i] != 0) {
data[i] = 0;
consumed++;
}
}
System.out.println(
" Retired " + numberElements + " " + Arrays.toString(data) );
mutex.release();
semProducer.release(numberElements);
}
Note also that this implementation is not very efficient (you'll need to iterate over the whole array both when inserting and deleting items, which can be expensive when MAX is large..)

Why do I get inconsistencies in this method when running

Sometimes the varible total will equal something else when run instead of 50005000 it's allways short like 50005001 when sometimes run why is this happening shouldn't synchronized(this) create a lock which can only be updated once the lock is released by the thread?
import java.util.concurrent.atomic.AtomicLong;
public class CurrentThread {
public static AtomicLong c = new AtomicLong(0L);
public static AtomicLong total = new AtomicLong(0L);
public static void main(String[] args) {
Thread t = Thread.currentThread();
System.out.println(t);
t.setName("My Thread");
System.out.println(t);
for (int x = 0; x < 10; x++) {
System.out.println("Instance " + x);
new Thread(new Updater(x, "Thread: " + String.valueOf(x))).start();
}
try {
Thread.sleep(1000);
}
catch (InterruptedException e) {
}
}
}
class Updater implements Runnable {
public int na;
private String threadName;
public Updater(int n, String threadName) {
this.na = n;
this.threadName = threadName;
}
#Override
public void run() {
this.updateCount();
if(CurrentThread.total.get() == 50005000) {
System.out.println("Passed");
}
else {
System.out.println("Failed");
}
}
public void updateCount() {
while (CurrentThread.c.get() < 10000) {
synchronized(this) {
CurrentThread.c.getAndIncrement();
CurrentThread.total.addAndGet(CurrentThread.c.get());
System.out.println(this.threadName + " " + String.valueOf(CurrentThread.c.get()) + " " + CurrentThread.total.get() + " " + System.nanoTime());
}
}
}
}

You're synchronizing on this, which is effectively not synchronizing at all, since each thread has a different Runnable instance.
Synchronize on something shared between all Updater instances instead, e.g. Updater.class.
Note, however, that synchronizing on AtomicLong is a bit of a code smell - it's meant to do things atomically already.
You can use compareAndSet instead, and avoid synchronizing entirely, e.g.:
while (CurrentThread.c.get() < 10000) {
while (true) {
long currValue = CurrentThread.c.get();
if (currValue >= 10000) break;
long newValue = currValue + 1;
// Only sets c to newValue if its value is still currValue.
if (CurrentThread.c.compareAndSet(currValue, newValue)) {
long total = CurrentThread.total.addAndGet(newValue);
System.out.println(
this.threadName + " " + newValue + " " + total + " " + System.nanoTime());
break;
}
}
}
Note that this makes use of "known" values, like newValue and total, rather than getting them again from the AtomicLong.

Java thread not responding to volatile boolean flag

I am new to Java concurrency, and I met a very strange problem:
I read from a large file and used several worker threads to work on the input (some complicated string matching tasks). I used a LinkedBlockingQueue to transmit the data to the worker threads, and a volatile boolean flag in the worker class to respond to the signal when the end-of-file is reached.
However, I cannot get the worker thread to stop properly. The CPU usage by this program is almost zero in the end, but the program won't terminate normally.
The simplified code is below. I have removed the real code and replaced them with a simple word counter. But the effect is the same. The worker thread won't terminate after the whole file is processed, and the boolean flag is set to true in the main thread.
The class with main
public class MultiThreadTestEntry
{
private static String inputFileLocation = "someFile";
private static int numbOfThread = 3;
public static void main(String[] args)
{
int i = 0;
Worker[] workers = new Worker[numbOfThread];
Scanner input = GetIO.getTextInput(inputFileLocation);
String temp = null;
ExecutorService es = Executors.newFixedThreadPool(numbOfThread);
LinkedBlockingQueue<String> dataQueue = new LinkedBlockingQueue<String>(1024);
for(i = 0 ; i < numbOfThread ; i ++)
{
workers[i] = new Worker(dataQueue);
workers[i].setIsDone(false);
es.execute(workers[i]);
}
try
{
while(input.hasNext())
{
temp = input.nextLine().trim();
dataQueue.put(temp);
}
}
catch (InterruptedException e)
{
Thread.currentThread().interrupt();
}
input.close();
for(i = 0 ; i < numbOfThread ; i ++)
{
workers[i].setIsDone(true);
}
es.shutdown();
try
{
es.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e)
{
Thread.currentThread().interrupt();
}
}
}
The worker class
public class Worker implements Runnable
{
private LinkedBlockingQueue<String> dataQueue = null;
private volatile boolean isDone = false;
public Worker(LinkedBlockingQueue<String> dataQueue)
{
this.dataQueue = dataQueue;
}
#Override
public void run()
{
String temp = null;
long count = 0;
System.out.println(Thread.currentThread().getName() + " running...");
try
{
while(!isDone || !dataQueue.isEmpty())
{
temp = dataQueue.take();
count = temp.length() + count;
if(count%1000 == 0)
{
System.out.println(Thread.currentThread().getName() + " : " + count);
}
}
System.out.println("Final result: " + Thread.currentThread().getName() + " : " + count);
}
catch (InterruptedException e)
{
}
}
public void setIsDone(boolean isDone)
{
this.isDone = isDone;
}
}
Any suggestions to why this happens?
Thank you very much.

As Dan Getz already said your worker take() waits until an element becomes available but the Queue may be empty.
In your code you check if the Queue is empty but nothing prevents the other Workers to read and remove an element from the element after the check.
If the Queue contains only one element and t1 and t2 are two Threads
the following could happen:
t2.isEmpty(); // -> false
t1.isEmpty(); // -> false
t2.take(); // now the queue is empty
t1.take(); // wait forever
in this case t1 would wait "forever".
You can avoid this by using pollinstead of take and check if the result is null
public void run()
{
String temp = null;
long count = 0;
System.out.println(Thread.currentThread().getName() + " running...");
try
{
while(!isDone || !dataQueue.isEmpty())
{
temp = dataQueue.poll(2, TimeUnit.SECONDS);
if (temp == null)
// re-check if this was really the last element
continue;
count = temp.length() + count;
if(count%1000 == 0)
{
System.out.println(Thread.currentThread().getName() + " : " + count);
}
}
System.out.println("Final result: " + Thread.currentThread().getName() + " : " + count);
}
catch (InterruptedException e)
{
// here it is important to restore the interrupted flag!
Thread.currentThread().interrupt();
}
}

Synchronizing multiple threads JAVA

I have 200 students waiting to enter a room with 200 seats (25 rows and 8 columns). The door capacity is 4 people. When a student enter the room, he chooses random seat (row and column). If the chosen seat is at 9th row or less it takes 1 second to sit, on 18th and less it takes 2 seconds, and if its from 18 to 25 it takes 3 seconds.
When any of them take a seat another person must come in the room.
The problem is that when the first 4 people enter the room they take seat one by one and not at once. How can I fix that?
For example if 2 people choose a seat at 5th row they both need to sit for 1 seconds and two new students must enter the room.
public class Student
{
int row;
int column;
volatile static int mutex;
//Generating random numbers for row and column
public Student(Seats[][] seats)
{
this.row = (int) Math.ceil(Math.random() * 25);
this.column = (int) Math.ceil(Math.random() * 8);
if (!seats[row][column].isTaken)
{
seats[row][column].isTaken = true;
} else
{
do
{
this.row = (int) Math.ceil(Math.random() * 25);
this.column = (int) Math.ceil(Math.random() * 8);
} while (!seats[row][column].isTaken);
seats[row][column].isTaken = true;
}
}
/*Check if the mutex is 4 (4 people are in the room) then wait
if someone enter the room increment mutex*/
synchronized void add() throws InterruptedException
{
while (mutex > 4)
wait();
Student.mutex++;
notifyAll();
}
/* Check if mutex is 0 (no one is in the room) then wait
if the student has sit - decrement mutex and notify*/
synchronized void takeSeat() throws InterruptedException
{
while (mutex == 0)
wait();
Student.mutex--;
notifyAll();
}
}
class Seats
{
int seat;
boolean isTaken;
public Seats(int seat)
{
this.seat = seat;
this.isTaken = false;
}
}
class StudentThread extends Thread
{
Seats[][] seats = new Seats[25][8];
StudentThread(Seats[][] seats)
{
this.seats = seats;
}
public void run()
{
try
{
Student student = new Student(seats);
synchronized (seats)
{
System.out.println("Student enter the room");
/*call the synchronized method from student
that increment the mutex*/
student.add();
if (Student.mutex == 4)
{
if (student.row <= 9)
{
sleep(1000);
student.takeSeat();
System.out.println("Student take a seat at "
+ student.row + " " + student.column);
}
if (student.row <= 18 && student.row > 9)
{
sleep(2000);
student.takeSeat();
System.out.println("Student take a seat at "
+ student.row + " " + student.column);
}
if (student.row <= 25 && student.row > 18)
{
sleep(3000);
student.takeSeat();
System.out.println("Student take a seat at "
+ student.row + " " + student.column);
}
}
}
} catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
class Main
{
public static void main(String[] args)
{
Seats[][] seats = new Seats[25][8];
//Initializing the seats
for (int i = 0; i < 25; i++)
for (int j = 0; j < 8; j++)
{
seats[i][j] = new Seats(i);
}
for (int i = 0; i < 200; i++)
{
StudentThread T1 = new StudentThread(seats);
T1.start();
}
}
}

Use a Semaphore, they are very practical for these kind of things.
To make the example a bit more realistic: imagine you need to do 200 HTTP get-requests, but the server will ban you if you run more than 4 requests at the same time. The example below shows how you can limit the number of requests running at the same time using a Semaphore.
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;
public class ResourceUsageLimiter {
static ExecutorService executor = Executors.newCachedThreadPool();
static int requests = 20;
static int maxRequestsConcurrent = 4;
static int maxRequestTime = 1000;
static Random randomizer = new Random();
static Semaphore openSlots = new Semaphore(maxRequestsConcurrent);
static long startTime = System.currentTimeMillis();
public static void main(String[] args) {
try {
for (int i = 0; i < requests; i++) {
openSlots.acquire();
executor.execute(new RequestRunner(i));
}
} catch (Exception e) {
e.printStackTrace();
} finally {
executor.shutdown();
}
}
static long time() {
return System.currentTimeMillis() - startTime;
}
static class RequestRunner implements Runnable {
int sleepTime, reqId;
public RequestRunner(int reqId) {
this.reqId = reqId;
sleepTime = randomizer.nextInt(maxRequestTime);
}
#Override
public void run() {
try {
System.out.println(time() + " " + reqId + " sleeping " + sleepTime);
Thread.sleep(sleepTime);
System.out.println(time() + " " + reqId + " sleep done");
} catch (Exception e) {
e.printStackTrace();
} finally {
openSlots.release();
}
}
}
}
Ofcourse, another way to limit the maximum number of requests running at the same time in the example is to use a thread pool with a fixed size of 4.

How to create a cyclic exchange of three threads?

How to create a cyclic exchange of three threads? That is: first thread must send data to second, second to third and third thread must send data to first.
I wrote some code, but threads exchange in random oder.
class DataClass {
int value;
String message;
DataClass(int v, String s) {
value = v;
message = s;
}
int getValue() {
return (value);
}
String getMassage() {
return (message);
}
}
class Loop implements Runnable {
int counter;
String name;
Exchanger<DataClass> exchanger;
Loop(int startValue, String id, Exchanger<DataClass> ex) {
counter = startValue;
name = id;
exchanger = ex;
System.out.println(name + ": created");
}
public void run() {
System.out.println(name + ": started");
DataClass data = new DataClass(counter, name);
for (int i = 0; i < 3; ++i) {
try {
DataClass newData = exchanger.exchange(data);
counter += newData.getValue();
System.out.println(name + ": from "
+ newData.getMassage() + ": data: "
+ newData.getValue() + ": state = " + counter);
} catch (InterruptedException e) {
System.err.println(e.toString());
}
}
System.out.println(name + ": ended");
}
}
public class ExchangerDemo {
public static void main(String args[]) {
System.out.println("Main process started");
Exchanger<DataClass> exchanger = new Exchanger<DataClass>();
Loop loop1 = new Loop(1, "First", exchanger);
Loop loop2 = new Loop(2, "Second", exchanger);
Loop loop3 = new Loop(3, "Third", exchanger);
new Thread(loop1).start();
new Thread(loop2).start();
new Thread(loop3).start();
System.out.println("Main process ended");
}
}

For your dependency you should make three classes, and have three distinct Exchange objects (one in each). So thread1 would be between 1 and 2 (output of 1 to 2), thread 2's would be between 2 and 3 and thread 3's exhanger would be between itself and 1. Remember the exchanger's would guard only until it had its input from its feeder, to till it passes to its receiver.
Also synchronized is not as bad as the books make out. use it. Watch http://www.youtube.com/watch?v=WTVooKLLVT8
Also for reference Best way of running two threads alternatively?
Also why do you need three threads? Can you use a thread pool and have each task to the 3 things ?