Develop Reference

java assignment synchronization producer consumer

I've written a Multithreading code for producer consumer problem in which I've written synchronized block inside the run method of consumer and producer thread which takes lock on shared list(I assumed)
So the point of question is that, will there be locking on the list, because as per each thread will have their own synchronized block but they are sharing the same list instance
public class Main {
static boolean finishFlag=false;
final int queueSize = 20;
List<Integer> queue = new LinkedList<>();
Semaphore semaphoreForList = new Semaphore(queueSize);
public Main(int producerCount,int consumerCount) {
while(producerCount!=0) {
new MyProducer(queue,semaphoreForList,queueSize).start(); //produces the producer
producerCount--;
}
while(consumerCount!=0) {
new MyConsumer(queue,semaphoreForList,queueSize).start(); //produces the consumer
consumerCount--;
}
}
public static void main(String args[]) {
/*
* input is from command line 1st i/p is number of producer and 2nd i/p is number of consumer
*/
try {
Main newMain = new Main(Integer.parseInt(args[0]),Integer.parseInt(args[1]));
try {
Thread.sleep(30000);
}
catch(InterruptedException e) {
}
System.out.println("exit");
finishFlag=true;
}
catch(NumberFormatException e) {
System.out.println(e.getMessage());
}
}
}
class MyProducer extends Thread{
private List<Integer> queue;
Semaphore semaphoreForList;
int queueSize;
public MyProducer(List<Integer> queue, Semaphore semaphoreForList,int queueSize) {
this.queue = queue;
this.semaphoreForList = semaphoreForList;
this.queueSize = queueSize;
}
public void run() {
while(!Main.finishFlag) {
try {
Thread.sleep((int)(Math.random()*1000));
}
catch(InterruptedException e) {
}
try {
if(semaphoreForList.availablePermits()==0) {//check if any space is left on queue to put the int
System.out.println("no more spaces left");
}
else {
synchronized(queue) {
semaphoreForList.acquire(); //acquire resource by putting int on the queue
int rand=(int)(Math.random()*10+1);
queue.add(rand);
System.out.println(rand+" was put on queue and now length is "+(queueSize-semaphoreForList.availablePermits()));
}
}
}
catch(InterruptedException m) {
System.out.println(m);
}
}
}
}
public class MyConsumer extends Thread{
private List<Integer> queue; //shared queue by consumer and producer
Semaphore semaphoreForList;
int queueSize;
public MyConsumer(List<Integer> queue, Semaphore semaphoreForList,int queueSize) {
this.queue = queue;
this.semaphoreForList = semaphoreForList;
this.queueSize = queueSize;
}
public void run() {
while(!Main.finishFlag) {//runs until finish flag is set to false by main
try {
Thread.sleep((int)(Math.random()*1000));//sleeps for random amount of time
}
catch(InterruptedException e) {
}
if((20-semaphoreForList.availablePermits())==0) {//checking if any int can be pulled from queue
System.out.println("no int on queue");
}
else {
synchronized(queue) {
int input=queue.remove(0);//releases the resource(position in queue) by pulling the int out of the queue and computing factorial
semaphoreForList.release();
int copyOfInput=input;
int fact=1;
while(copyOfInput!=0) {
fact = fact*copyOfInput;
copyOfInput--;
}
System.out.println(input+" was pulled out from queue and the computed factorial is "+fact+
" the remaining length of queue is "+(queueSize-semaphoreForList.availablePermits()));
}
}
}
}
}

I would rather recommend to use the java.lang.Object methods wait() and notify() to create a consumer-producer algorithm. Using this approach the queue won't be blocked by endlessly repeating and unnecessary synchronized statements which I think is a more performant and "event driven" solution.
This link might be helpful -
https://www.geeksforgeeks.org/producer-consumer-solution-using-threads-java/

Yes, the mutex/monitor is associated with the Java Object instance, which is the shared list in this instance. Which means all threads lock same mutex (associated with queue, and are synchronized through this.
So the good part: You program is actually thread-safe.
However the additional semaphore actually doesn't make a lot of sense in a variety of ways:
The checks (e.g. for availablePermits) happen outside of the lock, and are therefore only a best-guess about the state of your queue. It could be different shortly afterwards.
Trying to acquire a semaphore inside a lock, which can only be released inside the same lock, looks like a guaranteed recipe for a deadlock.
As AnDus has mentioned, this could probably be better solved via using the wait and notify methods which act as a condition variable. Most likely you need even two, one to unblock producers and one to unblock consumers.
In general, if this is not a coding exercise, use a class which already implements your desired functionality. In this case, java.util.concurrent.BlockingQueue seems like what you want.

Access thread variable that changes from main thread

So, I am new to threads, and I'm still learning how everything works. So, I couldn't find an answer that would provide an explanation for my problem (to my level of understanding).
I have a Runnable class that looks like so:
public class Request implements Runnable {
private Boolean ok = true;
public synchronized void setOk(Boolean ok) {
this.ok = ok;
}
public synchronized Boolean getOk() {
return ok;
}
private synchronized void foo() {
//if something happens
setOk(false);
}
#Override
public void run() {
while (true)
foo();
}
}
And then I have another class that does the following:
private static Request request;
private static void spawnThreads() {
ExecutorService e = new Executors.newFixedThreadPool(4);
request = new Request();
e.execute(request);
}
public static void main(String[] args) {
spawnThreads();
while (true) {
System.out.println(request.getOk());
if (!request.getOk())
request.setOk(true);
TimeUnit.SECONDS.sleep(10);
}
}
I need that if in the main thread, that getOk() returns false, do something and set it to true. Viceversa, set it to false in the thread (which I need to keep on going, no matter what the value of ok is at any given time).
As this code is, I can't get the value of request.getOk() in the main thread. If I remove the synchronized words from the getter and setter, I can access the value in the main thread until a point in time when it is changed by the thread, and never again.
Also, the executor is used because I would create multiple Request objects, and waiting for it to shutdown before accessing the variable would contradict my reason for doing this, as I would need all the threads to keep running.
That thread is making http requests to a server (that randomly times out, denies response, etc) and is used to retrieve some information. The ok variable is there to take a note when the thread acquires an ok response and some information from the server.
How do I solve it so that the thread can update that variable, but the main thread to be able to retrieve it whenever needed, no matter if it was changed by the thread in the meanwhile or not.
Would changing my Runnable to a Callable help? If yes, how?

Your example still leaves some holes in the thread-safety. Like mentioned by #Radiodef using AtomicBoolean can relieve you of most of the synchronisation if used properly.
Using your example, this is a thread safe Request class that accepts a message, like an answer to a http request.
public final class Request implements Runnable {
private final AtomicBoolean ok = new AtomicBoolean(false);
// volatile variables promote reference changes through all threads
private volatile String msg;
private boolean setMessage(String responseMessage) {
if (this.ok.compareAndSet(false, true)) {
this.msg = msg;
return true;
}
return false;
}
public boolean hasMessage() {
// *pure* getters don't need synchronisation!
return this.ok.get();
}
public String getMessageAndReset() {
// make a copy before resetting the OK
String msgCopy = this.msg;
this.ok.compareAndSet(true, false);
return msgCopy;
}
public void run() {
final Random rand = new Random();
try {
while(true) {
// sleep at random max 5 seconds
// (simulate unpredictable network)
TimeUnit.SECONDS.sleep(rand.nextInt(5));
while(!setMessage("Incoming message")) {
// busy waiting ... waits until the current value has
// been retrieved by the main thread
Thread.sleep(100);
}
}
} catch (Exception e) {
System.out.println(e);
}
}
}
And your main class:
public final class MainClazz implements Runnable {
private final ExecutorService exec;
private final Request request;
public void MainClazz() {
this.exec = new Executors.newFixedThreadPool(4);
this.request = new Request();
this.exec.execute(request);
}
public void run() {
while (true) {
if (request.hasMessage()) {
System.out.println(request.getMessageAndReset());
}
TimeUnit.SECONDS.sleep(10);
}
public static void main(String[] args) {
MainClazz main = new MainClazz();
main.run();
}
}
In this implementation, the Request class only holds a single value at a time. Depending the amount of data you expect you might want to think about using a buffer.
Also, like many others have mentioned, don't use while (true)! Get a synchronisation object from the java concurrent package!
More light reading on the AtomicBoolean object.

User Created Thread Pool Shutdown having different behaviour when a sysout is added

I was asked to create my own thread pool in an interview where I have to create the number of threads requested by the user. Allow user to submit there task and finally shutdown the pool. I wrote the below program which is working fine in all the cases other than shutdown.
public class ThreadPool
{
public final Queue<Runnable> workerQueue;
private static boolean isrunning = true;
private Thread[] workerThreads;
public ThreadPool(int N)
{
workerQueue = new LinkedList<>();
workerThreads = new Thread[N];
for (int i = 0; i < N; i++) {
workerThreads[i] = new Worker("Pool Thread " + i);
workerThreads[i].start();
}
}
public void shutdown()
{
while(isrunning){
if(workerQueue.isEmpty()){
isrunning = false;
}
}
}
public void submit(Runnable r) throws Exception
{
workerQueue.add(r);
}
private class Worker extends Thread
{
public Worker(String name)
{
super(name);
}
public void run()
{
while (isrunning) {
try {
if(!workerQueue.isEmpty())
{
Runnable r = workerQueue.poll();
r.run();
}
} catch (RuntimeException e) {
e.printStackTrace();
}
}
}
}
}
The Test method I wrote is like below
static public void main(String[] args) throws Exception
{
ClassA a1 = new ClassA();
ClassA a2 = new ClassA();
ClassA a3 = new ClassA();
ClassA a4 = new ClassA();
ClassA a5 = new ClassA();
ClassA a6 = new ClassA();
ThreadPool tp = new ThreadPool(5);
tp.submit(a1);
tp.submit(a2);
tp.submit(a3);
tp.submit(a4);
tp.submit(a5);
///////////////
tp.submit(a6);
tp.shutdown();
}
But the program is never ending, its running always and I have to stop it manually in eclipse. But if I add a simple System.out.print("") in my method shutdown, its working perfrectly(the Program is ended after the execution of all the threads).
Can you please tell me why its working with the sysout and why its not working with out it ?

You have two major problems with your code:
You are using an implementation of Queue that's not synchronized in a multithread environment. This leads to a Race Condition when multiple threads insert/remove from the queue. Please consider using something like:
workerQueue = new ConcurrentLinkedQueue()
You run some very tight loops, one in your Worker.run() method where you're not checking for NPE and not allowing for some "cooldown", and another one in shutdown():
while (isrunning) {
...
Runnable r = workerQueue.poll();
r.run();
...
}
This, coupled with 1, leads to the Queue being emptied without properly updating the size info: ie. isEmpty() returns false but poll() returns null. Your program will become stuck in a loop. This doesn't happen when you add System.out.print() due to different timing specific to your configuration (it can continue to fail in other environments - especially where there are more than 6 cpu cores available).

Returning a value from Runnable

The run method of Runnable has return type void and cannot return a value. I wonder however if there is any workaround of this.
I have a method like this:
public class Endpoint {
public method() {
Runnable runcls = new RunnableClass();
runcls.run()
}
}
The method run is like this:
public class RunnableClass implements Runnable {
public JaxbResponse response;
public void run() {
int id = inputProxy.input(chain);
response = outputProxy.input();
}
}
I want to have access to response variable in method. Is this possible?

Use Callable<V> instead of using Runnable interface.
Example:
public static void main(String args[]) throws Exception {
ExecutorService pool = Executors.newFixedThreadPool(3);
Set<Future<Integer>> set = new HashSet<>();
for (String word : args) {
Callable<Integer> callable = new WordLengthCallable(word);
Future<Integer> future = pool.submit(callable);
set.add(future);
}
int sum = 0;
for (Future<Integer> future : set) {
sum += future.get();
}
System.out.printf("The sum of lengths is %s%n", sum);
System.exit(sum);
}
In this example, you will also need to implement the class WordLengthCallable, which implements the Callable interface.

public void check() {
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<Integer> result = executor.submit(new Callable<Integer>() {
public Integer call() throws Exception {
return 10;
}
});
try {
int returnValue = result.get();
} catch (Exception exception) {
//handle exception
}
}

Have a look at the Callable class. This is usually submited via an executor service
It can return a future object which is returned when the thread completes

Yes, there are workaround. Just use queue and put into it value which you want to return. And take this value from another thread.
public class RunnableClass implements Runnable{
private final BlockingQueue<jaxbResponse> queue;
public RunnableClass(BlockingQueue<jaxbResponse> queue) {
this.queue = queue;
}
public void run() {
int id;
id =inputProxy.input(chain);
queue.put(outputProxy.input());
}
}
public class Endpoint{
public method_(){
BlockingQueue<jaxbResponse> queue = new LinkedBlockingQueue<>();
RunnableClass runcls = new RunnableClass(queue);
runcls.run()
jaxbResponse response = queue.take(); // waits until takes value from queue
}
}

If you add a field to RunnableClass you can set it in run and read it in method_. However, Runnable is a poor (the Java keyword) interface as it tells you nothing about the (the concept) interface (only useful line of the API docs: "The general contract of the method run is that it may take any action whatsoever."). Much better to use a more meaningful interface (that may return something).

One way is, we have to use Future - Callable approach.
Another way is, Instead of returning value, you can hold in object
Example:
class MainThread {
public void startMyThread() {
Object requiredObject = new Object(); //Map/List/OwnClass
Thread myThread = new Thread(new RunnableObject(requiredObject)).start();
myThread.join();
System.out.println(requiredObject.getRequiredValue());
}
}
class RunnableObject implements Runnable {
private Object requiredObject;
public RunnableObject(Object requiredObject) {
this.requiredObject = requiredObject;
}
public void run() {
requiredObject.setRequiredValue(xxxxx);
}
}
Because object scope is in the same scope so that you can pass object to thread and can retrieve in the main scope. But, most important thing is, we have to use join() method. Because main scope should be waiting for thread completion of its task.
For multiple thread case, you can use List/Map to hold the values from threads.

Try the following
public abstract class ReturnRunnable<T> implements Runnable {
public abstract T runForResult();
#Override
public void run() {
runForResult();
}
}

Take a look at the callable interface, perhaps this suites your needs. You can also try to get the value of the response field by calling a setter-method inside of your run() method
public void run() {
int id;
id =inputProxy.input(chain);
response = outputProxy.input();
OuterClass.setResponseData(response);
}

Using a shared ExecutorService as a task queue, how do I know when a job is complete?

I have a JobService that processes larger jobs. Jobs are dynamically subdivided into multiple tasks, tasks also might generate sub-tasks, etc, so its not possible to predict the total number of tasks for a job. Each task queues itself to run via ExecutorService.submit(...) The problem is it seems like I have to create a separate ExecutorService for each job, since the only way to tell when the 'job queue' is complete is to use ExecutorService.awaitTermination(...). This seems inefficient though, because I can't share a single threadpool between the jobs and their ExecutorService.
I'm looking for some alternatives, I was thinking of using an AtomicInteger for each job. Incrementing it when I submit a new task, decrementing it when a task finishes. But then I have to poll for when it is zero, and that seems messy, as well as some exception handling mess.
It seems like there must be a better solution?

Submit returns a Future object that can be used to wait on completion of tasks. You could keep track of these and add a method that recursively blocks until all subtasks are done. This way you can reuse the executor wherever you need to.
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.atomic.AtomicBoolean;
public class JobExecutor {
ExecutorService executorService = Executors.newFixedThreadPool(1);
private class Task implements Runnable {
private final String name;
private final Task[] subtasks;
private final ExecutorService executorService;
private volatile boolean started = false;
private Future<?> taskFuture;
// Separate list from subtasks because this is what you'll probably
// actually use as you may not be passing subtasks as constructor args
private final List<Task> subtasksToWaitOn = new ArrayList<Task>();
public Task(String name, ExecutorService executorService,
Task... subtasks) {
this.name = name;
this.executorService = executorService;
this.subtasks = subtasks;
}
public synchronized void start() {
if (!started) {
started = true;
taskFuture = executorService.submit(this);
}
}
public synchronized void blockTillDone() {
if (started) {
try {
taskFuture.get();
} catch (InterruptedException e) {
// TODO Handle
} catch (ExecutionException e) {
// TODO Handle
}
for (Task subtaskToWaitOn : subtasksToWaitOn) {
subtaskToWaitOn.blockTillDone();
}
} else {
// TODO throw exception
}
}
#Override
public void run() {
for (Task subtask : subtasks) {
subtask.start();
subtasksToWaitOn.add(subtask);
}
System.out.println("My name is: " + name);
}
}
void testSubmit() {
Task subsubTask1 = new Task("Subsubtask1", executorService);
Task subtask1 = new Task("Subtask1", executorService, subsubTask1);
Task subtask2 = new Task("Subtask2", executorService);
Task subtask3 = new Task("Subtask3", executorService);
Task job = new Task("Job", executorService, subtask1, subtask2,
subtask3);
job.start();
job.blockTillDone();
System.out.println("Job done!");
}
public static void main(String[] args) {
new JobExecutor().testSubmit();
}
}
Prints out:
My name is: Job
My name is: Subtask1
My name is: Subtask2
My name is: Subtask3
My name is: Subsubtask1
Job done!

If you're on java7 (or java6 with the backport library http://www.cs.washington.edu/homes/djg/teachingMaterials/grossmanSPAC_forkJoinFramework.html ), you might want to consider a Fork-Join pool for this sort of thing:
class MainTask extends RecursiveTask<Long> {
#Override
protected Long compute() {
SubTask subtask0 = new SubTask(0L);
SubTask subtask1 = new SubTask(1L);
SubTask subtask2 = new SubTask(2L);
SubTask subtask3 = new SubTask(3L);
SubTask subtask4 = new SubTask(4L);
SubTask subtask5 = new SubTask(5L);
subtask1.fork();
subtask2.fork();
subtask3.fork();
subtask4.fork();
subtask5.fork();
return subtask0.compute() +
subtask1.join() +
subtask2.join() +
subtask3.join() +
subtask4.join() +
subtask5.join();
}
}
class SubTask extends RecursiveTask<Long> {
private Long rawResult = null;
private Long expected = null;
public SubTask(long expected) {
this.expected = expected;
}
#Override
protected Long compute() {
return expected;
}
}
public static void main( String[] args )
{
ForkJoinPool forkJoinPool = new ForkJoinPool();
Long result = forkJoinPool.invoke(new MainTask());
System.out.println(result);
}
obviously this has hardcoded subtasks, but there's no reason you can't pass parameters to your main task, and use that to generate subtasks. The subtasks themselves don't all have to be of the same type, but they should all extend RecursiveTask. Realistically if a task generates subtasks (like MainTask above), at least one of the subtasks should have "compute" called directly on it (rather and a fork and a join), so that the current thread can execute some computations, and let other threads do the rest.