I tried to run example with CyclicBarrier from one of tutorials:
Service man should fill empty printers when the queue of empty printers is 3.
But when I ran the code it appears that printers are filled with 2, 3 or 4 empty printers in the queue:
Printer1 is empty
Printer12 is empty
Printer14 is empty
Printer13 is empty
Filling [Printer1, Printer12, Printer14, Printer13]
Printer2 is empty
Printer7 is empty
Filling [Printer2, Printer7]
So is the example wrong or my understanding of CyclicBarrier? I consider that queue should be exactly 3 elements size. What should I add to the code to fix that? Thanks in advance.
Code:
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.BrokenBarrierException;
import java.util.concurrent.CyclicBarrier;
import java.util.concurrent.TimeUnit;
public class PrinterRecharger {
public static void main(String args[]) {
ServiceMan serviceMan = new ServiceMan(3);
for (int i = 0; i < 15; i++) {
new Thread(new Printer(serviceMan, "Printer" + (i + 1))).start();
}
}
}
class ServiceMan {
private CyclicBarrier queue;
private List<String> inQueue;
public ServiceMan(int hardWorking) {
inQueue = new ArrayList<String>();
queue = new CyclicBarrier(hardWorking, new Runnable() {
#Override
public void run() {
System.out.println("Filling " + inQueue);
inQueue.clear();
}
});
}
public void recharge(String name) {
try {
inQueue.add(name);
queue.await();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}
}
class Printer implements Runnable {
private String name;
private Random rand;
private ServiceMan serviceMan;
public Printer(ServiceMan serviceMan, String name) {
this.name = name;
this.serviceMan = serviceMan;
this.rand = new Random();
}
public void run() {
try {
while (true) {
TimeUnit.SECONDS.sleep(rand.nextInt(10));
System.out.println(name + " is empty");
serviceMan.recharge(name);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Your code is thread-unsafe in several ways that I can immediately see, and probably some more which I have missed. You have data races as well as other types of race conditions.
ArrayList is not a thread-safe class, but you use it from multiple threads with no synchronization. Wrap the list into a Collections.synchronizedList() to see some improvement.
You lack any mutual exclusion between recharge() and the CyclicBarrier's action. A thread could add an item to the queue only to have it cleared away by the action.
Related
I'm trying to create a web crawler.
I've created a class to handle all URLs visited and to visit.
This class has to be accessed by multiple threads for retrieving and updating those lists.
The problem I'm facing, or at least I think, is in nextRandom() and probably also in next(). I think what is happening is the threads are interfering with each other since the function is somewhat synchronized but not atomic. Is there a way to make so this block of code is executed without any interruption by other threads?
The URL handler
import java.util.*;
import java.util.concurrent.ThreadLocalRandom;
public class UrlHandler {
private volatile Set<String> visited = new HashSet<String>();
private volatile List<String> toVisit = new ArrayList<String>();
public void addToVisit(String url) {
synchronized (this){
if (!visited.contains(url)) toVisit.add(url);
}
}
public void addToVisit(Collection<String> urls) {
synchronized (this){
for (String url : urls)
if (!visited.contains(url)) toVisit.add(url);
}
}
public void addVisited(String url){
synchronized (this){
visited.add(url);
}
}
public void addVisited(Collection<String> urls){
synchronized (this){
visited.addAll(urls);
}
}
public String next() {
while (toVisit.size() == 0) {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
synchronized (this){
String url = toVisit.get(0);
toVisit.remove(0);
return url;
}
}
public String nextRandom() {
synchronized (this){
int n = 0;
if (toVisit.size() > 1){
n = ThreadLocalRandom.current().nextInt(toVisit.size());
}
String url = toVisit.get(n);
toVisit.remove(n);
return url;
}
}
public List<String> getToVisit() {
synchronized (this){
return toVisit;
}
}
public Set<String> getVisited() {
synchronized (this){
return visited;
}
}
}
Web Crawler
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class WebCrawler {
private final ExecutorService executor;
public WebCrawler(int nOfThreads) {
this.executor = Executors.newFixedThreadPool(nOfThreads);
}
public void add(Runnable runnable) {
this.executor.execute(runnable);
}
//Used to shut down safely and wait also 5 of seconds for not finished tasks
public void shutdown() {
this.executor.shutdown();
try {
this.executor.awaitTermination(5, TimeUnit.SECONDS);
if (!this.executor.isTerminated()) {
System.err.println("Timed out waiting for executor to terminate cleanly. Shutting down.");
this.executor.shutdownNow();
}
} catch (final InterruptedException e) {
System.err.println("Interrupted while waiting for executor shutdown.");
Thread.currentThread().interrupt();
}
}
}
Failing test example
import org.junit.jupiter.api.Test;
import java.util.ArrayList;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertEquals;
public class UrlHandlerTest {
List<String> testList = new ArrayList<>(List.of("test1", "test2", "test3", "test3"));
List<String> uniqueTestList = new ArrayList<>(List.of("test1", "test2", "test3"));
UrlHandler urlHandler = new UrlHandler();
#Test
public void concurrentAccess(){
urlHandler.addToVisit(testList);
WebCrawler webCrawler = new WebCrawler(10);
for (int i = 0; i < urlHandler.getToVisit().size(); i++) {
webCrawler.add(new Runnable() {
#Override
public void run() {
String url = urlHandler.nextRandom();
urlHandler.addVisited(url);
System.out.println("Here thread " + Thread.currentThread().getId() + " working on: " + url);
}
});
}
webCrawler.shutdown();
System.out.println(urlHandler.getVisited());
assertEquals(true, urlHandler.getVisited().containsAll(uniqueTestList));
}
}
In the next method this code is a problem:
while (toVisit.size() == 0) {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
The lock isn't held for this part, so size can be stale. Instead of this, try something like
while (toVisit.size() == 0)
wait();
Do this in a synchronized block so you have the lock held while checking the collection size. Code that adds to the collection should notify in order to wake up the waiting threads.
This piece of code is problematic:
for (int i = 0; i < urlHandler.getToVisit().size(); i++) {
webCrawler.add(new Runnable() {
// ...
});
}
The urlHandler.getToVisit().size() is always changing during the traversal, and there is uncertainty (because the size will be changed asynchronously).
Change to:
int size = urlHandler.getToVisit().size();
for (int i = 0; i < size; i++) {
webCrawler.add(new Runnable() {
// ...
});
}
In the following Code there is a potential to enter a Deadlock similar to this Question "Deadlocks and Synchronized methods", now i understand why the two Threads are entering a
deadlock, but when i execute the code the Threads always enters a Deadlock so:
1 - When is a Deadlock not possible in this code ?
2 - How to prevent it from happening ?
I tried using wait() and notifyAll() like this :
wait()
waver.waveBack(this)
and then calling notifyAll() in waveBack(), but it didn't work what am i missing or misunderstood ?
package mainApp;
public class Wave {
static class Friend {
private final String name;
public Friend(String name) {
this.name = name;
}
public String getName() {
return this.name;
}
public synchronized void wave(Friend waver) {
String tmpname = waver.getName();
System.out.printf("%s : %s has waved to me!%n", this.name, tmpname);
waver.waveBack(this);
}
public synchronized void waveBack(Friend waver) {
String tmpname = waver.getName();
System.out.printf("%s : %s has waved back to me!%n", this.name, tmpname);
}
}
public static void main(String[] args) {
final Friend friendA = new Friend("FriendA");
final Friend friendB = new Friend("FriendB");
new Thread(new Runnable() {
public void run() {
friendA.wave(friendB);
}
}).start();
new Thread(new Runnable() {
public void run() {
friendB.wave(friendA);
}
}).start();
}
}
In this case, simply do not call another method that might need the lock while holding the lock. This ensures that there is always a moment in time where a method can get the lock and progress can be made.
Calling wait() before waver.waveBack(this) causes a chicken and egg problem: waveBack(this) is never called because the thread stops execution at the wait() statement and thus notifyAll() is never called to continue execution.
There are various ways to prevent deadlocks in the context of the example, but let's go with the advice from sarnold in one of the comments in his answer from the question you linked. To paraphrase sarnold: "it is usually easier to reason about locks on data".
Let's assume that the synchronized methods are synchronized to ensure some consistent update of state (i.e. some variables need to be updated but only one thread at any given time can modify these variables). For example, let's register the amount of waves send and waves received. The runnable code below should demonstrate this:
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
public class Wave {
static class Waves {
final Map<Friend, Integer> send = new HashMap<>();
final Map<Friend, Integer> received = new HashMap<>();
void addSend(Friend f) {
add(f, send);
}
void addReceived(Friend f) {
add(f, received);
}
void add(Friend f, Map<Friend, Integer> m) {
m.merge(f, 1, (i, j) -> i + j);
}
}
static class Friend {
final String name;
public Friend(String name) {
this.name = name;
}
final Waves waves = new Waves();
void wave(Friend friend) {
if (friend == this) {
return; // can't wave to self.
}
synchronized(waves) {
waves.addSend(friend);
}
friend.waveBack(this); // outside of synchronized block to prevent deadlock
}
void waveBack(Friend friend) {
synchronized(waves) {
waves.addReceived(friend);
}
}
String waves(boolean send) {
synchronized(waves) {
Map<Friend, Integer> m = (send ? waves.send : waves.received);
return m.keySet().stream().map(f -> f.name + " : " + m.get(f))
.sorted().collect(Collectors.toList()).toString();
}
}
#Override
public String toString() {
return name + ": " + waves(true) + " / " + waves(false);
}
}
final static int maxThreads = 4;
final static int maxFriends = 4;
final static int maxWaves = 50_000;
public static void main(String[] args) {
try {
List<Friend> friends = IntStream.range(0, maxFriends)
.mapToObj(i -> new Friend("F_" + i)).collect(Collectors.toList());
ExecutorService executor = Executors.newFixedThreadPool(maxThreads);
Random random = new Random();
List<Future<?>> requests = IntStream.range(0, maxWaves)
.mapToObj(i -> executor.submit(() ->
friends.get(random.nextInt(maxFriends))
.wave(friends.get(random.nextInt(maxFriends)))
)
).collect(Collectors.toList());
requests.stream().forEach(f ->
{ try { f.get(); } catch (Exception e) { e.printStackTrace(); } }
);
executor.shutdownNow();
System.out.println("Friend: waves send / waves received");
friends.stream().forEachOrdered(p -> System.out.println(p));
} catch (Exception e) {
e.printStackTrace();
}
}
}
I have two classes (Customer and Till). Customer thread waits until it is notified by a till thread. In my program, the customer thread is not executing it's code after being notified by the till thread. The till thread continues it's execution.
Customer.java (Customer thread extends Thread)
import java.util.concurrent.*;
import java.util.*;
public class Customer extends Thread
{
Random random_generator = new Random();
public int minimumQueueLength;
public Set set;
public Iterator iterator;
public boolean placed_in_queue;
public List<Integer> queue_length_list;
public CopyOnWriteArrayList till_set = new CopyOnWriteArrayList();
public Till till, till_to_join;
public final Object lock;
public Customer(CopyOnWriteArrayList till_set)
{
this.till_set = till_set;
this.placed_in_queue = false;
queue_length_list = new ArrayList<Integer>();
lock = new Object();
}
public void run()
{
try
{
place_in_queue();
}
catch (InterruptedException e1)
{
// TODO Auto-generated catch block
e1.printStackTrace();
}
if(placed_in_queue)
{
synchronized(this.lock)
{
System.out.println(this.getName()+" waiting");
try {
this.lock.wait();
System.out.println(this.getName()+" has been woken");
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
}
}
else
{
}
}
public void place_in_queue() throws InterruptedException
{
placed_in_queue = false;
iterator = till_set.iterator();
while(iterator.hasNext())
{
till = (Till)iterator.next();
queue_length_list.add(till.customer_queue.size());
}
minimumQueueLength =
queue_length_list.indexOf(Collections.min(queue_length_list));
if(minimumQueueLength < 5)
{
try
{
till_to_join = (Till)till_set.get(minimumQueueLength);
till_to_join.customer_queue.put(this);
placed_in_queue = true;
}
catch (InterruptedException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
Till.java (till thread extends Thread)
import java.util.HashMap;
import java.util.Iterator;
import java.util.Random;
import java.util.concurrent.*;
public class Till extends Thread
{
BlockingQueue<String> item_queue = new ArrayBlockingQueue<String>(200);
BlockingQueue<Customer> customer_queue = new ArrayBlockingQueue<Customer>(10);
public Random random;
public Customer c;
public Till(BlockingQueue<String> item_queue) throws InterruptedException
{
this.item_queue = item_queue;
random = new Random();
}
public void run()
{
while(true)
{
try
{
c = customer_queue.take();
synchronized(c.lock)
{
System.out.println(this.getName()+" Waking up : "+c.getName());
c.lock.notify();
System.out.println(c.getName()+" has been notified!");
}
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
}
CustomerGenerator.java
import java.util.*;
import java.util.concurrent.*;
public class CustomerGenerator extends Thread
{
public int customer_generation_rate;
//0 - slow
//1 - fast
public Random random_generator;
public static BlockingQueue<String> item_queue = new ArrayBlockingQueue<String>(200);
public static CopyOnWriteArrayList till_set = new CopyOnWriteArrayList();
public int i;
public CustomerGenerator(int customer_generation_rate, CopyOnWriteArrayList till_set)
{
this.customer_generation_rate = customer_generation_rate;
this.till_set = till_set;
this.i = 0;
random_generator = new Random();
}
public void run()
{
while(i<1)
{
switch(customer_generation_rate)
{
case 0 : try
{
Thread.sleep(random_generator.nextInt(1000));
}
catch (InterruptedException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
break;
case 1 : try
{
Thread.sleep(random_generator.nextInt(500));
}
catch(InterruptedException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
break;
default : customer_generation_rate = 0;
break;
}
Customer customer = new Customer(till_set);
customer.start();
total_customer_count++;
i++;
}
}
}
Driver.java
import java.util.Iterator;
import java.util.Set;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class Driver
{
public static BlockingQueue<String> item_queue = new ArrayBlockingQueue<>(200);
public static CopyOnWriteArrayList<Till> till_set = new CopyOnWriteArrayList<Till>();
public static Set set;
public static Iterator iterator;
public static int i;
public static final int till_count = 5;
public static Thread till_thread;
public static Till till_object;
public static ExecutorService till_service = Executors.newFixedThreadPool(5);
public static void main(final String[] args) throws InterruptedException
{
for(i=0; i<till_count; i++)
{
till_object = new Till(item_queue);
till_set.add(till_object);
}
final CustomerGenerator customer_generator = new CustomerGenerator(0, till_set);
customer_generator.start();
Thread.sleep(5000);
for(final Till t : till_set)
{
till_service.submit(t);
}
}
}
Output Obtained:
Thread-7 waiting
Thread-1 Waking up : Thread-7
Thread-7 has been notified!
Expected Output:
Thread-7 waiting
Thread-1 Waking up : Thread-7
Thread-7 has been notified!
Thread-7 has been woken
Please help. Thank you. :)
CustomerGenerator generates one customer only when invoked. Making a mcve version of it makes it very clear:
//i was initialized: i=0;
public void run()
{
while(i<1)
{
final Customer customer = new Customer(till_set);
customer.start();
i++;
}
}
I do not think that is what you meant.
I find mcve a very useful technique. Not only it makes helping much easier, it
is a powerful debugging tool. It many case, while preparing one, you are likely to find the problem. mcve should demonstrate the problem, and not your application.
There may be other issues in the code. For more help please post Mcve.
Some other comments:
In CustomerGenerator you pass a reference of all tills to a Customer by:
final Customer customer = new Customer(till_set); which is later used for selecting a till. I think till selection calculation would better be done in another class, say TillsManager which can have a stack of all customers waiting for a till.
In Driver defining
public static Till till_object;
for(i=0; i<5 ; i++)
{
till_object = new Till(item_queue);
till_set.add(till_object);
}
means you will end up with 5 times the same object in till_set. I assume you wanted :
for(i=0; i<till_count; i++)
{
Till till_object = new Till(item_queue);
till_set.add(till_object);
}
Just wait from the Till until it the Queue gets more than zero elements. From the customer thread after adding themself to the queue, notify the Till.
I have read almost all posts related to my problem but could not solve my problem. this code is a question in the Big Java - Early Object of Cay Horstmann. The question asks about counting the word of several files by using multithread programming and store the combined words counting which is my problem.
in order to get the combined counter I used a static variable which is incremented on each of iteration of counting words in threads. Also I used a ReentrantLock() to make incrementing available only for one thread at a time.
everything works fine except incrementing. it seems sometimes the static variable does not incremented. I tested the number of lock() and unlock()
by each thread and they match with my expected result however the static variable does not work properly.
is there any explanation for it? thank you for your time and help.
my task class:
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class WordCount implements Runnable
{
private String fileName;
Scanner inputFile;
private long counter;
public volatile static long combinedCounter = 0;
Lock ccLock;
public WordCount(String aName) throws FileNotFoundException
{
try
{
this.ccLock = new ReentrantLock();
this.counter = 0;
this.fileName = aName;
this.inputFile = new Scanner(new File(this.fileName));
}
catch (FileNotFoundException e)
{}
}
public void cCount()
{
ccLock.lock();
try
{
combinedCounter++;
/*synchronized (this)
{
combinedCounter++;
}*/
}
finally
{
ccLock.unlock();
}
}
#Override
public void run()
{
try
{
while (inputFile.hasNext() && !Thread.interrupted())
{
synchronized (this)
{
cCount();
}
counter++;
inputFile.next();
Thread.sleep(0);
}
System.out.printf("%s: %d\t\t%d\n", this.fileName,
this.counter,combinedCounter);
}
catch (InterruptedException e)
{}
}
}
This is my client class:
import java.io.FileNotFoundException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class WordCountRunner
{
public static void main(String[] args) throws FileNotFoundException,
InterruptedException
{
String a = "a.txt";
String b = "b.txt";
ExecutorService pool = Executors.newFixedThreadPool(2);
;
try
{
Runnable r1 = new WordCount(a);
Runnable r2 = new WordCount(b);
pool.execute(r1);
pool.execute(r2);
while (!pool.isTerminated())
{
pool.shutdown();
}
Thread.sleep(100);
System.out.print("***" + WordCount.combinedCounter);
}
catch (FileNotFoundException e)
{
}
finally
{
pool.shutdown();
}
}
}
The lock does not work because the ReentrantLock is an instance variable in the WordCount class. So, each instance of that class has its own private lock and they don't synchronize with each other. The easiest change would be to make the lock static, like the variable it's protecting.
Each of your Runnables has its own lock object. For your strategy to work, they all need to share exactly one lock.
For example,
// ...
static Lock ccLock = new ReentrantLock();
public WordCount(String aName) throws FileNotFoundException
{
try
{
// this.ccLock = new ReentrantLock();
this.counter = 0;
this.fileName = aName;
this.inputFile = new Scanner(new File(this.fileName));
// ...
}
I'm trying to run a simulation of negotiation between few people (lets say 6).
The negotiation is in pairs, meaning in this case: 3 simultaneous negotiation.
I thought of running 3 threads (neg1.run(), neg2.run(), neg3.run()), but the thing is I want to run them again afterwards once a negotiation is over, and they are free to negotiate again.
This has to be using a thread because few parameters are being changed during the negotiation and I need to have the most updated value of them.
I thought of creating a queue of negotiators that once one of them is finished with the run, he will return to the queue and wait for another "call".
Problem is, i'm not sure how to handle the recalling of a negotiation (launching another thread).
You can create "Negotiation" as a Runnable. And create a thread pool of 3 threads using java.util.concurrent.ThreadPoolExecutor.
Whenever you want to negotiate just call threadpoolexector.execute(). If any thread is available to carry negotiation. It will be done. Else pool will throw exception RejectedExecutionException . That will tell you that no thread is available to carry negotiation.
Try this java.util.concurrent.ThreadPoolExecutor. It might help what you want to achieve.
here's a quick and dirty version. I'm open to suggestions and criticism and will edit it accordingly (or you edit it if you want). Maybe I am creating too many threads?
import java.util.ArrayList;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
public class Main {
public static void main(String[] args) {
ArrayList<Person> personList = new ArrayList<Person>(6);
personList.add(new Person("George"));
personList.add(new Person("John"));
personList.add(new Person("Mary"));
personList.add(new Person("Linda"));
personList.add(new Person("William"));
personList.add(new Person("Albert"));
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors
.newFixedThreadPool(personList.size() / 2);
while (true) {
if (executor.getActiveCount() < executor.getMaximumPoolSize()) {
executor.execute(new Negotiation(personList));
} else {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
public class Person {
String name;
String lastContact = "";
boolean busy;
public Person(String name) {
this.name = name;
}
public synchronized boolean approach() {
if (!busy) {
return busy = true;
} else
return false;
}
public synchronized boolean leave() {
if (busy) {
busy = false;
return true;
} else
return false;
}
public String getName(){
return name;
}
public String getLastContact(){
return lastContact;
}
public void setLastContact(String partnerName){
lastContact = partnerName;
}
}
import java.util.ArrayList;
import java.util.Collections;
import java.util.Random;
public class Negotiation implements Runnable {
Random rand = new Random();
ArrayList<Person> personList = new ArrayList<Person>(6);
Person convStarter;
Person convPartner;
public Negotiation(ArrayList<Person> personList) {
this.personList = personList;
}
private Person findPerson(ArrayList<Person> personList) {
long seed = System.nanoTime();
Collections.shuffle(personList, new Random(seed));
for (Person person : personList) {
if (person.approach()) {
return person;
}
}
return null;
}
#Override
public void run() {
try {
if ((convStarter = findPerson(personList)) != null) {
if ((convPartner = findPerson(personList)) != null) {
if (!convStarter.getLastContact().equals(
convPartner.getName())) {
System.out.println(convStarter.getName()
+ " is talking to " + convPartner.getName());
convStarter.setLastContact(convPartner.getName());
convPartner.setLastContact(convStarter.getName());
Thread.sleep(randInt(2000, 10000)); //your actual negotiation
System.out.println(convStarter.getName()
+ " stopped talking to "
+ convPartner.getName());
}
convPartner.leave();
}
convStarter.leave();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public int randInt(int min, int max) {
return rand.nextInt((max - min) + 1) + min;
}
}