Develop Reference

How to make a block of code execute without interruption Java?

I'm trying to create a web crawler.
I've created a class to handle all URLs visited and to visit.
This class has to be accessed by multiple threads for retrieving and updating those lists.
The problem I'm facing, or at least I think, is in nextRandom() and probably also in next(). I think what is happening is the threads are interfering with each other since the function is somewhat synchronized but not atomic. Is there a way to make so this block of code is executed without any interruption by other threads?
The URL handler
import java.util.*;
import java.util.concurrent.ThreadLocalRandom;
public class UrlHandler {
private volatile Set<String> visited = new HashSet<String>();
private volatile List<String> toVisit = new ArrayList<String>();
public void addToVisit(String url) {
synchronized (this){
if (!visited.contains(url)) toVisit.add(url);
}
}
public void addToVisit(Collection<String> urls) {
synchronized (this){
for (String url : urls)
if (!visited.contains(url)) toVisit.add(url);
}
}
public void addVisited(String url){
synchronized (this){
visited.add(url);
}
}
public void addVisited(Collection<String> urls){
synchronized (this){
visited.addAll(urls);
}
}
public String next() {
while (toVisit.size() == 0) {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
synchronized (this){
String url = toVisit.get(0);
toVisit.remove(0);
return url;
}
}
public String nextRandom() {
synchronized (this){
int n = 0;
if (toVisit.size() > 1){
n = ThreadLocalRandom.current().nextInt(toVisit.size());
}
String url = toVisit.get(n);
toVisit.remove(n);
return url;
}
}
public List<String> getToVisit() {
synchronized (this){
return toVisit;
}
}
public Set<String> getVisited() {
synchronized (this){
return visited;
}
}
}
Web Crawler
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class WebCrawler {
private final ExecutorService executor;
public WebCrawler(int nOfThreads) {
this.executor = Executors.newFixedThreadPool(nOfThreads);
}
public void add(Runnable runnable) {
this.executor.execute(runnable);
}
//Used to shut down safely and wait also 5 of seconds for not finished tasks
public void shutdown() {
this.executor.shutdown();
try {
this.executor.awaitTermination(5, TimeUnit.SECONDS);
if (!this.executor.isTerminated()) {
System.err.println("Timed out waiting for executor to terminate cleanly. Shutting down.");
this.executor.shutdownNow();
}
} catch (final InterruptedException e) {
System.err.println("Interrupted while waiting for executor shutdown.");
Thread.currentThread().interrupt();
}
}
}
Failing test example
import org.junit.jupiter.api.Test;
import java.util.ArrayList;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertEquals;
public class UrlHandlerTest {
List<String> testList = new ArrayList<>(List.of("test1", "test2", "test3", "test3"));
List<String> uniqueTestList = new ArrayList<>(List.of("test1", "test2", "test3"));
UrlHandler urlHandler = new UrlHandler();
#Test
public void concurrentAccess(){
urlHandler.addToVisit(testList);
WebCrawler webCrawler = new WebCrawler(10);
for (int i = 0; i < urlHandler.getToVisit().size(); i++) {
webCrawler.add(new Runnable() {
#Override
public void run() {
String url = urlHandler.nextRandom();
urlHandler.addVisited(url);
System.out.println("Here thread " + Thread.currentThread().getId() + " working on: " + url);
}
});
}
webCrawler.shutdown();
System.out.println(urlHandler.getVisited());
assertEquals(true, urlHandler.getVisited().containsAll(uniqueTestList));
}
}

In the next method this code is a problem:
while (toVisit.size() == 0) {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
The lock isn't held for this part, so size can be stale. Instead of this, try something like
while (toVisit.size() == 0)
wait();
Do this in a synchronized block so you have the lock held while checking the collection size. Code that adds to the collection should notify in order to wake up the waiting threads.

This piece of code is problematic:
for (int i = 0; i < urlHandler.getToVisit().size(); i++) {
webCrawler.add(new Runnable() {
// ...
});
}
The urlHandler.getToVisit().size() is always changing during the traversal, and there is uncertainty (because the size will be changed asynchronously).
Change to:
int size = urlHandler.getToVisit().size();
for (int i = 0; i < size; i++) {
webCrawler.add(new Runnable() {
// ...
});
}

How to prevent deadlocks in synchronized methods?

In the following Code there is a potential to enter a Deadlock similar to this Question "Deadlocks and Synchronized methods", now i understand why the two Threads are entering a
deadlock, but when i execute the code the Threads always enters a Deadlock so:
1 - When is a Deadlock not possible in this code ?
2 - How to prevent it from happening ?
I tried using wait() and notifyAll() like this :
wait()
waver.waveBack(this)
and then calling notifyAll() in waveBack(), but it didn't work what am i missing or misunderstood ?
package mainApp;
public class Wave {
static class Friend {
private final String name;
public Friend(String name) {
this.name = name;
}
public String getName() {
return this.name;
}
public synchronized void wave(Friend waver) {
String tmpname = waver.getName();
System.out.printf("%s : %s has waved to me!%n", this.name, tmpname);
waver.waveBack(this);
}
public synchronized void waveBack(Friend waver) {
String tmpname = waver.getName();
System.out.printf("%s : %s has waved back to me!%n", this.name, tmpname);
}
}
public static void main(String[] args) {
final Friend friendA = new Friend("FriendA");
final Friend friendB = new Friend("FriendB");
new Thread(new Runnable() {
public void run() {
friendA.wave(friendB);
}
}).start();
new Thread(new Runnable() {
public void run() {
friendB.wave(friendA);
}
}).start();
}
}

In this case, simply do not call another method that might need the lock while holding the lock. This ensures that there is always a moment in time where a method can get the lock and progress can be made.
Calling wait() before waver.waveBack(this) causes a chicken and egg problem: waveBack(this) is never called because the thread stops execution at the wait() statement and thus notifyAll() is never called to continue execution.
There are various ways to prevent deadlocks in the context of the example, but let's go with the advice from sarnold in one of the comments in his answer from the question you linked. To paraphrase sarnold: "it is usually easier to reason about locks on data".
Let's assume that the synchronized methods are synchronized to ensure some consistent update of state (i.e. some variables need to be updated but only one thread at any given time can modify these variables). For example, let's register the amount of waves send and waves received. The runnable code below should demonstrate this:
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
public class Wave {
static class Waves {
final Map<Friend, Integer> send = new HashMap<>();
final Map<Friend, Integer> received = new HashMap<>();
void addSend(Friend f) {
add(f, send);
}
void addReceived(Friend f) {
add(f, received);
}
void add(Friend f, Map<Friend, Integer> m) {
m.merge(f, 1, (i, j) -> i + j);
}
}
static class Friend {
final String name;
public Friend(String name) {
this.name = name;
}
final Waves waves = new Waves();
void wave(Friend friend) {
if (friend == this) {
return; // can't wave to self.
}
synchronized(waves) {
waves.addSend(friend);
}
friend.waveBack(this); // outside of synchronized block to prevent deadlock
}
void waveBack(Friend friend) {
synchronized(waves) {
waves.addReceived(friend);
}
}
String waves(boolean send) {
synchronized(waves) {
Map<Friend, Integer> m = (send ? waves.send : waves.received);
return m.keySet().stream().map(f -> f.name + " : " + m.get(f))
.sorted().collect(Collectors.toList()).toString();
}
}
#Override
public String toString() {
return name + ": " + waves(true) + " / " + waves(false);
}
}
final static int maxThreads = 4;
final static int maxFriends = 4;
final static int maxWaves = 50_000;
public static void main(String[] args) {
try {
List<Friend> friends = IntStream.range(0, maxFriends)
.mapToObj(i -> new Friend("F_" + i)).collect(Collectors.toList());
ExecutorService executor = Executors.newFixedThreadPool(maxThreads);
Random random = new Random();
List<Future<?>> requests = IntStream.range(0, maxWaves)
.mapToObj(i -> executor.submit(() ->
friends.get(random.nextInt(maxFriends))
.wave(friends.get(random.nextInt(maxFriends)))
)
).collect(Collectors.toList());
requests.stream().forEach(f ->
{ try { f.get(); } catch (Exception e) { e.printStackTrace(); } }
);
executor.shutdownNow();
System.out.println("Friend: waves send / waves received");
friends.stream().forEachOrdered(p -> System.out.println(p));
} catch (Exception e) {
e.printStackTrace();
}
}
}

CyclicBarrier misunderstanding

I tried to run example with CyclicBarrier from one of tutorials:
Service man should fill empty printers when the queue of empty printers is 3.
But when I ran the code it appears that printers are filled with 2, 3 or 4 empty printers in the queue:
Printer1 is empty
Printer12 is empty
Printer14 is empty
Printer13 is empty
Filling [Printer1, Printer12, Printer14, Printer13]
Printer2 is empty
Printer7 is empty
Filling [Printer2, Printer7]
So is the example wrong or my understanding of CyclicBarrier? I consider that queue should be exactly 3 elements size. What should I add to the code to fix that? Thanks in advance.
Code:
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.BrokenBarrierException;
import java.util.concurrent.CyclicBarrier;
import java.util.concurrent.TimeUnit;
public class PrinterRecharger {
public static void main(String args[]) {
ServiceMan serviceMan = new ServiceMan(3);
for (int i = 0; i < 15; i++) {
new Thread(new Printer(serviceMan, "Printer" + (i + 1))).start();
}
}
}
class ServiceMan {
private CyclicBarrier queue;
private List<String> inQueue;
public ServiceMan(int hardWorking) {
inQueue = new ArrayList<String>();
queue = new CyclicBarrier(hardWorking, new Runnable() {
#Override
public void run() {
System.out.println("Filling " + inQueue);
inQueue.clear();
}
});
}
public void recharge(String name) {
try {
inQueue.add(name);
queue.await();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}
}
class Printer implements Runnable {
private String name;
private Random rand;
private ServiceMan serviceMan;
public Printer(ServiceMan serviceMan, String name) {
this.name = name;
this.serviceMan = serviceMan;
this.rand = new Random();
}
public void run() {
try {
while (true) {
TimeUnit.SECONDS.sleep(rand.nextInt(10));
System.out.println(name + " is empty");
serviceMan.recharge(name);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

Your code is thread-unsafe in several ways that I can immediately see, and probably some more which I have missed. You have data races as well as other types of race conditions.
ArrayList is not a thread-safe class, but you use it from multiple threads with no synchronization. Wrap the list into a Collections.synchronizedList() to see some improvement.
You lack any mutual exclusion between recharge() and the CyclicBarrier's action. A thread could add an item to the queue only to have it cleared away by the action.

How to implement forkjoin for a series of action which is done in for loop for now

I have a list of senders for them I have to parallely send mails individually.Currently I am iterating over the list construct the body (as it is different for different people) and then sending them. How can I use forkjoin for this. I tried using recusiveAction but I guess its only for recursive tasks.
All the examples available in internet are implemented with RecursiveAction. Is there any other class with which I can implement this.

ServiceExecutors work nicely for this. They come with Java.
import java.util.*;
import java.util.concurrent.*;
public class SendMailExample
{
public static void main(String[] args) throws Exception
{
ExecutorService executor = Executors.newFixedThreadPool(3);
Collection<Future> futures = new ArrayList<Future>();
futures.add(executor.submit(new Mailer("thread1")));
futures.add(executor.submit(new Mailer("thread2")));
futures.add(executor.submit(new Mailer("thread3")));
for (Future future : futures)
{
future.get();
}
executor.shutdown();
}
static class Mailer implements Runnable
{
private Object message;
public Mailer(Object message)
{
this.message = message;
}
public void run()
{
System.out.println("Sending message " + String.valueOf(message));
}
}
}

I browsed I got a better answer:
package Test1;
import java.util.*;
import java.util.concurrent.*;
import static java.util.Arrays.asList;
public class Sums
{
static class Sum implements Callable<Long>
{
private final long from;
private final long to;
Sum(long from, long to)
{
this.from = from;
this.to = to;
}
#Override
public Long call()
{
long acc = 0;
if(from == 0)
{
try
{
Thread.sleep(5000);
}
catch (InterruptedException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
System.out.println(from);
for (long i = from; i <= to; i++)
{
acc = acc + i;
}
return acc;
}
}
public static void main(String[] args) throws Exception
{
ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
List <Future<Long>> results = executor.invokeAll(asList(
new Sum(0, 10), new Sum(100, 1000), new Sum(10000, 1000000)
));
executor.shutdown();
for (Future<Long> result : results)
{
System.out.println(result.get());
}
}
}
With this code, you will be able to get the response and also any exceptions that are thrown.

how to restart a thread

I tried to write a file monitor which will check the file if a new line is appended,the monitor in fact is a thread which will read the line by a randomaccessfile all the time.
This is the monitor core codes:
public class Monitor {
public static Logger log = Logger.getLogger(Monitor.class);
public static final Monitor instance = new Monitor();
private static final ArrayList<Listener> registers = new ArrayList<Listener>();
private Runnable task = new MonitorTask();
private Thread monitorThread = new Thread(task);
private boolean beStart = true;
private static RandomAccessFile raf = null;
private File monitoredFile = null;
private long lastPos;
public void register(File f, Listener listener) {
this.monitoredFile = f;
registers.add(listener);
monitorThread.start();
}
public void replaceFile(File newFileToBeMonitored) {
this.monitoredFile = newFileToBeMonitored;
// here,how to restart the monitorThread?
}
private void setRandomFile() {
if (!monitoredFile.exists()) {
log.warn("File [" + monitoredFile.getAbsolutePath()
+ "] not exist,will try again after 30 seconds");
try {
Thread.sleep(30 * 1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
setRandomFile();
return;
}
try {
if (raf != null) {
raf.close();
lastPos = 0;
}
raf = new RandomAccessFile(monitoredFile, "r");
log.info("monitor file " + monitoredFile.getAbsolutePath());
} catch (FileNotFoundException e) {
// The file must exist now
} catch (IOException e) {}
}
private void startRead() {
beStart = true;
String line;
while (beStart) {
try {
raf.seek(lastPos);
while ((line = raf.readLine()) != null) {
fireEvent(new FileEvent(monitoredFile.getAbsolutePath(),
line));
}
lastPos = raf.getFilePointer();
} catch (IOException e1) {}
}
}
private void stopRead() {
this.beStart = false;
}
private void fireEvent(FileEvent event) {
for (Listener lis : registers) {
lis.lineAppended(event);
}
}
private class MonitorTask implements Runnable {
#Override
public void run() {
stopRead();
//why putting the resetReandomAccessFile in this thread method is that it will sleep if the file not exist.
setRandomFile();
startRead();
}
}
}
This is some help classes:
public interface Listener {
void lineAppended(FileEvent event);
}
public class FileEvent {
private String line;
private String source;
public FileEvent(String filepath, String addedLine) {
this.line = addedLine;
this.source = filepath;
}
//getter and setter
}
And this is a example to call the monitor:
public class Client implements Listener {
private static File f = new File("D:/ab.txt");
public static void main(String[] args) {
Monitor.instance.register(f, new Client());
System.out.println(" I am done in the main method");
try {
Thread.sleep(5000);
Monitor.instance.replaceFile(new File("D:/new.txt"));
} catch (InterruptedException e) {
System.out.println(e.getMessage());
}
}
#Override
public void lineAppended(FileEvent event) {
String line = event.getLine();
if (line.length() <= 0)
return;
System.err.println("found in listener:" + line + ":" + line.length());
}
}
Now,my probelm is the code work well if I just call:
Monitor.instance.register(file,listener);
This will monitor the file for line appending,and will notify the listener.
However it does not work when I call the :
Monitor.instance.replaceFile(anotherfile);
This means I want to monitor another file rather than before.
So in my Monitor I have to restart the thread,how to make it?
I have tried the:
monitorThread.interruppt();
It does not wrok.
Anyone can fix it for me or tell me how to do ?
Thanks.
Before I ask,I have googling the "restart java thread",so I know one can not restart a dead thread,but my thread does not return,so I think it can be restarted.

You don't restart a Thread, instead you create a new one each time you want to start a thread.
A better alternative may be to use Executors.newCachedThreadPool() which gives you a pool of thread which will be started/recycle for you.
BTW: You are using recursion rather than a loop to poll if the file exists. Using recursion can mean if you wait too long it will throw a StackOverflowError. IMHO you shouldn't wait at all, the polling thread should repeatedly attempt to open the file until it is told to stop (or the file appears)
Your current implementation also means if the file is replaced, you will have to reopen the file in the background thread anyway.

Instead of explaining, I just coded up a skeleton example. I did not test it terribly well, but it may be of some use.
In order to monitor a(nother) file, just create a new Monitor, passing it a ScheduledExecutorService. Starting and stopping monitoring is straightforward. You can (should) reuse the same executor for multiple monitors.
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
public interface Event
{
}
public interface Listener
{
void handle(Event event);
}
public class Monitor
{
private static final int CHECK_EVERY_SECONDS = 10;
private static final int RECHECK_AFTER_IF_NOT_EXISTS_SECONDS = 30;
private File file;
private ScheduledExecutorService executor;
private boolean active;
private List<Listener> listeners;
public Monitor(File file, ScheduledExecutorService executor)
{
super();
this.file = file;
this.executor = executor;
listeners = new ArrayList<Listener>();
}
public synchronized void start()
{
if (active)
{
return;
}
active = true;
executor.execute(new Runnable()
{
public void run()
{
synchronized (Monitor.this)
{
if (!active)
{
System.out.println("not active");
return;
}
}
if (!file.exists())
{
System.out.println("does not exist, rescheduled");
executor.schedule(this, RECHECK_AFTER_IF_NOT_EXISTS_SECONDS, TimeUnit.SECONDS);
return;
}
Event event = doStuff(file);
System.out.println("generated " + event);
updateListeners(event);
System.out.println("updated listeners and rescheduled");
executor.schedule(this, CHECK_EVERY_SECONDS, TimeUnit.SECONDS);
}
});
}
private Event doStuff(final File file)
{
return new Event()
{
public String toString()
{
return "event for " + file;
}
};
}
public synchronized void stop()
{
active = false;
}
public void addListener(Listener listener)
{
synchronized (listeners)
{
listeners.add(listener);
}
}
public void removeListener(Listener listener)
{
synchronized (listeners)
{
listeners.remove(listener);
}
}
private void updateListeners(Event event)
{
synchronized (listeners)
{
for (Listener listener : listeners)
{
listener.handle(event);
}
}
}
public static void main(String[] args) throws IOException
{
ScheduledExecutorService executor = Executors.newScheduledThreadPool(4);
File file = new File("test.png");
Monitor monitor = new Monitor(file, executor);
monitor.addListener(new Listener()
{
public void handle(Event event)
{
System.out.println("handling " + event);
}
});
monitor.start();
System.out.println("started...");
System.in.read();
monitor.stop();
System.out.println("done");
executor.shutdown();
}
}

See this post How to start/stop/restart a thread in Java?

I assume you answered your question
one can not restart a dead thread
This link may be helpful to you How to restart thread in java?

A thread in Java cannot be re-started. Every time you need to restart the thread you must make a new one.
That said, you might want to look at:
private void setRandomFile() {
if (!monitoredFile.exists()) {
log.warn("File [" + monitoredFile.getAbsolutePath()
+ "] not exist,will try again after 30 seconds");
try {
Thread.sleep(30 * 1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
setRandomFile();
return;
}
// ....
}
Here you sleep for 30 seconds if the file does not exist, then recursively call the same function. Now, I don't know what business requirements you have, but if this recursion ran long enough you will run out of stack space. Perhaps you will be better served with a while loop or even better, a little synchronisation like a Semaphore.