Develop Reference

How to make a block of code execute without interruption Java?

I'm trying to create a web crawler.
I've created a class to handle all URLs visited and to visit.
This class has to be accessed by multiple threads for retrieving and updating those lists.
The problem I'm facing, or at least I think, is in nextRandom() and probably also in next(). I think what is happening is the threads are interfering with each other since the function is somewhat synchronized but not atomic. Is there a way to make so this block of code is executed without any interruption by other threads?
The URL handler
import java.util.*;
import java.util.concurrent.ThreadLocalRandom;
public class UrlHandler {
private volatile Set<String> visited = new HashSet<String>();
private volatile List<String> toVisit = new ArrayList<String>();
public void addToVisit(String url) {
synchronized (this){
if (!visited.contains(url)) toVisit.add(url);
}
}
public void addToVisit(Collection<String> urls) {
synchronized (this){
for (String url : urls)
if (!visited.contains(url)) toVisit.add(url);
}
}
public void addVisited(String url){
synchronized (this){
visited.add(url);
}
}
public void addVisited(Collection<String> urls){
synchronized (this){
visited.addAll(urls);
}
}
public String next() {
while (toVisit.size() == 0) {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
synchronized (this){
String url = toVisit.get(0);
toVisit.remove(0);
return url;
}
}
public String nextRandom() {
synchronized (this){
int n = 0;
if (toVisit.size() > 1){
n = ThreadLocalRandom.current().nextInt(toVisit.size());
}
String url = toVisit.get(n);
toVisit.remove(n);
return url;
}
}
public List<String> getToVisit() {
synchronized (this){
return toVisit;
}
}
public Set<String> getVisited() {
synchronized (this){
return visited;
}
}
}
Web Crawler
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class WebCrawler {
private final ExecutorService executor;
public WebCrawler(int nOfThreads) {
this.executor = Executors.newFixedThreadPool(nOfThreads);
}
public void add(Runnable runnable) {
this.executor.execute(runnable);
}
//Used to shut down safely and wait also 5 of seconds for not finished tasks
public void shutdown() {
this.executor.shutdown();
try {
this.executor.awaitTermination(5, TimeUnit.SECONDS);
if (!this.executor.isTerminated()) {
System.err.println("Timed out waiting for executor to terminate cleanly. Shutting down.");
this.executor.shutdownNow();
}
} catch (final InterruptedException e) {
System.err.println("Interrupted while waiting for executor shutdown.");
Thread.currentThread().interrupt();
}
}
}
Failing test example
import org.junit.jupiter.api.Test;
import java.util.ArrayList;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertEquals;
public class UrlHandlerTest {
List<String> testList = new ArrayList<>(List.of("test1", "test2", "test3", "test3"));
List<String> uniqueTestList = new ArrayList<>(List.of("test1", "test2", "test3"));
UrlHandler urlHandler = new UrlHandler();
#Test
public void concurrentAccess(){
urlHandler.addToVisit(testList);
WebCrawler webCrawler = new WebCrawler(10);
for (int i = 0; i < urlHandler.getToVisit().size(); i++) {
webCrawler.add(new Runnable() {
#Override
public void run() {
String url = urlHandler.nextRandom();
urlHandler.addVisited(url);
System.out.println("Here thread " + Thread.currentThread().getId() + " working on: " + url);
}
});
}
webCrawler.shutdown();
System.out.println(urlHandler.getVisited());
assertEquals(true, urlHandler.getVisited().containsAll(uniqueTestList));
}
}

In the next method this code is a problem:
while (toVisit.size() == 0) {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
The lock isn't held for this part, so size can be stale. Instead of this, try something like
while (toVisit.size() == 0)
wait();
Do this in a synchronized block so you have the lock held while checking the collection size. Code that adds to the collection should notify in order to wake up the waiting threads.

This piece of code is problematic:
for (int i = 0; i < urlHandler.getToVisit().size(); i++) {
webCrawler.add(new Runnable() {
// ...
});
}
The urlHandler.getToVisit().size() is always changing during the traversal, and there is uncertainty (because the size will be changed asynchronously).
Change to:
int size = urlHandler.getToVisit().size();
for (int i = 0; i < size; i++) {
webCrawler.add(new Runnable() {
// ...
});
}

CyclicBarrier misunderstanding

I tried to run example with CyclicBarrier from one of tutorials:
Service man should fill empty printers when the queue of empty printers is 3.
But when I ran the code it appears that printers are filled with 2, 3 or 4 empty printers in the queue:
Printer1 is empty
Printer12 is empty
Printer14 is empty
Printer13 is empty
Filling [Printer1, Printer12, Printer14, Printer13]
Printer2 is empty
Printer7 is empty
Filling [Printer2, Printer7]
So is the example wrong or my understanding of CyclicBarrier? I consider that queue should be exactly 3 elements size. What should I add to the code to fix that? Thanks in advance.
Code:
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.BrokenBarrierException;
import java.util.concurrent.CyclicBarrier;
import java.util.concurrent.TimeUnit;
public class PrinterRecharger {
public static void main(String args[]) {
ServiceMan serviceMan = new ServiceMan(3);
for (int i = 0; i < 15; i++) {
new Thread(new Printer(serviceMan, "Printer" + (i + 1))).start();
}
}
}
class ServiceMan {
private CyclicBarrier queue;
private List<String> inQueue;
public ServiceMan(int hardWorking) {
inQueue = new ArrayList<String>();
queue = new CyclicBarrier(hardWorking, new Runnable() {
#Override
public void run() {
System.out.println("Filling " + inQueue);
inQueue.clear();
}
});
}
public void recharge(String name) {
try {
inQueue.add(name);
queue.await();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}
}
class Printer implements Runnable {
private String name;
private Random rand;
private ServiceMan serviceMan;
public Printer(ServiceMan serviceMan, String name) {
this.name = name;
this.serviceMan = serviceMan;
this.rand = new Random();
}
public void run() {
try {
while (true) {
TimeUnit.SECONDS.sleep(rand.nextInt(10));
System.out.println(name + " is empty");
serviceMan.recharge(name);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

Your code is thread-unsafe in several ways that I can immediately see, and probably some more which I have missed. You have data races as well as other types of race conditions.
ArrayList is not a thread-safe class, but you use it from multiple threads with no synchronization. Wrap the list into a Collections.synchronizedList() to see some improvement.
You lack any mutual exclusion between recharge() and the CyclicBarrier's action. A thread could add an item to the queue only to have it cleared away by the action.

Is my producer/consumer solution correct?

I'm trying to learn more about threads and thought that coming up with a solution to the producer/consumer problem would be a good start. One of the constraints I put on the solution was that the consumer does not know ahead of time how much the producer is producing. The code runs as expected and I've run it many many times, but that doesn't mean that it is free of flaws. Are there any problems with this solution?
package Multithreading.ProducerConsumer;
import java.util.LinkedList;
import java.util.concurrent.Semaphore;
public class ProducerConsumer
{
private class Producer implements Runnable
{
#Override
public void run()
{
for(int i = 0; i < 1000; i++)
{
try
{
canProduce.acquire();
mutex.acquire();
queue.add(i);
mutex.release();
canConsume.release();
}
catch (InterruptedException ex)
{
;
}
}
try
{
canConsume.acquire();
isTryingToFinish = true;
canConsume.release();
}
catch (InterruptedException ex)
{
;
}
}
}
private class Consumer implements Runnable
{
#Override
public void run()
{
while(!isDone)
{
try
{
canConsume.acquire();
mutex.acquire();
System.out.println(queue.pop());
if(isTryingToFinish && queue.isEmpty())
{
isDone = true;
}
mutex.release();
canProduce.release();
}
catch (InterruptedException ex)
{
;
}
}
}
}
Semaphore canProduce;
Semaphore canConsume;
Semaphore mutex;
boolean isTryingToFinish = false;
boolean isDone = false;
final static int bufferSize = 100;
LinkedList<Integer> queue;
public ProducerConsumer()
{
queue = new LinkedList<>();
canProduce = new Semaphore(bufferSize);
canConsume = new Semaphore(0);
mutex = new Semaphore(1);
}
public void Go() throws InterruptedException
{
Thread p = new Thread(new Producer());
Thread c = new Thread(new Consumer());
p.start();
c.start();
p.join();
c.join();
System.out.println("Job Complete!");
}
public static void main(String[] args) throws InterruptedException
{
ProducerConsumer p = new ProducerConsumer();
p.Go();
}
}

You could look at MSDN's 'Example 2: Synchronizing two threads: a producer and a consumer'. It's c# but that should not be a problem.

Timeout for a method

My Program looks like below
Main Program (Thread 1)
Create multiple simple java threads (Thead 1.1, 1.2...)
In each thread(1.1 or 1.2..) I'm doing some processing also calling one method which is
sometimes is not responding(CORBA calls). I want to define timer for
this method and thread(1.1 or 1.2 whoever is calling) should wait there itself till I get response or timer expired.
I have written following sample program. I don't think this is the right approach. Is there any better approach? In this prg I'm not sure when the interupt method is invoked.
public class MethodTimeout implements Runnable{
/**
* #param args
*/
public Thread t1 = null;
public int threadnum = 0;
public static void main(String[] args) {
for (int i=0; i<3; i++){
MethodTimeout mt =new MethodTimeout();
Thread t = new Thread(mt,"thread "+(i+1));
mt.t1 = t;
mt.threadnum = (i+1);
t.start();
}
System.out.println("stmt after execution");
}
public Object testTimeout(){
long startTime = System.currentTimeMillis();
try {
System.out.println("in side method start "+t1.getName()+" start time"+startTime);
Thread.sleep(5000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
long endtime = System.currentTimeMillis();
System.out.println("in side method end "+t1.getName()+" total time"+(endtime-startTime) );
return null;
}
#Override
public void run() {
Thread timeout = new Thread (){
public void run() {
testTimeout();
};
};
timeout.start();
try {
Thread.sleep(2000);
timeout.interrupt();
Thread.sleep(2000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println(t1.getName() + " is ending");
}
}

This very much sounds like you should implement Callable. This is just an example
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
public class Test {
public static void main(String[] args) throws Exception {
ExecutorService service = Executors.newFixedThreadPool(2);
Future<String> futureResult = service.submit(new MyCall());
try{
String result = futureResult.get(20, TimeUnit.MILLISECONDS);
} catch(TimeoutException timeout){
System.out.println("Timeout");
service.shutdownNow();
}
}
static class MyCall implements Callable<String> {
#Override
public String call() throws Exception {
try{
//Simulate some corba work
Thread.sleep(1000);
}catch(InterruptedException e){
Thread.currentThread().interrupt();
System.out.println("Shutting down the task!");
}
return "The result";
}
}
}

You can also make one minor change to #Eugene's answer, that is instead of calling the shutdownNow() on the ExecutorService itself you can just call cancel(true) on the futureResult that timed out. Here is the code snippet:
public class Test {
public static void main(String[] args) throws Exception {
ExecutorService service = Executors.newFixedThreadPool(2);
Future<String> futureResult = service.submit(new MyCall());
try{
String result = futureResult.get(20, TimeUnit.MILLISECONDS);
} catch(TimeoutException timeout){
System.out.println("Timeout");
} finally {
futureResult.cancel(true);
}
}
This is just to ensure that only the timed out thread is cancelled. As the shutdownNow() prevents waiting tasks from starting in addition to attempting to stop currently executing ones.

Wait and Notify issue

Earlier I posted a problem about implementing wait and notify, but I wasn't very clear, so here is a more specific question.
In the long code block below there is one wait and one notify. The notify is supposed to stop the wait and cause it to stop waiting. At the moment, i think the wait works, but the notify does not. Could someone explain why the notify doesn't notify the wait? Thanks!
Note: the rest of the code works i'm only interested in these two specific parts.
import com.fmr.ipgt.email.*;
import java.io.File;
import java.io.IOException;
import java.util.List;
import org.jdom.Document;
import org.jdom.Element;
import org.jdom.JDOMException;
import org.jdom.input.SAXBuilder;
import javax.mail.MessagingException;
class MyQuery {
synchronized void qQuery() throws Exception {
String query = ".z.k"; // The query that is used to query q; this can be changed here.
int version = 0;
c qConn = null;
qConn = new c(Main.host,Main.port); // Connect to the q database
while (Main.healthy) {
Object o = qConn.k(query); // Query q
version = c.t(o);
if(!(version==0)) {
System.out.println(version);
System.out.println("database healthy");
NewThread.suspendFlag = false;
notify();
break; // End the process if the database responds
}
}
System.out.println("reaches loop end");
}
}
class MyThread implements Runnable {
MyQuery myResource;
MyThread(String name, MyQuery so) {
myResource = so;
new Thread(this, name).start();
}
public void run() {
try {
myResource.qQuery(); // Begin a method to query q.
} catch (Exception e) {
e.printStackTrace();
}
}
}
class NewThread implements Runnable {
String name; // name of thread
Thread t;
static boolean suspendFlag;
private int minutes;
NewThread(int minutes) {
this.minutes = minutes;
System.out.println("reaches constructor");
t = new Thread(this);
suspendFlag = true;
t.start(); // Start the thread
}
// This is the entry point for thread.
public void run() {
try {
synchronized(this) {
while(suspendFlag) {
System.out.println("reaches wait");
wait(minutes*60000);
System.out.println("reaches end");
if(suspendFlag) {
Main.setHealth(false);
Main.sendMessages(); // The database has not responded for the given time. Report that it is unhealthy.
}
break;
}
}
} catch (InterruptedException e) {
System.out.println(name + " interrupted.");
} catch (MessagingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public class Main {
private static String[] recipients;
private static String subject = "Database Failure";
private static String message = "The database has failed or is in a hung state";
private static String from;
static String host;
static int port;
private static String emails;
private static int minutes;
static boolean healthy = true;
public static void main(String args[]) throws Exception {
// Import information from the configuration file
SAXBuilder builder = new SAXBuilder();
File xmlFile = new File("/export/home/rhadm/file.xml"); // Note: The directory for the configuration file may need to be changed
try {
Document document = (Document) builder.build(xmlFile);
Element rootNode = document.getRootElement();
List list = rootNode.getChildren("parameters");
Element node = (Element) list.get(0);
host = node.getChildText("host");
port = Integer.parseInt(node.getChildText("port"));
emails = node.getChildText("emails");
String delims = "[ ]+";
recipients = emails.split(delims); // parse email list
minutes = Integer.parseInt(node.getChildText("time"));
from = node.getChildText("from");
} catch (IOException io) {
System.out.println(io.getMessage());
} catch (JDOMException jdomex) {
System.out.println(jdomex.getMessage());
}
MyQuery unhealthy = new MyQuery();
NewThread ob1 = new NewThread(minutes);
new MyThread("MyThread", unhealthy); // Create new Thread
}
public static void setHealth(boolean health){
System.out.println("database unhealthy");
healthy = health;
}
public static void sendMessages() throws MessagingException {
System.out.println("sending emails");
FCAPMailSender.postMail(recipients,subject,message,from);
}
}

You are synchronizing on different objects. The notify will only effect objects synchronized-waiting on the same object & instance.
The waiting thread is synchronized & waiting on a NewThread while the notifying thread is doing so on a MyQuery instance
Have a shared object.
private final Object LOCK = new Object();
synchronized(LOCK){
LOCK.wait();
}
synchronized(LOCK){
LOCK.notify();
}