I have an executor service that submits x amount of threads concurrently to do a long task. I need to be able to stop all the current threads that are running and prevent queued tasks from starting. I am trying to implement a way to handle stopping threads that are waiting for a synchronized method in which the runnable passes a list of strings back to the interface that called it.
#Override
public synchronized void FilterResults(List<String> Results) {
//System.out.println("Result found: " + Results.size());
try {
Set<String> hs = new HashSet<>();
hs.addAll(Results);
Results.clear();
Results.addAll(hs);
for (String tempURL : Results) {
//System.out.println("Found url: " + tempURL);
if (!isCompleted(tempURL) && !isQueued(tempURL) && !isRunning(tempURL)) {
System.out.println("Added: " + tempURL + " to queue.");
queueLink(tempURL);
startNewThread(tempURL);
}
}
}catch(Exception e) {
}
return;
}
private synchronized void startNewThread(String seedURL) {
if (!isCompleted(seedURL) && !isRunning(seedURL) ) {
if (completedSize("") + runningSize() > 99) {
Stop();
}
String tempProxy = "";
String tempPort = "";
if (UseProxies) {
String Proxy = grabFreeProxy();
String[] splitProxy = Proxy.split(":");
tempProxy = splitProxy[0]; // 004
tempPort = splitProxy[1]; // 034556
}
//System.out.println("Proxy: " + tempProxy);
//System.out.println("Port: " + tempPort);
execService.submit(new Crawl(seedURL, this, tempProxy, tempPort, UseProxies));
removeFromQueue(url);
}
}
#Override
public Collection<String> Stop() {
try {
execService.shutdown();
if (execService.awaitTermination(45, TimeUnit.SECONDS)) {
System.out.println("task completed");
} else {
execService.shutdownNow();
}
} catch (InterruptedException e) {
}
return PROFILES;
}
The Runnable
public class Crawl implements Runnable{
public void run() {
while(!Thread.currentThread().isInterrupted() && shutdown == false) {
try {
//System.out.println(crawler.queueSize());
Thread.sleep(100);
Crawl(url);
}catch (InterruptedException e) {
Thread.currentThread().interrupt(); // set interrupt flag
}
}
public void crawl(){
try {
submitResults(urls); //Calls FilterResults()
} catch (InterruptedException e) {
// TODO Auto-generated catch block
//e.printStackTrace();
Thread.currentThread().interrupt();
}
crawler.removeUsedProxy(Proxy + ":" + Port);
this.shutdown();
}
}
When I call my shutdown method it takes 45 seconds+ is there anyway to reliably cancel the task without the long wait? This number grows as I have more threads, and since all the threads are blocking waiting to submit the results, it can take some time. If I cancel the task manually I do not care if the results are stored, I just need to be able to cancel. Any ideas?
Update I've tried ExecutorService#shutdownNow. It has not been reliable
when it comes to killing the tasks that are still blocked on the synchronized method.
Looks like you need to use ExecutorService#shutdownNow in case you don't want to wait and finish all the work and you'll receive a list with the tasks that weren't executed. You may use ExecutionService#awaitTermination (with different parameters than 45 seconds) if you want/need to provide a time to wait for the tasks to finish.
Related
I've written an endless loop in which I want to send a User Message every 5 seconds. Therefore I wrote a thread which waits for 5 seconds and then sends the Message received by the readLine() Method. If the user doesn't give any input the loop doesn't go on because of the readLine() Method waiting for input. So how can I cancel the readLine() Method?
while (true) {
new Thread() {
#Override
public void run() {
try {
long startTime = System.currentTimeMillis();
while ((System.currentTimeMillis() - startTime) < 5000) {
}
toClient.println(serverMessage);
clientMessage = fromClient.readLine();
System.out.println(clientName + ": " + clientMessage);
} catch (IOException e) {
e.printStackTrace();
}
}
}.start();
serverMessage = input.readLine();
}
This looks to be a producer-consumer type problem and I would structure this entirely differently since this fromClient.readLine(); is blocking and thus should be performed within another thread.
So consider reading the user input in another thread into a data structure, a Queue<String> such as a LinkedBlockingQueue<String>, and then retrieve String elements from the queue in the code above every 5 seconds, or nothing if no elements are held in the queue.
Something like....
new Thread(() -> {
while (true) {
try {
blockingQueue.put(input.readLine());
} catch (InterruptedException | IOException e) {
e.printStackTrace();
}
}
}).start();
new Thread(() -> {
try {
while (true) {
try {
TimeUnit.SECONDS.sleep(5);
} catch (InterruptedException e) {
e.printStackTrace();
}
String input = blockingQueue.poll();
input = input == null ? "" : input;
toClient.println(input);
}
} catch (IOException e) {
e.printStackTrace();
}
}).start();
Side notes: don't call .stop() on a thread as that is a dangerous thing to do. Also avoid extending Thread.
This is my Class. I am using a Quartz scheduler and in that once a job is toBeExecuted, I wanted to avoid concurrency..hence used Synchronize keyword.. and used wait for each thread but it seems that once job is executed..Notify doesnt call the waiting thread ..please help...stuck on this from last two days:
public class SJobListener implements JobListener {
public static final String LISTENER_NAME = "SchedulerJobListener";
ExecutingClass compSched = new ExecutingClass();
#Override
public String getName() {
return LISTENER_NAME; //must return a name
}
// Run this if job is about to be executed.
#Override
public void jobToBeExecuted(JobExecutionContext context) {
String jobName = context.getJobDetail().getKey().toString();
System.out.println("jobToBeExecuted");
System.out.println("Listener : Job : " + jobName + " is going to start...");
System.out.println("Thread running in jobToBeExecuted :"+Thread.currentThread().getName()+" "+Thread.currentThread().getId());
synchronized (compSched) {
if(!condition)
try {
System.out.println("Going to Wait");
Thread.currentThread().wait(200);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
//Run this after job has been executed
#Override
public void jobWasExecuted(JobExecutionContext context,
JobExecutionException jobException) {
System.out.println("jobWasExecuted");
String jobName = context.getJobDetail().getKey().toString();
System.out.println("Listener :Job : " + jobName + " is finished...");
System.out.println("Thread running in jobWasExecuted :"+Thread.currentThread().getName()+" "+Thread.currentThread().getId());
//synchronized (compSched) {
System.out.println("Notifying waiting threads");
//context.notifyAll();
Thread.currentThread().notifyAll();
if (!jobException.getMessage().equals("")) {
System.out.println("Exception thrown by: " + jobName
+ " Exception: " + jobException.getMessage());
jobException.printStackTrace();
}
System.out.println("Out Of jobWasExecuted");
}
}
Thanks in advance.
Please read on java concurrency:
Threads wait on a lock. This lock is what is used to notify other threads waiting on the same lock.
Consider:
public class SynchronizedExample{
private final Object LOCK = new Object();
public void doSomethingOr() {
if(somethingIsNotDone()) {
synchronize(LOCK) {
LOCK.wait(); //trycatch here
}
}
}
public void somethingSone() {
somethingIsDone = true;
synchronized(LOCK) {
LOCK.notifyAll(); //trycatch
}
}
}
Replace Thread.currentThread().wait(200); with compSched.wait(200).
And in jobWasExecuted you should call notify on compSched
The methods jobToBeExecuted and jobWasExecuted are running in different threads so you are waiting on a different object and expecting notifications on a different object. That is why it does not work.
If you explained your requirements a bit more succinctly, a different solution could be provided other than wait notify mechanism.
I refer to this link to create a fixed size threadpool. Then I have a method which allow submit Callable request and get the result, it look like this:
private ExecutorService threadPool = Executors.newFixedThreadPool(5);
private CompletionService<String> pool = new ExecutorCompletionService<String>(threadPool);
public void execute(Callable<String> request){
pool.submit(request);
// what happen if this method is called before get the result???
try {
String result = pool.take().get();
System.out.println("result is " + result);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
This execute method can be called many times and the request has difference execute time. The problem is that I want to get the result immediately when it finished. And I want to make sure when executing this method, other calls can be handled and allow add to thread poll.
Here is an example usage:
final Random rnd = new Random();
for (int i = 0; i < 5; i++) {
final String value = String.valueOf(i);
execute(new Callable<String>() {
#Override
public String call() throws Exception {
int sleep = rnd.nextInt(10) * 100;
System.out.println("sleep in " + sleep);
Thread.sleep(sleep);
return value;
}
});
}
And the results are always in order although they have difference execute time:
sleep in 900
result is 0
sleep in 300
result is 1
sleep in 0
result is 2
sleep in 500
result is 3
sleep in 600
result is 4
And I also used the future, but it doesn't work too.
private static void execute(Callable<String> request){
Future<String> future = threadPool.submit(request);
try {
String result = future.get();
System.out.println("result is " + result);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
Please tell me how can I do that? Thanks in advance.
You aren't using the CompletionService correctly. Your main thread is producing tasks and consuming results. A CompletionService is intended to decouple production and consumption; you'd use it when you have different threads playing these roles. The execute() method makes no sense; it is effectively doing this, but with a lot of obfuscation and overhead:
public void execute(Callable<String> request) {
try {
System.out.println("result is " + request.call());
} catch (Exception ex) {
ex.printStackTrace();
}
}
If you must consume the result as soon as it's ready, you have to make that part of the task. Otherwise, you need one application thread waiting for every task to complete, because if you don't, a task result might be ready and have to wait for a thread to be available to consume it. And if you have one thread per task already, why use a thread pool?
To be more explicit, if you want to guarantee no waiting, you need to do something like this:
final class MyTask implements Callable<Void> {
private final String value;
MyTask(String value) { this.value = value; }
#Override
public Void call() throws InterruptedException {
String result = doWork();
handleResult(result);
return null;
}
private String doWork() throws InterruptedException {
int sleep = ThreadLocalRandom.current().nextInt(10) * 100;
System.out.println("sleep in " + sleep);
Thread.sleep(sleep);
return value;
}
private void handleResult(String result) {
System.out.println("result is " + result);
}
}
If you want to use a CompletionService, you need some separate threads that take() from the service. But in this approach, if tasks are completed faster than they are consumed, some results will wait.
R4j,
the get() waits for the callable to return with the value from call: if you want to submit 5 requests you need to submit all requests and then call get
I have 4 threads witch are printing numbers from 15 to 0.I want to control executing of my threads for example I want first to thread D to finish and after him thread C and after him thread B and finally thread A. For now they are doing it all parallel.
How can I change that? any suggestions?
Here is my code:
// Suspending and resuming a thread for Java 2
class NewThread implements Runnable {
String name; // name of thread
Thread t;
boolean suspendFlag;
NewThread(String threadname) {
name = threadname;
t = new Thread(this, name);
System.out.println("New thread: " + t);
suspendFlag = false;
t.start(); // Start the thread
}
// This is the entry point for thread.
public void run() {
try {
for(int i = 15; i > 0; i--) {
System.out.println(name + ": " + i);
Thread.sleep(200);
synchronized(this) {
while(suspendFlag) {
wait();
}
}
}
} catch (InterruptedException e) {
System.out.println(name + " interrupted.");
}
System.out.println(name + " exiting.");
}
void mysuspend() {
suspendFlag = true;
}
synchronized void myresume() {
suspendFlag = false;
notify();
}
}
public class SuspendResume {
public static void main(String args[]) {
NewThread A = new NewThread("A");
NewThread B = new NewThread("B");
NewThread C = new NewThread("C");
NewThread D = new NewThread("D");
// try {
// System.out.println("****************************************************************");
// System.out.println(A.t.getState());
// System.out.println(B.t.getState());
// System.out.println(C.t.getState());
// System.out.println(D.t.getState());
//
// if(D.t.isAlive())
// {
// System.out.println("Bla bla bla");
// }
//
// Thread.sleep(1000);
// A.mysuspend();
// System.out.println("Suspending thread One");
// Thread.sleep(1000);
// A.myresume();
// System.out.println("Resuming thread One");
// B.mysuspend();
// System.out.println("Suspending thread Two");
// Thread.sleep(1000);
// B.myresume();
// System.out.println("Resuming thread Two");
//
//
//
// } catch (InterruptedException e) {
// System.out.println("Main thread Interrupted");
// }
// wait for threads to finish
try {
System.out.println("Waiting for threads to finish.");
A.t.join();
B.t.join();
C.t.join();
D.t.join();
} catch (InterruptedException e) {
System.out.println("Main thread Interrupted");
}
System.out.println("Main thread exiting.");
}
}
I think you should design structure of you service class first. I can suggest following:
public class Service {
private List<Service> dependencies;
// Starts service.
// It should wait until all dependencies started using awaitStart method and then start itself
public void start();
// Blocks current thread until service is started.
// If it is started returns immediately.
public void awaitStart();
// Stops service.
// Awaits until all dependencies are stopped using awaitStop.
public void stop();
// Blocks current thread until service is stopped.
// If it is already stops returns immediately
public void awaitStop();
// Actual code that has service specific code.
// This method may be invoked as last line in 'start' method.
public void run();
}
Next problem is to implement start and awaitStart (stop methods implemented similar). I recommend to use tools from java.util.concurrent for implementing awaitStart method. E.g. CountDownLatch. Each service has it's own latch that indicates that server is started. So code for awaitStart and start is following:
private CountDownLatch started = new CountDownLatch(1);
public void awaitStart() {
started.await();
}
public void start() {
for (Service service : dependencies) {
service.awaitStart();
}
System.out.println("Service " + name + " is started");
started.countDown();
run();
}
Sorry if the question is quite simple. I am a beginner.
I have to create thread that calulates something, while the first thread works the other one have to measure if the first thread calculate the function in specified time. If not, it has to throw exception. Else it returns the answer.
I'd take the java.util.concurrent components - simple example
public void myMethod() {
// select some executor strategy
ExecutorService executor = Executors.newFixedThreadPool(1);
Future f = executor.submit(new Runnable() {
#Override
public void run() {
heresTheMethodToBeExecuted();
}
});
try {
f.get(1000, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
// do something clever
} catch (ExecutionException e) {
// do something clever
} catch (TimeoutException e) {
// do something clever
}
}
Have your thread notify a synchronization object when it is done and have your other thread wait x number of milliseconds for it to finish.
public class Main {
private static final Object mThreadLock = new Object();
static class DoTaskThread extends Thread {
public void run() {
try {
int wait = new Random().nextInt(10000);
System.out.println("Waiting " + wait + " ms");
Thread.sleep(wait);
} catch (InterruptedException ex) {
}
synchronized (mThreadLock) {
mThreadLock.notifyAll();
}
}
}
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
synchronized (mThreadLock) {
DoTaskThread thread = new DoTaskThread();
thread.start();
try {
// Only wait 2 seconds for the thread to finish
mThreadLock.wait(2000);
} catch (InterruptedException ex) {
}
if (thread.isAlive()) {
throw new RuntimeException("thread took too long");
} else {
System.out.println("Thread finished in time");
}
}
}
}
join is a lot simpler than using a lock.
join (millis)
Waits at most millis milliseconds
for this thread to die. A timeout of 0
means to wait forever.
Example code:
Thread calcThread = new Thread(new Runnable(){
#Override
public void run() {
//some calculation
}
});
calcThread.start();
//wait at most 2secs for the calcThread to finish.
calcThread.join(2000);
//throw an exception if the calcThread hasn't completed.
if(calcThread.isAlive()){
throw new SomeException("calcThread is still running!");
}
Have a look at http://download.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ExecutorService.html#awaitTermination(long,%20java.util.concurrent.TimeUnit) which allows you to handle this without dealing with thread synchronization yourself.