Here's my code:
public static void main(String[] args) {
Observable.just("747", "737", "777")
.flatMap(
a -> {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return Observable.just(a).subscribeOn(Schedulers.newThread());
})
.subscribe(p -> System.out.println("Received " + p + " on thread " + Thread.currentThread().getName()));
}
As I have understood it, each of the items of the observable should will run in a separate thread (which does happen) and the results would be sent to the same thread that did the work (this happens as well). But what I cannot understand is that why is the main thread not exiting and waiting for the background threads to finish? The program continues for as long as each of the background threads is running.
If you look at a thread dump, you'll see that the main thread is actually stuck on the sleep statement. That's why it's not exiting.
This is because it is the thread that's executing the flatMap operator, so it's getting stuck. This also is why the code takes a long time to run. You can easily verify it by inserting a print statement just before sleep:
try {
System.out.println(Thread.currentThread().getName() + " is sleeping");
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
The output is something like this:
main is sleeping
main is sleeping
Received 747 on thread RxNewThreadScheduler-1
main is sleeping
Received 737 on thread RxNewThreadScheduler-2
Received 777 on thread RxNewThreadScheduler-3
I think you've meant to write something like this:
System.out.println(Thread.currentThread().getName() + " is creating the observable");
Observable.just("747", "737", "777")
.flatMap(a ->
Observable.fromCallable(() -> {
try {
System.out.println(Thread.currentThread().getName() + " is sleeping");
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return a;
}).subscribeOn(Schedulers.newThread())
).subscribe(p -> System.out.println("Received " + p + " on thread " + Thread.currentThread().getName()));
System.out.println(Thread.currentThread().getName() + " is going to exit");
Output:
main is creating the observable
main is going to exit
RxNewThreadScheduler-3 is sleeping
RxNewThreadScheduler-2 is sleeping
RxNewThreadScheduler-1 is sleeping
Received 777 on thread RxNewThreadScheduler-3
Received 747 on thread RxNewThreadScheduler-1
Received 737 on thread RxNewThreadScheduler-1
In this version main is exiting right after the Observable is created.
Related
I have the following code, and interrupted exception occurs at the arrow below. How should I rectify this ? I believe the ‘shutdown’ and ‘awaittermination’ are implemented correctly. It is the final if statement that is causing the interrupted exception. I just want to check that the executor is shutdown properly and all tasks are terminated or completed properly on the final if statement.
try {
cancelTaskAndDisconnectClient();
System.out.println("2. Disconnected # " + LocalDateTime.now() + " !!!");
// Disable new tasks from being submitted
executor.shutdown();
// wait a while for existing tasks to terminate
if(!executor.awaitTermination(10, TimeUnit.SECONDS)) {
// Cancel currently executing tasks
executor.shutdownNow();
if(!executor.awaitTermination(10, TimeUnit.SECONDS)) { <-- where exception occurred
System.err.println("Executor did not terminate");
}
}
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException ie) {
executor.shutdownNow();
ie.printStackTrace();
}
This is the exception i received.
java.lang.InterruptedException
at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2109)
at java.base/java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1454)
at samples.testbed.DataProcessing$CancelMktDataTask.run(DataProcessing.java:134) <- this is the line in my file
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
UPDATE: I have updated the code as follow. And I realised that issue is that even after I call shutdown,awaitTermination, shutdownNow, the Executor is shut down but never terminated.
public void shutdownshutdownAndAwaitTermination() {
// Disable new tasks from being submitted
executor.shutdownNow();
try {
// Wait a while for existing tasks to terminate
if (!executor.awaitTermination(60, TimeUnit.SECONDS)) {
executor.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!executor.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException e) {
// (Re-)Cancel if current thread also interrupted
executor.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
}
When I run the code above as below:
System.out.println("2. Disconnected # " + LocalDateTime.now() + " !!!");
System.err.println("BEFORE: Executor is Shut Down: " + executor.isShutdown() + "; Executor is Terminated: " + executor.isTerminated());
shutdownshutdownAndAwaitTermination();
System.err.println("AFTER: Executor is Shut Down: " + executor.isShutdown() + "; Executor is Terminated: " + executor.isTerminated());
And the output is:
BEFORE: Executor is Shut Down: false; Executor is Terminated: false
AFTER: Executor is Shut Down: true; Executor is Terminated: false
2. Disconnected # 2020-03-12T16:33:15.884436400 !!!
And the "Disconnected" print comes after the "before/after" when the order should really be reversed. What did I do wrong ?
So my threads are working as expected, and I just wanted to add some extra sauce to my homework.
I made a while loop that checks uses the isShutdown which returns false unless shutdown(); has been called.
So i call shutdown at the end of my method, but it won't ever exit the while loop.
public void runParrallel() throws InterruptedException {
System.out.println("Submitting Task ...");
ExecutorService executor = Executors.newFixedThreadPool(5);
List<Future<TagCounter>> counters = new ArrayList();
counters.add(executor.submit(new TagCounterCallable("https//www.fck.dk")));
counters.add(executor.submit(new TagCounterCallable("https://www.google.com")));
counters.add(executor.submit(new TagCounterCallable("https://politiken.dk")));
counters.add(executor.submit(new TagCounterCallable("https://cphbusiness.dk")));
System.out.println("Task is submitted");
while (!executor.isShutdown()) {
System.out.println("Task is not completed yet....");
Thread.sleep(1000);
}
for (Future<TagCounter> future : counters) {
try {
TagCounter tc = future.get();
System.out.println("Title: " + tc.getTitle());
System.out.println("Div's: " + tc.getDivCount());
System.out.println("Body's: " + tc.getBodyCount());
System.out.println("----------------------------------");
} catch (ExecutionException ex) {
System.out.println("Exception: " + ex);
}
}
executor.shutdown();
}
The while-loop is before you ever call shutdown(). The condition cannot possibly evaluate to false, thus you are stuck with an infinite loop. I'd suggest moving the while loop to the point after you call shutdown().
See also this question on how to shut down an ExecutorService.
Correct me if I'm wrong, but it looks like you want to wait until all tasks that were submitted to your ExecutorService have finished. If you know that they're going to finish in a timely manner, then you can use ExecutorService#shutdown in conjunction with ExecutorService#awaitTermination to block the executing thread until all tasks are complete.
This can be done with the following:
public void runParrallel() throws InterruptedException {
// Same code to submit tasks.
System.out.println("Task is submitted");
executor.shutdown();
executor.awaitTermination(1, TimeUnit.DAYS);
// At this point, the ExecutorService has been shut down successfully
// and all tasks have finished.
for (Future<TagCounter> future : counters) {
try {
TagCounter tc = future.get();
System.out.println("Title: " + tc.getTitle());
System.out.println("Div's: " + tc.getDivCount());
System.out.println("Body's: " + tc.getBodyCount());
System.out.println("----------------------------------");
} catch (ExecutionException ex) {
System.out.println("Exception: " + ex);
}
}
}
With this solution, the while loop can be removed.
Your while-loop is running infinitely because there is nothing that activates the executor.shutdown() inside the while-loop. The code wont progress to the end where you call executor.shutdown() because the while-loop's condition returns back to the start of the while-loop.
Put an if-statement inside the while-loop. The if-statement checks if the task is submitted, and if it is, the executor.shutdown() will be called.
Following is just an example:
while (!executor.isShutdown()) {
System.out.println("Task is not completed yet....");
Thread.sleep(1000);
if(TaskIsCompleted){
executor.shutdown();
}
}
I have an application that is running jobs that require two threads for every job. The two threads normally do some work and finish shortly after each other. Then after the second thread finishes I need to do some cleanup but since the threads are doing some network IO, it is possible for one thread to get blocked for a long time. In that case, I want the cleanup to take place a few seconds after the first thread finishes.
I implemented this behaviour with the following piece of code in a callback class:
private boolean first = true;
public synchronized void done() throws InterruptedException {
if (first) {
first = false;
wait(3000);
// cleanup here, as soon as possible
}
else {
notify();
}
}
Both threads invoke the done() method when they finish. The first one will then block in the wait() for at most 3 seconds but will be notified immediately when the seconds thread invokes the done() method.
I have tested this implementation and it seems to work well but I'm am curious if there's a better way of doing this. Even though this implementation doesn't look too complicated, I'm afraid that my program will deadlock or have some unsuspected synchronization issue.
I hope I understood your need. You want to wait for thread-a to complete and then wait either 3 seconds or for the end of thread-b.
It is better to use the newer Concurrent tools instead of the old wait/notify as there are so many edge cases to them.
// Two threads running so count down from 2.
CountDownLatch wait = new CountDownLatch(2);
class TestRun implements Runnable {
private final long waitTime;
public TestRun(long waitTime) {
this.waitTime = waitTime;
}
#Override
public void run() {
try {
// Wait a few seconds.
Thread.sleep(waitTime);
// Finished! Count me down.
wait.countDown();
System.out.println(new Date() + ": " + Thread.currentThread().getName() + " - Finished");
} catch (InterruptedException ex) {
System.out.println(Thread.currentThread().getName() + " - Interrupted");
}
}
}
public void test() throws InterruptedException {
// ThreadA
Thread threadA = new Thread(new TestRun(10000), "Thread A");
// ThreadB
Thread threadB = new Thread(new TestRun(30000), "Thread B");
// Fire them up.
threadA.start();
threadB.start();
// Wait for all to finish but threadA must finish.
threadA.join();
// Wait up to 3 seconds for B.
wait.await(3, TimeUnit.SECONDS);
System.out.println(new Date() + ": Done");
threadB.join();
}
happily prints:
Tue Sep 15 16:59:37 BST 2015: Thread A - Finished
Tue Sep 15 16:59:40 BST 2015: Done
Tue Sep 15 16:59:57 BST 2015: Thread B - Finished
Added
With the new clarity - that the end of any thread starts the timer - we can use a third thread for the cleanup. Each thread must call a method when it finishes to trigger the cleanup mechanism.
// Two threads running so count down from 2.
CountDownLatch wait = new CountDownLatch(2);
class TestRun implements Runnable {
private final long waitTime;
public TestRun(long waitTime) {
this.waitTime = waitTime;
}
#Override
public void run() {
try {
// Wait a few seconds.
Thread.sleep(waitTime);
// Finished! Count me down.
wait.countDown();
System.out.println(new Date() + ": " + Thread.currentThread().getName() + " - Finished");
// Record that I've finished.
finished();
} catch (InterruptedException ex) {
System.out.println(Thread.currentThread().getName() + " - Interrupted");
}
}
}
Runnable cleanup = new Runnable() {
#Override
public void run() {
try {
// Wait up to 3 seconds for both threads to clear.
wait.await(3, TimeUnit.SECONDS);
// Do your cleanup stuff here.
// ...
System.out.println(new Date() + ": " + Thread.currentThread().getName() + " - Finished");
} catch (InterruptedException ex) {
System.out.println(Thread.currentThread().getName() + " - Interrupted");
}
}
};
final AtomicBoolean cleanupStarted = new AtomicBoolean(false);
private void finished() {
// Make sure I only start the cleanup once.
if (cleanupStarted.compareAndSet(false, true)) {
new Thread(cleanup, "Cleanup").start();
}
}
public void test() throws InterruptedException {
// ThreadA
Thread threadA = new Thread(new TestRun(10000), "Thread A");
// ThreadB
Thread threadB = new Thread(new TestRun(30000), "Thread B");
// Fire them up.
threadA.start();
threadB.start();
System.out.println(new Date() + ": Done");
}
As done method is synchronized, so only one thread can execute at a time, with this second will wait to send notify until first finishes its whole job, which might cause performance bottleneck.
I would rather design it with short synchronized block which would primarily update the boolean first.
Good afternoon all,
I'm working with Java's semaphore and concurrency for a school project and had a few questions regarding how it works!
If there are no permits available, I need the thread to exit the "queue" - not just sleep until one is ready. Is this possible? As you can see in my try, catch, finally - there is no handle for this event:
try {
semaphore.acquire();
System.out.println(Thread.currentThread().getName() + " aquired for 3 seconds " + semaphore.toString());
Thread.sleep(3000);
}
catch (InterruptedException e) {
e.printStackTrace();
} finally {
semaphore.release();
System.out.println(Thread.currentThread().getName() + " released " + semaphore.toString());
}
Daniel brought up the tryAquire function - this looks great but the tutorials I have read state that semaphores require a try, catch, finally block to prevent a deadlock. My current code (implementing tryAquire) will release in the finally block even if that thread was never acquired. Do you have any suggestions?
public void seatCustomer(int numBurritos) {
try {
if(semaphore.tryAcquire()) {
System.out.println(Thread.currentThread().getName() + " aquired for 3 seconds " + semaphore.toString());
Thread.sleep(3000);
} else {
System.out.println(Thread.currentThread().getName() + " left due to full shop");
}
}
catch (InterruptedException e) {
e.printStackTrace();
} finally {
semaphore.release();
System.out.println(Thread.currentThread().getName() + " released " + semaphore.toString());
}
}
I suggest you read the JavaDocs for Semaphor. In particular, look at the tryAcquire method.
Acquires a permit from this semaphore, only if one is available at the time of invocation.
Acquires a permit, if one is available and returns immediately, with the value true, reducing the number of available permits by one.
If no permit is available then this method will return immediately with the value false.
What this means is you can try to acquire a permit if any are available. If none are available, this method returns false immediately instead of blocking.
You'll have to make your "finally" block a little smarter.
boolean hasPermit = false;
try {
hasPermit = semaphore.tryAcquire();
if (hasPermit) {
// do stuff.
}
} finally {
if (hasPermit) {
semaphore.release();
}
}
I want to start a group of 10 threads. In my main program constructor I am using:
executor = Executors.newFixedThreadPool(NTHREADS);
Callable<String> poller;
for (int i = 0; i < NTHREADS; ++i) {
Future<String> future = executor.submit(new Poller(0x3A, m_socket, ds_in, ds_out, socketLock));
set.add(future);
}
For the call() method in class Poller I have :
public String call()
{
// This has to be set here, otherwise all threads will have a name of "main".
myID = Thread.currentThread().getName();
boolean shutup_loop = true;
do {
System.out.println("Hey, I'm thread " + myID);
System.out.println("Hey, I'm thread " + Thread.currentThread().getName());
try {
Thread.sleep(10);
}
catch (java.lang.InterruptedException e) {
System.out.println("thread " + myID + ": " + e);
}
// Do if you want the printing to all match up on one line
synchronized (this) {
ByteArrayOutputStream baos = SendReceive(pollPacket);
System.out.print(myID + ": ");
if (baos != null) {
printStuff(baos);
System.out.println();
}
notify();
}
} while (shutup_loop);
return "poller is finished";
}
These Poller threads are calling SendReceive(), part of the Poller class:
public synchronized ByteArrayOutputStream SendReceive(byte[] toSend)
{
System.out.println("START");
System.out.println("SendReceive()1 " + myID);
System.out.println("SendReceive()2 " + Thread.currentThread().getName());
System.out.println("END");
try {
ds_out.write(toSend, 0, toSend.length);
ds_out.flush();
}
catch (java.io.IOException e) {
System.out.println("thread " + myID + ": " + e);
}
try {
m_socket.setSoTimeout(200); // <-- might need tweaking
}
catch (java.net.SocketException e) {
System.out.println("thread " + myID + ": " + e);
}
ByteArrayOutputStream baos = null;
try {
baos = getResponse(ds_in);
}
catch (java.io.IOException e) {
System.out.println("thread " + myID + ": " + e);
}
return baos;
}
Because this is a synchronized method I would expect the output to resemble:
START
SendReceive()1 pool-1-thread-1
SendReceive()2 pool-1-thread-1
END
START
SendReceive()1 pool-1-thread-2
SendReceive()2 pool-1-thread-2
END
Instead it is doing:
START
START
START
START
START
START
SendReceive()1 pool-1-thread-2
START
START
START
SendReceive()1 pool-1-thread-6
SendReceive()1 pool-1-thread-7
SendReceive()2 pool-1-thread-2
SendReceive()1 pool-1-thread-3
SendReceive()2 pool-1-thread-6
SendReceive()1 pool-1-thread-1
SendReceive()1 pool-1-thread-9
SendReceive()1 pool-1-thread-8
SendReceive()2 pool-1-thread-9
END
...
What gives?
synchronized uses this as a lock: in your case you have several Poller instances so each uses a different lock. To make it work you need a common lock:
either make the method static
or use a common private static final Object lock = new Object(); and use synchronized(lock) {...}
If what you are after is sharing the socket between pollers, you do not need to use an executor service. You need to serialize the use of the socket anyway, so you can just loop over the list of poll tasks and hand the current one the socket to work with.
On the other hand, if you really want to poll in parallel using the same socket, your network protocol should support that by allowing you to send a poll message without having to wait for a response to it; you would have multiple requests in flight. But I suspect that's a bigger and different problem that what you are trying to solve.