Develop Reference

Java awaiting Future result without blocking

I have an application where by clicking buttons (that number is defined) user creates tasks (Callable) that do some calculations. I want to be able to react when the task is finished. Using Future.get() blocks the application. Is there any way to be able to react when the Callable returns the result?
private static void startTask(int i){
try{
Future<Integer> future = executor.submit(callables.get(i-1));
ongoingTasks.put(i, future);
awaitResult(future, i);
}
catch(Exception e){
e.printStackTrace();
}
}
private static void awaitResult(Future<?> future, int taskNo) throws InterruptedException, ExecutionException{
System.out.println("result : " + future.get());
JButton b = buttons.get(taskNo);
b.setEnabled(false);
}

It sounds like you want a CompletableFuture. You have a function which is a "supplier" which supplies a value. This is the function that's actually doing the work.
You then have a function which accepts that value whenever the worker is done.
This is all asynchronous, so everything else carries on regardless of the outcome.
class Main
{
private static Integer work() {
System.out.println("work");
return 3;
}
private static void done(Integer i) {
System.out.println("done " + i);
}
public static void main (String... args)
{
CompletableFuture.supplyAsync(Main::work)
.thenAccept(Main::done);
System.out.println("end of main");
}
}
Sample output:
end of main
work
done 3

Editable queue of tasks running in background thread

I know this question was answered many times, but I'm struggling to understand how it works.
So in my application the user must be able to select items which will be added to a queue (displayed in a ListView using an ObservableList<Task>) and each item needs to be processed sequentially by an ExecutorService.
Also that queue should be editable (change the order and remove items from the list).
private void handleItemClicked(MouseEvent event) {
if (event.getClickCount() == 2) {
File item = listView.getSelectionModel().getSelectedItem();
Task<Void> task = createTask(item);
facade.getTaskQueueList().add(task); // this list is bound to a ListView, where it can be edited
Future result = executor.submit(task);
// where executor is an ExecutorService of which type?
try {
result.get();
} catch (Exception e) {
// ...
}
}
}
Tried it with executor = Executors.newFixedThreadPool(1) but I don't have control over the queue.
I read about ThreadPoolExecutor and queues, but I'm struggling to understand it as I'm quite new to Concurrency.
I need to run that method handleItemClicked in a background thread, so that the UI does not freeze, how can I do that the best way?
Summed up: How can I implement a queue of tasks, which is editable and sequentially processed by a background thread?
Please help me figure it out
EDIT
Using the SerialTaskQueue class from vanOekel helped me, now I want to bind the List of tasks to my ListView.
ListProperty<Runnable> listProperty = new SimpleListProperty<>();
listProperty.set(taskQueue.getTaskList()); // getTaskList() returns the LinkedList from SerialTaskQueue
queueListView.itemsProperty().bind(listProperty);
Obviously this doesn't work as it's expecting an ObservableList. There is an elegant way to do it?

The simplest solution I can think of is to maintain the task-list outside of the executor and use a callback to feed the executor the next task if it is available. Unfortunately, it involves synchronization on the task-list and an AtomicBoolean to indicate a task executing.
The callback is simply a Runnable that wraps the original task to run and then "calls back" to see if there is another task to execute, and if so, executes it using the (background) executor.
The synchronization is needed to keep the task-list in order and at a known state. The task-list can be modified by two threads at the same time: via the callback running in the executor's (background) thread and via handleItemClicked method executed via the UI foreground thread. This in turn means that it is never exactly known when the task-list is empty for example. To keep the task-list in order and at a known fixed state, synchronization of the task-list is needed.
This still leaves an ambiguous moment to decide when a task is ready for execution. This is where the AtomicBoolean comes in: a value set is always immediatly availabe and read by any other thread and the compareAndSet method will always ensure only one thread gets an "OK".
Combining the synchronization and the use of the AtomicBoolean allows the creation of one method with a "critical section" that can be called by both foreground- and background-threads at the same time to trigger the execution of a new task if possible. The code below is designed and setup in such a way that one such method (runNextTask) can exist. It is good practice to make the "critical section" in concurrent code as simple and explicit as possible (which, in turn, generally leads to an efficient "critical section").
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class SerialTaskQueue {
public static void main(String[] args) {
ExecutorService executor = Executors.newSingleThreadExecutor();
// all operations on this list must be synchronized on the list itself.
SerialTaskQueue tq = new SerialTaskQueue(executor);
try {
// test running the tasks one by one
tq.add(new SleepSome(10L));
Thread.sleep(5L);
tq.add(new SleepSome(20L));
tq.add(new SleepSome(30L));
Thread.sleep(100L);
System.out.println("Queue size: " + tq.size()); // should be empty
tq.add(new SleepSome(10L));
Thread.sleep(100L);
} catch (Exception e) {
e.printStackTrace();
} finally {
executor.shutdownNow();
}
}
// all lookups and modifications to the list must be synchronized on the list.
private final List<Runnable> tasks = new LinkedList<Runnable>();
// atomic boolean used to ensure only 1 task is executed at any given time
private final AtomicBoolean executeNextTask = new AtomicBoolean(true);
private final Executor executor;
public SerialTaskQueue(Executor executor) {
this.executor = executor;
}
public void add(Runnable task) {
synchronized(tasks) { tasks.add(task); }
runNextTask();
}
private void runNextTask() {
// critical section that ensures one task is executed.
synchronized(tasks) {
if (!tasks.isEmpty()
&& executeNextTask.compareAndSet(true, false)) {
executor.execute(wrapTask(tasks.remove(0)));
}
}
}
private CallbackTask wrapTask(Runnable task) {
return new CallbackTask(task, new Runnable() {
#Override public void run() {
if (!executeNextTask.compareAndSet(false, true)) {
System.out.println("ERROR: programming error, the callback should always run in execute state.");
}
runNextTask();
}
});
}
public int size() {
synchronized(tasks) { return tasks.size(); }
}
public Runnable get(int index) {
synchronized(tasks) { return tasks.get(index); }
}
public Runnable remove(int index) {
synchronized(tasks) { return tasks.remove(index); }
}
// general callback-task, see https://stackoverflow.com/a/826283/3080094
static class CallbackTask implements Runnable {
private final Runnable task, callback;
public CallbackTask(Runnable task, Runnable callback) {
this.task = task;
this.callback = callback;
}
#Override public void run() {
try {
task.run();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
callback.run();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
// task that just sleeps for a while
static class SleepSome implements Runnable {
static long startTime = System.currentTimeMillis();
private final long sleepTimeMs;
public SleepSome(long sleepTimeMs) {
this.sleepTimeMs = sleepTimeMs;
}
#Override public void run() {
try {
System.out.println(tdelta() + "Sleeping for " + sleepTimeMs + " ms.");
Thread.sleep(sleepTimeMs);
System.out.println(tdelta() + "Slept for " + sleepTimeMs + " ms.");
} catch (Exception e) {
e.printStackTrace();
}
}
private String tdelta() { return String.format("% 4d ", (System.currentTimeMillis() - startTime)); }
}
}
Update: if groups of tasks need to be executed serial, have a look at the adapted implementation here.

Simple queue and multi-threading

I have to process a lot of files. I wrote simple Java program that does the job, but it is too slow.
I need more than 1 working thread.
Im totally new with Java and Java multithreading.
Here is my code (simplified):
public static void main(String[] args)
{
// some queue here?
for (int i = 1; i < 8000000; i++)
{
processId(i);
}
}
public static void processId(int id)
{
try
{
// do work
System.out.println("Im working on: " + Integer.toString(id));
}
catch (Exception e)
{
// do something with errors
System.out.println("Error while working on: " + Integer.toString(id));
}
}
How can I add simple queue with 8 threads?

You should look into Executors.
You can create a thread pool of 8 threads using:
ExecutorService executor = Executors.newFixedThreadPool(8);
Then submit your tasks inside your loop the following way:
final int finalId = i; // final is necessary to be enclosed in lambda
executor.submit(() -> processId(finalId));
Or prior to java 8:
final int temp = i; // final is necessary to be enclosed in anonymous class
executor.submit(new Runnable() {
public void run() {
processId(finalId);
}
});
Don't forget to shutdown the thread pool when not needed anymore, as mentioned in the documentation. Here is an example from the doc:
private void shutdownAndAwaitTermination(ExecutorService pool) {
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
}

You should look into ExecutorService. This will make multithreading easy. An example:
Main code:
ExecutorService pool = Executors.newFixedThreadPool(8);
for (int i = 1; i < 8000000; i++) {
pool.submit(new intProcessingTask(i));
}
pool.shutdown();
pool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
// all tasks have now finished (unless an exception is thrown above)
intProcessingTask code:
private static class DownloadTask implements Runnable {
private int id;
public DownloadTask(int id) {
this.id = id;
}
#Override
public void run() {
System.out.println("Im working on: " + Integer.toString(id));
}
}
This is slightly longer than the the other answer, but does pretty much the same thing, and works on Java 7 and earlier.

There are many ways in Java for processing mulithreading. Base on your question that you need a queue, I think the most simple version is use Java ExecutorService. You can see this code:
public static void main(String[] args) {
// creating a thread pool with maximum thread will be 8
ExecutorService executorService = Executors.newFixedThreadPool(8);
for (int i = 0; i < 8000000; i++) {
final int threadId = i;
executorService.execute(new Runnable() {
public void run() {
processId(threadId);
}
});
}
}
ExecutorService has some methods:
execute(Runnable)
submit(Runnable)
submit(Callable)
invokeAny(...)
invokeAll(...)
I recommend you view this link: ExecutorService tutorial for clear explanation.
Hope this help :)

ThreadPoolExecutor runs even after ScheduledFuture is cancelled

I'm looking to create a ScheduledThreadPoolExecutor with an unknown pool size. Pool size is determined at run-time, will likely be between 1-5, and for this example I used size 2. We use a custom Task that simply executes a method every so often, but that method will eventually throw an exception (which I've simulated with a simple numTimes variable and if statement). If an exception is thrown, I only want to cancel execution of THAT specific thread! If all threads are cancelled, I want to shut down the ScheduledThreadPoolExecutor. Once numTimes == 5 I simulate the exception to cancel the thread), and I can manage to cancel the thread a number of ways but they just don't feel right.
As a side note, I placed ScheduledFuture everywhere just to play around with cancelling it.
public class Test
{
static ScheduledThreadPoolExecutor stpe = new ScheduledThreadPoolExecutor(2);
public static void main(String[] args)
{ stpe.scheduleWithFixedDelay(new UpdateTask(1), 0, 1000, TimeUnit.MILLISECONDS);
stpe.scheduleWithFixedDelay(new UpdateTask(2), 0, 5000, TimeUnit.MILLISECONDS);
// stpe.shutdown();
}
public static class UpdateTask implements Runnable
{
int id;
int numTimes = 0;
ScheduledFuture<?> t;
public UpdateTask(int id)
{ this.id = id;
}
public void run()
{ System.out.println("Hello " + id + " num: " + numTimes);
String fn = "C:\\lib" + id;
if (numTimes++ == 5)
{ File f = new File(fn);
f.mkdir();
t.cancel(false);
}
}
}
}
Calling t.cancel() from run() or from main() have the same effect, in that the thread stops executing but the program does not stop running. Naturally, this is because the ThreadPoolExecutor is still doing stuff, despite both threads no longer being scheduled.
I tried invoking shutdown on stpe, but it doesn't finish thread execution. Two directories are created with stpe.shutdown commented out, and they are not otherwise.
I can't figure out an elegant way to cancel ScheduledFuture, then ScheduledThreadPoolExecutor when all ScheduledFuture's are cancelled.
Final approach ##
I was not able to get s1.get() to work as described in the answer below, so I simply created my own class to handle it.
public class Test
{
static ScheduledThreadPoolExecutor stpe = new ScheduledThreadPoolExecutor(2);
static CancelUpdateTasks canceller;
public static void main(String[] args)
{ Test t = new Test();
canceller.add(0, stpe.scheduleWithFixedDelay(new UpdateTask(0), 0, 1000, TimeUnit.MILLISECONDS));
canceller.add(1, stpe.scheduleWithFixedDelay(new UpdateTask(1), 0, 5000, TimeUnit.MILLISECONDS));
canceller.waitForSchedules();
stpe.shutdown();
}
public Test()
{ canceller = new CancelUpdateTasks();
}
public static class UpdateTask implements Runnable
{
int id;
int numTimes = 0;
public UpdateTask(int id)
{ this.id = id;
}
public void run()
{ System.out.println("Hello " + id + " num: " + numTimes);
if (numTimes++ == 5)
{ canceller.cancel(id);
}
}
}
public class CancelUpdateTasks
{ List<ScheduledFuture<?>> scheduler;
boolean isScheduled;
public CancelUpdateTasks()
{ scheduler = new ArrayList<ScheduledFuture<?>>();
isScheduled = false;
}
public void waitForSchedules()
{ int schedId = 0;
while(isScheduled)
{ ScheduledFuture<?> schedule = scheduler.get(schedId);
if (schedule.isCancelled())
{ if (schedId == scheduler.size() - 1)
return;
schedId++;
}
else
{ try
{ Thread.sleep(1000);
}
catch (InterruptedException e)
{ e.printStackTrace();
}
}
}
}
public void add(int id, ScheduledFuture<?> schedule)
{ scheduler.add(id, schedule);
if (!isScheduled)
isScheduled = true;
}
public void cancel(int id)
{ scheduler.get(id).cancel(false);
}
public void cancelNow(int id)
{ scheduler.get(id).cancel(true);
}
}
}

You'll want to issue a shutdown on the pool. The JVM will continue to run until there are only daemon threads alive. A ThreadPoolExecutor by default will create non-daemon threads.
Just invoke stpe.shutdown();
edit: Based on OPs update
shutdown admittedly is different for ScheduledThreadPoolExecugtor than a plain ThreadPoolExecutor. In this case shutdown prevents any scheduled task to become re scheduled. To make it work correctly you will have to wait on the futures completion. You can do so by get()ing on the ScheduledFuture
ScheduledFuture sf1 = stpe.scheduleWithFixedDelay(new UpdateTask(1), 0, 1000, TimeUnit.MILLISECONDS);
ScheduledFuture sf2 = stpe.scheduleWithFixedDelay(new UpdateTask(2), 0, 5000, TimeUnit.MILLISECONDS);
sf1.get();
sf2.get();
stpe.shutdown();
In this case both tasks are run asynchronously, the main thread will first wait for sf1 to complete then will wait for sf2 to complete and finally shutdown.

Two threads deadlocking but can't see why, lock released with notifyAll()

using JConsole it seems i get a deadlock situation when 2 threads try to modify this object.
package com.steven.concurrent.assignment2.memoryallocator;
/*
* This seems to deadlock... cant see why though.
*/
public class MemAllocMonitor implements IMemoryAllocator {
private final int MAX_FREE = 50;
private int freePages = MAX_FREE;
//I think this would work, without even the need for sync blocks.....
// But only in the situaion where i would not have to check the bounds of the updates. If it was just modification, this would be
// fine....
//private volatile int freePages = 50;
public MemAllocMonitor(int pages){
assert(pages < MAX_FREE);
this.freePages = pages;
}
public MemAllocMonitor(){
}
#Override
public synchronized void request(int number) {
if(number < 0)
throw new IllegalArgumentException();
while(freePages - number < 0) {
System.out.println("No space....waiting...");
try {
this.wait();
} catch (Exception e) {}
}
freePages -= number;
System.out.println("Requested : " + number + " remaining " + freePages);
this.notifyAll();
}
#Override
public synchronized void release(int number) {
if(number < 0)
throw new IllegalArgumentException();
while(freePages + number > MAX_FREE) {
System.out.println("page table full....would be " + (number + freePages) );
try {
this.wait();
} catch (Exception e) {}
}
freePages += number;
System.out.println("Released : " + number + " remaining " + freePages);
this.notifyAll();
}
#Override
public int getFreePages() {
return freePages;
}
}
This object is accessed via a simple wrapper that implements runnable, and calls either method as shown below.
package com.steven.concurrent.assignment2.memoryallocator;
import concurrent.RandomGenerator;
import concurrent.Time;
public class MemAllocRequester implements Runnable, MemoryAllocatorAction{
private IMemoryAllocator memoryAllocator;
private volatile boolean shutdown = false;;
public MemAllocRequester(IMemoryAllocator memAlloc){
this.memoryAllocator = memAlloc;
}
#Override
public void run() {
while(!shutdown){
Time.delay(500);
memoryAllocator.request(RandomGenerator.integer(0, 30));
}
}
public void ShutDown(){
this.shutdown = true;
}
}
and
package com.steven.concurrent.assignment2.memoryallocator;
import concurrent.RandomGenerator;
import concurrent.Time;
public class MemAllocReleaser implements Runnable, MemoryAllocatorAction{
private IMemoryAllocator memoryAllocator;
private volatile boolean shutdown = false;;
public MemAllocReleaser(IMemoryAllocator memAlloc){
this.memoryAllocator = memAlloc;
}
#Override
public void run() {
while(!shutdown){
Time.delay(500);
memoryAllocator.release(RandomGenerator.integer(0, 30));
}
}
public void ShutDown(){
this.shutdown = true;
}
}
It is started off as such...
package com.steven.concurrent.assignment2.memoryallocator;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class MemAllocMain {
public static void main(String[] args){
ExecutorService executor = Executors.newFixedThreadPool(10);
//IMemoryAllocator memoryAllocator = new MemAllocSemaphore();
IMemoryAllocator memoryAllocator = new MemAllocMonitor();
System.out.println("Starting app with " + memoryAllocator.getFreePages() + " pages...");
Thread t1 = new Thread(new MemAllocRequester(memoryAllocator));
Thread t2 = new Thread(new MemAllocReleaser(memoryAllocator));
t1.setName("MEMORY REQUESTER £££££££££££££££££££");
t2.setName("MEMORY RELEASER £££££££££££££££££££");
executor.submit(t1);
executor.submit(t2);
}
}
I have implemented a solution using the semaphore class, but for some reason this is causing trouble using the default java monitor solution. It runs for about 30 seconds, then both threads go into their waiting state, even though the lock should be enforced.

The problem is that both threads are hitting the upper and lower bounds (50 and 0 respectively) at the same time. Both examples below highlight the deadlock.
Scenario 1
request(29) - freePages=21
request(30) - under 0 so waits
release(30) - over 50 so waits : deadlock
Scenario 2
request(29) - freePages=21
release(30) - over 50 so waits
request(30) - under 0 so waits : deadlock
I am not sure what the exact requirements are for the homework problem but you need to revisit the release and request methods. I see two viable solutions:
Change the release method so that it only releases up to MAX_FREE but will still return
Change the release method so that it can release a subset of the amount requested, notifyAll, reenter the wait so it can release the remaining amount.
Also, you are kind of using the ExecutionService wrong. The ExecutionService is what creates the Threads so there is no reason for you to create the threads like you are doing.
Thread t1 = new Thread(new MemAllocRequester(memoryAllocator));
Thread t2 = new Thread(new MemAllocReleaser(memoryAllocator));
The threads you are creating will actually never be 'started' as Threads. It is still working for you because the ExecutionService threads will call your Thread.run() which will call MemAlloc*.run(). i.e. your t1 and t2 threads just pass the run() call along and provide no value.
Your MemAllocRequester and MemAllocReleaser are Runnables so just pass those into the ExecutionService directly.
executor.submit(new MemAllocRequester(memoryAllocator));
executor.submit(new MemAllocReleaser(memoryAllocator));