Parallel Iterator - java

I've looked through these forums to give me insight on how to create a ParallelIterator, but no luck. I want one Collection to be on one Thread and the another Collection to be on another Thread, but I wish to use a ThreadPool to do the job so that it can split up into parts equal to the amount of cores you have using Runtime.getRuntime().availableProcessors(). I also want to return an Actor rather than a Collection of Actors but the next() method is stopping me from doing so. Here is my code so far:
package com.atem.util;
public class ParallelIterator<Collection<Actor>> implements Iterator<Collection<Actor>> {
private Iterator<Actor> firstActorIterator;
private Iterator<Actor> secondActorIterator;
public ParallelIterator(final Collection<Actor> firstCollection, final Collection<Actor> secondCollection) {
this.firstActorIterator = firstCollection.iterator();
this.secondActorIterator = secondCollection.iterator();
}
#Override
public boolean hasNext() {
return this.getFirstActorIterator().hasNext() && this.getSecondActorIterator().hasNext();
}
#Override
public Collection<Actor> next() {
return new ParallelCollection(this.getFirstActorIterator().next(), this.getSecondActorIterator().next());
}
#Override
public void remove() {
this.getFirstActorIterator().remove();
this.getSecondActorIterator().remove();
}
public Iterator<Actor> getFirstIterator() {
return this.firstIterator;
}
public Iterator<Actor> getSecondIterator() {
return this.secondIterator;
}
public ParallelCollection<Actor> getParallelActorCollection() {
return this.parallelActorCollection;
}
}
I could change the next() method to return an Actor but then the ParallelCollection would get in the way. If there's anything else you wish to know, please let me know. Help is very much appreciated. Also, I'm somewhat of a beginner so if you could explain it in simple terms that would be great. Thanks!

I agree with Louis's comment that you shouldn't try to write something like this on your own, but I wrote something that might help you get started.
WARNING: This code was not thoroughly tested:
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class Parallel {
//modify the arg for the number of threads you want to use
final static ExecutorService service = Executors.newFixedThreadPool(2);
public static interface Operation<R, E> {
public R perform(E input);
}
public static <R, E> List<R> forEach(final Iterable<E> inputs,
final Operation<R,E> operation) {
final List<Future<R>> futures = Collections.synchronizedList(new ArrayList<Future<R>>());
for (final E input : inputs) {
final Callable<R> callable = new Callable<R>() {
public R call() throws Exception {
return operation.perform(input);
}
};
futures.add(service.submit(callable));
}
final List<R> outputs = new ArrayList<>();
try {
for (final Future<R> future : futures) {
outputs.add(future.get());
}
}
catch (Exception e) {
e.printStackTrace();
}
return outputs;
}
}

Related

ConcurrentSkipListMap firstKey() throws NoSuchElementException even though it contains data

I wrote a small application that receives data from a web socket, which I store in static ConcurrentSkipListMap.
The application initially creates a new thread where it runs infinitely while loop calling ConcurrentSkipListMap.firstKey(). After a while, this call throws a NoSuchElementException, even though the ConcurrentSkipListMap contains data.
break point in catch block
Example of my application:
I have cacher class that contains websocket implementation and NavigableMap init:
package solvethat.net.triobot.Example;
import com.binance.api.client.BinanceApiCallback;
import com.binance.api.client.BinanceApiClientFactory;
import com.binance.api.client.domain.event.DepthEvent;
import com.binance.api.client.domain.market.OrderBook;
import com.binance.api.client.domain.market.OrderBookEntry;
import java.math.BigDecimal;
import java.util.Comparator;
import java.util.List;
import java.util.NavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;
public class AskCacher {
private long updateId;
private final BinanceApiClientFactory factory;
public AskCacher() {
factory = BinanceApiClientFactory.newInstance();
initAsks();
runWebsocket();
}
/**
* Init data getting order book snapshot
*/
private void initAsks() {
try {
OrderBook orderBook = factory.newRestClient().getOrderBook("btcusdt".toUpperCase(), 10);
updateId = orderBook.getLastUpdateId();
NavigableMap<Double, Double> asks = new ConcurrentSkipListMap<>(Comparator.naturalOrder());
for (OrderBookEntry ask : orderBook.getAsks()) {
asks.put(Double.parseDouble(ask.getPrice()), Double.parseDouble(ask.getQty()));
}
StaticData.ask = asks;
} catch (Exception e) {
System.err.println(e.getMessage());
}
}
private void runWebsocket() {
factory.newWebSocketClient().onDepthEvent("btcusdt", new BinanceApiCallback<>() {
/**
* Set ask price and call analysis method
*/
#Override
public void onResponse(DepthEvent depthEvent) {
if (depthEvent.getFinalUpdateId() > updateId) {
updateId = depthEvent.getFinalUpdateId();
updateOrderBook(depthEvent.getAsks());
}
}
/**
* Just print err message
*/
#Override
public void onFailure(final Throwable cause) {
System.err.println(cause.getMessage());
}
});
}
/**
* Updates an order book (asks) with a delta received from the server.
* Whenever the qty specified is ZERO, it means the price should was removed from the order book.
*/
private void updateOrderBook(List<OrderBookEntry> orderBookDeltas) {
for (OrderBookEntry orderBookDelta : orderBookDeltas) {
Double price = Double.parseDouble(orderBookDelta.getPrice());
BigDecimal qty = new BigDecimal(orderBookDelta.getQty());
if (qty.compareTo(BigDecimal.ZERO) == 0) {
// qty=0 means remove this level
StaticData.ask.remove(price);
} else {
StaticData.ask.put(price, Double.parseDouble(orderBookDelta.getQty()));
}
}
// Print best ask to see if cacher is alive
System.out.println("btc-usdt best ask: " + StaticData.ask.firstKey());
// Edit map length
if (StaticData.ask.size() > 10) {
StaticData.ask.tailMap((Double) StaticData.ask.keySet().toArray()[10], true).clear();
}
}}
Then infinite loop:
package solvethat.net.triobot.Example;
public class InfiniteLoop {
public void loopProcess() {
Analyzer analyzer = new Analyzer();
while (true) {
analyzer.analyze(StaticData.ask.firstEntry());
}
}}
And analyzer class:
package solvethat.net.triobot.Example;
import java.util.Map;
public class Analyzer {
public void analyze(Map.Entry<Double, Double> entry) {
StaticData.AnalyzeObject analyzeObject = new StaticData.AnalyzeObject();
analyzeObject.setBestAsk(entry.getKey());
if (analyzeObject.getBestAsk() > 50000) {
System.out.println("It is a good price!!");
}
}
}
Static data model:
package solvethat.net.triobot.Example;
import java.util.NavigableMap;
public class StaticData {
public static NavigableMap<Double, Double> ask;
public static class AnalyzeObject {
double bestAsk;
public double getBestAsk() {
return bestAsk;
}
public void setBestAsk(double bestAsk) {
this.bestAsk = bestAsk;
}
}
}
Main class for example run:
package solvethat.net.triobot.Example;
public class Main {
public static void main(String[] arguments) {
new AskCacher();
new Thread(new InfiniteLoop()::loopProcess).start();
}
}
The example only shows how the application is composed, but I was not able to use it to raise an error but I opened my repo as public:
https://github.com/Sick-E/TrioBot
Can anyone please help me?
Thank you.
Tomas
You can replace your code with something like that (no exception handling is required)
Optional.ofNullable(trio.getThirdPair().getBids().firstEntry())
.map(Map.Entry::getKey)
.ifPresent(trio.getTrioAnalysis()::setBidThird);

How to implement extended apply method? Java

My question has to do with implementing the FlatApply class that I have created previously into this class called FilteringFlatApplyFunction. I cannot seem to implement the apply correctly because the static class continues to tell me it needs to be abstract and the #Override is not working the way it is supposed to. The end goal I am looking for is a way to use inheritance to borrow most of the functionality from the FlatApply class and implement the filter class. I have tried many different things but still can't get it, the predicate "pred" checks if the given predicate is true and if so, indicates to return the element, I thought that would implement FlatApply,I have been getting this error what seems like forever now. Thanks
Error:
FilteringFlatApplyFunction is not abstract and does not override abstract method apply(T) in FlatApplyFunction
where T is a type-variable:
T extends Object declared in class FilteringFlatApplyFunction
package iterators;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import iterators.Apply;
// Iterator that uses a Predicate to filter out elements from the input
public class Filter<T> extends FlatApply<T,T> {
public Filter(Predicate<T> p, Iterator<T> input) {
super(new FilteringFlatApplyFunction<T>(p),input);
}
// uses a Predicate to decide whether the input element is output or not
private static class FilteringFlatApplyFunction<T> implements FlatApplyFunction<T,T> {
private final Predicate pred;
public FilteringFlatApplyFunction(Predicate<T> p) {
this.pred = p;
}
#Override
public T apply(Iterator T) {
T result = null;
if((!T.hasNext())) throw new IllegalStateException();
if (pred.check(T.next()) == true){
result = (T) T.next();
}
else{
return (T) T;
}
}
Here is the FlatApply
package iterators;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Queue;
public class FlatApply<InT,OutT> implements Iterator<OutT> {
private final Iterator<InT> input;
private final FlatApplyFunction<InT,OutT> f;
private final Queue<OutT> q;
public FlatApply(FlatApplyFunction<InT,OutT> f, Iterator<InT> input) {
this.input = input;
this.f = f;
this.q = new LinkedList<OutT>();
}
#Override
public boolean hasNext() {
if (!q.isEmpty()) {
return true;
}
else {
while(q.isEmpty() && input.hasNext()) {
List<OutT> result = f.apply(input.next());
q.addAll(result);
}
if(q.isEmpty()) return false;
else return true;
}
}
#Override
public OutT next() {
if((!hasNext())) throw new IllegalStateException();
return q.poll();
}
}
Here is the FlatApplyFunction
package iterators;
import java.util.List;
public interface FlatApplyFunction<InT, OutT> {
public List<OutT> apply(InT x);
}
Here is the apply class
package iterators;
import java.util.Iterator;
public class Apply<InT,OutT> implements Iterator<OutT> {
// The function that will be applied to each input element to make an output element
private final ApplyFunction<InT,OutT> f;
// The Iterator that this Apply object will get its input from
private final Iterator<InT> input;
public Apply(ApplyFunction<InT, OutT> f, Iterator<InT> input) {
this.input = input;
this.f = f;
}
#Override
public boolean hasNext() {
return input.hasNext();
}
#Override
public OutT next() {
if((!hasNext())) throw new IllegalStateException();
OutT result = f.apply(input.next());
return result;
}
}
The FlatApplyFunction interface says this:
public List<OutT> apply(InT x);
But the FilteringFlatApplyFunction implementation of that interface says this:
public T apply(Iterator T) {
The interface requires a List to be returned, by you're just returning T. Also, the parameter is required to be a T, but you have it as an Iterator.
Make those match, and this compiler error should go away.

Getting a result in the future?

I'm looking to get a result from a method which can take a while to complete and doesn't actually return the object, so I'd like to deal with it as effectively as possible. Here's an example of what I'm trying to achieve:
public static void main (String[] args) {
Object obj = someMethod();
System.out.println("The object is" + obj + ", wooh!");
}
public void callObject() {
// Sends request for the object
}
public void receiveObject(Object object) {
// Received the object
}
public Object someMethod() {
callObject();
// delay whilst the object is being received
// return received object once received, but how?
}
The method callObject will call to get the object, however a different method is called with the object in. I want someMethod() to be able to call for the object, and then return what it eventually receives, even though the actual call and receive are separate methods.
I've looked into using FutureTasks and Callables which I think is the way forward, I'm just not too sure how to implement it.
Sorry if I didn't explain myself too well, I'll give more information if necessary.
Thanks!
You could write a method, that kicks of some long running task asynchronously. You would then return a future object, that is empty but gets filled when the long running task is completed. In other programming languages, this is called a promise.
Here is an simple example. I created a method called someLongAsyncOperation which executes something that takes a while. To simulate this, I just sleep for 3 seconds before generating an answer.
import java.util.UUID;
import java.util.concurrent.*;
public class Test {
private static final ExecutorService executorService = Executors.newSingleThreadExecutor();
public Future<MyAnswer> someLongAsyncOperation(){
Future<MyAnswer> future = executorService.submit(() -> {
Thread.sleep(3000);
return new MyAnswer(UUID.randomUUID().toString());
});
return future;
}
public static void main(String[] args) throws Exception {
System.out.println("calling someLongAsyncOperation ...");
Future<MyAnswer> future = new Test().someLongAsyncOperation();
System.out.println("calling someLongAsyncOperation done.");
// do something else
System.out.println("wait for answer ...");
MyAnswer myAnswer = future.get();
System.out.printf("wait for answer done. Answer is: %s", myAnswer.value);
executorService.shutdown();
}
static class MyAnswer {
final String value;
MyAnswer(String value) {
this.value = value;
}
}
}
If you execute this little test class, you'll see, that someLongAsyncOperation returns fast, but when calling future.get(); we wait for the operation to complete.
You could now do something like starting of more than one longAsyncOperation, so they would run in parallel. And then wait until all of them are done.
Does this work as a starting point for you?
EDIT
You could implement someMethod like this:
public MyAnswer someMethod() throws ExecutionException, InterruptedException {
Future<MyAnswer> future = someLongAsyncOperation(); // kick of async operation
return future.get(); // wait for result
}
Which will make the async operation synchron again, by calling it and waiting for the result.
EDIT2
Here's another example that uses wait/notify:
import java.util.UUID;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class Test2 {
private static final ExecutorService executorService = Executors.newSingleThreadExecutor();
private Object receivedObject;
private final Object mutex = new Object();
public static void main (String[] args) throws InterruptedException {
Object obj = new Test2().someMethod();
System.out.println("The object is" + obj + ", wooh!");
executorService.shutdown();
}
public void callObject() {
System.out.println("callObject ...");
// Sends request for the object asynchronously!
executorService.submit(() -> {
// some wait time to simulate slow request
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
// provide object to callback
receiveObject(UUID.randomUUID().toString());
});
System.out.println("callObject done.");
}
public void receiveObject(Object object) {
System.out.println("receiveObject ...");
synchronized (mutex) {
this.receivedObject = object;
mutex.notify();
}
System.out.println("receiveObject done.");
}
public Object someMethod() throws InterruptedException {
System.out.println("someMethod ...");
synchronized (mutex) {
callObject();
while(this.receivedObject == null){
mutex.wait();
}
}
System.out.println("someMethod done.");
return this.receivedObject;
}
}
someMethod waits until receivedObject exists. receiveObject notifies upon arrival.
You need a callback:
private abstract class Callback<T>{
run(T object);
}
public Object someMethod() {
callObject(new Callback<Object>()
{
#Override
public void run(Object object)
{
System.out.println("The object is" + object + ", wooh!");
}
})
}
public void callObject(Callback<Object> callback) {
// Sends request for the object
callback.run(object);
}
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
class ThreadExample implements Callable<String>{
#Override
public String call() throws Exception {
// TODO Auto-generated method stub
return "Ashish";
}
}
public class FutureThreadExample {
public static void main(String a[]) throws InterruptedException, ExecutionException {
ExecutorService executorService=Executors.newFixedThreadPool(1);
List <Future<String>>objList=new ArrayList<Future<String>>();
for(int i=0;i<10;i++) {
Future<String> obj=executorService.submit(new ThreadExample());
objList.add(obj);
}
for( Future<String> fut:objList) {
System.out.println(fut.get());
}
executorService.shutdown();
}
}

Producer-Consumer with Predicate

I'm looking for a java collection that supports blocking read()s on a predicate. I wrote a simple version but it seems like this must have been invented already?
For example:
interface PredicateConsumerCollection<T> {
public void put(T t);
#Nullable
public T get(Predicate<T> p, long millis) throws InterruptedException;
}
put() delivers its argument to a waiting consumer with a matching predicate, or stashes it in a store. A get() returns immediately if a suitable T is already in the store, or blocks till a suitable value is put(), or times out. Consumers compete but fairness isn't critical in my case.
Anyone aware of a such a collection?
There is no immediate class that can solve your problem, but a combination of a ConcurrentHashMap and a BlockingQueue could be a solution.
The hash map is defined as:
final ConcurrentHashMap<Predicate, LinkedBlockingQueue<Result>> lookup;
The put needs to ensure, that for each Predicate a queue is added to the map, this can be done thread-safe using putIfAbsent.
If you have a fixed set of Predicates, you can simply pre-fill the list, then a Consumer can simply call lookup.get(Predicate).take()
If the amount of Predicates is unknown/too many, you need to write a wait/notify implementation for Consumers in case a Predicate is not yet in the list on your own.
I also need something very similar for testing that a certain JMS asynchronous message has been received within a certain timeout. It turns out that your question is relatively easy to implement by using basic wait/notify as explained in the Oracle tutorials. The idea is to make the put and query methods synchronized and let the query method do a wait. The put method calls notifyAll to wake up any waiting threads in the query method. The query method must then check if the predicate is matched. The most tricky thing is getting the timeout right due to waking up when the predicate does not match and due to possible " spurious wakeups". I found this stackoverflow post that provides the answer.
Here is the implementation I came up with:
import java.util.ArrayList;
import java.util.List;
// import net.jcip.annotations.GuardedBy;
import com.google.common.base.Predicate;
import com.google.common.collect.Iterables;
public class PredicateConsumerCollectionImpl<T> implements
PredicateConsumerCollection<T> {
// #GuardedBy("this")
private List<T> elements = new ArrayList<>();
#Override
public synchronized void put(T t) {
elements.add(t);
notifyAll();
}
#Override
public synchronized T query(Predicate<T> p, long millis)
throws InterruptedException {
T match = null;
long nanosOfOneMilli = 1000000L;
long endTime = System.nanoTime() + millis * nanosOfOneMilli;
while ((match = Iterables.find(elements, p, null)) == null) {
long sleepTime = endTime - System.nanoTime();
if (sleepTime <= 0) {
return null;
}
wait(sleepTime / nanosOfOneMilli,
(int) (sleepTime % nanosOfOneMilli));
}
return match;
}
synchronized boolean contains(T t) {
return elements.contains(t);
}
}
And here is a JUnit test that proves that the code works as intended:
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
import org.junit.Before;
import org.junit.Test;
import com.google.common.base.Predicate;
/**
* Unit test for the {#link PredicateConsumerCollection} implementation.
*
* <p>
* The tests act as consumers waiting for the test Producer to put a certain
* String.
*/
public class PredicateConsumerCollectionTest {
private static class Producer implements Runnable {
private PredicateConsumerCollection<String> collection;
public Producer(PredicateConsumerCollection<String> collection) {
this.collection = collection;
collection.put("Initial");
}
#Override
public void run() {
try {
int millis = 50;
collection.put("Hello");
Thread.sleep(millis);
collection.put("I");
Thread.sleep(millis);
collection.put("am");
Thread.sleep(millis);
collection.put("done");
Thread.sleep(millis);
collection.put("so");
Thread.sleep(millis);
collection.put("goodbye!");
} catch (InterruptedException e) {
e.printStackTrace();
fail("Unexpected InterruptedException");
}
}
}
private PredicateConsumerCollectionImpl<String> collection;
private Producer producer;
#Before
public void setup() {
collection = new PredicateConsumerCollectionImpl<>();
producer = new Producer(collection);
}
#Test(timeout = 2000)
public void wait_for_done() throws InterruptedException {
assertTrue(collection.contains("Initial"));
assertFalse(collection.contains("Hello"));
Thread producerThread = new Thread(producer);
producerThread.start();
String result = collection.query(new Predicate<String>() {
#Override
public boolean apply(String s) {
return "done".equals(s);
}
}, 1000);
assertEquals("done", result);
assertTrue(collection.contains("Hello"));
assertTrue(collection.contains("done"));
assertTrue(producerThread.isAlive());
assertFalse(collection.contains("goodbye!"));
producerThread.join();
assertTrue(collection.contains("goodbye!"));
}
#Test(timeout = 2000)
public void wait_for_done_immediately_happens() throws InterruptedException {
Thread producerThread = new Thread(producer);
producerThread.start();
String result = collection.query(new Predicate<String>() {
#Override
public boolean apply(String s) {
return "Initial".equals(s);
}
}, 1000);
assertEquals("Initial", result);
assertFalse(collection.contains("I"));
producerThread.join();
assertTrue(collection.contains("goodbye!"));
}
#Test(timeout = 2000)
public void wait_for_done_never_happens() throws InterruptedException {
Thread producerThread = new Thread(producer);
producerThread.start();
assertTrue(producerThread.isAlive());
String result = collection.query(new Predicate<String>() {
#Override
public boolean apply(String s) {
return "DONE".equals(s);
}
}, 1000);
assertEquals(null, result);
assertFalse(producerThread.isAlive());
assertTrue(collection.contains("goodbye!"));
}
}

How to know when a CompletionService is finished delivering results?

I want to use a CompletionService to process the results from a series of threads as they are completed. I have the service in a loop to take the Future objects it provides as they become available, but I don't know the best way to determine when all the threads have completed (and thus to exit the loop):
import java.util.concurrent.Callable;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.ThreadPoolExecutor;
public class Bar {
final static int MAX_THREADS = 4;
final static int TOTAL_THREADS = 20;
public static void main(String[] args) throws Exception{
final ThreadPoolExecutor threadPool = (ThreadPoolExecutor) Executors.newFixedThreadPool(MAX_THREADS);
final CompletionService<Integer> service = new ExecutorCompletionService<Integer>(threadPool);
for (int i=0; i<TOTAL_THREADS; i++){
service.submit(new MyCallable(i));
}
int finished = 0;
Future<Integer> future = null;
do{
future = service.take();
int result = future.get();
System.out.println(" took: " + result);
finished++;
}while(finished < TOTAL_THREADS);
System.out.println("Shutting down");
threadPool.shutdown();
}
public static class MyCallable implements Callable<Integer>{
final int id;
public MyCallable(int id){
this.id = id;
System.out.println("Submitting: " + id);
}
#Override
public Integer call() throws Exception {
Thread.sleep(1000);
System.out.println("finished: " + id);
return id;
}
}
}
I've tried checking the state of the ThreadPoolExecutor, but I know the getCompletedTaskCount and getTaskCount methods are only approximations and shouldn't be relied upon. Is there a better way to ensure that I've retrieved all the Futures from the CompletionService than counting them myself?
Edit: Both the link that Nobeh provided, and this link suggest that counting the number of tasks submitted, then calling take() that many times, is the way to go. I'm just surprised there isn't a way to ask the CompletionService or its Executor what's left to be returned.
See http://www.javaspecialists.eu/archive/Issue214.html for a decent suggestion on how to extend the ExecutorCompletionService to do what you're looking for. I've pasted the relevant code below for your convenience. The author also suggests making the service implement Iterable, which I think would be a good idea.
FWIW, I agree with you that this really should be part of the standard implementation, but alas, it's not.
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
public class CountingCompletionService<V> extends ExecutorCompletionService<V> {
private final AtomicLong submittedTasks = new AtomicLong();
private final AtomicLong completedTasks = new AtomicLong();
public CountingCompletionService(Executor executor) {
super(executor);
}
public CountingCompletionService(
Executor executor, BlockingQueue<Future<V>> queue) {
super(executor, queue);
}
public Future<V> submit(Callable<V> task) {
Future<V> future = super.submit(task);
submittedTasks.incrementAndGet();
return future;
}
public Future<V> submit(Runnable task, V result) {
Future<V> future = super.submit(task, result);
submittedTasks.incrementAndGet();
return future;
}
public Future<V> take() throws InterruptedException {
Future<V> future = super.take();
completedTasks.incrementAndGet();
return future;
}
public Future<V> poll() {
Future<V> future = super.poll();
if (future != null) completedTasks.incrementAndGet();
return future;
}
public Future<V> poll(long timeout, TimeUnit unit)
throws InterruptedException {
Future<V> future = super.poll(timeout, unit);
if (future != null) completedTasks.incrementAndGet();
return future;
}
public long getNumberOfCompletedTasks() {
return completedTasks.get();
}
public long getNumberOfSubmittedTasks() {
return submittedTasks.get();
}
public boolean hasUncompletedTasks() {
return completedTasks.get() < submittedTasks.get();
}
}
The code below is inspired by #Mark's answer, but I find it more convenient to use:
package com.example;
import java.util.Iterator;
import java.util.concurrent.Callable;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
public class CompletionIterator<T> implements Iterator<T>, AutoCloseable {
private AtomicInteger count = new AtomicInteger(0);
private CompletionService<T> completer;
private ExecutorService executor = Executors.newWorkStealingPool(100);
public CompletionIterator() {
this.completer = new ExecutorCompletionService<>(executor);
}
public void submit(Callable<T> task) {
completer.submit(task);
count.incrementAndGet();
}
#Override
public boolean hasNext() {
return count.decrementAndGet() >= 0;
}
#Override
public T next() {
try {
return completer.take().get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
}
#Override
public void close() {
try {
executor.shutdown();
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
executor = null;
completer = null;
count = null;
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
This is how it can be used :
try(CompletionIterator service = new CompletionIterator()) {
service.submit(task1);
service.submit(task2);
// all tasks must be submitted before iterating, to avoid race condition
for (Future<Integer> future : service) {
System.out.printf("Job %d is done%n", future.get());
}
}
Answering to these questions gives you the answer?
Do your asynchronous tasks create other tasks submitted to CompletionService?
Is service the only object that is supposed to handle the tasks created in your application?
Based on reference documentation, CompletionService acts upon a consumer/producer approach and takes advantage of an internal Executor. So, as long as, you produce the tasks in one place and consume them in another place, CompletionService.take() will denote if there are any more results to give out.
I believe this question also helps you.
My take based on Alex R' variant. Implying this will only be called in one thread, so no atomics just plain int counter
public class CompletionIterator<T> implements Iterable<T> {
private int _count = 0;
private final CompletionService<T> _completer;
public CompletionIterator(ExecutorService executor) {
this._completer = new ExecutorCompletionService<>(executor);
}
public void submit(Callable<T> task) {
_completer.submit(task);
_count++;
}
#Override
public Iterator<T> iterator() {
return new Iterator<T>() {
#Override
public boolean hasNext() {
return _count > 0;
}
#Override
public T next() {
try {
T ret = _completer.take().get();
_count--;
return ret;
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
}
};
}
}

Categories

Resources