I have been using custom blockingqueue inside ThreadExecutorPool, but some times task workers do not take task and dispacher thread does not put a new task into queue.
I wonder following custom blocking queue implementation causes deadlock. Is there any wrong with this code?
Is is better to and synchronized block for add() and take() methods.
import java.util.Collection;
import java.util.concurrent.LinkedBlockingQueue;
import org.apache.log4j.Logger;
import com.ttech.utils.alarm.Alarm;
import com.ttech.utils.alarm.AlarmInterface;
import com.ttech.utils.counter.Counter;
import com.ttech.utils.counter.SNMPAgent;
public class WorkerQueue<E> extends LinkedBlockingQueue<E> {
private static final long serialVersionUID = 1L;
public Integer lowThreshold;
public Integer highThreshold;
public Integer capacity;
public String name;
public String type;
public Counter counter = null;
public boolean writeAlarmLog;
public static final Logger logger = Logger.getLogger(WorkerQueue.class);
public static Alarm HighThresholdAlarm = null;
public static Alarm CapacityAlarm = null;
// Check the size here and clear capacity and high threshold alarms in case
public E take() throws InterruptedException {
E data = super.take();
counter.setNewValue(super.size());
if (super.size() == lowThreshold) {
if(!this.writeAlarmLog) {
HighThresholdAlarm.clear(name);
CapacityAlarm.clear(name);
} else {
HighThresholdAlarm.clearLog(name, "Queue High Threshold");
CapacityAlarm.clearLog(name, "Queue Capacity Overload");
}
}
return data;
}
public E poll() {
E data = super.poll();
counter.setNewValue(super.size());
if (super.size() == lowThreshold) {
if(!this.writeAlarmLog) {
HighThresholdAlarm.clear(name);
CapacityAlarm.clear(name);
} else {
HighThresholdAlarm.clearLog(name, "Queue High Threshold");
CapacityAlarm.clearLog(name, "Queue Capacity Overload");
}
}
return data;
}
public int drainTo(Collection<? super E> c, int maxElements){
int size = super.drainTo(c,maxElements);
counter.setNewValue(super.size());
return size;
}
// During adding the data to queue check capacity and high threshold raise alarm in case
public boolean add(E data) {
Boolean rc = true;
if (capacity > 0) {
if (this.size() >= capacity) {
logger.error("Queue " + name + " is over capacity");
if(!this.writeAlarmLog)
CapacityAlarm.raise(name);
else
CapacityAlarm.raiseLog(AlarmInterface.AS_CRITICAL, name, "Queue Capacity Overload");
return false;
}
}
if (!super.add(data)) {
logger.error("Cannot add data to queue:" + name);
rc = false;
} else {
counter.setNewValue(super.size());
}
if (highThreshold == super.size()) {
if(!this.writeAlarmLog)
HighThresholdAlarm.raise(name);
else
HighThresholdAlarm.raiseLog(AlarmInterface.AS_CRITICAL, name, "Queue High Threshold");
}
return rc;
}
}
ThreadPoolExecutor does not add tasks to its work queue. It offers them and if not accepted passes them to the configured RejectedExecutionHandler. By default this is the abort policy handler, which causes a RejectedExecutionException to be thrown.
The add method in your custom queue will never be called.
If you want to track changes in the number of in-flight tasks you have, I would suggest overriding the beforeExecute or afterExecute method of the executor itself. The number of active tasks can be obtained from getActiveCount.
Related
I am trying to write a simple queue like ArrayBlockingQueue in which the head of the queue will be removed if the queue is full while adding an element. The class should just have the below public methods
To get the size of the Queue
To get an element from the head of the queue. If no element available block.
To add an element at the tail of the queue
Can someone review the below code and let me know if there is a better way of doing this?
public class CircularArrayNonBlockingQueue<E> {
private ArrayBlockingQueue<E> blockingQueue;
public CircularArrayNonBlockingQueue(int size) {
blockingQueue = new ArrayBlockingQueue<>(size);
}
public synchronized int size() {
return blockingQueue.size();
}
public synchronized void add(E element) {
if(blockingQueue.remainingCapacity() <= 0) {
blockingQueue.poll();
}
blockingQueue.add(element);
}
public synchronized E poll() {
return blockingQueue.poll();
}
}
EDIT
Based on the discussion in the comments I don't need to make all the methods synchronized. The updated code looks like below -
public class CircularNonBlockingQueue<E> {
private final ArrayBlockingQueue<E> blockingQueue;
public CircularNonBlockingQueue(int size) {
blockingQueue = new ArrayBlockingQueue<>(size);
}
public int size() {
return blockingQueue.size();
}
public synchronized void add(E element) {
if(blockingQueue.remainingCapacity() <= 0) {
blockingQueue.poll();
}
blockingQueue.add(element);
}
public E take() throws InterruptedException {
return blockingQueue.take();
}
}
Having a thread-safe backend collection does not necessarily make a correct program. When only your add method is synchronized, the take() method may run concurrently to it, so it is possible that after your if(blockingQueue.remainingCapacity() <= 0) test within add, a concurrently running take() removes an element, so the poll() within add may remove an element unnecessarily. There is a perceivable difference to the situation where add() would complete before the take(), as the consuming thread would receive a different item. It other words, the effect would be as if add would sometimes not remove the oldest item, but the second oldest one.
On the other hand, if you use synchronized for all of your methods consistently, there is no need to have a thread-safe backend collection:
import java.util.ArrayDeque;
public class CircularBlockingQueue<E> {
private final ArrayDeque<E> blockingQueue;
private final int maxSize;
public CircularBlockingQueue(int size) {
if(size<1) throw new IllegalArgumentException("size == "+size);
blockingQueue = new ArrayDeque<>(size);
maxSize = size;
}
public synchronized int size() {
return blockingQueue.size();
}
public synchronized void add(E element) {
if(blockingQueue.size() == maxSize) {
blockingQueue.poll();
}
blockingQueue.add(element);
notify();
}
public synchronized E take() throws InterruptedException {
while(blockingQueue.isEmpty()) wait();
return blockingQueue.remove();
}
}
However, if you can live with weaker guarantees regarding the oldest element, you can use a BlockingQueue and don’t need any synchronized:
public class CircularBlockingQueue<E> {
private final ArrayBlockingQueue<E> blockingQueue;
public CircularBlockingQueue(int size) {
blockingQueue = new ArrayBlockingQueue<>(size);
}
public int size() {
return blockingQueue.size();
}
public void add(E element) {
while(!blockingQueue.offer(element)) {
blockingQueue.poll();
}
}
public E take() throws InterruptedException {
return blockingQueue.take();
}
}
It must be noted that neither of these solutions provides “fairness”. So if the number of producer and consumer threads is large compared to the queue’s capacity, there is the risk that producers repeatedly remove items without reactivating threads blocked in take(). So you should always ensure to have a sufficiently large capacity.
Suppose a field in an object that changes from null to non-null and back and forth, etc., depending on the operation of one thread.
A second thread shall lazily take some action whenever it happens to get hold of a non-null value. In particular the second thread shall wait until the value switches to non-null. If it falls out of the wait, I want to be sure that it has a non-null in its hand.
This does not seem like a queue situation, because the second thread will not take the element away, it just uses it if one happens to be available.
It also does not fit for semaphore use, because again it would not .acquire() a permit.
Rather it reminds of of compare-and-get with built in wait, but this does not seem to exist.
Is there a predefined device for this in java.util.concurrent that I miss to identify. How can this be done?
This is similar but does not have an accepted answer or one that would help here.
Here's an implementation relying on ReentrantLock to manage a volatile field. This borrows heavily from the double-checked locking idiom, but instead of creating the value itself, a read operation waits on a condition to signal that a value has been set.
The get() method is overloaded with a version that accepts a timeout. Both versions are interruptible.
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class BlockingRef<V> {
private final Lock lock = new ReentrantLock(true);
private final Condition signal = lock.newCondition();
private volatile V value;
public BlockingRef() {
this(null);
}
public BlockingRef(V initialValue) {
this.value = initialValue;
}
public final void set(V value) {
lock.lock();
try {
this.value = value;
signal.signalAll();
} finally {
lock.unlock();
}
}
public final V get() throws InterruptedException {
V result = value;
if (result == null) {
lock.lockInterruptibly();
try {
for (result = value; result == null; result = value)
signal.await();
} finally {
lock.unlock();
}
}
return result;
}
public final V get(long time, TimeUnit unit)
throws TimeoutException, InterruptedException
{
V result = value;
if (result == null) {
long start = System.nanoTime();
if (!lock.tryLock(time, unit)) throw new TimeoutException();
try {
time = unit.toNanos(time);
for (result = value; result == null; result = value) {
long wait = time - (System.nanoTime() - start);
if (wait <= 0) throw new TimeoutException();
signal.await(wait, TimeUnit.NANOSECONDS);
}
} finally {
lock.unlock();
}
}
return result;
}
#Override
public String toString() {
return String.valueOf(value);
}
}
I'm currently working on my first multithreaded software - a program, which calculates prime numbers...
Basically I create n (number of Threads) runnables. These runnables are added to an ArrayList. They check, whether a number is a prime. If the number is a prime I add it into an long array for later use. Since I want the primes to be in correct order in this array I need specific Threads to wait for others. I do this by looping through the ArrayList (see above) and wait for the threads, which check a lower number.
After a thread is done I want to remove it from the given ArrayList, but I cant since the other threads are still looping through it (This is the reason why the ConcurrentModificationException occurs I guess - This is my first time working with threads...).
I honestly hope that any of you guys can help me :)
Thank your really much!
Matthias
My runnable class (I just create four objects of this class in the main method):
import java.util.ArrayList;
public class PrimeRunnable implements Runnable {
//Static Util
public static ArrayList<PrimeRunnable> runningThreads = new ArrayList<PrimeRunnable>();
public static long[] primes;
public static int nextFreeIndex = 1;
public static long nextPossiblePrime = 3;
//Object specific
private long numberToCheck;
private Thread primeThread;
private String threadName;
private long threadID;
public PrimeRunnable() {
numberToCheck = nextPossiblePrime;
increaseNextPossiblePrime();
threadName = "ThreadToCheck" + numberToCheck;
threadID = numberToCheck;
runningThreads.add(this);
}
#Override
public void run() {
boolean isPrime = true;
double sqrtOfPossiblePrime = Math.sqrt(numberToCheck);
long lastDevider = 0;
for(int index = 0; index < nextFreeIndex; index++) {
lastDevider = primes[index];
if(numberToCheck%primes[index] == 0) {
isPrime = false;
break;
}
if(primes[index] > sqrtOfPossiblePrime) {
break;
}
}
while(lastDevider < sqrtOfPossiblePrime) {
lastDevider += 1;
if(numberToCheck%lastDevider == 0) {
isPrime = false;
break;
}
}
if(isPrime) {
//Wait for lower Threads.
for(PrimeRunnable runnable : runningThreads) {
if(runnable.getThreadID() < this.getThreadID()) {
try {
runnable.primeThread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
primes[nextFreeIndex] = numberToCheck;
increaseNextFreeIndex();
System.out.println(numberToCheck);
}
runningThreads.remove(this);
}
public void start() {
if(primeThread == null) {
primeThread = new Thread(this, threadName);
}
primeThread.start();
}
public void reset() {
numberToCheck = nextPossiblePrime;
increaseNextPossiblePrime();
threadName = "ThreadToCheck" + numberToCheck;
threadID = numberToCheck;
//No need to readd into runningThread, since we only manipulate an already existing object.
primeThread = new Thread(this, threadName);
primeThread.start();
}
public static void setUpperBorder(int upperBorder) {
if(primes == null) {
primes = new long[upperBorder];
primes[0] = 2;
} else {
System.err.println("You are not allowed to set the upper border while running.");
}
}
public long getNumberToCheck() {
return numberToCheck;
}
private void increaseNextPossiblePrime() {
nextPossiblePrime += 2;
}
private void increaseNextFreeIndex() {
nextFreeIndex += 2;
}
public long getThreadID() {
return threadID;
}
public boolean isAlive() {
return primeThread.isAlive();
}
}
I was able to replicate the issue and fix it using Java implementation of a concurrent list CopyOnWriteArrayList
Here's my main class
public class PrimeRunnableMain {
public static void main(String[] args) {
PrimeRunnable.setUpperBorder(10);
PrimeRunnable primeRunnable1 = new PrimeRunnable();
PrimeRunnable primeRunnable2 = new PrimeRunnable();
PrimeRunnable primeRunnable3 = new PrimeRunnable();
PrimeRunnable primeRunnable4 = new PrimeRunnable();
primeRunnable1.start();
primeRunnable2.start();
primeRunnable3.start();
primeRunnable4.start();
}
}
and here's PrimeRunnable
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
public class PrimeRunnable implements Runnable {
// Static Util
public static List<PrimeRunnable> runningThreads = new CopyOnWriteArrayList<PrimeRunnable>();
public static long[] primes;
public static int nextFreeIndex = 1;
public static long nextPossiblePrime = 3;
// Object specific
private long numberToCheck;
private Thread primeThread;
private String threadName;
private long threadID;
public PrimeRunnable() {
numberToCheck = nextPossiblePrime;
increaseNextPossiblePrime();
threadName = "ThreadToCheck" + numberToCheck;
threadID = numberToCheck;
runningThreads.add(this);
}
#Override
public void run() {
boolean isPrime = true;
double sqrtOfPossiblePrime = Math.sqrt(numberToCheck);
long lastDevider = 0;
for (int index = 0; index < nextFreeIndex; index++) {
lastDevider = primes[index];
if (numberToCheck % primes[index] == 0) {
isPrime = false;
break;
}
if (primes[index] > sqrtOfPossiblePrime) {
break;
}
}
while (lastDevider < sqrtOfPossiblePrime) {
lastDevider += 1;
if (numberToCheck % lastDevider == 0) {
isPrime = false;
break;
}
}
if (isPrime) {
// Wait for lower Threads.
for (PrimeRunnable runnable : runningThreads) {
if (runnable.getThreadID() < this.getThreadID()) {
try {
runnable.primeThread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
primes[nextFreeIndex] = numberToCheck;
increaseNextFreeIndex();
System.out.println(numberToCheck);
}
runningThreads.remove(this);
}
public void start() {
if (primeThread == null) {
primeThread = new Thread(this, threadName);
}
primeThread.start();
}
public void reset() {
numberToCheck = nextPossiblePrime;
increaseNextPossiblePrime();
threadName = "ThreadToCheck" + numberToCheck;
threadID = numberToCheck;
// No need to readd into runningThread, since we only manipulate an
// already existing object.
primeThread = new Thread(this, threadName);
primeThread.start();
}
public static void setUpperBorder(int upperBorder) {
if (primes == null) {
primes = new long[upperBorder];
primes[0] = 2;
} else {
System.err
.println("You are not allowed to set the upper border while running.");
}
}
public long getNumberToCheck() {
return numberToCheck;
}
private void increaseNextPossiblePrime() {
nextPossiblePrime += 2;
}
private void increaseNextFreeIndex() {
nextFreeIndex += 2;
}
public long getThreadID() {
return threadID;
}
public boolean isAlive() {
return primeThread.isAlive();
}
}
What about a PrimeListener class that contains a synchronized method publishPrime that inserts the prime in the correct position in the list? Inserting at the right position into the list should not take too much time, if you start at the last index of a LinkedList.
Alternatively you could also insert it into a SortedSet (implementation: TreeSet). I presume you don't want any duplicate primes anyway. In that case synchronizedSortedSet may be directly used instead of the listener.
Note that you still seem rather stuck on lower level structures. When programming concurrently on Java it pays off to use the higher level constructs (executors, futures, concurrent queue's etc. etc.).
The main distinction between fail-fast and fail-safe iterators is
whether or not the collection can be modified while it is being
iterated. Fail-safe iterators allow this; fail-fast iterators do not.
Fail-fast iterators operate directly on the collection itself. During
iteration, fail-fast iterators fail as soon as they realize that the
collection has been modified (i.e., upon realizing that a member has
been added, modified, or removed) and will throw a
ConcurrentModificationException. Some examples include ArrayList,
HashSet, and HashMap (most JDK1.4 collections are implemented to be
fail-fast). Fail-safe iterates operate on a cloned copy of the
collection and therefore do not throw an exception if the collection
is modified during iteration. Examples would include iterators
returned by ConcurrentHashMap or CopyOnWriteArrayList.
When using synchronized block or method, we synchronized by mutable object. But I don't understand how to use Locks with Conditions from j.u.c.. I'm trying to solve puzzle with two workers and a cart with lock and conditions. When first worker add weight to the cart - second wait. When cart is full, than first worker wait and second releases the cart.
I create two threads for each worker and use one cart. But in reality only one thread performs (worker, that add weight) until cart is full. Than program blocks. What I'm doing wrong and what I misunderstand?
That's my implementation of this puzzle.
package puzzles.workers;
public enum WorkerType {
ADDER, REDUCER;
}
Cart class
package puzzles.workers;
public class Cart {
private static final int INITIAL_CAPACITY = 10;
private static final int INITIAL_WEIGHT = 0;
private int capacity;
private int weight;
public Cart() {
this(INITIAL_CAPACITY);
}
public Cart(int capacity) {
this.capacity = capacity;
weight = INITIAL_WEIGHT;
}
public void addWeight() {
weight++;
}
public void reduceWeight() {
weight--;
}
public int getCapacity() {
return capacity;
}
public int getWeight() {
return weight;
}
}
Worker class.
package puzzles.workers;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class WorkerWithLock implements Runnable {
private final Cart cart;
private WorkerType workerType;
final Lock lock = new ReentrantLock();
final Condition whenEmpty = lock.newCondition();
final Condition whenFull = lock.newCondition();
public WorkerWithLock(Cart cart, WorkerType workerType) {
this.cart = cart;
this.workerType = workerType;
}
#Override
public void run() {
while (true) {
if (workerType == WorkerType.ADDER) {
try {
addWeight();
} catch (InterruptedException e) {
e.printStackTrace();
Thread.currentThread().interrupt();
}
} else {
try {
reduceWeight();
} catch (InterruptedException e) {
e.printStackTrace();
Thread.currentThread().interrupt();
}
}
}
}
public void addWeight() throws InterruptedException {
lock.lock();
try {
while (cart.getWeight() == (cart.getCapacity() - 1)) {
whenFull.await();
}
cart.addWeight();
System.out.println("++ weight is: " + cart.getWeight());
whenEmpty.signalAll();
Thread.sleep(500);
} finally {
lock.unlock();
}
}
public void reduceWeight() throws InterruptedException {
lock.lock();
try {
while (cart.getWeight() == 0) {
whenEmpty.await();
}
cart.reduceWeight();
System.out.println("-- weight is: " + cart.getWeight());
whenFull.signalAll();
Thread.sleep(500);
} finally {
lock.unlock();
}
}
public static void main(String[] args) throws InterruptedException {
Cart cart = new Cart(5);
WorkerWithLock adder = new WorkerWithLock(cart, WorkerType.ADDER);
WorkerWithLock reducer = new WorkerWithLock(cart, WorkerType.REDUCER);
new Thread(reducer).start();
new Thread(adder).start();
}
}
It is a kind of a race condition
That is because both threads are waiting.
One in addWeightand one in reduceWeight.
First the reducer stops, if weight is 0. At this time, the adder is may be not already started.
Than the adder stops, if weight = cpacity - 1
Now, both are waiting for an interrupt().
EDIT1. See my comments in the code
public void addWeight() throws InterruptedException {
lock.lock();
try {
while (cart.getWeight() == (cart.getCapacity() - 1)) {
whenFull.await(); //<-- ADDER waits here
}
cart.addWeight();
System.out.println("++ weight is: " + cart.getWeight());
whenEmpty.signalAll(); //<-- Never called since both are waiting
Thread.sleep(500);
} finally {
lock.unlock();
}
}
public void reduceWeight() throws InterruptedException {
lock.lock();
try {
while (cart.getWeight() == 0) {
whenEmpty.await(); //<-- REDUCER waits here
}
cart.reduceWeight();
System.out.println("-- weight is: " + cart.getWeight());
whenFull.signalAll(); //<-- Never called since both are waiting
Thread.sleep(500);
} finally {
lock.unlock();
}
}
EDIT2: Ok, now I understand the behaviour.
Your code is designed to synchronize ONE object for multiple threads, but your are using TWO objects.
Every of your both WorkerWithLock Objects has its own Lock and Condition objects. So calls of lock.lock() and whenFull.signalAll() in object ADDER does not effect Object REDUCER.
Your code will work if you make the lock and condition variables static, so that both objects are working with the same lock and the same condition
final static Lock lock = new ReentrantLock();
final static Condition whenEmpty = lock.newCondition();
final static Condition whenFull = lock.newCondition();
Sometimes multithreading is hard :)
I have the following code:
while(slowIterator.hasNext()) {
performLengthTask(slowIterator.next());
}
Because both iterator and task are slow it makes sense to put those into separate threads. Here is a quick and dirty attempt for an Iterator wrapper:
class AsyncIterator<T> implements Iterator<T> {
private final BlockingQueue<T> queue = new ArrayBlockingQueue<T>(100);
private AsyncIterator(final Iterator<T> delegate) {
new Thread() {
#Override
public void run() {
while(delegate.hasNext()) {
queue.put(delegate.next()); // try/catch removed for brevity
}
}
}.start();
}
#Override
public boolean hasNext() {
return true;
}
#Override
public T next() {
return queue.take(); // try/catch removed for brevity
}
// ... remove() throws UnsupportedOperationException
}
However this implementation lacks support for "hasNext()". It would be ok of course for the hasNext() method to block until it knows whether to return true or not. I could have a peek object in my AsyncIterator and I could change hasNext() to take an object from the queue and have next() return this peek. But this would cause hasNext() to block indefinitely if the delegate iterator's end has been reached.
Instead of utilizing the ArrayBlockingQueue I could of course do thread communication myself:
private static class AsyncIterator<T> implements Iterator<T> {
private final Queue<T> queue = new LinkedList<T>();
private boolean delegateDone = false;
private AsyncIterator(final Iterator<T> delegate) {
new Thread() {
#Override
public void run() {
while (delegate.hasNext()) {
final T next = delegate.next();
synchronized (AsyncIterator.this) {
queue.add(next);
AsyncIterator.this.notify();
}
}
synchronized (AsyncIterator.this) {
delegateDone = true;
AsyncIterator.this.notify();
}
}
}.start();
}
#Override
public boolean hasNext() {
synchronized (this) {
while (queue.size() == 0 && !delegateDone) {
try {
wait();
} catch (InterruptedException e) {
throw new Error(e);
}
}
}
return queue.size() > 0;
}
#Override
public T next() {
return queue.remove();
}
#Override
public void remove() {
throw new UnsupportedOperationException();
}
}
However all the extra synchronizations, waits and notifys don't really make the code any more readable and it is easy to hide a race condition somewhere.
Any better ideas?
Update
Yes I do know about common observer/observable patterns. However the usual implementations don't foresee an end to the flow of data and they are not iterators.
I specifically want an iterator here, because actually the above mentioned loop exists in an external library and it wants an Iterator.
This is a tricky one, but I think I got the right answer this time. (I deleted my first answer.)
The answer is to use a sentinel. I haven't tested this code, and I removed try/catches for clarity:
public class AsyncIterator<T> implements Iterator<T> {
private BlockingQueue<T> queue = new ArrayBlockingQueue<T>(100);
private T sentinel = (T) new Object();
private T next;
private AsyncIterator(final Iterator<T> delegate) {
new Thread() {
#Override
public void run() {
while (delegate.hasNext()) {
queue.put(delegate.next());
}
queue.put(sentinel);
}
}.start();
}
#Override
public boolean hasNext() {
if (next != null) {
return true;
}
next = queue.take(); // blocks if necessary
if (next == sentinel) {
return false;
}
return true;
}
#Override
public T next() {
T tmp = next;
next = null;
return tmp;
}
}
The insight here is that hasNext() needs to block until the next item is ready. It also needs some kind of quit condition, and it can't use an empty queue or a boolean flag for that because of threading issues. A sentinel solves the problem without any locking or synchronization.
Edit: cached "next" so hasNext() can be called more than once.
Or save yourself the headache and use RxJava:
import java.util.Iterator;
import rx.Observable;
import rx.Scheduler;
import rx.observables.BlockingObservable;
import rx.schedulers.Schedulers;
public class RxAsyncIteratorExample {
public static void main(String[] args) throws InterruptedException {
final Iterator<Integer> slowIterator = new SlowIntegerIterator(3, 7300);
// the scheduler you use here will depend on what behaviour you
// want but io is probably what you want
Iterator<Integer> async = asyncIterator(slowIterator, Schedulers.io());
while (async.hasNext()) {
performLengthTask(async.next());
}
}
public static <T> Iterator<T> asyncIterator(
final Iterator<T> slowIterator,
Scheduler scheduler) {
final Observable<T> tObservable = Observable.from(new Iterable<T>() {
#Override
public Iterator<T> iterator() {
return slowIterator;
}
}).subscribeOn(scheduler);
return BlockingObservable.from(tObservable).getIterator();
}
/**
* Uninteresting implementations...
*/
public static void performLengthTask(Integer integer)
throws InterruptedException {
log("Running task for " + integer);
Thread.sleep(10000l);
log("Finished task for " + integer);
}
private static class SlowIntegerIterator implements Iterator<Integer> {
private int count;
private final long delay;
public SlowIntegerIterator(int count, long delay) {
this.count = count;
this.delay = delay;
}
#Override
public boolean hasNext() {
return count > 0;
}
#Override
public Integer next() {
try {
log("Starting long production " + count);
Thread.sleep(delay);
log("Finished long production " + count);
}
catch (InterruptedException e) {
throw new IllegalStateException(e);
}
return count--;
}
#Override
public void remove() {
throw new UnsupportedOperationException();
}
}
private static final long startTime = System.currentTimeMillis();
private static void log(String s) {
double time = ((System.currentTimeMillis() - startTime) / 1000d);
System.out.println(time + ": " + s);
}
}
Gives me:
0.031: Starting long production 3
7.332: Finished long production 3
7.332: Starting long production 2
7.333: Running task for 3
14.633: Finished long production 2
14.633: Starting long production 1
17.333: Finished task for 3
17.333: Running task for 2
21.934: Finished long production 1
27.334: Finished task for 2
27.334: Running task for 1
37.335: Finished task for 1