Develop Reference

How to implement a Trie Iterator with Thread run()

I have an uni assignment where I need to implement a Trie and it's nodes(Node) and the iterator.
The iterator should iterate over the nodes using a StringBuffer to maintain the status of the word and it should implement the Runnable interface, meaning I have to implement a run() method!
I have tried this implementation:
class NodeIterator implements Iterator<String>, Runnable {
String nextWord;
boolean terminated;
Thread thread;
public NodeIterator() {
thread = new Thread(this,"Node iterator");
thread.start();
}
#Override
public void run() {
terminated = false;
visit(Trie.this.root);
synchronized (this) {
terminated = true;
handshake();
}
}
private void visit(Node node) {
}
private void handshake() {
notify();
try {
wait();
} catch (InterruptedException e) {
}
}
#Override
public boolean hasNext() {
synchronized (this) {
if(!terminated)
handshake();
}
return nextWord != null;
}
#Override
public String next() {
String word = nextWord;
synchronized (this) {
word = null;
}
return word;
}
}
I am missing an implementation for the visit() method as I'm not sure how to "visit" the nodes and I'm not sure the rest is correct as I've never worked with threads.
Should I be doing something differently?
Edit:
public class Trie implements Iterable<String> {
Node root;
...
private static class Node {
private HashMap<Character, Node> children;
private boolean endOfWord;
...
}
}

One way you can implement visit() is with recursion. The recursive method needs to keep track of the prefix of the string built so far, and if it finds a node that ends a word, publishes the string it has found so far. Assuming you can't change the signature for visit(Node), you'll need a helper method:
void visit(Node root) {
visitRecursive("", root);
}
private void visitRecursive(String prefix, Node node) {
if (node.endOfWord) {
nextWord = prefix;
synchronized (this) {
handshake();
}
}
for (Map.Entry<Character, Node> entry : node.children.entrySet()) {
visitRecursive(prefix + entry.getKey(), entry.getValue());
}
}
I'm not sure all your multithreading code is correct, but the very least this will iterate over all strings stored in the trie.

Elements not removed from Set

I have a TreeSet with a comparator:
private final TreeSet<TimedTask> sortedEvents;
public TimedUpdatableTaskList(){
Comparator<TimedTask> comparator = new TimedTaskComparator();
sortedEvents = new TreeSet<>(comparator);
}
The class implements a method to add elements to the set:
public void add(TimedTask task) {
synchronized (sortedEvents) {
sortedEvents.add(task);
log.info("Add task {}:{}", task.getClass().getName(), task);
}
}
The TimedTasks added are like this:
class AIRepairTask extends TimedTask {
private AsyncEventBus clientServerEventBus;
private final IShip ship;
private final IShipyard shipyard;
public AIRepairTask(LocalDateTime executionTime, IShip ship, IShipyard shipyard) {
super();
setExecutionTime(executionTime);
this.ship = ship;
this.shipyard = shipyard;
}
#Override
public void run() {
ship.repair();
shipyard.removeCompletedRepair(ship);
ship.setAvailable(true);
clientServerEventBus.post(new RepairFinishedEvent(ship));
}
}
And then there is a method that iterates over the elements and eventually removes them:
public void handleClockTick(ClockTick event) {
LocalDateTime now = date.getCurrentDate();
int nbHandledTasks = 0;
synchronized (sortedEvents) {
int initialSize = sortedEvents.size();
boolean moreEvents = true;
while(moreEvents && !sortedEvents.isEmpty()) {
TimedTask task = sortedEvents.first();
Preconditions.checkNotNull(task.getExecutionTime(), "The exectution time of the task may not be null");
if (task.getExecutionTime().isBefore(now)) {
try {
task.run();
boolean removed = sortedEvents.remove(task);
Preconditions.checkArgument(removed, "The Task "+task+" was not removed.");
nbHandledTasks++;
} catch (Exception e) {
e.printStackTrace()
}
} else {
moreEvents = false;
}
}
Preconditions.checkArgument(initialSize - nbHandledTasks == sortedEvents.size(), "List size did not become smaller: initial size "+initialSize+", handled tasks "+nbHandledTasks+", actual list size: "+sortedEvents.size());
}
}
As the essential parts are wrapped in a synchronized block, the set should not change while iterating over the elements.
Given that there are no exceptions in the try block, the precondition check at the end, should never fail, but that is exactly what happens:
java.lang.IllegalArgumentException: List size did not become smaller: initial size 41, handled tasks 2, actual list size: 41
How can this happen?

Puzzle about two workers and a cart. Usage lock and conditions doesn't work

When using synchronized block or method, we synchronized by mutable object. But I don't understand how to use Locks with Conditions from j.u.c.. I'm trying to solve puzzle with two workers and a cart with lock and conditions. When first worker add weight to the cart - second wait. When cart is full, than first worker wait and second releases the cart.
I create two threads for each worker and use one cart. But in reality only one thread performs (worker, that add weight) until cart is full. Than program blocks. What I'm doing wrong and what I misunderstand?
That's my implementation of this puzzle.
package puzzles.workers;
public enum WorkerType {
ADDER, REDUCER;
}
Cart class
package puzzles.workers;
public class Cart {
private static final int INITIAL_CAPACITY = 10;
private static final int INITIAL_WEIGHT = 0;
private int capacity;
private int weight;
public Cart() {
this(INITIAL_CAPACITY);
}
public Cart(int capacity) {
this.capacity = capacity;
weight = INITIAL_WEIGHT;
}
public void addWeight() {
weight++;
}
public void reduceWeight() {
weight--;
}
public int getCapacity() {
return capacity;
}
public int getWeight() {
return weight;
}
}
Worker class.
package puzzles.workers;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class WorkerWithLock implements Runnable {
private final Cart cart;
private WorkerType workerType;
final Lock lock = new ReentrantLock();
final Condition whenEmpty = lock.newCondition();
final Condition whenFull = lock.newCondition();
public WorkerWithLock(Cart cart, WorkerType workerType) {
this.cart = cart;
this.workerType = workerType;
}
#Override
public void run() {
while (true) {
if (workerType == WorkerType.ADDER) {
try {
addWeight();
} catch (InterruptedException e) {
e.printStackTrace();
Thread.currentThread().interrupt();
}
} else {
try {
reduceWeight();
} catch (InterruptedException e) {
e.printStackTrace();
Thread.currentThread().interrupt();
}
}
}
}
public void addWeight() throws InterruptedException {
lock.lock();
try {
while (cart.getWeight() == (cart.getCapacity() - 1)) {
whenFull.await();
}
cart.addWeight();
System.out.println("++ weight is: " + cart.getWeight());
whenEmpty.signalAll();
Thread.sleep(500);
} finally {
lock.unlock();
}
}
public void reduceWeight() throws InterruptedException {
lock.lock();
try {
while (cart.getWeight() == 0) {
whenEmpty.await();
}
cart.reduceWeight();
System.out.println("-- weight is: " + cart.getWeight());
whenFull.signalAll();
Thread.sleep(500);
} finally {
lock.unlock();
}
}
public static void main(String[] args) throws InterruptedException {
Cart cart = new Cart(5);
WorkerWithLock adder = new WorkerWithLock(cart, WorkerType.ADDER);
WorkerWithLock reducer = new WorkerWithLock(cart, WorkerType.REDUCER);
new Thread(reducer).start();
new Thread(adder).start();
}
}

It is a kind of a race condition
That is because both threads are waiting.
One in addWeightand one in reduceWeight.
First the reducer stops, if weight is 0. At this time, the adder is may be not already started.
Than the adder stops, if weight = cpacity - 1
Now, both are waiting for an interrupt().
EDIT1. See my comments in the code
public void addWeight() throws InterruptedException {
lock.lock();
try {
while (cart.getWeight() == (cart.getCapacity() - 1)) {
whenFull.await(); //<-- ADDER waits here
}
cart.addWeight();
System.out.println("++ weight is: " + cart.getWeight());
whenEmpty.signalAll(); //<-- Never called since both are waiting
Thread.sleep(500);
} finally {
lock.unlock();
}
}
public void reduceWeight() throws InterruptedException {
lock.lock();
try {
while (cart.getWeight() == 0) {
whenEmpty.await(); //<-- REDUCER waits here
}
cart.reduceWeight();
System.out.println("-- weight is: " + cart.getWeight());
whenFull.signalAll(); //<-- Never called since both are waiting
Thread.sleep(500);
} finally {
lock.unlock();
}
}
EDIT2: Ok, now I understand the behaviour.
Your code is designed to synchronize ONE object for multiple threads, but your are using TWO objects.
Every of your both WorkerWithLock Objects has its own Lock and Condition objects. So calls of lock.lock() and whenFull.signalAll() in object ADDER does not effect Object REDUCER.
Your code will work if you make the lock and condition variables static, so that both objects are working with the same lock and the same condition
final static Lock lock = new ReentrantLock();
final static Condition whenEmpty = lock.newCondition();
final static Condition whenFull = lock.newCondition();
Sometimes multithreading is hard :)

how to terminate retrieval from a blocking queue

I have some code where i execute a several tasks using Executors and a Blocking Queue. The results have to be returned as an iterator because that is what the application that i work on expects. However, there is a 1:N relationship between the task and the results added to the queue, so i cannot use the ExecutorCompletionService. While calling hasNext(), i need to know when all the tasks have finished and added all the results to the queue, so that i can stop the retrieval of results from the queue. Note, that once items are put on the queue, another thread should be ready to consume (Executor.invokeAll(), blocks until all tasks have completed, which is not what i want, nor a timeout). This was my first attempt, i am using an AtomicInteger just to demonstrate the point even though it will not work. Could someone help me in undestanding how i can solve this issue?
public class ResultExecutor<T> implements Iterable<T> {
private BlockingQueue<T> queue;
private Executor executor;
private AtomicInteger count;
public ResultExecutor(Executor executor) {
this.queue = new LinkedBlockingQueue<T>();
this.executor = executor;
count = new AtomicInteger();
}
public void execute(ExecutorTask task) {
executor.execute(task);
}
public Iterator<T> iterator() {
return new MyIterator();
}
public class MyIterator implements Iterator<T> {
private T current;
public boolean hasNext() {
if (count.get() > 0 && current == null)
{
try {
current = queue.take();
count.decrementAndGet();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
return current != null;
}
public T next() {
final T ret = current;
current = null;
return ret;
}
public void remove() {
throw new UnsupportedOperationException();
}
}
public class ExecutorTask implements Runnable{
private String name;
public ExecutorTask(String name) {
this.name = name;
}
private int random(int n)
{
return (int) Math.round(n * Math.random());
}
#SuppressWarnings("unchecked")
public void run() {
try {
int random = random(500);
Thread.sleep(random);
queue.put((T) (name + ":" + random + ":1"));
queue.put((T) (name + ":" + random + ":2"));
queue.put((T) (name + ":" + random + ":3"));
queue.put((T) (name + ":" + random + ":4"));
queue.put((T) (name + ":" + random + ":5"));
count.addAndGet(5);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
And the calling code looks like:
Executor e = Executors.newFixedThreadPool(2);
ResultExecutor<Result> resultExecutor = new ResultExecutor<Result>(e);
resultExecutor.execute(resultExecutor.new ExecutorTask("A"));
resultExecutor.execute(resultExecutor.new ExecutorTask("B"));
Iterator<Result> iter = resultExecutor.iterator();
while (iter.hasNext()) {
System.out.println(iter.next());
}

Use "poison" objects in the Queue to signal that a task will provide no more results.
class Client
{
public static void main(String... argv)
throws Exception
{
BlockingQueue<String> queue = new LinkedBlockingQueue<String>();
ExecutorService workers = Executors.newFixedThreadPool(2);
workers.execute(new ExecutorTask("A", queue));
workers.execute(new ExecutorTask("B", queue));
Iterator<String> results =
new QueueMarkersIterator<String>(queue, ExecutorTask.MARKER, 2);
while (results.hasNext())
System.out.println(results.next());
}
}
class QueueMarkersIterator<T>
implements Iterator<T>
{
private final BlockingQueue<? extends T> queue;
private final T marker;
private int count;
private T next;
QueueMarkersIterator(BlockingQueue<? extends T> queue, T marker, int count)
{
this.queue = queue;
this.marker = marker;
this.count = count;
this.next = marker;
}
public boolean hasNext()
{
if (next == marker)
next = nextImpl();
return (next != marker);
}
public T next()
{
if (next == marker)
next = nextImpl();
if (next == marker)
throw new NoSuchElementException();
T tmp = next;
next = marker;
return tmp;
}
/*
* Block until the status is known. Interrupting the current thread
* will cause iteration to cease prematurely, even if elements are
* subsequently queued.
*/
private T nextImpl()
{
while (count > 0) {
T o;
try {
o = queue.take();
}
catch (InterruptedException ex) {
count = 0;
Thread.currentThread().interrupt();
break;
}
if (o == marker) {
--count;
}
else {
return o;
}
}
return marker;
}
public void remove()
{
throw new UnsupportedOperationException();
}
}
class ExecutorTask
implements Runnable
{
static final String MARKER = new String();
private static final Random random = new Random();
private final String name;
private final BlockingQueue<String> results;
public ExecutorTask(String name, BlockingQueue<String> results)
{
this.name = name;
this.results = results;
}
public void run()
{
int random = ExecutorTask.random.nextInt(500);
try {
Thread.sleep(random);
}
catch (InterruptedException ignore) {
}
final int COUNT = 5;
for (int idx = 0; idx < COUNT; ++idx)
results.add(name + ':' + random + ':' + (idx + 1));
results.add(MARKER);
}
}

I believe a Future is what you're looking for. It allows you to associate asynchronous tasks with a result object, and query the status of that result. For each task you begin, keep a reference to its Future and use that to determine whether or not it has completed.

If I understand your problem correctly (which I'm not sure I do), you can prevent an infinite wait on an empty queue by using [BlockingQueue.poll][1] instead of take(). This lets you specify a timeout, after which time null will be returned if the queue is empty.
If you drop this straight into your hasNext implementation (with an appropriately short timeout), the logic will be correct. An empty queue will return false while a queue with
entities remaining will return true.
[1]: http://java.sun.com/javase/6/docs/api/java/util/concurrent/BlockingQueue.html#poll(long, java.util.concurrent.TimeUnit)

Here is an alternate solution that uses a non-blocking queue with wait/notify, AtomicInteger and a callback.
public class QueueExecutor implements CallbackInterface<String> {
public static final int NO_THREADS = 26;
private Object syncObject = new Object();
private AtomicInteger count;
Queue<String> queue = new LinkedList<String>();
public void execute() {
count = new AtomicInteger(NO_THREADS);
ExecutorService executor = Executors.newFixedThreadPool(NO_THREADS/2);
for(int i=0;i<NO_THREADS;i++)
executor.execute(new ExecutorTask<String>("" + (char) ('A'+i), queue, this));
Iterator<String> iter = new QueueIterator<String>(queue, count);
int count = 0;
while (iter.hasNext()) {
System.out.println(iter.next());
count++;
}
System.out.println("Handled " + count + " items");
}
public void callback(String result) {
System.out.println(result);
count.decrementAndGet();
synchronized (syncObject) {
syncObject.notify();
}
}
public class QueueIterator<T> implements Iterator<T> {
private Queue<T> queue;
private AtomicInteger count;
public QueueIterator(Queue<T> queue, AtomicInteger count) {
this.queue = queue;
this.count = count;
}
public boolean hasNext() {
while(true) {
synchronized (syncObject) {
if(queue.size() > 0)
return true;
if(count.get() == 0)
return false;
try {
syncObject.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
public T next() {
synchronized (syncObject) {
if(hasNext())
return queue.remove();
else
return null;
}
}
public void remove() {
throw new UnsupportedOperationException();
}
}
class ExecutorTask<T> implements Runnable {
private String name;
private Queue<T> queue;
private CallbackInterface<T> callback;
public ExecutorTask(String name, Queue<T> queue,
CallbackInterface<T> callback) {
this.name = name;
this.queue = queue;
this.callback = callback;
}
#SuppressWarnings("unchecked")
public void run() {
try {
Thread.sleep(1000);
Random randomX = new Random();
for (int i = 0; i < 5; i++) {
synchronized (syncObject) {
Thread.sleep(randomX.nextInt(10)+1);
queue.add((T) (name + ":" + ":" + i));
syncObject.notify();
}
}
callback.callback((T) (name + ": Done"));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
public interface CallbackInterface<T> {
void callback(T result);
}
And the calling code is simply:
QueueExecutor exec = new QueueExecutor();
exec.execute();

I am not sure I understand you, but why can't the worker threads put themselves Lists onto the Queue. You can then make a custom iterator that goes over the queue in an outer loop and through the subiterators. All without concurrency magic.

Concurrent Set Queue

Maybe this is a silly question, but I cannot seem to find an obvious answer.
I need a concurrent FIFO queue that contains only unique values. Attempting to add a value that already exists in the queue simply ignores that value. Which, if not for the thread safety would be trivial. Is there a data structure in Java or maybe a code snipit on the interwebs that exhibits this behavior?

If you want better concurrency than full synchronization, there is one way I know of to do it, using a ConcurrentHashMap as the backing map. The following is a sketch only.
public final class ConcurrentHashSet<E> extends ForwardingSet<E>
implements Set<E>, Queue<E> {
private enum Dummy { VALUE }
private final ConcurrentMap<E, Dummy> map;
ConcurrentHashSet(ConcurrentMap<E, Dummy> map) {
super(map.keySet());
this.map = Preconditions.checkNotNull(map);
}
#Override public boolean add(E element) {
return map.put(element, Dummy.VALUE) == null;
}
#Override public boolean addAll(Collection<? extends E> newElements) {
// just the standard implementation
boolean modified = false;
for (E element : newElements) {
modified |= add(element);
}
return modified;
}
#Override public boolean offer(E element) {
return add(element);
}
#Override public E remove() {
E polled = poll();
if (polled == null) {
throw new NoSuchElementException();
}
return polled;
}
#Override public E poll() {
for (E element : this) {
// Not convinced that removing via iterator is viable (check this?)
if (map.remove(element) != null) {
return element;
}
}
return null;
}
#Override public E element() {
return iterator().next();
}
#Override public E peek() {
Iterator<E> iterator = iterator();
return iterator.hasNext() ? iterator.next() : null;
}
}
All is not sunshine with this approach. We have no decent way to select a head element other than using the backing map's entrySet().iterator().next(), the result being that the map gets more and more unbalanced as time goes on. This unbalancing is a problem both due to greater bucket collisions and greater segment contention.
Note: this code uses Guava in a few places.

There's not a built-in collection that does this. There are some concurrent Set implementations that could be used together with a concurrent Queue.
For example, an item is added to the queue only after it was successfully added to the set, and each item removed from the queue is removed from the set. In this case, the contents of the queue, logically, are really whatever is in the set, and the queue is just used to track the order and provide efficient take() and poll() operations found only on a BlockingQueue.

I would use a synchronized LinkedHashSet until there was enough justification to consider alternatives. The primary benefit that a more concurrent solution could offer is lock splitting.
The simplest concurrent approach would be a a ConcurrentHashMap (acting as a set) and a ConcurrentLinkedQueue. The ordering of operations would provide the desired constraint. An offer() would first perform a CHM#putIfAbsent() and if successful insert into the CLQ. A poll() would take from the CLQ and then remove it from the CHM. This means that we consider an entry in our queue if it is in the map and the CLQ provides the ordering. The performance could then be adjusted by increasing the map's concurrencyLevel. If you are tolerant to additional racy-ness, then a cheap CHM#get() could act as a reasonable precondition (but it can suffer by being a slightly stale view).

A java.util.concurrent.ConcurrentLinkedQueue gets you most of the way there.
Wrap the ConcurrentLinkedQueue with your own class that checks for the uniqueness of an add. Your code has to be thread safe.

What do you mean by a concurrent queue with Set semantics? If you mean a truly concurrent structure (as opposed to a thread-safe structure) then I would contend that you are asking for a pony.
What happens for instance if you call put(element) and detect that something is already there which immediately is removed? For instance, what does it mean in your case if offer(element) || queue.contains(element) returns false?
These kinds of things often need to thought about slightly differently in a concurrent world as often nothing is as it seems unless you stop the world (lock it down). Otherwise you are usually looking at something in the past. So, what are you actually trying to do?

Perhaps extend ArrayBlockingQueue. In order to get access to the (package-access) lock, I had to put my sub-class within the same package. Caveat: I haven't tested this.
package java.util.concurrent;
import java.util.Collection;
import java.util.concurrent.locks.ReentrantLock;
public class DeDupingBlockingQueue<E> extends ArrayBlockingQueue<E> {
public DeDupingBlockingQueue(int capacity) {
super(capacity);
}
public DeDupingBlockingQueue(int capacity, boolean fair) {
super(capacity, fair);
}
public DeDupingBlockingQueue(int capacity, boolean fair, Collection<? extends E> c) {
super(capacity, fair, c);
}
#Override
public boolean add(E e) {
final ReentrantLock lock = this.lock;
lock.lock();
try {
if (contains(e)) return false;
return super.add(e);
} finally {
lock.unlock();
}
}
#Override
public boolean offer(E e) {
final ReentrantLock lock = this.lock;
lock.lock();
try {
if (contains(e)) return true;
return super.offer(e);
} finally {
lock.unlock();
}
}
#Override
public void put(E e) throws InterruptedException {
final ReentrantLock lock = this.lock;
lock.lockInterruptibly(); //Should this be lock.lock() instead?
try {
if (contains(e)) return;
super.put(e); //if it blocks, it does so without holding the lock.
} finally {
lock.unlock();
}
}
#Override
public boolean offer(E e, long timeout, TimeUnit unit) throws InterruptedException {
final ReentrantLock lock = this.lock;
lock.lock();
try {
if (contains(e)) return true;
return super.offer(e, timeout, unit); //if it blocks, it does so without holding the lock.
} finally {
lock.unlock();
}
}
}

A simple answer for a queue of unique objects can be as follow:
import java.util.concurrent.ConcurrentLinkedQueue;
public class FinalQueue {
class Bin {
private int a;
private int b;
public Bin(int a, int b) {
this.a = a;
this.b = b;
}
#Override
public int hashCode() {
return a * b;
}
public String toString() {
return a + ":" + b;
}
#Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
Bin other = (Bin) obj;
if ((a != other.a) || (b != other.b))
return false;
return true;
}
}
private ConcurrentLinkedQueue<Bin> queue;
public FinalQueue() {
queue = new ConcurrentLinkedQueue<Bin>();
}
public synchronized void enqueue(Bin ipAddress) {
if (!queue.contains(ipAddress))
queue.add(ipAddress);
}
public Bin dequeue() {
return queue.poll();
}
public String toString() {
return "" + queue;
}
/**
* #param args
*/
public static void main(String[] args) {
FinalQueue queue = new FinalQueue();
Bin a = queue.new Bin(2,6);
queue.enqueue(a);
queue.enqueue(queue.new Bin(13, 3));
queue.enqueue(queue.new Bin(13, 3));
queue.enqueue(queue.new Bin(14, 3));
queue.enqueue(queue.new Bin(13, 9));
queue.enqueue(queue.new Bin(18, 3));
queue.enqueue(queue.new Bin(14, 7));
Bin x= queue.dequeue();
System.out.println(x.a);
System.out.println(queue.toString());
System.out.println("Dequeue..." + queue.dequeue());
System.out.println("Dequeue..." + queue.dequeue());
System.out.println(queue.toString());
}
}