I would like to store tuples objects in a concurent java collection and then have an efficient, blocking query method that returns the first element matching a pattern. If no such element is available, it would block until such element is present.
For instance if I have a class:
public class Pair {
public final String first;
public final String Second;
public Pair( String first, String second ) {
this.first = first;
this.second = second;
}
}
And a collection like:
public class FunkyCollection {
public void add( Pair p ) { /* ... */ }
public Pair get( Pair p ) { /* ... */ }
}
I would like to query it like:
myFunkyCollection.get( new Pair( null, "foo" ) );
which returns the first available pair with the second field equalling "foo" or blocks until such element is added. Another query example:
myFunkyCollection.get( new Pair( null, null ) );
should return the first available pair whatever its values.
Does a solution already exists ? If it is not the case, what do you suggest to implement the get( Pair p ) method ?
Clarification: The method get( Pair p) must also remove the element. The name choice was not very smart. A better name would be take( ... ).
Here's some source code. It basically the same as what cb160 said, but having the source code might help to clear up any questions you may still have. In particular the methods on the FunkyCollection must be synchronized.
As meriton pointed out, the get method performs an O(n) scan for every blocked get every time a new object is added. It also performs an O(n) operation to remove objects. This could be improved by using a data structure similar to a linked list where you can keep an iterator to the last item checked. I haven't provided source code for this optimization, but it shouldn't be too difficult to implement if you need the extra performance.
import java.util.*;
public class BlockingQueries
{
public class Pair
{
public final String first;
public final String second;
public Pair(String first, String second)
{
this.first = first;
this.second = second;
}
}
public class FunkyCollection
{
final ArrayList<Pair> pairs = new ArrayList<Pair>();
public synchronized void add( Pair p )
{
pairs.add(p);
notifyAll();
}
public synchronized Pair get( Pair p ) throws InterruptedException
{
while (true)
{
for (Iterator<Pair> i = pairs.iterator(); i.hasNext(); )
{
Pair pair = i.next();
boolean firstOk = p.first == null || p.first.equals(pair.first);
boolean secondOk = p.second == null || p.second.equals(pair.second);
if (firstOk && secondOk)
{
i.remove();
return pair;
}
}
wait();
}
}
}
class Producer implements Runnable
{
private FunkyCollection funkyCollection;
public Producer(FunkyCollection funkyCollection)
{
this.funkyCollection = funkyCollection;
}
public void run()
{
try
{
for (int i = 0; i < 10; ++i)
{
System.out.println("Adding item " + i);
funkyCollection.add(new Pair("foo" + i, "bar" + i));
Thread.sleep(1000);
}
}
catch (InterruptedException e)
{
Thread.currentThread().interrupt();
}
}
}
public void go() throws InterruptedException
{
FunkyCollection funkyCollection = new FunkyCollection();
new Thread(new Producer(funkyCollection)).start();
System.out.println("Fetching bar5.");
funkyCollection.get(new Pair(null, "bar5"));
System.out.println("Fetching foo2.");
funkyCollection.get(new Pair("foo2", null));
System.out.println("Fetching foo8, bar8");
funkyCollection.get(new Pair("foo8", "bar8"));
System.out.println("Finished.");
}
public static void main(String[] args) throws InterruptedException
{
new BlockingQueries().go();
}
}
Output:
Fetching bar5.
Adding item 0
Adding item 1
Adding item 2
Adding item 3
Adding item 4
Adding item 5
Fetching foo2.
Fetching foo8, bar8
Adding item 6
Adding item 7
Adding item 8
Finished.
Adding item 9
Note that I put everything into one source file to make it easier to run.
I know of no existing container that will provide this behavior. One problem you face is the case where no existing entry matches the query. In that case, you'll have to wait for new entries to arrive, and those new entries are supposed to arrive at the tail of the sequence. Given that you're blocking, you don't want to have to examine all the entries that precede the latest addition, because you've already inspected them and determined that they don't match. Hence, you need some way to record your current position, and be able to search forward from there whenever a new entry arrives.
This waiting is a job for a Condition. As suggested in cb160's answer, you should allocate a Condition instance inside your collection, and block on it via Condition#await(). You should also expose a companion overload to your get() method to allow timed waiting:
public Pair get(Pair p) throws InterruptedException;
public Pair get(Pair p, long time, TimeUnit unit) throws InterruptedException;
Upon each call to add(), call on Condition#signalAll() to unblock the threads waiting on unsatisfied get() queries, allowing them to scan the recent additions.
You haven't mentioned how or if items are ever removed from this container. If the container only grows, that simplifies how threads can scan its contents without worrying about contention from other threads mutating the container. Each thread can begin its query with confidence as to the minimum number of entries available to inspect. However, if you allow removal of items, there are many more challenges to confront.
In your FunkyCollection add method you could call notifyAll on the collection itself every time you add an element.
In the get method, if the underlying container (Any suitable conatiner is fine) doesn't contain the value you need, wait on the FunkyCollection. When the wait is notified, check to see if the underlying container contains the result you need. If it does, return the value, otherwise, wait again.
It appears you are looking for an implementation of Tuple Spaces. The Wikipedia article about them lists a few implementations for Java, perhaps you can use one of those. Failing that, you might find an open source implementation to imitate, or relevant research papers.
Related
I implemented a queue using array. Now I want to remove a element by searching, if the element were there it must be removed from the queue.
public static void deleteFromQueue(PassengerQueue passengerQueue){
//passengerQueue.dequeue();
Scanner scan = new Scanner(System.in);
System.out.print("please Enter the Passenger name: ");
String name = scan.nextLine();
for (int i=0; i<passengerQueue.getPassenger().length; i++){
if (passengerQueue.getPassenger()[i].getName().equals(name)){
//
}
}
}
here my method of removing
Why do you specifically want to use an array. Please find an example attached using an ArrayList:
package stackoverflow;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import java.util.ArrayList;
import java.util.Iterator;
public class QuickTest {
#Test
public void test() throws Exception {
PassengerQueue passengerQueue = new PassengerQueue();
passengerQueue.add(new Passenger("testName1"));
passengerQueue.add(new Passenger("testName2"));
Assertions.assertEquals(2, passengerQueue.size());
PassengerUtil.removeByName(passengerQueue, "testName1");
Assertions.assertEquals(passengerQueue.size(), 1);
System.out.println("All done");
}
private static class PassengerUtil {
/** #param passengerQueue Modified by reference. **/
private static void removeByName(PassengerQueue passengerQueue, String specifiedName) {
// Using an Iterator so that I don't trigger ConcurrentModificationException.
for (Iterator<Passenger> it = passengerQueue.iterator() ; it.hasNext() ; ) {
Passenger currPassenger = it.next();
if (currPassenger.getName().equals(specifiedName)) {
it.remove();
}
}
}
}
private class PassengerQueue extends ArrayList<Passenger> {
}
private class Passenger {
private String name;
public Passenger(String name) {
if (name == null) {
throw new NullPointerException("The " + Passenger.class.getSimpleName() + " cannot have a Name equal to NULL!");
}
this.name = name;
}
public String getName() {
return this.name;
}
}
}
Please note the following:
My PassengerQueue object extends ArrayList. So I have a type-safe list of Passengers just by extending ArrayList - I don't need to do anything else.
I use an Iterator to iterate over the list. Its a bit more verbose than your normal for-each loop, but its necessary to not trigger a ConcurrentModificationException. Java doesn't always like when you iterate over a list and then for example delete things from that list while you're iterating over it. (Maybe simple examples won't trigger the ConcurrentModificationException)
You called your list PassengerQueue. Please note that Java does have Queue(https://docs.oracle.com/javase/8/docs/api/java/util/Queue.html) collections. Similar to me extending ArrayList you can look at Queue subclasses and extend that instead if you really need your Collection to function like a queue.
Your code, and mine, can currently delete multiple elements from the list if the list contains Passengers with the same name.
Your question title asked about deleting from an array using an index position. You can consider adding the Apache Commons Lang project to your classpath and using methods from their ArrayUtils
Actually, my answer can be improved to not even use an Iterator:
private static class PassengerUtil {
/** #param passengerQueue Modified by reference. **/
private static void removeByName(PassengerQueue passengerQueue, String specifiedName) {
passengerQueue.removeIf(currPassenger -> currPassenger.getName().equals(specifiedName));
}
}
Some reading on the latter code example here.
A 'Queue' is defined as a data structure that holds a sequence of items where you can only add something to the end (the 'tail') and where you can take something from the beginning, the 'head'. And sometimes it is said that you can get the current size of the sequence, ask whether the sequence is empty, and that you can look ('peek') at the first item without taking it.
That's the basics. And you can implement that in various ways.
There is an interface in Java (java.util.Queue) that provides the basic features described above. So when you declare
java.util.Queue myQueue = …
then you cannot search your queue for an item and remove it (ok, you can take all elements from your queue, one by one, and add again those you want to keep, but that's tedious).
But the implementation for java.util.Queue is java.util.LinkedList, and a list can be searched.
So you write
java.util.Queue myQueue = new java.util.LinkedList();
and as you now know that the implementation of your queue is in fact a list, you can write
…
for( var i = ((java.util.List) myQueue).iterator(); i.hasNext(); )
{
if( matchesCriteriaForRemoval( i.next() ) i.remove();
}
…
But this works only because you know some implementation details of myQueue – but that was what you want to hide when you chose to define it as java.util.Queue.
So when you have to be able to remove entries from your PassengerQueue, that data structure should provide a method to do so instead of revealing its internal implementation.
This means your code have to look like this:
public static void deleteFromQueue( PassengerQueue passengerQueue )
{
Scanner scan = new Scanner( System.in );
System.out.print( "please Enter the Passenger name: " );
String name = scan.nextLine();
passengerQueue.removeByName( name );
}
How this method PassengerQueue.removeByName() is implemented depends from the internal implementation of PassengerQueue; if it uses the java.util.List with the name passengers to store the passengers, it may look like this:
public final void removeByName( final String name )
{
for( var i = passengers.iterator(); i.hasNext(); )
{
if( passengerNameMatches( name, i.next() ) ) i.remove();
}
}
If you use another container for your passengers, that removal method has to be implemented differently …
Obviously I omitted all error handling, and the collections are generic types, but I used them as raw because of brevity.
I want to implement a util getting an Enum object by its string value. Here is my implementation.
IStringEnum.java
public interface IStringEnum {
String getValue();
}
StringEnumUtil.java
public class StringEnumUtil {
private volatile static Map<String, Map<String, Enum>> stringEnumMap = new HashMap<>();
private StringEnumUtil() {}
public static <T extends Enum<T>> Enum fromString(Class<T> enumClass, String symbol) {
final String enumClassName = enumClass.getName();
if (!stringEnumMap.containsKey(enumClassName)) {
synchronized (enumClass) {
if (!stringEnumMap.containsKey(enumClassName)) {
System.out.println("aaa:" + stringEnumMap.get(enumClassName));
Map<String, Enum> innerMap = new HashMap<>();
EnumSet<T> set = EnumSet.allOf(enumClass);
for (Enum e: set) {
if (e instanceof IStringEnum) {
innerMap.put(((IStringEnum) e).getValue(), e);
}
}
stringEnumMap.put(enumClassName, innerMap);
}
}
}
return stringEnumMap.get(enumClassName).get(symbol);
}
}
I wrote a unit test in order to test whether it works in multi-thread case.
StringEnumUtilTest.java
public class StringEnumUtilTest {
enum TestEnum implements IStringEnum {
ONE("one");
TestEnum(String value) {
this.value = value;
}
#Override
public String getValue() {
return this.value;
}
private String value;
}
#Test
public void testFromStringMultiThreadShouldOk() {
final int numThread = 100;
CountDownLatch startLatch = new CountDownLatch(1);
CountDownLatch doneLatch = new CountDownLatch(numThread);
List<Boolean> resultList = new LinkedList<>();
for (int i = 0; i < numThread; ++i) {
new Thread(() -> {
try {
startLatch.await();
} catch (Exception e) {
e.printStackTrace();
}
resultList.add(StringEnumUtil.fromString(TestEnum.class, "one") != null);
doneLatch.countDown();
}).start();
}
startLatch.countDown();
try {
doneLatch.await();
} catch (Exception e) {
e.printStackTrace();
}
assertEquals(numThread, resultList.stream().filter(item -> item.booleanValue()).count());
}
}
The testing result is:
aaa:null
java.lang.AssertionError:
Expected :100
Actual :98
It denotes that only one thread execute this line of code:
System.out.println("aaa:" + stringEnumMap.get(enumClassName));
So the initialization codes should be executed by only one thread.
The strange thing is, the result of some thread will be null after executing this line of code:
return stringEnumMap.get(enumClassName).get(symbol);
Since there is no NullPointerException, stringEnumMap.get(enumClassName) must return the reference of innerMap. But why it will get null after calling get(symbol) of innerMap?
Please help, it drive me crazy the whole day!
The problem is due to the line
List<Boolean> resultList = new LinkedList<>();
From JavaDoc of LinkedList:
Note that this implementation is not synchronized.If multiple threads access a linked list concurrently, and at least one of the threads modifies the list structurally, it must be synchronized externally. (A structural modification is any operation that adds or deletes one or more elements; merely setting the value of an element is not a structural modification.) This is typically accomplished by synchronizing on some object that naturally encapsulates the list.If no such object exists, the list should be "wrapped" using the Collections.synchronizedListmethod. This is best done at creation time, to prevent accidental unsynchronized access to the list:
List list = Collections.synchronizedList(new LinkedList(...));
As LinkedList is not thread safe, and unexpected behavior may happens during the add operation.
Which cause the resultList size less than the thread count, and hence the expected count is less than the result count.
To get correct result, add Collections.synchronizedList as suggested.
Although you implementation is fine, I suggest you to follow Matt Timmermans answer for simpler and robust solution.
stringEnumMap should be a ConcurrentHashMap<String, Map<String,Enum>>, and use computeIfAbsent to do the lazy initialization.
ConcurrentMap interface
As others noted, if manipulating a Map across threads you must account for concurrency.
You could handle concurrent access yourself. But there is no need. Java comes with two implementations of Map that are built to internally handle concurrency. These implementations implement the ConcurrentMap interface.
ConcurrentSkipListMap
ConcurrentHashMap
The first maintains the keys in sorted order, implementing the NavigableMap interface.
Here is a table I authored to show the characteristics of all the implementations of Map bundled with Java 11.
You might find other third-party implementations of the ConcurrentMap interface.
try moving
if (!stringEnumMap.containsKey(enumClassName))
and the
return stringEnumMap.get(enumClassName).get(symbol);
into the synchronized block.
So I'm making a search algorithm. I'm using a queue to store all of my objects
This is how I initialised it
Queue<Node> queue = new LinkedList<Node>();
I want to compare a variable in each object and order to queue. My plan is to use a for loop to compare the first object with each of the other objects and whichever object has the lowest variable is sent to the front of the queue. Then move onto the next object and repeat the process. My issue is I'm not sure how to retrieve an object from the queue that isn't the first object in the queue....
You could do a for loop through the Queue:
for (Node n : queue) {
do stuff with n
}
However, you aren't going to be able to remove items from the middle of the queue. Might I suggest a structure like an ArrayList?
In my opinion the best way is to use PriorityQueue. You can specify implementation of Comparator interface that will impose how elements should be sorted inside of queue.
Here is an example:
Let's say that this is your Node class:
public class Node {
// this field will be used to sort in queue
private int value;
public Node(int value) {
this.value = value;
}
public int getValue() {
return value;
}
#Override
public String toString() {
return "My value is: " + value;
}
}
And here is example of adding Nodes into queue:
import java.util.PriorityQueue;
import java.util.Random;
public class QueueExample {
public static void main(String[] args) {
Random r = new Random();
// Priority queue with custom comparator
PriorityQueue<Node> queue = new PriorityQueue<Node>(10, new SampleNodeComparator());
// adding 100 nodes with random value
for(int i = 0; i < 100; ++i) {
queue.add( new Node(r.nextInt(1000)));
}
// nodes will be removed from queue in order given by comparator
while(queue.size() != 0) {
System.out.println(queue.remove());
}
}
}
And the most important part - implementation of our custom comparator
import java.util.Comparator;
// our comparator needs to implements Comparator interface
public class SampleNodeComparator implements Comparator<Node> {
#Override
public int compare(Node o1, Node o2) {
/*
value that should be return from compare method should follow rules:
if o1 == o2 - return 0
if o1 > o2 - return any positive value
if o1 < 02 - return any negative value
*/
return o1.getValue() - o2.getValue();
}
}
When you run main method from QueueExample class you will see on console that values are removed from queue sorted by Node.value value.
Use Queue<E>#peek () to retrieve an object without removing it.
Some example code:
import java.util.*;
class Example {
public static void main (String[] args) throws Exception {
Queue<String> list = new PriorityQueue<>();
{ // Initialize the Queue
list.add ("Hello ");
list.add ("Mrs. ");
list.add ("DoubtFire! ");
}
System.out.println (list);
// Iterating through the Queue
String element;
while ( (element = list.peek()) != null) {
if (element.equals ("Mrs. ")) {
System.out.println ("\"Mrs\" found!");
}
System.out.println (element);
list.remove (element);
}
System.out.println (list); // Empty by now...
}
}
Output:
[DoubtFire! , Mrs. , Hello ]
DoubtFire!
Hello
"Mrs" found!
Mrs.
[]
Queue interface does not guarantee any particular order while iterating or polling so theoretically this task is impossible to implement with Queue.
Seeing your response to my comment, I think that in your case, you should use the PriorityQueue because it does what you need without needing you to reinvent the wheel, which is usually not recommended.
By default, the priority queue will use the default implementation of the compareTo method. Assuming that you have a composite type, you have two options:
You can make your custom class implement the Comparabale interface and have your sorting logic there.
Alternatively, you could pass your own comparator:
PriorityQueue<..> p = new PriorityQueue<..>(5, new Comparator<..>()
{
#override
public int compare(.. type1, .. type2)
{
//comparison logic done here.
}
}
You can take a look at this short tutorial for more information.
So I tried to implement a stack with just one queue and it appears to work, but I'm not sure if there's something wrong with it since most of the solutions I've seen online use two queues. Can anyone tell if me if there are problems with my implementation?
public class MyStack<T> {
/**
* #param args
*/
private Queue<T> q = new LinkedList<T>();
public MyStack(){
}
public static void main(String[] args) {
// TODO Auto-generated method stub
MyStack<String> s = new MyStack<String>();
s.push("1");
s.push("2");
s.push("3");
s.push("4");
System.out.println(s.pop());
System.out.println(s.pop());
System.out.println(s.pop());
System.out.println(s.pop());
System.out.println(s.pop());
}
public void push(T s){
q.offer(s);
}
public T pop(){
int n = q.size();
for(int i = 0; i < n-1; i++){
q.offer(q.poll());
}
return q.poll();
}
}
Output:
4
3
2
1
null
Your solution is inefficient because you have to loop through the whole stack every time you pop something from it. (Effectively you have to traverse the whole linked list, before removing the element that was at the end.)
Edit: Java's linked list is doubly linked anyway, so this is entirely pointless.
You should use either a Stack or a Deque or even a LinkedList.
Implementing your own is just ... pointless. Unless of course (as #bas suggests) you are doing a course on data structures in which case you should go Commando and implement your own structure from scratch. Using another structure because it is nearly like the one you are trying to make is like using a hammer with screws.
If you really need to implement something yourself something like this should work:
public class Stack<T> {
private Entry top = null;
private class Entry {
final Entry up;
final T it;
public Entry(Entry up, T it) {
this.up = up;
this.it = it;
}
}
public void push ( T it ) {
top = new Entry(top, it);
}
public T pop () {
if ( top == null ) {
throw new EmptyStackException();
}
T it = top.it;
top = top.up;
return it;
}
}
NB: This may not be thread safe.
There is absolutely no reason a stack should use two queues. As a matter of fact, it only needs to keep track of one top-node that references the nodes below it.
The code seems to work, but as nachokk said, this is not the site for code review. This site is ment if you run into errors and require assistance.
You must use two queues ONLY when you have basic queues operations, like enqueue and dequeue. When you can use other methods, especially iterating over queue, you can do it with only one queue, like you did.
I have set of connection objects (library code I cannot change) that have a send method. If the sending fails, they call back a generic onClosed listener which I implement that calls removeConnection() in my code, which will remove the connection from the collection.
The onClosed callback is generic and can be called at any time. It is called when the peer closes the connection, for example, and not just when a write fails.
However, if I have some code that loops over my connections and sends, then the onClosed callback will attempt to modify a collection during iteration.
My current code creates a copy of the connections list before each iteration over it; however, in profiling this has shown to be very expensive.
Set<Connection> connections = new ....;
public void addConnection(Connection conn) {
connections.add(conn);
conn.addClosedListener(this);
}
#Override void onClosed(Connection conn) {
connections.remove(conn);
}
void send(Message msg) {
// how to make this so that the onClosed callback can be safely invoked, and efficient?
for(Connection conn: connections)
conn.send(msg);
}
How can I efficiently cope with modifying collections during iteration?
To iterate a collection with the concurrent modification without any exceptions use List Iterator.
http://www.mkyong.com/java/how-do-loop-iterate-a-list-in-java/ - example
If you use simple for or foreach loops, you will receive ConcurrentModificationException during the element removing - be careful on that.
As an addition, you could override the List Iterator with your own one and add the needed logic. Just implement the java.util.Iterator interface.
A ConcurrentSkipListSet is probably what you want.
You could also use a CopyOnWriteArraySet. This of course will still make a copy, however, it will only do so when the set is modified. So as long as Connection objects are not added or removed regularly, this would be more efficient.
You can also use ConcurrentHashMap.
ConcurrentHashMap is thread-safe, so you don't need to make a copy in order to be able to iterate.
Take a look at this implementation.. http://www.java2s.com/Tutorial/Java/0140__Collections/Concurrentset.htm
I would write a collection wrapper that:
Keeps a set of objects that are to be removed. If the iteration across the underlying collection comes across one of these it is skipped.
On completion of iteration, takes a second pass across the list to remove all of the gathered objects.
Perhaps something like this:
class ModifiableIterator<T> implements Iterator<T> {
// My iterable.
final Iterable<T> it;
// The Iterator we are walking.
final Iterator<T> i;
// The removed objects.
Set<T> removed = new HashSet<T>();
// The next actual one to return.
T next = null;
public ModifiableIterator(Iterable<T> it) {
this.it = it;
i = it.iterator();
}
#Override
public boolean hasNext() {
while ( next == null && i.hasNext() ) {
// Pull a new one.
next = i.next();
if ( removed.contains(next)) {
// Not that one.
next = null;
}
}
if ( next == null ) {
// Finished! Close.
close();
}
return next != null;
}
#Override
public T next() {
T n = next;
next = null;
return n;
}
// Close down - remove all removed.
public void close () {
if ( !removed.isEmpty() ) {
Iterator<T> i = it.iterator();
while ( i.hasNext() ) {
if ( removed.contains(i.next())) {
i.remove();
}
}
// Clear down.
removed.clear();
}
}
#Override
public void remove() {
throw new UnsupportedOperationException("Not supported.");
}
public void remove(T t) {
removed.add(t);
}
}
public void test() {
List<String> test = new ArrayList(Arrays.asList("A","B","C","D","E"));
ModifiableIterator i = new ModifiableIterator(test);
i.remove("A");
i.remove("E");
System.out.println(test);
while ( i.hasNext() ) {
System.out.println(i.next());
}
System.out.println(test);
}
You may need to consider whether your list could contain null values, in which case you will need to tweak it somewhat.
Please remember to close the iterator if you abandon the iteration before it completes.