I need to use a FIFO structure in my application. It needs to have at most 5 elements.
I'd like to have something easy to use (I don't care for concurrency) that implements the Collection interface.
I've tried the LinkedList, that seems to come from Queue, but it doesn't seem to allow me to set it's maximum capacity. It feels as if I just want at max 5 elements but try to add 20, it will just keep increasing in size to fit it. I'd like something that'd work the following way:
XQueue<Integer> queue = new XQueue<Integer>(5); //where 5 is the maximum number of elements I want in my queue.
for (int i = 0; i < 10; ++i) {
queue.offer(i);
}
for (int i = 0; i < 5; ++i) {
System.out.println(queue.poll());
}
That'd print:
5
6
7
8
9
Thanks
Create your own subclass of the one you want, and override the add method so that it
checks if the new object will fit, and fails if not
calls super.add()
(and the constructors).
If you want it to block when inserting if full, it is a different matter.
I haven't seen any limitation like that in the API. You can use ArrayList by changing the behavior of the add method with anonymous class feature:
new ArrayList<Object>(){
public boolean add(Object o){ /*...*/ }
}
Looks like what you want is a limited size FIFO structure, that evicts oldest items when new ones are added. I recommend a solution based on a cyclic array implementation, where you should track the index of the queue tail and queue head, and increase them (in cyclic manner) as needed.
EDIT:
Here is my implementation (note that it IS a Collection). It works fine with your test scenario.
public class XQueue <T> extends AbstractQueue<T>{
private T[] arr;
private int headPos;
private int tailPos;
private int size;
#SuppressWarnings("unchecked")
public XQueue(int n){
arr = (T[]) new Object[n];
}
private int nextPos(int pos){
return (pos + 1) % arr.length;
}
#Override
public T peek() {
if (size == 0)
return null;
return arr[headPos];
}
public T poll(){
if (size == 0)
return null;
size--;
T res = arr[headPos];
headPos = nextPos(headPos);
return res;
}
#Override
public boolean offer(T e) {
if (size < arr.length)
size++;
else
if (headPos == tailPos)
headPos = nextPos(headPos);
arr[tailPos] = e;
tailPos = nextPos(tailPos);
return true;
}
#Override
public Iterator<T> iterator() {
return null; //TODO: Implement
}
#Override
public int size() {
return size;
}
}
Perhaps an ArrayBlockingQueue might do the trick. Look here. Try something like this:
BlockingQueue<Integer> queue = new ArrayBlockingQueue<Integer>(5);
for (int i = 0; i < 10; i++) {
while (!queue.offer(i)) {
queue.poll();
}
}
for (int i = 0; i < 5; i++) {
System.out.println(queue.poll());
}
You have three choices
1) Subclass an Abstract Collection
2) Limit the size to five and do the logic around the code where you are doing the insert.
3) Use LinkedListHashMap The removeEldestEntry(Map.Entry) method may be overridden to impose a policy for removing stale mappings automatically when new mappings are added to the map. (You would then use an Iterator to get the values - which will be returned in order of insertion)
Your best bet is #1 - It is real easy if you look at the link.
Did you have a look at the Apache Commons Collections library? The BoundedFifoBuffer should exactly meet your needs.
If I remember correctly, I've done exactly what you want using a LinkedList.
What you need to do is check the size of the List, if it's 5 and you want to add objects, just delete the first element and keep doing so if the size is 5.
Related
I want a List of n Sets of Integers and initially this list should be filled with null.
A lot of the Sets will be initialised later, and some will remain null.
I have tried different methods to implement this, some of them are included here:
List<HashSet<Integer>> List_of_Sets = Arrays.asList(new HashSet[n]);
ArrayList<HashSet<Integer>> List_of_Sets = new ArrayList<>(n);
while(n-- > 0) List_of_Sets.add(null);
Is there a faster way to do this?
For clarification an example for arrays would be Arrays.fill() used to be slower than:
/*
* initialize a smaller piece of the array and use the System.arraycopy
* call to fill in the rest of the array in an expanding binary fashion
*/
public static void bytefill(byte[] array, byte value) {
int len = array.length;
if (len > 0){
array[0] = value;
}
//Value of i will be [1, 2, 4, 8, 16, 32, ..., len]
for (int i = 1; i < len; i += i) {
System.arraycopy(array, 0, array, i, ((len - i) < i) ? (len - i) : i);
}
}
^above code is from Ross Drew's answer to Fastest way to set all values of an array?
Is there a faster way to do this?
As far as I am aware, no. Certainly, there is no easy way that is faster.
Based on how it works, I think (but I have not tested) that the Arrays.asList(new HashSet[n]) should be the fastest solution.
It would be possible to implement a custom List implementation that is like an ArrayList but is pre-initialized to N null values. But under the hood the initialization will be pretty much identical with what happens in the List implementation that asList returns. So I doubt that any performance improvements would be significant ... or worth the effort.
If you want to be sure of this, you could write a benchmark of the various options. However, I don't think this is the right approach in this case.
Instead I would recommend benchmarking and profiling your entire application to determine if operations on this list are a real performance hotspot.
If it is not a hotspot, my recommendation would be to just use the Arrays.asList approach and spend your time on something more important.
If it is a hotspot, you should consider replacing the List with an array. From your earlier description it seemed you are going to use the List like an array; i.e. using positional get and set operations, and no operations that change the list size. If that is the case, then using a real array should be more efficient. It saves memory, and avoids a level of indirection and (possibly) some bounds checking.
One reason not to do this would be if you needed to pass the array to some other code that requires a List.
If resizing is not important to you then implementing your own list might be fast. It might also be buggy. It would at least be interesting to benchmark compared to Java's lists. One strange effect that you might see is that standard lists might be optimised by the JIT sooner, as they could be used internally by Java's standard library.
Here is my attempt, although I suggest you don't use it. Use a standard list implementation instead.
import java.util.*;
public class FastListOfNullsDemo {
public static void main(String[] args) {
Set<Integer>[] arr = new Set[100_000]; // all set to null by default.
List<Set<Integer>> myList = new ArrayBackedList<>(arr);
myList.set(3, new TreeSet<Integer>());
myList.get(3).add(5);
myList.get(3).add(4);
myList.get(3).add(3);
myList.get(3).add(2);
myList.get(3).add(1);
// Let's just print some because 100,000 is a lot!
for (int i = 0; i < 10; i++) {
System.out.println(myList.get(i));
}
}
}
class ArrayBackedList<T> extends AbstractList<T> {
private final T[] arr;
ArrayBackedList(T[] arr) {
this.arr = arr;
}
#Override
public T get(int index) {
return arr[index];
}
#Override
public int size() {
return arr.length;
}
#Override
public T set(int index, T value) {
T result = arr[index];
arr[index] = value;
return result;
}
}
Another possibility would be implementing an always-null, fixed-size list. Use that to initialise the ArrayList. I won't promise that it is fast but you could try it out.
import java.util.*;
public class FastListOfNullsDemo {
public static void main(String[] args) {
List<Set<Integer>> allNull = new NullList<>(100_000);
List<Set<Integer>> myList = new ArrayList<>(allNull);
myList.set(3, new TreeSet<Integer>());
myList.get(3).add(5);
myList.get(3).add(4);
myList.get(3).add(3);
myList.get(3).add(2);
myList.get(3).add(1);
System.out.println(myList.size());
// Let's just print some because 100,000 is a lot!
for (int i = 0; i < 10; i++) {
System.out.println(myList.get(i));
}
}
}
class NullList<T> extends AbstractList<T> {
private int count;
NullList(int count) {
this.count = count;
}
#Override
public T get(int index) {
return null;
}
#Override
public int size() {
return count;
}
}
A very simple & quick question on Java libraries: is there a ready-made class that implements a Queue with a fixed maximum size - i.e. it always allows addition of elements, but it will silently remove head elements to accomodate space for newly added elements.
Of course, it's trivial to implement it manually:
import java.util.LinkedList;
public class LimitedQueue<E> extends LinkedList<E> {
private int limit;
public LimitedQueue(int limit) {
this.limit = limit;
}
#Override
public boolean add(E o) {
super.add(o);
while (size() > limit) { super.remove(); }
return true;
}
}
As far as I see, there's no standard implementation in Java stdlibs, but may be there's one in Apache Commons or something like that?
Apache commons collections 4 has a CircularFifoQueue<> which is what you are looking for. Quoting the javadoc:
CircularFifoQueue is a first-in first-out queue with a fixed size that replaces its oldest element if full.
import java.util.Queue;
import org.apache.commons.collections4.queue.CircularFifoQueue;
Queue<Integer> fifo = new CircularFifoQueue<Integer>(2);
fifo.add(1);
fifo.add(2);
fifo.add(3);
System.out.println(fifo);
// Observe the result:
// [2, 3]
If you are using an older version of the Apache commons collections (3.x), you can use the CircularFifoBuffer which is basically the same thing without generics.
Update: updated answer following release of commons collections version 4 that supports generics.
Guava now has an EvictingQueue, a non-blocking queue which automatically evicts elements from the head of the queue when attempting to add new elements onto the queue and it is full.
import java.util.Queue;
import com.google.common.collect.EvictingQueue;
Queue<Integer> fifo = EvictingQueue.create(2);
fifo.add(1);
fifo.add(2);
fifo.add(3);
System.out.println(fifo);
// Observe the result:
// [2, 3]
I like #FractalizeR solution. But I would in addition keep and return the value from super.add(o)!
public class LimitedQueue<E> extends LinkedList<E> {
private int limit;
public LimitedQueue(int limit) {
this.limit = limit;
}
#Override
public boolean add(E o) {
boolean added = super.add(o);
while (added && size() > limit) {
super.remove();
}
return added;
}
}
Use composition not extends (yes I mean extends, as in a reference to the extends keyword in java and yes this is inheritance). Composition is superier because it completely shields your implementation, allowing you to change the implementation without impacting the users of your class.
I recommend trying something like this (I'm typing directly into this window, so buyer beware of syntax errors):
public LimitedSizeQueue implements Queue
{
private int maxSize;
private LinkedList storageArea;
public LimitedSizeQueue(final int maxSize)
{
this.maxSize = maxSize;
storageArea = new LinkedList();
}
public boolean offer(ElementType element)
{
if (storageArea.size() < maxSize)
{
storageArea.addFirst(element);
}
else
{
... remove last element;
storageArea.addFirst(element);
}
}
... the rest of this class
A better option (based on the answer by Asaf) might be to wrap the Apache Collections CircularFifoBuffer with a generic class. For example:
public LimitedSizeQueue<ElementType> implements Queue<ElementType>
{
private int maxSize;
private CircularFifoBuffer storageArea;
public LimitedSizeQueue(final int maxSize)
{
if (maxSize > 0)
{
this.maxSize = maxSize;
storateArea = new CircularFifoBuffer(maxSize);
}
else
{
throw new IllegalArgumentException("blah blah blah");
}
}
... implement the Queue interface using the CircularFifoBuffer class
}
The only thing I know that has limited space is the BlockingQueue interface (which is e.g. implemented by the ArrayBlockingQueue class) - but they do not remove the first element if filled, but instead block the put operation until space is free (removed by other thread).
To my knowledge your trivial implementation is the easiest way to get such an behaviour.
You can use a MinMaxPriorityQueue from Google Guava, from the javadoc:
A min-max priority queue can be configured with a maximum size. If so, each time the size of the queue exceeds that value, the queue automatically removes its greatest element according to its comparator (which might be the element that was just added). This is different from conventional bounded queues, which either block or reject new elements when full.
An LRUMap is another possibility, also from Apache Commons.
http://commons.apache.org/collections/apidocs/org/apache/commons/collections/map/LRUMap.html
Ok I'll share this option. This is a pretty performant option - it uses an array internally - and reuses entries. It's thread safe - and you can retrieve the contents as a List.
static class FixedSizeCircularReference<T> {
T[] entries
FixedSizeCircularReference(int size) {
this.entries = new Object[size] as T[]
this.size = size
}
int cur = 0
int size
synchronized void add(T entry) {
entries[cur++] = entry
if (cur >= size) {
cur = 0
}
}
List<T> asList() {
int c = cur
int s = size
T[] e = entries.collect() as T[]
List<T> list = new ArrayList<>()
int oldest = (c == s - 1) ? 0 : c
for (int i = 0; i < e.length; i++) {
def entry = e[oldest + i < s ? oldest + i : oldest + i - s]
if (entry) list.add(entry)
}
return list
}
}
public class ArrayLimitedQueue<E> extends ArrayDeque<E> {
private int limit;
public ArrayLimitedQueue(int limit) {
super(limit + 1);
this.limit = limit;
}
#Override
public boolean add(E o) {
boolean added = super.add(o);
while (added && size() > limit) {
super.remove();
}
return added;
}
#Override
public void addLast(E e) {
super.addLast(e);
while (size() > limit) {
super.removeLast();
}
}
#Override
public boolean offerLast(E e) {
boolean added = super.offerLast(e);
while (added && size() > limit) {
super.pollLast();
}
return added;
}
}
I have a question about java collections such as Set or List. More generally objects that you can use in a for-each loop. Is there any requirement that the elements of them actually has to be stored somewhere in a data structure or can they be described only from some sort of requirement and calculated on the fly when you need them? It feels like this should be possible to be done, but I don't see any of the java standard collection classes doing anything like this. Am I breaking any sort of contract here?
The thing I'm thinking about using these for is mainly mathematics. Say for example I want to have a set representing all prime numbers under 1 000 000. It might not be a good idea to save these in memory but to instead have a method check if a particular number is in the collection or not.
I'm also not at all an expert at java streams, but I feel like these should be usable in java 8 streams since the objects have very minimal state (the objects in the collection doesn't even exist until you try to iterate over them or check if a particular object exists in the collection).
Is it possible to have Collections or Iterators with virtually infinitely many elements, for example "all numbers on form 6*k+1", "All primes above 10" or "All Vectors spanned by this basis"? One other thing I'm thinking about is combining two sets like the union of all primes below 1 000 000 and all integers on form 2^n-1 and list the mersenne primes below 1 000 000. I feel like it would be easier to reason about certain mathematical objects if it was done this way and the elements weren't created explicitly until they are actually needed. Maybe I'm wrong.
Here's two mockup classes I wrote to try to illustrate what I want to do. They don't act exactly as I would expect (see output) which make me think I am breaking some kind of contract here with the iterable interface or implementing it wrong. Feel free to point out what I'm doing wrong here if you see it or if this kind of code is even allowed under the collections framework.
import java.util.AbstractSet;
import java.util.Iterator;
public class PrimesBelow extends AbstractSet<Integer>{
int max;
int size;
public PrimesBelow(int max) {
this.max = max;
}
#Override
public Iterator<Integer> iterator() {
return new SetIterator<Integer>(this);
}
#Override
public int size() {
if(this.size == -1){
System.out.println("Calculating size");
size = calculateSize();
}else{
System.out.println("Accessing calculated size");
}
return size;
}
private int calculateSize() {
int c = 0;
for(Integer p: this)
c++;
return c;
}
public static void main(String[] args){
PrimesBelow primesBelow10 = new PrimesBelow(10);
for(int i: primesBelow10)
System.out.println(i);
System.out.println(primesBelow10);
}
}
.
import java.util.Iterator;
import java.util.NoSuchElementException;
public class SetIterator<T> implements Iterator<Integer> {
int max;
int current;
public SetIterator(PrimesBelow pb) {
this.max= pb.max;
current = 1;
}
#Override
public boolean hasNext() {
if(current < max) return true;
else return false;
}
#Override
public Integer next() {
while(hasNext()){
current++;
if(isPrime(current)){
System.out.println("returning "+current);
return current;
}
}
throw new NoSuchElementException();
}
private boolean isPrime(int a) {
if(a<2) return false;
for(int i = 2; i < a; i++) if((a%i)==0) return false;
return true;
}
}
Main function gives the output
returning 2
2
returning 3
3
returning 5
5
returning 7
7
Exception in thread "main" java.util.NoSuchElementException
at SetIterator.next(SetIterator.java:27)
at SetIterator.next(SetIterator.java:1)
at PrimesBelow.main(PrimesBelow.java:38)
edit: spotted an error in the next() method. Corrected it and changed the output to the new one.
Well, as you see with your (now fixed) example, you can easily do it with Iterables/Iterators. Instead of having a backing collection, the example would've been nicer with just an Iterable that takes the max number you wish to calculate primes to. You just need to make sure that you handle the hasNext() method properly so you don't have to throw an exception unnecessarily from next().
Java 8 streams can be used easier to perform these kinds of things nowadays, but there's no reason you can't have a "virtual collection" that's just an Iterable. If you start implementing Collection it becomes harder, but even then it wouldn't be completely impossible, depending on the use cases: e.g. you could implement contains() that checks for primes, but you'd have to calculate it and it would be slow for large numbers.
A (somewhat convoluted) example of a semi-infinite set of odd numbers that is immutable and stores no values.
public class OddSet implements Set<Integer> {
public boolean contains(Integer o) {
return o % 2 == 1;
}
public int size() {
return Integer.MAX_VALUE;
}
public boolean add(Integer i) {
throw new OperationNotSupportedException();
}
public boolean equals(Object o) {
return o instanceof OddSet;
}
// etc. etc.
}
As DwB stated, this is not possible to do with Java's Collections API, as every element must be stored in memory. However, there is an alternative: this is precisely why Java's Stream API was implemented!
Streams allow you to iterate across an infinite amount of objects that are not stored in memory unless you explicitly collect them into a Collection.
From the documentation of IntStream#iterate:
Returns an infinite sequential ordered IntStream produced by iterative application of a function f to an initial element seed, producing a Stream consisting of seed, f(seed), f(f(seed)), etc.
The first element (position 0) in the IntStream will be the provided seed. For n > 0, the element at position n, will be the result of applying the function f to the element at position n - 1.
Here are some examples that you proposed in your question:
public class Test {
public static void main(String[] args) {
IntStream.iterate(1, k -> 6 * k + 1);
IntStream.iterate(10, i -> i + 1).filter(Test::isPrime);
IntStream.iterate(1, n -> 2 * n - 1).filter(i -> i < 1_000_000);
}
private boolean isPrime(int a) {
if (a < 2) {
return false;
}
for(int i = 2; i < a; i++) {
if ((a % i) == 0) {
return false;
}
return true;
}
}
}
E.g.
I have a queue
void someMethod() {
history.add(new Sample(time, data));
...
traverse(history);
}
void traverse(Queue<Sample> history) {
for(int i=0; i<history.size(); i=i+10) {
history.get(i)... // ???
}
}
class Sample {
long time;
double data;
}
The concerns are that
I don't want to destroy this queue by calling traverse().
Traverse the queue in a given step, say 10 here.
Any simple and nice solution?
for (Sample s : history)
doStuff(s);
This is called the enhanced for-loop; you can read more about it here.
Queue implements Iterable, so a simple loop will traverse it:
for (Sample sample : history)
An Iterator is another way to do it, with more control (can destroy it if you want to), but more verbose.
If you just want to iterate, use a for-each loop or directly a for loop with an Iterator. This doesn't consume the queue.
If you need to iterate with a step, you can use this pattern. It works generally with any Iterable. Putting the skipping into a separate reusable method makes the code more clear than having two nested for loops.
public static void main(String[] args) {
Queue<Sample> history = ...
int step = 10;
for (Iterator<Sample> it = history.iterator();
it.hasNext(); skip(it, step - 1)) {
// note that we skipped 1 less elements than the size of the step
Sample sample = it.next();
// stuff
}
}
static void skip(Iterator<?> iterator, int count) {
for (int i = 0; i < count && iterator.hasNext(); i++) {
iterator.next();
}
}
LinkedList<Sample> h = (LinkedList<Sample>) history;
for(int i=0; i < h.size(); i+=step) {
h.get(i).memory ...
}
I just realized this approach, haven't tried it yet.
As nullptr pointed out, the condition for above code is that the Queue is implemented as a LinkedList. (which is my case: Queue<Sample> history = new LinkedList<Sample>();)
For my data structures class our homework is to create a generic heap ADT. In the siftUp() method I need to do comparison and if the parent is smaller I need to do a swap. The problem I am having is that the comparison operators are not valid on generic types. I believe I need to use the Comparable interface but from what I read it’s not a good idea to use with Arrays. I have also search this site and I have found good information that relates to this post none of them helped me find the solution
I removed some of the code that wasn’t relevant
Thanks
public class HeapQueue<E> implements Cloneable {
private int highest;
private Integer manyItems;
private E[] data;
public HeapQueue(int a_highest) {
data = (E[]) new Object[10];
highest = a_highest;
}
public void add(E item, int priority) {
// check to see is priority value is within range
if(priority < 0 || priority > highest) {
throw new IllegalArgumentException
("Priority value is out of range: " + priority);
}
// increase the heaps capacity if array is out of space
if(manyItems == data.length)
ensureCapacity();
manyItems++;
data[manyItems - 1] = item;
siftUp(manyItems - 1);
}
private void siftUp(int nodeIndex) {
int parentIndex;
E tmp;
if (nodeIndex != 0) {
parentIndex = parent(nodeIndex);
if (data[parentIndex] < data[nodeIndex]) { <-- problem ****
tmp = data[parentIndex];
data[parentIndex] = data[nodeIndex];
data[nodeIndex] = tmp;
siftUp(parentIndex);
}
}
}
private int parent(int nodeIndex) {
return (nodeIndex - 1) / 2;
}
}
Technically you're using the comparable interface on on item, not an array. One item in the array specifically. I think the best solution here is to accept, in the constructor, a Comparator that the user can pass to compare his generic objects.
Comparator<E> comparator;
public HeapQueue(int a_highest, Comparator<E> compare)
{
this.comparator = compare;
Then, you would store that comparator in a member function and use
if (comparator.compare(data[parentIndex],data[nodeIndex]) < 0)
In place of the less than operator.
If I am reading this right, E simply needs to extend Comparable and then your problem line becomes...
if (data[parentIndex].compareTo(ata[nodeIndex]) < 0)
This is not breaking any bet-practice rules that I know of.