Develop Reference

Efficient way to save last X amount of integers [duplicate]

A very simple & quick question on Java libraries: is there a ready-made class that implements a Queue with a fixed maximum size - i.e. it always allows addition of elements, but it will silently remove head elements to accomodate space for newly added elements.
Of course, it's trivial to implement it manually:
import java.util.LinkedList;
public class LimitedQueue<E> extends LinkedList<E> {
private int limit;
public LimitedQueue(int limit) {
this.limit = limit;
}
#Override
public boolean add(E o) {
super.add(o);
while (size() > limit) { super.remove(); }
return true;
}
}
As far as I see, there's no standard implementation in Java stdlibs, but may be there's one in Apache Commons or something like that?

Apache commons collections 4 has a CircularFifoQueue<> which is what you are looking for. Quoting the javadoc:
CircularFifoQueue is a first-in first-out queue with a fixed size that replaces its oldest element if full.
import java.util.Queue;
import org.apache.commons.collections4.queue.CircularFifoQueue;
Queue<Integer> fifo = new CircularFifoQueue<Integer>(2);
fifo.add(1);
fifo.add(2);
fifo.add(3);
System.out.println(fifo);
// Observe the result:
// [2, 3]
If you are using an older version of the Apache commons collections (3.x), you can use the CircularFifoBuffer which is basically the same thing without generics.
Update: updated answer following release of commons collections version 4 that supports generics.

Guava now has an EvictingQueue, a non-blocking queue which automatically evicts elements from the head of the queue when attempting to add new elements onto the queue and it is full.
import java.util.Queue;
import com.google.common.collect.EvictingQueue;
Queue<Integer> fifo = EvictingQueue.create(2);
fifo.add(1);
fifo.add(2);
fifo.add(3);
System.out.println(fifo);
// Observe the result:
// [2, 3]

I like #FractalizeR solution. But I would in addition keep and return the value from super.add(o)!
public class LimitedQueue<E> extends LinkedList<E> {
private int limit;
public LimitedQueue(int limit) {
this.limit = limit;
}
#Override
public boolean add(E o) {
boolean added = super.add(o);
while (added && size() > limit) {
super.remove();
}
return added;
}
}

Use composition not extends (yes I mean extends, as in a reference to the extends keyword in java and yes this is inheritance). Composition is superier because it completely shields your implementation, allowing you to change the implementation without impacting the users of your class.
I recommend trying something like this (I'm typing directly into this window, so buyer beware of syntax errors):
public LimitedSizeQueue implements Queue
{
private int maxSize;
private LinkedList storageArea;
public LimitedSizeQueue(final int maxSize)
{
this.maxSize = maxSize;
storageArea = new LinkedList();
}
public boolean offer(ElementType element)
{
if (storageArea.size() < maxSize)
{
storageArea.addFirst(element);
}
else
{
... remove last element;
storageArea.addFirst(element);
}
}
... the rest of this class
A better option (based on the answer by Asaf) might be to wrap the Apache Collections CircularFifoBuffer with a generic class. For example:
public LimitedSizeQueue<ElementType> implements Queue<ElementType>
{
private int maxSize;
private CircularFifoBuffer storageArea;
public LimitedSizeQueue(final int maxSize)
{
if (maxSize > 0)
{
this.maxSize = maxSize;
storateArea = new CircularFifoBuffer(maxSize);
}
else
{
throw new IllegalArgumentException("blah blah blah");
}
}
... implement the Queue interface using the CircularFifoBuffer class
}

The only thing I know that has limited space is the BlockingQueue interface (which is e.g. implemented by the ArrayBlockingQueue class) - but they do not remove the first element if filled, but instead block the put operation until space is free (removed by other thread).
To my knowledge your trivial implementation is the easiest way to get such an behaviour.

You can use a MinMaxPriorityQueue from Google Guava, from the javadoc:
A min-max priority queue can be configured with a maximum size. If so, each time the size of the queue exceeds that value, the queue automatically removes its greatest element according to its comparator (which might be the element that was just added). This is different from conventional bounded queues, which either block or reject new elements when full.

An LRUMap is another possibility, also from Apache Commons.
http://commons.apache.org/collections/apidocs/org/apache/commons/collections/map/LRUMap.html

Ok I'll share this option. This is a pretty performant option - it uses an array internally - and reuses entries. It's thread safe - and you can retrieve the contents as a List.
static class FixedSizeCircularReference<T> {
T[] entries
FixedSizeCircularReference(int size) {
this.entries = new Object[size] as T[]
this.size = size
}
int cur = 0
int size
synchronized void add(T entry) {
entries[cur++] = entry
if (cur >= size) {
cur = 0
}
}
List<T> asList() {
int c = cur
int s = size
T[] e = entries.collect() as T[]
List<T> list = new ArrayList<>()
int oldest = (c == s - 1) ? 0 : c
for (int i = 0; i < e.length; i++) {
def entry = e[oldest + i < s ? oldest + i : oldest + i - s]
if (entry) list.add(entry)
}
return list
}
}

public class ArrayLimitedQueue<E> extends ArrayDeque<E> {
private int limit;
public ArrayLimitedQueue(int limit) {
super(limit + 1);
this.limit = limit;
}
#Override
public boolean add(E o) {
boolean added = super.add(o);
while (added && size() > limit) {
super.remove();
}
return added;
}
#Override
public void addLast(E e) {
super.addLast(e);
while (size() > limit) {
super.removeLast();
}
}
#Override
public boolean offerLast(E e) {
boolean added = super.offerLast(e);
while (added && size() > limit) {
super.pollLast();
}
return added;
}
}

Can you have collections without storing the values in Java?

I have a question about java collections such as Set or List. More generally objects that you can use in a for-each loop. Is there any requirement that the elements of them actually has to be stored somewhere in a data structure or can they be described only from some sort of requirement and calculated on the fly when you need them? It feels like this should be possible to be done, but I don't see any of the java standard collection classes doing anything like this. Am I breaking any sort of contract here?
The thing I'm thinking about using these for is mainly mathematics. Say for example I want to have a set representing all prime numbers under 1 000 000. It might not be a good idea to save these in memory but to instead have a method check if a particular number is in the collection or not.
I'm also not at all an expert at java streams, but I feel like these should be usable in java 8 streams since the objects have very minimal state (the objects in the collection doesn't even exist until you try to iterate over them or check if a particular object exists in the collection).
Is it possible to have Collections or Iterators with virtually infinitely many elements, for example "all numbers on form 6*k+1", "All primes above 10" or "All Vectors spanned by this basis"? One other thing I'm thinking about is combining two sets like the union of all primes below 1 000 000 and all integers on form 2^n-1 and list the mersenne primes below 1 000 000. I feel like it would be easier to reason about certain mathematical objects if it was done this way and the elements weren't created explicitly until they are actually needed. Maybe I'm wrong.
Here's two mockup classes I wrote to try to illustrate what I want to do. They don't act exactly as I would expect (see output) which make me think I am breaking some kind of contract here with the iterable interface or implementing it wrong. Feel free to point out what I'm doing wrong here if you see it or if this kind of code is even allowed under the collections framework.
import java.util.AbstractSet;
import java.util.Iterator;
public class PrimesBelow extends AbstractSet<Integer>{
int max;
int size;
public PrimesBelow(int max) {
this.max = max;
}
#Override
public Iterator<Integer> iterator() {
return new SetIterator<Integer>(this);
}
#Override
public int size() {
if(this.size == -1){
System.out.println("Calculating size");
size = calculateSize();
}else{
System.out.println("Accessing calculated size");
}
return size;
}
private int calculateSize() {
int c = 0;
for(Integer p: this)
c++;
return c;
}
public static void main(String[] args){
PrimesBelow primesBelow10 = new PrimesBelow(10);
for(int i: primesBelow10)
System.out.println(i);
System.out.println(primesBelow10);
}
}
.
import java.util.Iterator;
import java.util.NoSuchElementException;
public class SetIterator<T> implements Iterator<Integer> {
int max;
int current;
public SetIterator(PrimesBelow pb) {
this.max= pb.max;
current = 1;
}
#Override
public boolean hasNext() {
if(current < max) return true;
else return false;
}
#Override
public Integer next() {
while(hasNext()){
current++;
if(isPrime(current)){
System.out.println("returning "+current);
return current;
}
}
throw new NoSuchElementException();
}
private boolean isPrime(int a) {
if(a<2) return false;
for(int i = 2; i < a; i++) if((a%i)==0) return false;
return true;
}
}
Main function gives the output
returning 2
2
returning 3
3
returning 5
5
returning 7
7
Exception in thread "main" java.util.NoSuchElementException
at SetIterator.next(SetIterator.java:27)
at SetIterator.next(SetIterator.java:1)
at PrimesBelow.main(PrimesBelow.java:38)
edit: spotted an error in the next() method. Corrected it and changed the output to the new one.

Well, as you see with your (now fixed) example, you can easily do it with Iterables/Iterators. Instead of having a backing collection, the example would've been nicer with just an Iterable that takes the max number you wish to calculate primes to. You just need to make sure that you handle the hasNext() method properly so you don't have to throw an exception unnecessarily from next().
Java 8 streams can be used easier to perform these kinds of things nowadays, but there's no reason you can't have a "virtual collection" that's just an Iterable. If you start implementing Collection it becomes harder, but even then it wouldn't be completely impossible, depending on the use cases: e.g. you could implement contains() that checks for primes, but you'd have to calculate it and it would be slow for large numbers.
A (somewhat convoluted) example of a semi-infinite set of odd numbers that is immutable and stores no values.
public class OddSet implements Set<Integer> {
public boolean contains(Integer o) {
return o % 2 == 1;
}
public int size() {
return Integer.MAX_VALUE;
}
public boolean add(Integer i) {
throw new OperationNotSupportedException();
}
public boolean equals(Object o) {
return o instanceof OddSet;
}
// etc. etc.
}

As DwB stated, this is not possible to do with Java's Collections API, as every element must be stored in memory. However, there is an alternative: this is precisely why Java's Stream API was implemented!
Streams allow you to iterate across an infinite amount of objects that are not stored in memory unless you explicitly collect them into a Collection.
From the documentation of IntStream#iterate:
Returns an infinite sequential ordered IntStream produced by iterative application of a function f to an initial element seed, producing a Stream consisting of seed, f(seed), f(f(seed)), etc.
The first element (position 0) in the IntStream will be the provided seed. For n > 0, the element at position n, will be the result of applying the function f to the element at position n - 1.
Here are some examples that you proposed in your question:
public class Test {
public static void main(String[] args) {
IntStream.iterate(1, k -> 6 * k + 1);
IntStream.iterate(10, i -> i + 1).filter(Test::isPrime);
IntStream.iterate(1, n -> 2 * n - 1).filter(i -> i < 1_000_000);
}
private boolean isPrime(int a) {
if (a < 2) {
return false;
}
for(int i = 2; i < a; i++) {
if ((a % i) == 0) {
return false;
}
return true;
}
}
}

Iterating through elements of a data structure instead of Collection

My problem is this: I have an iterator class which is supposed to iterate through elements in a given data structure, <E> let's say, but what I have managed to accomplish is that when I pass in the data structure it will iterate the data structure itself.
ie. DynamicIterator it = new DynamicIterator(da);
say da is an array the output will be [1,2,3,4,5,6] instead of 1,2,3,4,5,6
My issue is, more than anything, understanding the generally accepted practice for dealing with this more than the issue itself.
edit for code:
public class X<E>
{
private final E[] rray;
private int currentIndex = 0;
public X(E... a)
{
//if the incoming array is null, don't start
if(a == null)
{
System.out.println("Array is null");
System.exit(1);
}
//set the temp array (rray) to the incoming array (a)
this.rray = a;
}
//hasNext element?
public boolean hasNext()
{
return rray.length > currentIndex;
}
//next element (depends on hasNext())
public E next()
{
if (!hasNext())
{
System.out.println("Element doesn't exist, done");
System.exit(1);
}
return rray[currentIndex++];
}
//return array
public E[] access()
{
return rray;
}
}

You won't be able to do this with a completely generic parameter <E> - how would you iterate through a Throwable, for example? What your class X does at the moment is accept any number of objects in its constructor, and then simply returns each of those objects in turn.
If you restricted the bounds of the objects passed in to implement e.g. Iterable, then you can actually start to "look inside" them and return their contents:
public class X<E> {
private final Iterator<E> it;
public X(Iterable<E> a) {
it = a.iterator();
}
public boolean hasNext() {
return it.hasNext();
}
public E next() {
return it.next();
}
}
Although this doesn't really accomplish anything different to just using a.iterator() directly instead of an instance of X...

Comparison operators in generic heap

For my data structures class our homework is to create a generic heap ADT. In the siftUp() method I need to do comparison and if the parent is smaller I need to do a swap. The problem I am having is that the comparison operators are not valid on generic types. I believe I need to use the Comparable interface but from what I read it’s not a good idea to use with Arrays. I have also search this site and I have found good information that relates to this post none of them helped me find the solution
I removed some of the code that wasn’t relevant
Thanks
public class HeapQueue<E> implements Cloneable {
private int highest;
private Integer manyItems;
private E[] data;
public HeapQueue(int a_highest) {
data = (E[]) new Object[10];
highest = a_highest;
}
public void add(E item, int priority) {
// check to see is priority value is within range
if(priority < 0 || priority > highest) {
throw new IllegalArgumentException
("Priority value is out of range: " + priority);
}
// increase the heaps capacity if array is out of space
if(manyItems == data.length)
ensureCapacity();
manyItems++;
data[manyItems - 1] = item;
siftUp(manyItems - 1);
}
private void siftUp(int nodeIndex) {
int parentIndex;
E tmp;
if (nodeIndex != 0) {
parentIndex = parent(nodeIndex);
if (data[parentIndex] < data[nodeIndex]) { <-- problem ****
tmp = data[parentIndex];
data[parentIndex] = data[nodeIndex];
data[nodeIndex] = tmp;
siftUp(parentIndex);
}
}
}
private int parent(int nodeIndex) {
return (nodeIndex - 1) / 2;
}
}

Technically you're using the comparable interface on on item, not an array. One item in the array specifically. I think the best solution here is to accept, in the constructor, a Comparator that the user can pass to compare his generic objects.
Comparator<E> comparator;
public HeapQueue(int a_highest, Comparator<E> compare)
{
this.comparator = compare;
Then, you would store that comparator in a member function and use
if (comparator.compare(data[parentIndex],data[nodeIndex]) < 0)
In place of the less than operator.

If I am reading this right, E simply needs to extend Comparable and then your problem line becomes...
if (data[parentIndex].compareTo(ata[nodeIndex]) < 0)
This is not breaking any bet-practice rules that I know of.

Which Queue implementation to use in Java?

I need to use a FIFO structure in my application. It needs to have at most 5 elements.
I'd like to have something easy to use (I don't care for concurrency) that implements the Collection interface.
I've tried the LinkedList, that seems to come from Queue, but it doesn't seem to allow me to set it's maximum capacity. It feels as if I just want at max 5 elements but try to add 20, it will just keep increasing in size to fit it. I'd like something that'd work the following way:
XQueue<Integer> queue = new XQueue<Integer>(5); //where 5 is the maximum number of elements I want in my queue.
for (int i = 0; i < 10; ++i) {
queue.offer(i);
}
for (int i = 0; i < 5; ++i) {
System.out.println(queue.poll());
}
That'd print:
5
6
7
8
9
Thanks

Create your own subclass of the one you want, and override the add method so that it
checks if the new object will fit, and fails if not
calls super.add()
(and the constructors).
If you want it to block when inserting if full, it is a different matter.

I haven't seen any limitation like that in the API. You can use ArrayList by changing the behavior of the add method with anonymous class feature:
new ArrayList<Object>(){
public boolean add(Object o){ /*...*/ }
}

Looks like what you want is a limited size FIFO structure, that evicts oldest items when new ones are added. I recommend a solution based on a cyclic array implementation, where you should track the index of the queue tail and queue head, and increase them (in cyclic manner) as needed.
EDIT:
Here is my implementation (note that it IS a Collection). It works fine with your test scenario.
public class XQueue <T> extends AbstractQueue<T>{
private T[] arr;
private int headPos;
private int tailPos;
private int size;
#SuppressWarnings("unchecked")
public XQueue(int n){
arr = (T[]) new Object[n];
}
private int nextPos(int pos){
return (pos + 1) % arr.length;
}
#Override
public T peek() {
if (size == 0)
return null;
return arr[headPos];
}
public T poll(){
if (size == 0)
return null;
size--;
T res = arr[headPos];
headPos = nextPos(headPos);
return res;
}
#Override
public boolean offer(T e) {
if (size < arr.length)
size++;
else
if (headPos == tailPos)
headPos = nextPos(headPos);
arr[tailPos] = e;
tailPos = nextPos(tailPos);
return true;
}
#Override
public Iterator<T> iterator() {
return null; //TODO: Implement
}
#Override
public int size() {
return size;
}
}

Perhaps an ArrayBlockingQueue might do the trick. Look here. Try something like this:
BlockingQueue<Integer> queue = new ArrayBlockingQueue<Integer>(5);
for (int i = 0; i < 10; i++) {
while (!queue.offer(i)) {
queue.poll();
}
}
for (int i = 0; i < 5; i++) {
System.out.println(queue.poll());
}

You have three choices
1) Subclass an Abstract Collection
2) Limit the size to five and do the logic around the code where you are doing the insert.
3) Use LinkedListHashMap The removeEldestEntry(Map.Entry) method may be overridden to impose a policy for removing stale mappings automatically when new mappings are added to the map. (You would then use an Iterator to get the values - which will be returned in order of insertion)
Your best bet is #1 - It is real easy if you look at the link.

Did you have a look at the Apache Commons Collections library? The BoundedFifoBuffer should exactly meet your needs.

If I remember correctly, I've done exactly what you want using a LinkedList.
What you need to do is check the size of the List, if it's 5 and you want to add objects, just delete the first element and keep doing so if the size is 5.