Develop Reference

Collectors.groupingBy functionality and summing a field

.collect(Collectors.groupingBy(Point::getName, Collectors.summingInt(Point::getCount)));
I have a list of Point objects that I want to group by a certain key (the name field) and sum by the count field of that class. The code above does the job but returns a map of Point objects. However, I want a list of Point objects returned - not a map.
What is the cleanest way to do this with java 8 streams?
Example:
input = [pt("jack", 1), pt("jack", 1), pt("jack", 1)]
result = [pt("jack", 3)]
Thanks

You can use Collectors.toMap() with a merge function as parameter.
If you add a function to sum count fields:
public class Point {
//...
public static Point sum(Point p1, Point p2) {
return new Point(p1.getName(), p1.getCount()+p2.getCount());
}
}
Then you can use it in toMap():
List<Point> list = Collections.nCopies(10, new Point("jack", 1));
Collection<Point> output = list.stream()
.collect(Collectors.toMap(Point::getName, Function.identity(), Point::sum)) // results as Map<String, Point> {"jack", Point("jack",10)}
.values(); // to get the Point instances
System.out.println(output);
Output:
[Point [name=jack, count=10]]

import java.util.ArrayList;
import java.util.List;
import java.util.function.BiConsumer;
import java.util.function.Supplier;
public class Pointers {
private String name;
private int count;
public Pointers(String name, int count) {
this.name = name;
this.count = count;
}
public String getName() {
return name;
}
public int getCount() {
return count;
}
public void incrementCount(int amount) {
count += amount;
}
public boolean equals(Object obj) {
boolean equal = false;
if (obj instanceof Pointers) {
Pointers other = (Pointers) obj;
equal = name.equals(other.getName());
}
return equal;
}
public String toString() {
return name + count;
}
public static void main(String[] args) {
List<Pointers> list = List.of(new Pointers("Jack", 1),
new Pointers("Jack", 1),
new Pointers("Jack", 1));
Supplier<List<Pointers>> supplier = () -> new ArrayList<Pointers>();
BiConsumer<List<Pointers>, Pointers> accumulator = (l, p) -> {
if (l.contains(p)) {
Pointers elem = l.get(l.indexOf(p));
elem.incrementCount(p.getCount());
}
else {
l.add(p);
}
};
BiConsumer<List<Pointers>, List<Pointers>> combiner = (l1, l2) -> {
};
List<Pointers> lst = list.stream()
.collect(supplier, accumulator, combiner);
System.out.println(lst);
}
}

Actually, you were close. You can take the key (name) and value (point sum) and repackage it into a new Point object and return as a list. Note that by re-assiging to list, you destroy the original one which will of course be garbage collected. This approach does not require a modification of your current class.
list = list.stream()
.collect(Collectors.groupingBy(Point::getName,
Collectors.summingInt(Point::getCount)))
.entrySet().stream()
.map(e -> new Point(e.getKey(), e.getValue()))
.collect(Collectors.toList());

How to implement Java 8 generic method that returns zipped stream where objects from two streams appear in turn?

I tried to implement a generic method that returns a zipped stream from two streams. The returned stream should be a concatenation of two streams where elements appear in turn, and are based on two streams that are passed to this method. If one stream is longer than another, it should contain values from the longer stream in the end. The code I have come up with is:
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.stream.Stream;
public class Task12 {
public static void main(String[] args) {
Stream<Integer> stream1 = Stream.iterate(0, integer -> integer)
.limit(20);
Stream<Integer> stream2 = Stream.iterate(1, integer -> integer)
.limit(20);
Stream<Integer> zippedStreams = zip(stream1, stream2);
zippedStreams.forEach(System.out::println);
}
private static <T> Stream<T> zip(Stream<T> first, Stream<T> second) {
Iterator<T> iterator1 = first.iterator();
Iterator<T> iterator2 = second.iterator();
List<T> elements = new LinkedList<>();
while (iterator1.hasNext() || iterator2.hasNext()) {
if (iterator1.hasNext()) {
elements.add(iterator1.next());
}
if(iterator2.hasNext()) {
elements.add(iterator2.next());
}
}
return elements.stream();
}
I have a few questions about this implementation :
is there a better way to implement it?
what if one of the streams would be an infinite Stream? Is there a way to limit the returned stream to contain exact number of elements - other than using "limit" method and passing limit as a parameter to "zip" method?

There is a way to construct the stream out of an iterator/iterable. That's very close to your solution but that way, it handles infinite stream also
public static void main(String[] args) {
Stream<Integer> stream1 = Stream.iterate(0, integer -> integer);
Stream<Integer> stream2 = Stream.iterate(1, integer -> integer)
.limit(4);
Stream<Integer> zippedStreams = StreamSupport.stream(new Ziperator<>(stream1, stream2).spliterator(), false);
zippedStreams.limit(15).forEach(System.out::println);
}
public static class Ziperator<T> implements Iterator<T>, Iterable<T>{
Iterator<T> iterator1;
Iterator<T> iterator2;
boolean even = true;
#Override
public Iterator<T> iterator() {
return this;
}
public Ziperator(Stream<T> first, Stream<T> second) {
iterator1 = first.iterator();
iterator2 = second.iterator();
}
public boolean hasNext(){
return iterator1.hasNext() || iterator2.hasNext();
}
#Override
public T next() {
if(!iterator2.hasNext()){
even = true;
}
if(!iterator1.hasNext()){
even = false;
}
if (even) {
even = false;
return iterator1.next();
} else {
even = true;
return iterator2.next();
}
}
#Override
public void remove() {
}
}
Output for that is
0
1
0
1
0
1
0
1
0
0
0
0
0
0
0
This code can be clean-up/optimized but it's a working base :)

Collect pairs from a stream

I have a stream of objects like this:
"0", "1", "2", "3", "4", "5",
How can I transform it to stream of pairs :
{ new Pair("0", "1"), new Pair("2", "3"), new Pair("4", "5")}.
The stream size is unknown. I am reading data from a file that might be big. I have only iterator to collection and I transform this iterator to stream using spliterator.
I know that here is a answer for processing adjacent pairs with StreamEx :
Collect successive pairs from a stream
Can this be done in java or StreamEx ?
Thanks

It's not a natural fit but you can do
List input = ...
List<Pair> pairs = IntStream.range(0, input.size() / 2)
.map(i -> i * 2)
.mapToObj(i -> new Pair(input.get(i), input.get(i + 1)))
.collect(Collectors.toList());
To create Pairs as you go in a stream you need a stateful lambdas which should be generally avoided but can be done. Note: this will only works if the stream is single threaded. i.e. not parallel.
Stream<?> stream =
assert !stream.isParallel();
Object[] last = { null };
List<Pair> pairs = stream.map(a -> {
if (last[0] == null) {
last[0] = a;
return null;
} else {
Object t = last[0];
last[0] = null;
return new Pair(t, a);
}
}).filter(p -> p != null)
.collect(Collectors.toList());
assert last[0] == null; // to check for an even number input.

If you don't want to collect the elements
The title of the question says collect pairs from a stream, so I'd assume that you want to actually collect these, but you commented:
Your solution works, the problem is that it loads the data from file to PairList and then I may use stream from this collection to process pairs. I can't do it because the data might be too big to store in the memory.
so here's a way to do this without collecting the elements.
It's relatively straightforward to transform an Iterator<T> into an Iterator<List<T>>, and from that to transform a stream into a stream of pairs.
/**
* Returns an iterator over pairs of elements returned by the iterator.
*
* #param iterator the base iterator
* #return the paired iterator
*/
public static <T> Iterator<List<T>> paired(Iterator<T> iterator) {
return new Iterator<List<T>>() {
#Override
public boolean hasNext() {
return iterator.hasNext();
}
#Override
public List<T> next() {
T first = iterator.next();
if (iterator.hasNext()) {
return Arrays.asList(first, iterator.next());
} else {
return Arrays.asList(first);
}
}
};
}
/**
* Returns an stream of pairs of elements from a stream.
*
* #param stream the base stream
* #return the pair stream
*/
public static <T> Stream<List<T>> paired(Stream<T> stream) {
return StreamSupport.stream(Spliterators.spliteratorUnknownSize(paired(stream.iterator()), Spliterator.ORDERED),
false);
}
#Test
public void iteratorAndStreamsExample() {
List<String> strings = Arrays.asList("a", "b", "c", "d", "e", "f");
Iterator<List<String>> pairs = paired(strings.iterator());
while (pairs.hasNext()) {
System.out.println(pairs.next());
// [a, b]
// [c, d]
// [e, f]
}
paired(Stream.of(1, 2, 3, 4, 5, 6, 7, 8)).forEach(System.out::println);
// [1, 2]
// [3, 4]
// [5, 6]
// [7, 8]
}
If you want to collect the elements...
I'd do this by collecting into a list, and using an AbstractList to provide a view of the elements as pairs.
First, the PairList. This is a simple AbstractList wrapper around any list that has an even number of elements. (This could easily be adapted to handle odd length lists, once the desired behavior is specified.)
/**
* A view on a list of its elements as pairs.
*
* #param <T> the element type
*/
static class PairList<T> extends AbstractList<List<T>> {
private final List<T> elements;
/**
* Creates a new pair list.
*
* #param elements the elements
*
* #throws NullPointerException if elements is null
* #throws IllegalArgumentException if the length of elements is not even
*/
public PairList(List<T> elements) {
Objects.requireNonNull(elements, "elements must not be null");
this.elements = new ArrayList<>(elements);
if (this.elements.size() % 2 != 0) {
throw new IllegalArgumentException("number of elements must have even size");
}
}
#Override
public List<T> get(int index) {
return Arrays.asList(elements.get(index), elements.get(index + 1));
}
#Override
public int size() {
return elements.size() / 2;
}
}
Then we can define the collector that we need. This is essentially shorthand for collectingAndThen(toList(), PairList::new):
/**
* Returns a collector that collects to a pair list.
*
* #return the collector
*/
public static <E> Collector<E, ?, PairList<E>> toPairList() {
return Collectors.collectingAndThen(Collectors.toList(), PairList::new);
}
Note that it could be worthwhile defining a PairList constructor that doesn't defensively copy the list, for the use case that we know the backing list is freshly generated (as in this case). That's not really essential right now, though. But once we did that, this method would be collectingAndThen(toCollection(ArrayList::new), PairList::newNonDefensivelyCopiedPairList).
And now we can use it:
/**
* Creates a pair list with collectingAndThen, toList(), and PairList::new
*/
#Test
public void example() {
List<List<Integer>> intPairs = Stream.of(1, 2, 3, 4, 5, 6)
.collect(toPairList());
System.out.println(intPairs); // [[1, 2], [2, 3], [3, 4]]
List<List<String>> stringPairs = Stream.of("a", "b", "c", "d")
.collect(toPairList());
System.out.println(stringPairs); // [[a, b], [b, c]]
}
Here's a complete source file with a runnable example (as a JUnit test):
package ex;
import java.util.AbstractList;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Objects;
import java.util.stream.Collector;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.junit.Test;
public class PairCollectors {
/**
* A view on a list of its elements as pairs.
*
* #param <T> the element type
*/
static class PairList<T> extends AbstractList<List<T>> {
private final List<T> elements;
/**
* Creates a new pair list.
*
* #param elements the elements
*
* #throws NullPointerException if elements is null
* #throws IllegalArgumentException if the length of elements is not even
*/
public PairList(List<T> elements) {
Objects.requireNonNull(elements, "elements must not be null");
this.elements = new ArrayList<>(elements);
if (this.elements.size() % 2 != 0) {
throw new IllegalArgumentException("number of elements must have even size");
}
}
#Override
public List<T> get(int index) {
return Arrays.asList(elements.get(index), elements.get(index + 1));
}
#Override
public int size() {
return elements.size() / 2;
}
}
/**
* Returns a collector that collects to a pair list.
*
* #return the collector
*/
public static <E> Collector<E, ?, PairList<E>> toPairList() {
return Collectors.collectingAndThen(Collectors.toList(), PairList::new);
}
/**
* Creates a pair list with collectingAndThen, toList(), and PairList::new
*/
#Test
public void example() {
List<List<Integer>> intPairs = Stream.of(1, 2, 3, 4, 5, 6)
.collect(toPairList());
System.out.println(intPairs); // [[1, 2], [2, 3], [3, 4]]
List<List<String>> stringPairs = Stream.of("a", "b", "c", "d")
.collect(toPairList());
System.out.println(stringPairs); // [[a, b], [b, c]]
}
}

Assuming there is a Pair with left, right and getters and a constructor:
static class Paired<T> extends AbstractSpliterator<Pair<T>> {
private List<T> list = new ArrayList<>(2);
private final Iterator<T> iter;
public Paired(Iterator<T> iter) {
super(Long.MAX_VALUE, 0);
this.iter = iter;
}
#Override
public boolean tryAdvance(Consumer<? super Pair<T>> consumer) {
getBothIfPossible(iter);
if (list.size() == 2) {
consumer.accept(new Pair<>(list.remove(0), list.remove(0)));
return true;
}
return false;
}
private void getBothIfPossible(Iterator<T> iter) {
while (iter.hasNext() && list.size() < 2) {
list.add(iter.next());
}
}
}
Usage would be:
Iterator<Integer> iterator = List.of(1, 2, 3, 4, 5).iterator();
Paired<Integer> p = new Paired<>(iterator);
StreamSupport.stream(p, false)
.forEach(pair -> System.out.println(pair.getLeft() + " " + pair.getRight()));

I know I'm late to the party, but all of the answers seem to be really complicated or have a lot of GC overhead/short-lived objects (which is not a big deal with modern JVMs), but why not do it simply like this?
public class PairCollaterTest extends TestCase {
static class PairCollater<T> implements Function<T, Stream<Pair<T, T>>> {
T prev;
#Override
public Stream<Pair<T, T>> apply(T curr) {
if (prev == null) {
prev = curr;
return Stream.empty();
}
try {
return Stream.of(Pair.of(prev, curr));
} finally {
prev = null;
}
}
}
public void testPairCollater() {
Stream.of("0", "1", "2", "3", "4", "5").sequential().flatMap(new PairCollater<>()).forEach(System.out::println);
}
}
Prints:
(0,1)
(2,3)
(4,5)

Just replace IntStream.range(1, 101) with your stream (you don't need to know your stream's size) -
import java.util.ArrayList;
import java.util.List;
import java.util.stream.IntStream;
public class TestClass {
public static void main(String[] args) {
final Pair pair = new Pair();
final List<Pair> pairList = new ArrayList<>();
IntStream.range(1, 101)
.map(i -> {
if (pair.a == null) {
pair.a = i;
return 0;
} else {
pair.b = i;
return 1;
}
})
.filter(i -> i == 1)
.forEach(i -> {
pairList.add(new Pair(pair));
pair.reset();
});
pairList.stream().forEach(p -> System.out.print(p + " "));
}
static class Pair {
public Object a;
public Object b;
public Pair() {
}
public Pair(Pair orig) {
this.a = orig.a;
this.b = orig.b;
}
void reset() {
a = null;
b = null;
}
#Override
public String toString() {
return "{" + a + "," + b + '}';
}
}
}

Double a stream

I want to double a Stream (no DoubleStream). Meaning I start with a stream and want to get a new stream where each element of the old stream is streamed twice. So 1,2,3,4,4,5 gives us 1,1,2,2,3,3,4,4,4,4,5,5. Is there such a stream operation?

Create an inner stream which will contain current element two times and flatMap this stream.
stream.flatMap(e -> Stream.of(e,e))
If you want to multiply the number of elements by n you can create an utility method like this one:
public static <T> Stream<T> multiplyElements(Stream<T> in, int n) {
return in.flatMap(e -> IntStream.range(0, n).mapToObj(i -> e));
// we can also use IntStream.rangeClosed(1, n)
// but I am used to iterating from 0 to n (where n is excluded)
}
(but try to use a better name for this method, since the current one may be ambiguous)
Usage example:
multiplyElements(Stream.of(1,2), 3).forEach(System.out::println);
Output:
1
1
1
2
2
2

You can create a stream of 2 elements for each original element and flatMap it:
List<Integer> list = Arrays.asList(1, 2, 3, 4, 4, 5);
List<Integer> doubled = list.stream().flatMap(i -> Stream.of(i, i)).collect(toList());

Here's a simple example of what biziclop has described in the comments.
static <E> Collection<E> multiply(Collection<E> source, int count) {
return new AbstractCollection<E>() {
#Override
public int size() {
return count * source.size();
}
#Override
public Iterator<E> iterator() {
return new Iterator<E>() {
final Iterator<E> it = source.iterator();
E next;
int i = 0;
#Override
public boolean hasNext() {
return i < size();
}
#Override
public E next() {
if (hasNext()) {
if ((i % count) == 0) {
next = it.next();
}
++i;
return next;
} else {
throw new NoSuchElementException();
}
}
};
}
};
}
(Working example on Ideone.)
CW'd since it wasn't my idea and the flatMap suggestions more directly answer the question.

How to create an infinite Stream<E> out of an Iterator<E>?

Looking at the following class I've made:
public class FibonacciSupplier implements Iterator<Integer> {
private final IntPredicate hasNextPredicate;
private int beforePrevious = 0;
private int previous = 1;
private FibonacciSupplier(final IntPredicate hasNextPredicate) {
this.hasNextPredicate = hasNextPredicate;
}
#Override
public boolean hasNext() {
return hasNextPredicate.test(previous);
}
#Override
public Integer next() {
int result = beforePrevious + previous;
beforePrevious = previous;
previous = result;
return result;
}
public static FibonacciSupplier infinite() {
return new FibonacciSupplier(i -> true);
}
public static FibonacciSupplier finite(final IntPredicate predicate) {
return new FibonacciSupplier(predicate);
}
}
And the usage of it in:
public class Problem2 extends Problem<Integer> {
#Override
public void run() {
result = toList(FibonacciSupplier.finite(i -> (i <= 4_000_000)))
.stream()
.filter(i -> (i % 2 == 0))
.mapToInt(i -> i)
.sum();
}
#Override
public String getName() {
return "Problem 2";
}
private static <E> List<E> toList(final Iterator<E> iterator) {
List<E> list = new ArrayList<>();
while (iterator.hasNext()) {
list.add(iterator.next());
}
return list;
}
}
How would I be able to create an infinite Stream<E>?
If I were to use Stream<Integer> infiniteStream = toList(FibonacciSupplier.infinite()).stream(), I would, possibly surprisingly, never get an infinite stream.
Instead the code would loop forever in the creation of the list in an underlying method.
This so far is purely theoretical, but I can definately understand the need for it if I would want to first skip the first x numbers from an infinite stream, and then limit it by the last y numbers, something like:
int x = MAGIC_NUMBER_X;
int y = MAGIC_NUMBER_y;
int sum = toList(FibonacciSupplier.infinite())
.stream()
.skip(x)
.limit(y)
.mapToInt(i -> i)
.sum();
The code would not ever return a result, how should it be done?

Your mistake is to think that you need an Iterator or a Collection to create a Stream. For creating an infinite stream, a single method providing one value after another is enough. So for your class FibonacciSupplier the simplest use is:
IntStream s=IntStream.generate(FibonacciSupplier.infinite()::next);
or, if you prefer boxed values:
Stream<Integer> s=Stream.generate(FibonacciSupplier.infinite()::next);
Note that in this case the method does not have to be named next nor fulfill the Iterator interface. But it doesn’t matter if it does as with your class. Further, as we just told the stream to use the next method as a Supplier, the hasNext method will never be called. It’s just infinite.
Creating a finite stream using your Iterator is a bit more complicated:
Stream<Integer> s=StreamSupport.stream(
Spliterators.spliteratorUnknownSize(
FibonacciSupplier.finite(intPredicate), Spliterator.ORDERED),
false);
In this case if you want a finite IntStream with unboxed int values your FibonacciSupplier should implement PrimitiveIterator.OfInt.

In Java 8 there are no public, concrete classes implementing the interface Stream. However, there are some static factory methods. One of the most important is StreamSupport.stream. In particular, it is used in the default method Collection.stream –inherited by most collection classes:
default Stream<E> stream() {
return StreamSupport.stream(spliterator(), false);
}
The default implementation of this method creates a Spliterator by invoking spliterator(), and passes the created object to the factory method. Spliterator is a new interface introduced with Java 8 to support parallel streams. It is similar to Iterator, but in contrast to the latter, a Spliterator can be divided into parts, that can be processed independently. See Spliterator.trySplit for details.
The default method Iterable.spliterator was also added in Java 8, so that every Iterable class automatically supports Spliterators. The implementation looks as follows:
default Spliterator<T> spliterator() {
return Spliterators.spliteratorUnknownSize(iterator(), 0);
}
The method creates the Spliterator from an arbitrary Iterator. If you combine these two steps, you can create a Stream from an arbitrary Iterator:
<T> Stream<T> stream(Iterator<T> iterator) {
Spliterator<T> spliterator
= Spliterators.spliteratorUnknownSize(iterator, 0);
return StreamSupport.stream(spliterator, false);
}
To get an impression of Spliterators, here is a very simple example without using collections. The following class implements Spliterator to iterate over a half-open interval of integers:
public final class IntRange implements Spliterator.OfInt {
private int first, last;
public IntRange(int first, int last) {
this.first = first;
this.last = last;
}
public boolean tryAdvance(IntConsumer action) {
if (first < last) {
action.accept(first++);
return true;
} else {
return false;
}
}
public OfInt trySplit() {
int size = last - first;
if (size >= 10) {
int temp = first;
first += size / 2;
return new IntRange(temp, first);
} else {
return null;
}
}
public long estimateSize() {
return Math.max(last - first, 0);
}
public int characteristics() {
return ORDERED | DISTINCT | SIZED | NONNULL
| IMMUTABLE | CONCURRENT | SUBSIZED;
}
}

You can use the low level stream support primitives and the Spliterators library to make a stream out of an Iterator.
The last parameter to StreamSupport.stream() says that the stream is not parallel. Be sure to let it like that because your Fibonacci iterator depends on previous iterations.
return StreamSupport.stream( Spliterators.spliteratorUnknownSize( new Iterator<Node>()
{
#Override
public boolean hasNext()
{
// to implement
return ...;
}
#Override
public ContentVersion next()
{
// to implement
return ...;
}
}, Spliterator.ORDERED ), false );

To add another answer, perhaps AbstractSpliterator is a better choice, especially given the example code. Generate is inflexible as there is no [good] way to give a stop condition except by using limit. Limit only accepts a number of items rather than a predicate, so we have to know how many items we want to generate - which might not be possible, and what if the generator is a black box passed to us?
AbstractSpliterator is a halfway house between having to write a whole spliterator, and using Iterator/Iterable. AbstractSpliterator lacks just the tryAdvance method where we first check our predicate for being done, and if not pass the generated value to an action. Here's an example of a Fibonacci sequence using AbstractIntSpliterator:
public class Fibonacci {
private static class FibonacciGenerator extends Spliterators.AbstractIntSpliterator
{
private IntPredicate hasNextPredicate;
private int beforePrevious = 0;
private int previous = 0;
protected FibonacciGenerator(IntPredicate hasNextPredicate)
{
super(Long.MAX_VALUE, 0);
this.hasNextPredicate = hasNextPredicate;
}
#Override
public boolean tryAdvance(IntConsumer action)
{
if (action == null)
{
throw new NullPointerException();
}
int next = Math.max(1, beforePrevious + previous);
beforePrevious = previous;
previous = next;
if (!hasNextPredicate.test(next))
{
return false;
}
action.accept(next);
return true;
}
#Override
public boolean tryAdvance(Consumer<? super Integer> action)
{
if (action == null)
{
throw new NullPointerException();
}
int next = Math.max(1, beforePrevious + previous);
beforePrevious = previous;
previous = next;
if (!hasNextPredicate.test(next))
{
return false;
}
action.accept(next);
return true;
}
}
public static void main(String args[])
{
Stream<Integer> infiniteStream = StreamSupport.stream(
new FibonacciGenerator(i -> true), false);
Stream<Integer> finiteStream = StreamSupport.stream(
new FibonacciGenerator(i -> i < 100), false);
// Print with a side-effect for the demo
infiniteStream.limit(10).forEach(System.out::println);
finiteStream.forEach(System.out::println);
}
}
For more details I've covered generators in Java 8 in my blog http://thecannycoder.wordpress.com/