I'm playing with java reflection and learning more about Stream.collect.
I have an annotation MyTag that has two properties (id and type enum[Normal|Failure]).
Also, I have a list of annotated methods with MyTag and I was able to group those methods by the id property of the MyTag annotation using Collectors.groupingBy:
List<Method> ml = getMethodsAnnotatedWith(anClass.getClass(),
MyTag.class);
Map<String, List<Method>> map = ml.stream().collect(groupingBy(m -> {
var ann = m.getDeclaredAnnotation(MyTag.class);
return ann.anId();
}, TreeMap::new, toList()));
Now I need to reduce the resulting List to one single object composed of ONLY TWO items of the same MyTag.id, one with a MyTag.type=Normal and the other with a MyTag.type=Failure. So it would result in something like a Map<String, Pair<Method, Method>>. If there are more than two occurrences, I must just pick the first ones, log and ignore the rest.
How could I achieve that ?
You can use
Map<String, Map<Type, Method>> map = Arrays.stream(anClass.getClass().getMethods())
.filter(m -> m.isAnnotationPresent(MyTag.class))
.collect(groupingBy(m -> m.getDeclaredAnnotation(MyTag.class).anId(),
TreeMap::new,
toMap(m -> m.getDeclaredAnnotation(MyTag.class).aType(),
m -> m, (first, last) -> first,
() -> new EnumMap<>(Type.class))));
The result maps the annotations ID property to a Map from Type (the enum constants NORMAL and FAILURE) to the first encountered method with a matching annotation. Though “first” has not an actual meaning when iterating over the methods discovered by Reflection, as it doesn’t guaranty any specific order.
The () -> new EnumMap<>(Type.class) map factory is not necessary, it would also work with the general purpose map used by default when you don’t specify a factory. But the EnumMap will handle your case of having only two constants to map in a slightly more efficient way and its iteration order will match the declaration order of the enum constants.
I think, the EnumMap is better than a Pair<Method, Method> that requires to remember which method is associated with “normal” and which with “failure”. It’s also easier to adapt to more than two constants. Also, the EnumMap is built-in and doesn’t require a 3rd party library.
The following example can easily be adapted to your code:
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.TreeMap;
import java.util.stream.Collectors;
import org.apache.commons.lang3.tuple.Pair;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class Test {
private static final Logger logger = LoggerFactory.getLogger(Test.class);
public static void main(String[] args) {
List<Pair<String, String>> ml = Arrays.asList(
Pair.of("key1", "value1"),
Pair.of("key1", "value1"),
Pair.of("key1", "value2"),
Pair.of("key2", "value1"),
Pair.of("key2", "value3"));
Map<String, Pair<String, String>> map = ml.stream().collect(
Collectors.groupingBy(m -> {
return m.getKey();
}, TreeMap::new, Collectors.toList()))
.entrySet()
.stream()
.collect(Collectors.toMap(
Map.Entry::getKey, e -> convert(e.getValue())));
System.out.println(map.values());
}
private static Pair<String, String> convert(List<Pair<String, String>> original) {
long count1 = original.stream().filter(e -> Objects.equals(e.getValue(), "value1")).count();
long count2 = original.stream().filter(e -> Objects.equals(e.getValue(), "value2")).count();
if (count1 > 1) {
logger.warn("More than one occurrence of value1");
}
if (count2 > 1) {
logger.warn("More than one occurrence of value2");
}
return Pair.of(count1 > 0 ? "value1" : null,
count2 > 0 ? "value2" : null);
}
}
Instead of Pair<String, String> use Method
m.getDeclaredAnnotation(MyTag.class).anId() corresponds to pair.getKey()
The folowing result is printed to the console:
01:23:27.959 [main] WARN syglass.Test2 - More than one occurrence of value1
[(value1,value2), (value1,null)]
First, create your own MethodPair class:
class MethodPair {
private final Method failure;
private final Method normal;
public MethodPair(Method failure, Method normal) {
this.failure = failure;
this.normal = normal;
}
public Method getFailure() {
return failure;
}
public Method getNormal() {
return normal;
}
public MethodPair combinedWith(MethodPair other) {
return new MethodPair(
this.failure == null ? other.failure : this.failure,
this.normal == null ? other.normal : this.normal)
);
}
}
Notice the combinedWith method. This is going to useful in the reduction that we are going to do.
Instead of toList, use the reducing collector:
Map<String, MethodPair> map = ml.stream().collect(groupingBy(m -> {
var ann = m.getDeclaredAnnotation(MyTag.class);
return ann.anId();
}, TreeMap::new,
Collectors.reducing(new MethodPair(null, null), method -> {
var type = method.getDeclaredAnnotation(MyTag.class).type();
if (type == Type.NORMAL) {
return new MethodPair(null, method);
} else {
return new MethodPair(method, null);
}
}, MethodPair::combinedWith)
));
If you are fine with doing this in two steps, I would suggest that you create the Map<String, List<Method>> first, then map its values to a new map. IMO this is more readable:
Map<String, List<Method>> map = ml.stream().collect(groupingBy(m -> {
var ann = m.getDeclaredAnnotation(MyTag.class);
return ann.anId();
}, TreeMap::new, toList()));
var result = map.entrySet().stream().collect(Collectors.toMap(entry -> entry.getKey(), entry -> {
Method normal = null;
Method failure = null;
for (var m : entry.getValue()) {
var type = m.getDeclaredAnnotation(MyTag.class).type();
if (type == Type.NORMAL && normal == null) {
normal = m;
} else if (type == Type.FAILURE && failure == null) {
failure = m;
}
if (normal != null && failure != null) {
break;
}
}
return new MethodPair(failure, normal);
}));
Related
I'm trying to use Java stream to filter some values based on certain conditions. I am able to achieve the same using traditional for loops and a little bit of streams, but I want to rewrite the same logic fully in streams.
Original code:
public List <String> getProductNames(Hub hub, String requestedGroup) {
List <SupportedProduct> configuredProducts = repo.getSupportedProducts(hub);
List <String> productNames = new ArrayList <> ();
for (SupportedProduct supportedProduct: configuredProducts) {
List < String > categoryNameList = new ArrayList <> ();
String activeCategoryName = supportedProduct.getCategoryDetails().getActiveCategoryName();
if (activeCategoryName == null) {
Optional.ofNullable(supportedProduct.getCategoryDetails().getCategories())
.orElse(Collections.emptyList())
.forEach(category - > categoryNameList.add(category.getName()));
} else {
categoryNameList.add(activeCategoryName);
}
for (String catName: categoryNameList) {
Division division = divisionRepo.getDivisionByCatName(catName);
if (division != null && division.getGroup() == requestedGroup) {
productNames.add(supportedProduct.getProductName());
}
}
}
return productNames;
}
My try:
return Optional.ofNullable(configuredProducts).orElse(Collections.emptyList()).stream()
.map(supportedProduct -> {
List<String> categoryNameList = new ArrayList<>();
String activeCategoryName = supportedProduct.getCategoryDetails().getActiveCategoryName();
if (activeCategoryName == null) {
Optional.ofNullable(supportedProduct.getCategoryDetails().getCategories())
.orElse(Collections.emptyList())
.forEach(category -> categoryNameList.add(category.getName()));
} else {
categoryNameList.add(activeCategoryName);
}
return categoryNameList;
})
.filter(catName ->{
Division division = divisionRepo.getDivisionByCatName(catName);
return division != null && division.getGroup() == requestedGroup;
})........
But I'm lost beyond this.
Please help me to write the same using streams.
EDIT: Added IDEOne for testing - Link
The logic inside is quite complicated, however, try this out:
public List <String> getProductNames(Hub hub, String requestedGroup) {
List<SupportedProduct> configuredProducts = repo.getSupportedProducts(hub);
// extract pairs:
// key=SupportedProduct::getProductName
// values=List with one activeCategoryName OR names of all the categories
Map<String, List<String>> namedActiveCategoryNamesMap = configuredProducts.stream()
.collect(Collectors.toMap(
SupportedProduct::getProductName,
p -> Optional.ofNullable(p.getCategoryDetails().getActiveCategoryName())
.map(Collections::singletonList)
.orElse(Optional.ofNullable(p.getCategoryDetails().getCategories())
.stream()
.flatMap(Collection::stream)
.map(Category::getName)
.collect(Collectors.toList()))));
// look-up based on the categories' names, group equality comparison and returning a List
return namedActiveCategoryNamesMap.entrySet().stream()
.filter(entry -> entry.getValue().stream()
.map(catName -> divisionRepo.getDivisionByCatName(catName))
.filter(Objects::nonNull)
.map(Division::getGroup)
.anyMatch(requestedGroup::equals))
.map(Map.Entry::getKey)
.collect(Collectors.toList());
}
I recommend splitting into separate methods for sake of readability (the best way to go).
The verbose logics of Optional chains including two orElse calls can be surely simplified, however, it gives you the idea.
You can perform within one Stream using Collectors.collectingAndThen. In that case, I'd extract the Function finisher elsewhere, example:
public List<String> getProductNames(Hub hub, String requestedGroup) {
return repo.getSupportedProducts(hub).stream()
.collect(Collectors.collectingAndThen(
Collectors.toMap(
SupportedProduct::getProductName,
categoryNamesFunction()),
productNamesFunction(requestedGroup)));
}
private Function<Map<String, List<String>>, List<String>> productNamesFunction(String requestedGroup) {
return map -> map.entrySet().stream()
.filter(entry -> entry.getValue().stream()
.map(divisionRepo::getDivisionByCatName)
.filter(Objects::nonNull)
.map(Division::getGroup)
.anyMatch(requestedGroup::equals))
.map(Map.Entry::getKey)
.collect(Collectors.toList());
}
private Function<SupportedProduct, List<String>> categoryNamesFunction() {
return p -> Optional.ofNullable(p.getCategoryDetails().getActiveCategoryName())
.map(Collections::singletonList)
.orElse(Optional.ofNullable(p.getCategoryDetails().getCategories())
.stream()
.flatMap(Collection::stream)
.map(Category::getName)
.collect(Collectors.toList()));
}
I have the following class:
public static class GenerateMetaAlert implements WindowFunction<Tuple2<String, Boolean>, Tuple2<String, Boolean>, Tuple, TimeWindow> {
#Override
public void apply(Tuple key, TimeWindow timeWindow, Iterable<Tuple2<String, Boolean>> iterable, Collector<Tuple2<String, Boolean>> collector) throws Exception {
//code
}
}
What I'm trying to do is is for each element of the collection there are any other with the opposite value in a field.
An example:
Iterable: [<val1,val2>,<val3,val4>,<val5,val6>,...,<valx,valy>]
|| || || ||
elem1 elem2 elem3 elemn
What I would like to test:
foreach(element)
if elem(i).f0 = elem(i+1).f0 then ...
if elem(i).f0 = elem(i+2).f0 then ...
<...>
if elem(i+1).f0 = elem(i+2).f0 then ...
<...>
if elem(n-1).f0 = elem(n).f0 then ...
I think this would be possible using something like this:
Tuple2<String, Boolean> tupla = iterable.iterator().next();
iterable.iterator().forEachRemaining((e)->{
if ((e.f0 == tupla.f0) && (e.f1 != tupla.f1)) collector.collect(e);});
But like i'm new with Java, I don't know how I could do it in an optimal way.
This is a part of a Java program which use Apache Flink:
.keyBy(0, 1)
.timeWindow(Time.seconds(60))
.apply(new GenerateMetaAlert())
Testing:
Using the following code:
public static class GenerateMetaAlert implements WindowFunction<Tuple2<String, Boolean>, Tuple2<String, Boolean>, Tuple, TimeWindow> {
#Override
public void apply(Tuple key, TimeWindow timeWindow, Iterable<Tuple2<String, Boolean>> iterable, Collector<Tuple2<String, Boolean>> collector) throws Exception {
System.out.println("key: " +key);
StreamSupport.stream(iterable.spliterator(), false)
.collect(Collectors.groupingBy(t -> t.f0)) // yields a Map<String, List<Tuple2<String, Boolean>>>
.values() // yields a Collection<List<Tuple2<String, Boolean>>>
.stream()
.forEach(l -> {
System.out.println("l.size: " +l.size());
// l is the list of tuples for some common f0
while (l.size() > 1) {
Tuple2<String, Boolean> t0 = l.get(0);
System.out.println("t0: " +t0);
l = l.subList(1, l.size());
l.stream()
.filter(t -> t.f1 != t0.f1)
.forEach(t -> System.out.println("t: "+ t));
}
});
}
}
The result is:
key: (868789022645948,true)
key: (868789022645948,false)
l.size: 2
l.size: 2
t0: (868789022645948,true)
t0: (868789022645948,false)
Conclusion of this test: is like the condition .filter(t -> t.f1 != t0.f1) is never met
If I change .filter(t -> t.f1 != t0.f1) for .filter(t -> t.f1 != true) (or false) the filter works
I also use the following:
final Boolean[] aux = new Boolean[1];
<...>
Tuple2<String, Boolean> t0 = l.get(0);
aux[0] = t0.f1;
<...>
.filter(t -> !t.f1.equals(aux[0]))
But even with that, I don't have any output (I only have it when I use t.f1.equals(aux[0])
An Iterable allows you to obtain as many Iterators over its elements as you like, but each of them iterates over all the elements, and only once. Thus, your idea for using forEachRemaining() will not work as you hope. Because you're generating a new Iterator to invoke that method on, it will start at the beginning instead of after the element most recently provided by the other iterator.
What you can do instead is create a Stream by use of the Iterable's Spliterator, and use a grouping-by Collector to group the iterable's tuples by their first value. You can then process the tuple lists as you like.
For example, although I have some doubts as to whether it's what you actually want, this implements the logic described in the question:
StreamSupport.stream(iterable.spliterator(), false)
.collect(Collectors.groupingBy(t -> t.f0)) // yields a Map<String, List<Tuple2<String, Boolean>>>
.values() // yields a Collection<List<Tuple2<String, Boolean>>>
.stream()
.forEach(l -> {
// l is the list of tuples for some common f0
while (l.size() > 1) {
Tuple2<String, Boolean> t0 = l.get(0);
l = l.subList(1, l.size());
l.stream()
.filter(t -> t.f1 != t0.f1)
.forEach(t -> collect(t));
}
});
Note well that that can collect the same tuple multiple times, as follows from your pseudocode. If you wanted something different, such as collecting only tuples representing a flip of f1 value for a given f0, once each, then you would want a different implementation of the lambda in the outer forEach() operation.
Following class:
public class Foo {
private Date date;
private String name;
private Long number;
}
I now have a List<Foo> which I want to convert to Map<Date, Map<String,Long>> (Long should be a sum of numbers). What makes this hard is that I want exactly 26 entries in the inner map, where the 26th is called "Others" which sums up everything that has a number lower than the other 25.
I came up with following code:
data.stream().collect(Collectors.groupingBy(e -> e.getDate(), Collectors.groupingBy(e -> {
if (/*get current size of inner map*/>= 25) {
return e.getName();
} else {
return "Other";
}
}, Collectors.summingLong(e -> e.getNumber()))));
As you can see, I have no idea how to check the number of elements which are already in the inner map. How can I get the current size of the inner map or is there another way to achieve what I want?
My Java 7 code:
Map<Date, Map<String, Long>> result = new LinkedHashMap<Date, Map<String, Long>>();
for (Foo fr : data) {
if (result.get(fr.getDate()) == null) {
result.put(fr.getDate(), new LinkedHashMap<String, Long>());
}
if (result.get(fr.getDate()) != null) {
if (result.get(fr.getDate()).size() >= 25) {
if (result.get(fr.getDate()).get("Other") == null) {
result.get(fr.getDate()).put("Other", 0l);
}
if (result.get(fr.getDate()).get("Other") != null) {
long numbers= result.get(fr.getDate()).get("Other");
result.get(fr.getDate()).replace("Other", numbers+ fr.getNumbers());
}
} else {
result.get(fr.getDate()).put(fr.getName(), fr.getNumbers());
}
}
}
Edit:
The map should help me to realize a table like this:
But I need to sum the "Others" first.
If you need any more infos feel free to ask
I don’t think that this operation will benefit from using the Stream API. Still, you can improve the operation with Java 8 features:
Map<Date, Map<String, Long>> result = new LinkedHashMap<>();
for(Foo fr : data) {
Map<String, Long> inner
= result.computeIfAbsent(fr.getDate(), date -> new LinkedHashMap<>());
inner.merge(inner.size()>=25?"Other":fr.getAirlineName(), fr.getNumbers(), Long::sum);
}
This code assumes that the airline names are already unique for each date. Otherwise, you would have to extend the code to
Map<Date, Map<String, Long>> result = new LinkedHashMap<>();
for(Foo fr : data) {
Map<String, Long> inner
= result.computeIfAbsent(fr.getDate(), date -> new LinkedHashMap<>());
inner.merge(inner.size() >= 25 && !inner.containsKey(fr.getAirlineName())?
"Other": fr.getAirlineName(), fr.getNumbers(), Long::sum);
}
to accumulate the values for the airline correctly.
For completeness, here is how to implement it as a stream operation.
Since the custom collector has some complexity, it’s worth writing it as reusable code:
public static <T,K,V> Collector<T,?,Map<K,V>> toMapWithLimit(
Function<? super T, ? extends K> key, Function<? super T, ? extends V> value,
int limit, K fallBack, BinaryOperator<V> merger) {
return Collector.of(LinkedHashMap::new, (map, t) ->
mergeWithLimit(map, key.apply(t), value.apply(t), limit, fallBack, merger),
(map1,map2) -> {
if(map1.isEmpty()) return map2;
if(map1.size()+map2.size() < limit)
map2.forEach((k,v) -> map1.merge(k, v, merger));
else
map2.forEach((k,v) ->
mergeWithLimit(map1, k, v, limit, fallBack, merger));
return map1;
});
}
private static <T,K,V> void mergeWithLimit(Map<K,V> map, K key, V value,
int limit, K fallBack, BinaryOperator<V> merger) {
map.merge(map.size() >= limit && !map.containsKey(key)? fallBack: key, value, merger);
}
This is like Collectors.toMap, but supporting a limit and a fallback key for additional entries. You may recognize the Map.merge call, similar to the loop solution as the crucial element.
Then, you may use the collector as
Map<Date, Map<String, Long>> result = data.stream().collect(
Collectors.groupingBy(Foo::getDate, LinkedHashMap::new,
toMapWithLimit(Foo::getAirlineName, Foo::getNumbers, 25, "Other", Long::sum)));
A bit too late :) But I come with this Java 8 solution without using the for loop or the custom collector. It is based on collectingAndThen which allows you to transform the result of collecting operation.
It allows me to divide the stream in finisher operation based on treshold.
However, I am not sure about the performance.
int treshold = 25
Map<Date, Map<String, Long>> result = data.stream().collect(groupingBy(Foo::getDate,
collectingAndThen(Collectors.toList(), x -> {
if (x.size() >= treshold) {
Map<String, Long> resultMap = new HashMap<>();
resultMap.putAll(x.subList(0, treshold).stream().collect(groupingBy(Foo::getName, Collectors.summingLong(Foo::getNumber))));
resultMap.putAll(x.subList(treshold, x.size()).stream().collect(groupingBy(y -> "Other", Collectors.summingLong(Foo::getNumber))));
return resultMap;
} else {
return x.stream().collect(groupingBy(Foo::getName, Collectors.summingLong(Foo::getNumber)));
}
})));
First of all, let's simplify the original problem by adapting it to java 8 without using Streams.
Map<Date, Map<String, Long>> result = new LinkedHashMap();
for (Foo fr : data) {
Map<String, Long> map = result.getOrDefault(fr.getDate(), new LinkedHashMap());
if (map.size() >= 25) {
Long value = map.getOrDefault("Other", 0L); // getOrDefault from 1.8
map.put("Other", value + 1);
} else {
map.put(fr.getName(), fr.getNumber());
}
result.put(fr.getDate(), map);
}
And now using Stream
int limit = 25;
Map<Date, Map<String, Long>> collect = data.stream()
.collect(Collectors.groupingBy(Foo::getDate))
.entrySet().stream()
.collect(Collectors.toMap(Map.Entry::getKey, v -> {
Map<String, Long> c = v.getValue().stream()
.limit(limit)
.collect(Collectors.toMap(Foo::getName, Foo::getNumber));
long remaining = v.getValue().size() - limit;
if (remaining > 0) {
c.put("Other", remaining);
}
return c;
}));
I have to filter a Collection of Objects by a Map, which holds key value pairs of the Objects field names and field values. I am trying to apply all filters by stream().filter().
The Objects are actually JSON, therefore the Map holds the names of its variables as well as the value they have to contain in order to be accepted, but for simplicity reasons and because its not relevant to the question I wrote a simple Testclass for simulating the behaviour:
public class TestObject {
private int property1;
private int property2;
private int property3;
public TestObject(int property1, int property2, int property3) {
this.property1 = property1;
this.property2 = property2;
this.property3 = property3;
}
public int getProperty(int key) {
switch(key) {
case 1: return property1;
case 2: return property2;
default: return property3;
}
}
}
What I have tried so far:
public static void main(String[] args) {
List<TestObject> list = new ArrayList<>();
Map<Integer, Integer> filterMap = new HashMap<>();
list.add(new TestObject(1, 2, 3));
list.add(new TestObject(1, 2, 4));
list.add(new TestObject(1, 4, 3));
filterMap.put(3, 3); //Filter property3 == 3
filterMap.put(2, 2); //Filter property2 == 2
//Does not apply the result
filterMap.forEach((key, value) -> list.stream()
.filter(testObject -> testObject.getProperty(key) == value)
.collect(Collectors.toList())
);
/* Gives error: boolean can not be converted to void
list = list.stream()
.filter(testObject -> filterMap.forEach((key, value) -> testObject.getProperty(key) == value))
.collect(Collectors.toList()
);
*/
//Printing result
list.forEach(obj -> System.out.println(obj.getProperty(1) + " " + obj.getProperty(2) + " " + obj.getProperty(3)));
}
I tried putting forEach of the Map first and the stream of the Collection first, but both solutions did not work as intended. The desired output of this example would be only to print the object with the values property1=1, property2=2 and property3=3.
How can I apply all filters correctly like when you would put them one after another in the code with a fixed amount of filters?
With a known amount of filters:
list.stream().filter(...).filter(...)
Edit:
Sweeper summed my question up very well in his answer, so just for clarification (and probably future readers) here again: I want to keep all Objects that satisfy all filters.
I suppose you want to keep all the TestObjects that satisfy all the conditions specified by the map?
This will do the job:
List<TestObject> newList = list.stream()
.filter(x ->
filterMap.entrySet().stream()
.allMatch(y ->
x.getProperty(y.getKey()) == y.getValue()
)
)
.collect(Collectors.toList());
Translated into "English",
filter the list list by keeping all the elements x that:
all of the key value pairs y of filterMap must satisfy:
x.getProperty(y.getKey()) == y.getValue()
(I don't think I did a good job at making this human readable...) If you want a more readable solution, I recommend Jeroen Steenbeeke's answer.
To apply a variable number of filter steps to a stream (that only become known at runtime), you could use a loop to add filter steps.
Stream<TestObject> stream = list.stream();
for (Predicate<TestObject> predicate: allPredicates) {
stream = stream.filter(predicate);
}
list = stream.collect(Collectors.toList());
It has nothing to do with filter. Actually the filter never work as per your code. Look at
//Does not apply the result
filterMap.forEach((key, value) -> list.stream()
.filter(testObject -> testObject.getProperty(key) == value)
.collect(Collectors.toList())
);
List has been filtered but nothing is changed here. No element has been deleted and No object address has been changed either. Try removeIf
// Does not apply the result
filterMap.forEach((key, value) -> list.removeIf(testObject -> testObject.getProperty(key) != value));
output is
1 2 3
A more general approach is to create a multi filter (Predicate) which is concatenated using Predicate.and(...) or Predicate.or(...). This is applicable to anything using Predicate - first of all Stream and Optional.
Since the result is a Predicate itself one can continue with Predicate.and(...) or Predicate.or(...) or with building more complex predicates using MultiPredicate again.
class MultiPredicate {
public static <T> Predicate<T> matchingAll(Collection<Predicate<T>> predicates) {
Predicate<T> multiPredicate = to -> true;
for (Predicate<T> predicate : predicates) {
multiPredicate = multiPredicate.and(predicate);
}
return multiPredicate;
}
#SafeVarargs
public static <T> Predicate<T> matchingAll(Predicate<T> first, Predicate<T>... other) {
if (other == null || other.length == 0) {
return first;
}
Predicate<T> multiPredicate = first;
for (Predicate<T> predicate : other) {
multiPredicate = multiPredicate.and(predicate);
}
return multiPredicate;
}
public static <T> Predicate<T> matchingAny(Collection<Predicate<T>> predicates) {
Predicate<T> multiPredicate = to -> false;
for (Predicate<T> predicate : predicates) {
multiPredicate = multiPredicate.or(predicate);
}
return multiPredicate;
}
#SafeVarargs
public static <T> Predicate<T> matchingAny(Predicate<T> first, Predicate<T>... other) {
if (other == null || other.length == 0) {
return first;
}
Predicate<T> multiPredicate = first;
for (Predicate<T> predicate : other) {
multiPredicate = multiPredicate.or(predicate);
}
return multiPredicate;
}
}
Applying to the question:
public static void main(String... args) {
List<TestObject> list = new ArrayList<>();
Map<Integer, Integer> filterMap = new HashMap<>();
list.add(new TestObject(1, 2, 3));
list.add(new TestObject(1, 2, 4));
list.add(new TestObject(1, 4, 3));
filterMap.put(3, 3); // Filter property3 == 3
filterMap.put(2, 2); // Filter property2 == 2
List<Predicate<TestObject>> filters = filterMap.entrySet().stream()
.map(filterMapEntry -> mapToFilter(filterMapEntry))
.collect(Collectors.toList());
Predicate<TestObject> multiFilter = MultiPredicate.matchingAll(filters);
List<TestObject> filtered = list.stream()
.filter(multiFilter)
.collect(Collectors.toList());
for (TestObject to : filtered) {
System.out.println("(" + to.getProperty(1) + "|" + to.getProperty(2) + "|" + to.getProperty(3) + ")");
}
}
private static Predicate<TestObject> mapToFilter(Entry<Integer,Integer> filterMapEntry) {
return to -> to.getProperty(filterMapEntry.getKey()) == filterMapEntry.getValue();
}
In this case all filters have to match. The result is:
(1|2|3)
If we use MultiPredicate.matchingAny(...) the result is:
(1|2|3)
(1|2|4)
(1|4|3)
In Java 8, this works:
Stream<Class> stream = Stream.of(ArrayList.class);
HashMap<Class, List<Class>> map = (HashMap)stream.collect(Collectors.groupingBy(Class::getSuperclass));
But this doesn't:
Stream<Class> stream = Stream.of(List.class);
HashMap<Class, List<Class>> map = (HashMap)stream.collect(Collectors.groupingBy(Class::getSuperclass));
Maps allows a null key, and List.class.getSuperclass() returns null. But Collectors.groupingBy emits a NPE, at Collectors.java, line 907:
K key = Objects.requireNonNull(classifier.apply(t), "element cannot be mapped to a null key");
It works if I create my own collector, with this line changed to:
K key = classifier.apply(t);
My questions are:
1) The Javadoc of Collectors.groupingBy doesn't say it shouldn't map a null key. Is this behavior necessary for some reason?
2) Is there another, easier way, to accept a null key, without having to create my own collector?
I had the same kind of problem.
This failed, because groupingBy performs Objects.requireNonNull on the value returned from the classifier:
Map<Long, List<ClaimEvent>> map = events.stream()
.filter(event -> eventTypeIds.contains(event.getClaimEventTypeId()))
.collect(groupingBy(ClaimEvent::getSubprocessId));
Using Optional, this works:
Map<Optional<Long>, List<ClaimEvent>> map = events.stream()
.filter(event -> eventTypeIds.contains(event.getClaimEventTypeId()))
.collect(groupingBy(event -> Optional.ofNullable(event.getSubprocessId())));
For the first question, I agree with skiwi that it shouldn't be throwing a NPE. I hope they will change that (or else at least add it to the javadoc). Meanwhile, to answer the second question I decided to use Collectors.toMap instead of Collectors.groupingBy:
Stream<Class<?>> stream = Stream.of(ArrayList.class);
Map<Class<?>, List<Class<?>>> map = stream.collect(
Collectors.toMap(
Class::getSuperclass,
Collections::singletonList,
(List<Class<?>> oldList, List<Class<?>> newEl) -> {
List<Class<?>> newList = new ArrayList<>(oldList.size() + 1);
newList.addAll(oldList);
newList.addAll(newEl);
return newList;
}));
Or, encapsulating it:
/** Like Collectors.groupingBy, but accepts null keys. */
public static <T, A> Collector<T, ?, Map<A, List<T>>>
groupingBy_WithNullKeys(Function<? super T, ? extends A> classifier) {
return Collectors.toMap(
classifier,
Collections::singletonList,
(List<T> oldList, List<T> newEl) -> {
List<T> newList = new ArrayList<>(oldList.size() + 1);
newList.addAll(oldList);
newList.addAll(newEl);
return newList;
});
}
And use it like this:
Stream<Class<?>> stream = Stream.of(ArrayList.class);
Map<Class<?>, List<Class<?>>> map = stream.collect(groupingBy_WithNullKeys(Class::getSuperclass));
Please note rolfl gave another, more complicated answer, which allows you to specify your own Map and List supplier. I haven't tested it.
Use filter before groupingBy##
Filter out the null instances before groupingBy.
Here is an example
MyObjectlist.stream()
.filter(p -> p.getSomeInstance() != null)
.collect(Collectors.groupingBy(MyObject::getSomeInstance));
To your 1st question, from the docs:
There are no guarantees on the type, mutability, serializability, or thread-safety of the Map or List objects returned.
Because not all Map implementations allow null keys they probably added this to reduce to the most common allowable definition of a map to get maximum flexibility when choosing a type.
To your 2nd question, you just need a supplier, wouldn't a lambda work? I'm still getting acquainted with Java 8, maybe a smarter person can add a better answer.
I figured I would take a moment and try to digest this issue you have. I put together a SSCE for what I would expect if I did it manually, and what the groupingBy implementation actually does.
I don't think this is an answer, but it is a 'wonder why it is a problem' thing. Also, if you want, feel free to hack this code to have a null-friendly collector.
Edit: A generic-friendly implementation:
/** groupingByNF - NullFriendly - allows you to specify your own Map and List supplier. */
private static final <T,K> Collector<T,?,Map<K,List<T>>> groupingByNF (
final Supplier<Map<K,List<T>>> mapsupplier,
final Supplier<List<T>> listsupplier,
final Function<? super T,? extends K> classifier) {
BiConsumer<Map<K,List<T>>, T> combiner = (m, v) -> {
K key = classifier.apply(v);
List<T> store = m.get(key);
if (store == null) {
store = listsupplier.get();
m.put(key, store);
}
store.add(v);
};
BinaryOperator<Map<K, List<T>>> finalizer = (left, right) -> {
for (Map.Entry<K, List<T>> me : right.entrySet()) {
List<T> target = left.get(me.getKey());
if (target == null) {
left.put(me.getKey(), me.getValue());
} else {
target.addAll(me.getValue());
}
}
return left;
};
return Collector.of(mapsupplier, combiner, finalizer);
}
/** groupingByNF - NullFriendly - otherwise similar to Java8 Collections.groupingBy */
private static final <T,K> Collector<T,?,Map<K,List<T>>> groupingByNF (Function<? super T,? extends K> classifier) {
return groupingByNF(HashMap::new, ArrayList::new, classifier);
}
Consider this code (the code groups String values based on the String.length(), (or null if the input String is null)):
public static void main(String[] args) {
String[] input = {"a", "a", "", null, "b", "ab"};
// How we group the Strings
final Function<String, Integer> classifier = (a) -> {return a != null ? Integer.valueOf(a.length()) : null;};
// Manual implementation of a combiner that accumulates a string value based on the classifier.
// no special handling of null key values.
BiConsumer<Map<Integer,List<String>>, String> combiner = (m, v) -> {
Integer key = classifier.apply(v);
List<String> store = m.get(key);
if (store == null) {
store = new ArrayList<String>();
m.put(key, store);
}
store.add(v);
};
// The finalizer merges two maps together (right into left)
// no special handling of null key values.
BinaryOperator<Map<Integer, List<String>>> finalizer = (left, right) -> {
for (Map.Entry<Integer, List<String>> me : right.entrySet()) {
List<String> target = left.get(me.getKey());
if (target == null) {
left.put(me.getKey(), me.getValue());
} else {
target.addAll(me.getValue());
}
}
return left;
};
// Using a manual collector
Map<Integer, List<String>> manual = Arrays.stream(input).collect(Collector.of(HashMap::new, combiner, finalizer));
System.out.println(manual);
// using the groupingBy collector.
Collector<String, ?, Map<Integer, List<String>>> collector = Collectors.groupingBy(classifier);
Map<Integer, List<String>> result = Arrays.stream(input).collect(collector);
System.out.println(result);
}
The above code produces the output:
{0=[], null=[null], 1=[a, a, b], 2=[ab]}
Exception in thread "main" java.lang.NullPointerException: element cannot be mapped to a null key
at java.util.Objects.requireNonNull(Objects.java:228)
at java.util.stream.Collectors.lambda$groupingBy$135(Collectors.java:907)
at java.util.stream.Collectors$$Lambda$10/258952499.accept(Unknown Source)
at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at CollectGroupByNull.main(CollectGroupByNull.java:49)
First of all, you are using lots of raw objects. This is not a good idea at all, first convert the following:
Class to Class<?>, ie. instead of a raw type, a parametrized type with an unknown class.
Instead of forcefully casting to a HashMap, you should supply a HashMap to the collector.
First the correctly typed code, without caring about a NPE yet:
Stream<Class<?>> stream = Stream.of(ArrayList.class);
HashMap<Class<?>, List<Class<?>>> hashMap = (HashMap<Class<?>, List<Class<?>>>)stream
.collect(Collectors.groupingBy(Class::getSuperclass));
Now we get rid of the forceful cast there, and instead do it correctly:
Stream<Class<?>> stream = Stream.of(ArrayList.class);
HashMap<Class<?>, List<Class<?>>> hashMap = stream
.collect(Collectors.groupingBy(
Class::getSuperclass,
HashMap::new,
Collectors.toList()
));
Here we replace the groupingBy which just takes a classifier, to one that takes a classifier, a supplier and a collector. Essentially this is the same as what there was before, but now it is correctly typed.
You are indeed correct that in the javadoc it is not stated that it will throw a NPE, and I do not think it should be throwing one, as I am allowed to supply whatever map I want, and if my map allows null keys, then it should be allowed.
I do not see any other way to do it simpler as of now, I'll try to look more into it.
You can use Stream#collect(Supplier<R> supplier, BiConsumer<R,? super T> accumulator, BiConsumer<R,R> combiner) instead.
https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#collect-java.util.function.Supplier-java.util.function.BiConsumer-java.util.function.BiConsumer-
When you have a list of objects of a self-defined POJO type:
package code;
import static java.util.Arrays.asList;
import static java.util.stream.Collectors.toList;
import static lombok.AccessLevel.PRIVATE;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Stream;
import lombok.Data;
import lombok.experimental.Accessors;
import lombok.experimental.FieldDefaults;
public class MainGroupListIntoMap {
public static void main(String[] args) throws Exception {
final List<Item> items = Arrays.asList(
new Item().setName("One").setType("1"),
new Item().setName("Two").setType("1"),
new Item().setName("Three").setType("1"),
new Item().setName("Four").setType("2"),
new Item().setName("Same").setType(null),
new Item().setName("Same").setType(null),
new Item().setName(null).setType(null)
);
final Map<String, List<Item>> grouped = items
.stream()
.collect(HashMap::new,
(m, v) -> m.merge(v.getType(),
asList(v),
(oldList, newList) -> Stream.concat(oldList.stream(),
newList.stream())
.collect(toList())),
HashMap::putAll);
grouped.entrySet().forEach(System.out::println);
}
}
#Data
#Accessors(chain = true)
#FieldDefaults(level = PRIVATE)
class Item {
String name;
String type;
}
Output:
null=[Item(name=Same, type=null), Item(name=Same, type=null), Item(name=null, type=null)]
1=[Item(name=One, type=1), Item(name=Two, type=1), Item(name=Three, type=1)]
2=[Item(name=Four, type=2)]
In your case:
package code;
import static java.util.Arrays.asList;
import static java.util.stream.Collectors.toList;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Stream;
public class MainGroupListIntoMap2 {
public static void main(String[] args) throws Exception {
group(asList(ArrayList.class, List.class))
.entrySet()
.forEach(System.out::println);
}
private static Map<Class<?>, List<Class<?>>> group(List<Class<?>> classes) {
final Map<Class<?>, List<Class<?>>> grouped = classes
.stream()
.collect(HashMap::new,
(m, v) -> m.merge(v.getSuperclass(),
asList(v),
(oldList, newList) -> Stream.concat(oldList.stream(),
newList.stream())
.collect(toList())),
HashMap::putAll);
return grouped;
}
}
Output:
null=[interface java.util.List]
class java.util.AbstractList=[class java.util.ArrayList]