Related
In Java 8 how can I filter a collection using the Stream API by checking the distinctness of a property of each object?
For example I have a list of Person object and I want to remove people with the same name,
persons.stream().distinct();
Will use the default equality check for a Person object, so I need something like,
persons.stream().distinct(p -> p.getName());
Unfortunately the distinct() method has no such overload. Without modifying the equality check inside the Person class is it possible to do this succinctly?
Consider distinct to be a stateful filter. Here is a function that returns a predicate that maintains state about what it's seen previously, and that returns whether the given element was seen for the first time:
public static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
Set<Object> seen = ConcurrentHashMap.newKeySet();
return t -> seen.add(keyExtractor.apply(t));
}
Then you can write:
persons.stream().filter(distinctByKey(Person::getName))
Note that if the stream is ordered and is run in parallel, this will preserve an arbitrary element from among the duplicates, instead of the first one, as distinct() does.
(This is essentially the same as my answer to this question: Java Lambda Stream Distinct() on arbitrary key?)
An alternative would be to place the persons in a map using the name as a key:
persons.collect(Collectors.toMap(Person::getName, p -> p, (p, q) -> p)).values();
Note that the Person that is kept, in case of a duplicate name, will be the first encontered.
You can wrap the person objects into another class, that only compares the names of the persons. Afterward, you unwrap the wrapped objects to get a person stream again. The stream operations might look as follows:
persons.stream()
.map(Wrapper::new)
.distinct()
.map(Wrapper::unwrap)
...;
The class Wrapper might look as follows:
class Wrapper {
private final Person person;
public Wrapper(Person person) {
this.person = person;
}
public Person unwrap() {
return person;
}
public boolean equals(Object other) {
if (other instanceof Wrapper) {
return ((Wrapper) other).person.getName().equals(person.getName());
} else {
return false;
}
}
public int hashCode() {
return person.getName().hashCode();
}
}
Another solution, using Set. May not be the ideal solution, but it works
Set<String> set = new HashSet<>(persons.size());
persons.stream().filter(p -> set.add(p.getName())).collect(Collectors.toList());
Or if you can modify the original list, you can use removeIf method
persons.removeIf(p -> !set.add(p.getName()));
There's a simpler approach using a TreeSet with a custom comparator.
persons.stream()
.collect(Collectors.toCollection(
() -> new TreeSet<Person>((p1, p2) -> p1.getName().compareTo(p2.getName()))
));
We can also use RxJava (very powerful reactive extension library)
Observable.from(persons).distinct(Person::getName)
or
Observable.from(persons).distinct(p -> p.getName())
You can use groupingBy collector:
persons.collect(Collectors.groupingBy(p -> p.getName())).values().forEach(t -> System.out.println(t.get(0).getId()));
If you want to have another stream you can use this:
persons.collect(Collectors.groupingBy(p -> p.getName())).values().stream().map(l -> (l.get(0)));
You can use the distinct(HashingStrategy) method in Eclipse Collections.
List<Person> persons = ...;
MutableList<Person> distinct =
ListIterate.distinct(persons, HashingStrategies.fromFunction(Person::getName));
If you can refactor persons to implement an Eclipse Collections interface, you can call the method directly on the list.
MutableList<Person> persons = ...;
MutableList<Person> distinct =
persons.distinct(HashingStrategies.fromFunction(Person::getName));
HashingStrategy is simply a strategy interface that allows you to define custom implementations of equals and hashcode.
public interface HashingStrategy<E>
{
int computeHashCode(E object);
boolean equals(E object1, E object2);
}
Note: I am a committer for Eclipse Collections.
Similar approach which Saeed Zarinfam used but more Java 8 style:)
persons.collect(Collectors.groupingBy(p -> p.getName())).values().stream()
.map(plans -> plans.stream().findFirst().get())
.collect(toList());
You can use StreamEx library:
StreamEx.of(persons)
.distinct(Person::getName)
.toList()
I recommend using Vavr, if you can. With this library you can do the following:
io.vavr.collection.List.ofAll(persons)
.distinctBy(Person::getName)
.toJavaSet() // or any another Java 8 Collection
Extending Stuart Marks's answer, this can be done in a shorter way and without a concurrent map (if you don't need parallel streams):
public static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
final Set<Object> seen = new HashSet<>();
return t -> seen.add(keyExtractor.apply(t));
}
Then call:
persons.stream().filter(distinctByKey(p -> p.getName());
My approach to this is to group all the objects with same property together, then cut short the groups to size of 1 and then finally collect them as a List.
List<YourPersonClass> listWithDistinctPersons = persons.stream()
//operators to remove duplicates based on person name
.collect(Collectors.groupingBy(p -> p.getName()))
.values()
.stream()
//cut short the groups to size of 1
.flatMap(group -> group.stream().limit(1))
//collect distinct users as list
.collect(Collectors.toList());
Distinct objects list can be found using:
List distinctPersons = persons.stream()
.collect(Collectors.collectingAndThen(
Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(Person:: getName))),
ArrayList::new));
I made a generic version:
private <T, R> Collector<T, ?, Stream<T>> distinctByKey(Function<T, R> keyExtractor) {
return Collectors.collectingAndThen(
toMap(
keyExtractor,
t -> t,
(t1, t2) -> t1
),
(Map<R, T> map) -> map.values().stream()
);
}
An exemple:
Stream.of(new Person("Jean"),
new Person("Jean"),
new Person("Paul")
)
.filter(...)
.collect(distinctByKey(Person::getName)) // return a stream of Person with 2 elements, jean and Paul
.map(...)
.collect(toList())
Another library that supports this is jOOλ, and its Seq.distinct(Function<T,U>) method:
Seq.seq(persons).distinct(Person::getName).toList();
Under the hood, it does practically the same thing as the accepted answer, though.
Set<YourPropertyType> set = new HashSet<>();
list
.stream()
.filter(it -> set.add(it.getYourProperty()))
.forEach(it -> ...);
While the highest upvoted answer is absolutely best answer wrt Java 8, it is at the same time absolutely worst in terms of performance. If you really want a bad low performant application, then go ahead and use it. Simple requirement of extracting a unique set of Person Names shall be achieved by mere "For-Each" and a "Set".
Things get even worse if list is above size of 10.
Consider you have a collection of 20 Objects, like this:
public static final List<SimpleEvent> testList = Arrays.asList(
new SimpleEvent("Tom"), new SimpleEvent("Dick"),new SimpleEvent("Harry"),new SimpleEvent("Tom"),
new SimpleEvent("Dick"),new SimpleEvent("Huckle"),new SimpleEvent("Berry"),new SimpleEvent("Tom"),
new SimpleEvent("Dick"),new SimpleEvent("Moses"),new SimpleEvent("Chiku"),new SimpleEvent("Cherry"),
new SimpleEvent("Roses"),new SimpleEvent("Moses"),new SimpleEvent("Chiku"),new SimpleEvent("gotya"),
new SimpleEvent("Gotye"),new SimpleEvent("Nibble"),new SimpleEvent("Berry"),new SimpleEvent("Jibble"));
Where you object SimpleEvent looks like this:
public class SimpleEvent {
private String name;
private String type;
public SimpleEvent(String name) {
this.name = name;
this.type = "type_"+name;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
}
And to test, you have JMH code like this,(Please note, im using the same distinctByKey Predicate mentioned in accepted answer) :
#Benchmark
#OutputTimeUnit(TimeUnit.SECONDS)
public void aStreamBasedUniqueSet(Blackhole blackhole) throws Exception{
Set<String> uniqueNames = testList
.stream()
.filter(distinctByKey(SimpleEvent::getName))
.map(SimpleEvent::getName)
.collect(Collectors.toSet());
blackhole.consume(uniqueNames);
}
#Benchmark
#OutputTimeUnit(TimeUnit.SECONDS)
public void aForEachBasedUniqueSet(Blackhole blackhole) throws Exception{
Set<String> uniqueNames = new HashSet<>();
for (SimpleEvent event : testList) {
uniqueNames.add(event.getName());
}
blackhole.consume(uniqueNames);
}
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(MyBenchmark.class.getSimpleName())
.forks(1)
.mode(Mode.Throughput)
.warmupBatchSize(3)
.warmupIterations(3)
.measurementIterations(3)
.build();
new Runner(opt).run();
}
Then you'll have Benchmark results like this:
Benchmark Mode Samples Score Score error Units
c.s.MyBenchmark.aForEachBasedUniqueSet thrpt 3 2635199.952 1663320.718 ops/s
c.s.MyBenchmark.aStreamBasedUniqueSet thrpt 3 729134.695 895825.697 ops/s
And as you can see, a simple For-Each is 3 times better in throughput and less in error score as compared to Java 8 Stream.
Higher the throughput, better the performance
I would like to improve Stuart Marks answer. What if the key is null, it will through NullPointerException. Here I ignore the null key by adding one more check as keyExtractor.apply(t)!=null.
public static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
Set<Object> seen = ConcurrentHashMap.newKeySet();
return t -> keyExtractor.apply(t)!=null && seen.add(keyExtractor.apply(t));
}
This works like a charm:
Grouping the data by unique key to form a map.
Returning the first object from every value of the map (There could be multiple person having same name).
persons.stream()
.collect(groupingBy(Person::getName))
.values()
.stream()
.flatMap(values -> values.stream().limit(1))
.collect(toList());
The easiest way to implement this is to jump on the sort feature as it already provides an optional Comparator which can be created using an element’s property. Then you have to filter duplicates out which can be done using a statefull Predicate which uses the fact that for a sorted stream all equal elements are adjacent:
Comparator<Person> c=Comparator.comparing(Person::getName);
stream.sorted(c).filter(new Predicate<Person>() {
Person previous;
public boolean test(Person p) {
if(previous!=null && c.compare(previous, p)==0)
return false;
previous=p;
return true;
}
})./* more stream operations here */;
Of course, a statefull Predicate is not thread-safe, however if that’s your need you can move this logic into a Collector and let the stream take care of the thread-safety when using your Collector. This depends on what you want to do with the stream of distinct elements which you didn’t tell us in your question.
There are lot of approaches, this one will also help - Simple, Clean and Clear
List<Employee> employees = new ArrayList<>();
employees.add(new Employee(11, "Ravi"));
employees.add(new Employee(12, "Stalin"));
employees.add(new Employee(23, "Anbu"));
employees.add(new Employee(24, "Yuvaraj"));
employees.add(new Employee(35, "Sena"));
employees.add(new Employee(36, "Antony"));
employees.add(new Employee(47, "Sena"));
employees.add(new Employee(48, "Ravi"));
List<Employee> empList = new ArrayList<>(employees.stream().collect(
Collectors.toMap(Employee::getName, obj -> obj,
(existingValue, newValue) -> existingValue))
.values());
empList.forEach(System.out::println);
// Collectors.toMap(
// Employee::getName, - key (the value by which you want to eliminate duplicate)
// obj -> obj, - value (entire employee object)
// (existingValue, newValue) -> existingValue) - to avoid illegalstateexception: duplicate key
Output - toString() overloaded
Employee{id=35, name='Sena'}
Employee{id=12, name='Stalin'}
Employee{id=11, name='Ravi'}
Employee{id=24, name='Yuvaraj'}
Employee{id=36, name='Antony'}
Employee{id=23, name='Anbu'}
Here is the example
public class PayRoll {
private int payRollId;
private int id;
private String name;
private String dept;
private int salary;
public PayRoll(int payRollId, int id, String name, String dept, int salary) {
super();
this.payRollId = payRollId;
this.id = id;
this.name = name;
this.dept = dept;
this.salary = salary;
}
}
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collector;
import java.util.stream.Collectors;
public class Prac {
public static void main(String[] args) {
int salary=70000;
PayRoll payRoll=new PayRoll(1311, 1, "A", "HR", salary);
PayRoll payRoll2=new PayRoll(1411, 2 , "B", "Technical", salary);
PayRoll payRoll3=new PayRoll(1511, 1, "C", "HR", salary);
PayRoll payRoll4=new PayRoll(1611, 1, "D", "Technical", salary);
PayRoll payRoll5=new PayRoll(711, 3,"E", "Technical", salary);
PayRoll payRoll6=new PayRoll(1811, 3, "F", "Technical", salary);
List<PayRoll>list=new ArrayList<PayRoll>();
list.add(payRoll);
list.add(payRoll2);
list.add(payRoll3);
list.add(payRoll4);
list.add(payRoll5);
list.add(payRoll6);
Map<Object, Optional<PayRoll>> k = list.stream().collect(Collectors.groupingBy(p->p.getId()+"|"+p.getDept(),Collectors.maxBy(Comparator.comparingInt(PayRoll::getPayRollId))));
k.entrySet().forEach(p->
{
if(p.getValue().isPresent())
{
System.out.println(p.getValue().get());
}
});
}
}
Output:
PayRoll [payRollId=1611, id=1, name=D, dept=Technical, salary=70000]
PayRoll [payRollId=1811, id=3, name=F, dept=Technical, salary=70000]
PayRoll [payRollId=1411, id=2, name=B, dept=Technical, salary=70000]
PayRoll [payRollId=1511, id=1, name=C, dept=HR, salary=70000]
Late to the party but I sometimes use this one-liner as an equivalent:
((Function<Value, Key>) Value::getKey).andThen(new HashSet<>()::add)::apply
The expression is a Predicate<Value> but since the map is inline, it works as a filter. This is of course less readable but sometimes it can be helpful to avoid the method.
Building on #josketres's answer, I created a generic utility method:
You could make this more Java 8-friendly by creating a Collector.
public static <T> Set<T> removeDuplicates(Collection<T> input, Comparator<T> comparer) {
return input.stream()
.collect(toCollection(() -> new TreeSet<>(comparer)));
}
#Test
public void removeDuplicatesWithDuplicates() {
ArrayList<C> input = new ArrayList<>();
Collections.addAll(input, new C(7), new C(42), new C(42));
Collection<C> result = removeDuplicates(input, (c1, c2) -> Integer.compare(c1.value, c2.value));
assertEquals(2, result.size());
assertTrue(result.stream().anyMatch(c -> c.value == 7));
assertTrue(result.stream().anyMatch(c -> c.value == 42));
}
#Test
public void removeDuplicatesWithoutDuplicates() {
ArrayList<C> input = new ArrayList<>();
Collections.addAll(input, new C(1), new C(2), new C(3));
Collection<C> result = removeDuplicates(input, (t1, t2) -> Integer.compare(t1.value, t2.value));
assertEquals(3, result.size());
assertTrue(result.stream().anyMatch(c -> c.value == 1));
assertTrue(result.stream().anyMatch(c -> c.value == 2));
assertTrue(result.stream().anyMatch(c -> c.value == 3));
}
private class C {
public final int value;
private C(int value) {
this.value = value;
}
}
Maybe will be useful for somebody. I had a little bit another requirement. Having list of objects A from 3rd party remove all which have same A.b field for same A.id (multiple A object with same A.id in list). Stream partition answer by Tagir Valeev inspired me to use custom Collector which returns Map<A.id, List<A>>. Simple flatMap will do the rest.
public static <T, K, K2> Collector<T, ?, Map<K, List<T>>> groupingDistinctBy(Function<T, K> keyFunction, Function<T, K2> distinctFunction) {
return groupingBy(keyFunction, Collector.of((Supplier<Map<K2, T>>) HashMap::new,
(map, error) -> map.putIfAbsent(distinctFunction.apply(error), error),
(left, right) -> {
left.putAll(right);
return left;
}, map -> new ArrayList<>(map.values()),
Collector.Characteristics.UNORDERED)); }
I had a situation, where I was suppose to get distinct elements from list based on 2 keys.
If you want distinct based on two keys or may composite key, try this
class Person{
int rollno;
String name;
}
List<Person> personList;
Function<Person, List<Object>> compositeKey = personList->
Arrays.<Object>asList(personList.getName(), personList.getRollno());
Map<Object, List<Person>> map = personList.stream().collect(Collectors.groupingBy(compositeKey, Collectors.toList()));
List<Object> duplicateEntrys = map.entrySet().stream()`enter code here`
.filter(settingMap ->
settingMap.getValue().size() > 1)
.collect(Collectors.toList());
A variation of the top answer that handles null:
public static <T, K> Predicate<T> distinctBy(final Function<? super T, K> getKey) {
val seen = ConcurrentHashMap.<Optional<K>>newKeySet();
return obj -> seen.add(Optional.ofNullable(getKey.apply(obj)));
}
In my tests:
assertEquals(
asList("a", "bb"),
Stream.of("a", "b", "bb", "aa").filter(distinctBy(String::length)).collect(toList()));
assertEquals(
asList(5, null, 2, 3),
Stream.of(5, null, 2, null, 3, 3, 2).filter(distinctBy(x -> x)).collect(toList()));
val maps = asList(
hashMapWith(0, 2),
hashMapWith(1, 2),
hashMapWith(2, null),
hashMapWith(3, 1),
hashMapWith(4, null),
hashMapWith(5, 2));
assertEquals(
asList(0, 2, 3),
maps.stream()
.filter(distinctBy(m -> m.get("val")))
.map(m -> m.get("i"))
.collect(toList()));
In my case I needed to control what was the previous element. I then created a stateful Predicate where I controled if the previous element was different from the current element, in that case I kept it.
public List<Log> fetchLogById(Long id) {
return this.findLogById(id).stream()
.filter(new LogPredicate())
.collect(Collectors.toList());
}
public class LogPredicate implements Predicate<Log> {
private Log previous;
public boolean test(Log atual) {
boolean isDifferent = previouws == null || verifyIfDifferentLog(current, previous);
if (isDifferent) {
previous = current;
}
return isDifferent;
}
private boolean verifyIfDifferentLog(Log current, Log previous) {
return !current.getId().equals(previous.getId());
}
}
My solution in this listing:
List<HolderEntry> result ....
List<HolderEntry> dto3s = new ArrayList<>(result.stream().collect(toMap(
HolderEntry::getId,
holder -> holder, //or Function.identity() if you want
(holder1, holder2) -> holder1
)).values());
In my situation i want to find distinct values and put their in List.
I've been working on a class where i encountered Maps and List with very huge Type values.
for e.g.
Map<String, Map<String, Map<String, ...>>> map = new HashMap<>();
and
List<List<Map<String,...>>> list = new ArrayList<>();
looping over such datatypes makes boilerplate code looks very ugly!
for(Map<String, Map<String, ...>> e : map.entrySet()){
//do something...
for(Map<String, ..> e1 : e.entrySet()){
..more such loops
}
}
I came up with a solution to reduce the size of the template using '?' keyword
below is my solution
Map<String, Map<String, Map<String, Long>>> map = ....
for(Entry<String, ?> e : map.entrySet()){
Map<String , ?> mapLevel1 = (Map<String, ?>)e.getValue();
for(Entry<String, ?> e1 : mapLevel1.entrySet()){
Map<String, ?> mapLevel2Map = (Map<String, ?>) e1.getValue();
for(Entry<String, ?> e2 : mapLevel2Map.entrySet()){
Long data = (Long)e2.getValue();
.....
}
}
}
could there be any potential problems in going with this approach ?
Thanks in Anticipation !
? means "some type, but I don't know which" (approximately). Since you do know what the type is, it isn't really suitable.
could there be any potential problems in going with this approach
The casts you need everywhere after getValue (and don't need without ?) are a very significant problem and I wouldn't even call it "potential". If any part of the type changes, good luck finding which casts you need to change and to what.
EDIT: since Java 10, you can just do
for(var e : map.entrySet()){
var mapLevel1 = e.getValue();
for(var e1 : mapLevel1.entrySet()){
var mapLevel2Map = e1.getValue();
for(var e2 : mapLevel2Map.entrySet()){
// or var again
Long data = e2.getValue();
.....
}
}
}
and let the compiler deduce types and check everything makes sense.
As others have pointed out, using ? is worse. Don’t do it.
You should enable all compiler warnings in your IDE (or use -Xlint, if building on the command line). That will inform you that casting to generic types is an unsafe operation.
A good way to keep things clean is to create actual data classes which encapsulate those Maps.
For instance, you might replace this:
Map<String, Map<String, Map<String, Boolean>>> map = new HashMap<>();
with this:
Person person = new Person();
supported by these three classes:
public class Person {
private final Map<String, Address> addressesByType = new HashMap<>();
public Set<String> getAddressTypes() {
return new HashSet<>(addressesByType.keySet());
}
public Address getAddress(String type) {
return addressesByType.get(type);
}
public void addAddress(String type,
Address address) {
Objects.requireNonNull(type);
Objects.requireNonNull(address);
addressesByType.put(type, address);
}
}
and:
public class Address {
public static final String TYPE_HOME = "Home";
public static final String TYPE_WORK = "Work";
private final Map<String, Vehicle> vehiclesByType = new HashMap<>();
public Set<String> getVehicleTypes() {
return new HashSet<>(vehiclesByType.keySet());
}
public Vehicle getVehicle(String type) {
return vehiclesByType.get(type);
}
public void addVehicle(String type,
Vehicle vehicle) {
Objects.requireNonNull(type);
Objects.requireNonNull(vehicle);
vehiclesByType.put(type, vehicle);
}
}
and finally:
public class Vehicle {
public static final String TYPE_PERSONAL = "Personal";
public static final String TYPE_BUSINESS = "Business";
private final Map<String, Boolean> inspectionsByDate = new HashMap<>();
public Set<String> getInspectionDates() {
return inspectionsByDate.keySet();
}
public Boolean getInspectionOutcome(String date) {
return inspectionsByDate.get(date);
}
public void addInspection(String date,
boolean outcome) {
Objects.requireNonNull(date);
inspectionsByDate.put(date, outcome);
}
}
Your loops would then look like this:
for (String addressType : person.getAddressTypes()) {
Address address = person.getAddress(addressType);
for (String vehicleType : address.getVehicleTypes()) {
Vehicle vehicle = address.getVehicle(vehicleType);
for (String date : vehicle.getInspectionDates()) {
boolean outcome = vehicle.getInspectionOutcome(date);
// ...
}
}
}
(The above is just an example. Obviously, in real life, the keys would be enum values, the dates would be LocalDate or Date objects, and people can have more than one address and more than one vehicle for a particular purpose.)
You can encapsulate Lists in a similar manner; see, for example, NodeList.
could there be any potential problems in going with this approach ?
Thanks in Anticipation !
A not readable code and a losing of the generics benefit as you have to cast values returned by Map methods, as if you used a raw type.
In any case, having a so important deep structure may create runtime errors because of potential instantiation missing and makes code complex to read and maintain.
You should improve your design and introduce custom classes that wrap the maps and provide logic methods to add and retrieve data.
You should also consider library as Guava.
Table is for example a good candidate to bring a some abstraction in your manipulated types.
I would like to have a method that maps a List to a NavigableMap. The method call expects an parameter that is used as map key. This parameter is an attribute of the list objects.
Something like this, so both calls are ok:
List<MyObject> list = new ArrayList<>();
NavigableMap<String, MyObject> stringKeyMap = asNavMap(list, MyObject:.getString());
NavigableMap<Date, MyObject> dateKeyMap = asNavMap(list, MyObject::getDate());
I dont know how to define the second parameter (MyObject::getDate()). Do I have to use a lambda expression (p -> p.getDate()) or something like Predicate or Function?
I've tried to derive a solution from Approach 8 (or simular) from http://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html, but I don't know how to do.
This is what I have done so far:
The concrete implementation:
public class ConcreteConverter {
public static NavigableMap<Integer, Pair<Integer, String>> asNavMap(List<Pair<Integer, String>> pairs) {
NavigableMap<Integer, Pair<Integer, String>> navMap = new TreeMap<>();
for (Pair<Integer, String> pair : pairs) {
navMap.put(pair.getKey(), pair);
}
return navMap;
}
public static void main(String[] args) {
List<Pair<Integer, String>> pairs = new ArrayList<>();
pairs.add(new Pair<Integer, String>(1, "one"));
NavigableMap<Integer, Pair<Integer, String>> map = ConcreteConverter.asNavMap(pairs);
}
}
class Pair<K, V> {
K key;
V val;
// constructor, getter, setter
}
Here I stuck (??? is an attribute of the Pair object):
public static <K, V> NavigableMap<K, V> asNavMap(List<V> items, ???) {
NavigableMap<K, V> navMap = new TreeMap<>();
for (V item : items) {
navMap.put(???, item);
}
return navMap;
}
Please notice I have barely experiences writing generic methods or using lambda functions/interfaces.
Any help is appreciated.
Edit 1
As Nick Vanderhofen mentioned I didn't clarify the search for a generic solution.
You can do that with a Function. You keep the code you wanted:
List<MyObject> list = new ArrayList<>();
NavigableMap<String, MyObject> stringKeyMap = asNavMap(list, MyObject::getKey);
The method asNavMap can then take a Function:
private NavigableMap<String,MyObject> asNavMap(List<MyObject> list, Function<MyObject, String> getKey) {
//the actual mapping goes here
}
The getKey method you are specifying can either be a simple getter on the MyObject:
public String getKey(){
return key;
}
Or you could create a static method to get the same result:
public static String getKey(MyObject myObject){
return myObject.getKey();
}
To apply the function you can just use the apply method:
String key = getKey.apply(someObject);
For the actual mapping implementation you can keep your for loop, or you could rewrite it using java 8 and re-use the Function that you got as a parameter in the collector. However, since you want a TreeMap, the syntax is quite verbose:
items.stream().collect(Collectors.toMap(mapper, Function.identity(), (a,b) -> a, TreeMap::new));
Just figured out a working solution!
Still reading http://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html#approach7 I've tried to use Function, and now this is my solution:
public static <K, V> NavigableMap<K, V> asNavigableMap(List<V> items, Function<V, K> mapper) {
NavigableMap<K, V> navMap = new TreeMap<>();
for (V item : items)
navMap.put(mapper.apply(item), item);
return navMap;
}
And these calls work:
List<Pair<Integer, String>> pairs = new ArrayList<>();
pairs.add(new Pair<Integer, String>(1, "one"));
NavigableMap<Integer, Pair<Integer, String>> navI2P1 = GenericConverter.asNavigableMap(pairs, Pair::getKey);
NavigableMap<String, Pair<Integer, String>> navI2P2 = GenericConverter.asNavigableMap(pairs, Pair::getVal);
It was hard for me to understand the Function functional interface and the apply method.
Thanks to anyone!
In Java 8, this works:
Stream<Class> stream = Stream.of(ArrayList.class);
HashMap<Class, List<Class>> map = (HashMap)stream.collect(Collectors.groupingBy(Class::getSuperclass));
But this doesn't:
Stream<Class> stream = Stream.of(List.class);
HashMap<Class, List<Class>> map = (HashMap)stream.collect(Collectors.groupingBy(Class::getSuperclass));
Maps allows a null key, and List.class.getSuperclass() returns null. But Collectors.groupingBy emits a NPE, at Collectors.java, line 907:
K key = Objects.requireNonNull(classifier.apply(t), "element cannot be mapped to a null key");
It works if I create my own collector, with this line changed to:
K key = classifier.apply(t);
My questions are:
1) The Javadoc of Collectors.groupingBy doesn't say it shouldn't map a null key. Is this behavior necessary for some reason?
2) Is there another, easier way, to accept a null key, without having to create my own collector?
I had the same kind of problem.
This failed, because groupingBy performs Objects.requireNonNull on the value returned from the classifier:
Map<Long, List<ClaimEvent>> map = events.stream()
.filter(event -> eventTypeIds.contains(event.getClaimEventTypeId()))
.collect(groupingBy(ClaimEvent::getSubprocessId));
Using Optional, this works:
Map<Optional<Long>, List<ClaimEvent>> map = events.stream()
.filter(event -> eventTypeIds.contains(event.getClaimEventTypeId()))
.collect(groupingBy(event -> Optional.ofNullable(event.getSubprocessId())));
For the first question, I agree with skiwi that it shouldn't be throwing a NPE. I hope they will change that (or else at least add it to the javadoc). Meanwhile, to answer the second question I decided to use Collectors.toMap instead of Collectors.groupingBy:
Stream<Class<?>> stream = Stream.of(ArrayList.class);
Map<Class<?>, List<Class<?>>> map = stream.collect(
Collectors.toMap(
Class::getSuperclass,
Collections::singletonList,
(List<Class<?>> oldList, List<Class<?>> newEl) -> {
List<Class<?>> newList = new ArrayList<>(oldList.size() + 1);
newList.addAll(oldList);
newList.addAll(newEl);
return newList;
}));
Or, encapsulating it:
/** Like Collectors.groupingBy, but accepts null keys. */
public static <T, A> Collector<T, ?, Map<A, List<T>>>
groupingBy_WithNullKeys(Function<? super T, ? extends A> classifier) {
return Collectors.toMap(
classifier,
Collections::singletonList,
(List<T> oldList, List<T> newEl) -> {
List<T> newList = new ArrayList<>(oldList.size() + 1);
newList.addAll(oldList);
newList.addAll(newEl);
return newList;
});
}
And use it like this:
Stream<Class<?>> stream = Stream.of(ArrayList.class);
Map<Class<?>, List<Class<?>>> map = stream.collect(groupingBy_WithNullKeys(Class::getSuperclass));
Please note rolfl gave another, more complicated answer, which allows you to specify your own Map and List supplier. I haven't tested it.
Use filter before groupingBy##
Filter out the null instances before groupingBy.
Here is an example
MyObjectlist.stream()
.filter(p -> p.getSomeInstance() != null)
.collect(Collectors.groupingBy(MyObject::getSomeInstance));
To your 1st question, from the docs:
There are no guarantees on the type, mutability, serializability, or thread-safety of the Map or List objects returned.
Because not all Map implementations allow null keys they probably added this to reduce to the most common allowable definition of a map to get maximum flexibility when choosing a type.
To your 2nd question, you just need a supplier, wouldn't a lambda work? I'm still getting acquainted with Java 8, maybe a smarter person can add a better answer.
I figured I would take a moment and try to digest this issue you have. I put together a SSCE for what I would expect if I did it manually, and what the groupingBy implementation actually does.
I don't think this is an answer, but it is a 'wonder why it is a problem' thing. Also, if you want, feel free to hack this code to have a null-friendly collector.
Edit: A generic-friendly implementation:
/** groupingByNF - NullFriendly - allows you to specify your own Map and List supplier. */
private static final <T,K> Collector<T,?,Map<K,List<T>>> groupingByNF (
final Supplier<Map<K,List<T>>> mapsupplier,
final Supplier<List<T>> listsupplier,
final Function<? super T,? extends K> classifier) {
BiConsumer<Map<K,List<T>>, T> combiner = (m, v) -> {
K key = classifier.apply(v);
List<T> store = m.get(key);
if (store == null) {
store = listsupplier.get();
m.put(key, store);
}
store.add(v);
};
BinaryOperator<Map<K, List<T>>> finalizer = (left, right) -> {
for (Map.Entry<K, List<T>> me : right.entrySet()) {
List<T> target = left.get(me.getKey());
if (target == null) {
left.put(me.getKey(), me.getValue());
} else {
target.addAll(me.getValue());
}
}
return left;
};
return Collector.of(mapsupplier, combiner, finalizer);
}
/** groupingByNF - NullFriendly - otherwise similar to Java8 Collections.groupingBy */
private static final <T,K> Collector<T,?,Map<K,List<T>>> groupingByNF (Function<? super T,? extends K> classifier) {
return groupingByNF(HashMap::new, ArrayList::new, classifier);
}
Consider this code (the code groups String values based on the String.length(), (or null if the input String is null)):
public static void main(String[] args) {
String[] input = {"a", "a", "", null, "b", "ab"};
// How we group the Strings
final Function<String, Integer> classifier = (a) -> {return a != null ? Integer.valueOf(a.length()) : null;};
// Manual implementation of a combiner that accumulates a string value based on the classifier.
// no special handling of null key values.
BiConsumer<Map<Integer,List<String>>, String> combiner = (m, v) -> {
Integer key = classifier.apply(v);
List<String> store = m.get(key);
if (store == null) {
store = new ArrayList<String>();
m.put(key, store);
}
store.add(v);
};
// The finalizer merges two maps together (right into left)
// no special handling of null key values.
BinaryOperator<Map<Integer, List<String>>> finalizer = (left, right) -> {
for (Map.Entry<Integer, List<String>> me : right.entrySet()) {
List<String> target = left.get(me.getKey());
if (target == null) {
left.put(me.getKey(), me.getValue());
} else {
target.addAll(me.getValue());
}
}
return left;
};
// Using a manual collector
Map<Integer, List<String>> manual = Arrays.stream(input).collect(Collector.of(HashMap::new, combiner, finalizer));
System.out.println(manual);
// using the groupingBy collector.
Collector<String, ?, Map<Integer, List<String>>> collector = Collectors.groupingBy(classifier);
Map<Integer, List<String>> result = Arrays.stream(input).collect(collector);
System.out.println(result);
}
The above code produces the output:
{0=[], null=[null], 1=[a, a, b], 2=[ab]}
Exception in thread "main" java.lang.NullPointerException: element cannot be mapped to a null key
at java.util.Objects.requireNonNull(Objects.java:228)
at java.util.stream.Collectors.lambda$groupingBy$135(Collectors.java:907)
at java.util.stream.Collectors$$Lambda$10/258952499.accept(Unknown Source)
at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at CollectGroupByNull.main(CollectGroupByNull.java:49)
First of all, you are using lots of raw objects. This is not a good idea at all, first convert the following:
Class to Class<?>, ie. instead of a raw type, a parametrized type with an unknown class.
Instead of forcefully casting to a HashMap, you should supply a HashMap to the collector.
First the correctly typed code, without caring about a NPE yet:
Stream<Class<?>> stream = Stream.of(ArrayList.class);
HashMap<Class<?>, List<Class<?>>> hashMap = (HashMap<Class<?>, List<Class<?>>>)stream
.collect(Collectors.groupingBy(Class::getSuperclass));
Now we get rid of the forceful cast there, and instead do it correctly:
Stream<Class<?>> stream = Stream.of(ArrayList.class);
HashMap<Class<?>, List<Class<?>>> hashMap = stream
.collect(Collectors.groupingBy(
Class::getSuperclass,
HashMap::new,
Collectors.toList()
));
Here we replace the groupingBy which just takes a classifier, to one that takes a classifier, a supplier and a collector. Essentially this is the same as what there was before, but now it is correctly typed.
You are indeed correct that in the javadoc it is not stated that it will throw a NPE, and I do not think it should be throwing one, as I am allowed to supply whatever map I want, and if my map allows null keys, then it should be allowed.
I do not see any other way to do it simpler as of now, I'll try to look more into it.
You can use Stream#collect(Supplier<R> supplier, BiConsumer<R,? super T> accumulator, BiConsumer<R,R> combiner) instead.
https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#collect-java.util.function.Supplier-java.util.function.BiConsumer-java.util.function.BiConsumer-
When you have a list of objects of a self-defined POJO type:
package code;
import static java.util.Arrays.asList;
import static java.util.stream.Collectors.toList;
import static lombok.AccessLevel.PRIVATE;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Stream;
import lombok.Data;
import lombok.experimental.Accessors;
import lombok.experimental.FieldDefaults;
public class MainGroupListIntoMap {
public static void main(String[] args) throws Exception {
final List<Item> items = Arrays.asList(
new Item().setName("One").setType("1"),
new Item().setName("Two").setType("1"),
new Item().setName("Three").setType("1"),
new Item().setName("Four").setType("2"),
new Item().setName("Same").setType(null),
new Item().setName("Same").setType(null),
new Item().setName(null).setType(null)
);
final Map<String, List<Item>> grouped = items
.stream()
.collect(HashMap::new,
(m, v) -> m.merge(v.getType(),
asList(v),
(oldList, newList) -> Stream.concat(oldList.stream(),
newList.stream())
.collect(toList())),
HashMap::putAll);
grouped.entrySet().forEach(System.out::println);
}
}
#Data
#Accessors(chain = true)
#FieldDefaults(level = PRIVATE)
class Item {
String name;
String type;
}
Output:
null=[Item(name=Same, type=null), Item(name=Same, type=null), Item(name=null, type=null)]
1=[Item(name=One, type=1), Item(name=Two, type=1), Item(name=Three, type=1)]
2=[Item(name=Four, type=2)]
In your case:
package code;
import static java.util.Arrays.asList;
import static java.util.stream.Collectors.toList;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Stream;
public class MainGroupListIntoMap2 {
public static void main(String[] args) throws Exception {
group(asList(ArrayList.class, List.class))
.entrySet()
.forEach(System.out::println);
}
private static Map<Class<?>, List<Class<?>>> group(List<Class<?>> classes) {
final Map<Class<?>, List<Class<?>>> grouped = classes
.stream()
.collect(HashMap::new,
(m, v) -> m.merge(v.getSuperclass(),
asList(v),
(oldList, newList) -> Stream.concat(oldList.stream(),
newList.stream())
.collect(toList())),
HashMap::putAll);
return grouped;
}
}
Output:
null=[interface java.util.List]
class java.util.AbstractList=[class java.util.ArrayList]
So I have the following HashMap:
HashMap<String, List<someDataType>> map;
I want to create a new HashMap that is only composed of the k/v pairs in map that have a value (the list) whose length is less than a certain "x". The only way I know how to do this is to iterate through the HashMap and put k/v pairs into a new HashMap. Is there a more concise way to achieve what I'm looking for? Thanks.
Using guava:
Map<String, List<String>> newMap =
Maps.filterEntries(originalMap, new MyEntryPredicate(10));
where:
private static class MyEntryPredicate implements Predicate<Map.Entry<String, List<String>>> {
// max list length, exclusive
private int maxLength;
private MyEntryPredicate(int maxLength) {
this.maxLength = maxLength;
}
#Override
public boolean apply(Map.Entry<String, List<String>> input) {
return input != null && input.getValue().size() < maxLength;
}
}
If the Guava library is available to your project, you could use Maps.filterValues (somewhat echoing Keith's answer):
final int x = 42;
Map<String, List<String>> filteredMap =
Maps.filterValues(map, new Predicate<Collection<?>>() {
#Override
public boolean apply(final Collection<?> collection) {
return collection.size() < x;
}
});
Map<String, List<String>> filteredMapCopy = ImmutableMap.copyOf(filteredMap);
Note the need for a copy because filterValues returns a filtered view of the original map.
Update: with Java 8 you can simplify the predicate to a lambda expression:
Map<String, List<String>> filteredMap = Maps.filterValues(map, list -> list.size() < x);
Nowadays (Java 8+) this could be done with streams:
Predicate<Map.Entry<String, List<String>>> test = entry -> entry.getValue().size() <= x; // note this is java.util.function.Predicate
Map<String, List<String>> filteredMap = map.entrySet().stream().filter(test)
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
This helps to avoid the dependency to guava which might be undesired.
You may want to look at the Guava library from Google. There's an enormous number of Collections and Map related utils in there, which let you do complex stuff quite concisely. An example of what you can do is:
Iterable<Long> list =
Iterables.limit(
Iterables.filter(
Ordering.natural()
.reverse()
.onResultOf(new Function<Long, Integer>() {
public Integer apply(Long id) {
return // result of this is for sorting purposes
}
})
.sortedCopy(
Multisets.intersection(set1, set2)),
new Predicate<Long>() {
public boolean apply(Long id) {
return // whether to filter this id
}
}), limit);
I'm sure you can find something in there which can do what you're looking for :-)
Going along with the other Guava examples, you can use Guava's MultiMaps:
final MultiMap<K, V> mmap = ArrayListMultiMap.create();
// do stuff.
final int limit = 10;
final MultiMap<K, V> mmapView =
MultiMaps.filterKeys(mmap, new Predicate<K>(){
public boolean apply(K k) {
return mmap.get(k).size() <= limit;
}
});
The MultiMaps.newListMultiMap method takes arguments you don't want to provide. You can't use MultiMaps.filterValues or .filterEntries here because those use the individual values, not the lists of values. On the other hand, mmap.get(k) never returns null. You cam, of course, use a static inner class that you pass mmap and limit to instead of using anonymous inner classes.
Alternatevely you can make a copy of the original map and iterate over the values removing those whose length is less than x.