Develop Reference

Java8 Stream List<Map<String,Object>> groupingBy and counting value

i want get result like { "key1" : 4 ,"key2" :2 }
i known i can use map and groupby and such as
list.stream()
.map(map -> map2Entity(map))
.collect(Collectors.groupingBy(Entity::getKey,Collectors.summarizingInt(Entity::getCnt)) )
This is my code and how implements (todo) code
public void test() {
List<Map<String, Object>> list = Arrays.asList(
createNewMap("key1", 1),
createNewMap("key2", 2),
createNewMap("key1", 3)
);
// i want get result like {"key1":4,"key2":2}
// how can i get the result don't use map()
list.stream()
.collect(Collectors.groupingBy(this::getKey),....(todo));
}
private String getKey(Map<String,Object> map){
return (String) map.get("key");
}
private Map<String, Object> createNewMap(String key, Integer val) {
Map<String, Object> map = new HashMap<>();
map.put("key", key);
map.put(key, val);
return map;
}

You have to use the flatMap operator along with the groupingBy collector. Here's how it looks.
Map<String, Integer> keyToSumValuesMap = list.stream()
.flatMap(m -> m.entrySet().stream())
.collect(Collectors.groupingBy(Map.Entry::getKey,
Collectors.summingInt(Map.Entry::getValue)));
Moreover do not use Object type to represent integers since that is not type safe. Consider declaring method level generics to overcome that issue. Here's how it looks.
private static <S, T> Map<S, T> createNewMap(S key, T val) {
Map<S, T> map = new HashMap<>();
map.put(key, val);
return map;
}
And the output now looks like this:
{key1=4, key2=2}

Java 8 stream to collect a Map of List of items

I have a List of Maps which stores the roles and the names of people. For ex:
List<Map<String, String>> listOfData
1) Role: Batsman
Name: Player1
2)Role: Batsman
Name: Player2
3)Role: Bowler
Name: Player3
Role and Name are the Keys of the map.
I want to convert this into a Map<String, List<String>> result, which will give me a list of names for each role, i.e
k1: Batsman v1: [Player1, Player2]
k2: Bowler v2: [Player3]
listOfData
.stream()
.map(entry -> new AbstractMap.SimpleEntry<>(entry.get("Role"), entry.get("Name"))
.collect(Collectors.toList());
Doing this way will not give me a list of names for the role, it will give me a single name. How do i keep collecting the elements of the list and then add it to a key ?
Java code to create base structure:
Map<String, String> x1 = ImmutableMap.of("Role", "Batsman", "Name", "Player1");
Map<String, String> y1 = ImmutableMap.of("Role", "Batsman", "Name", "Player2");
Map<String, String> z1 = ImmutableMap.of("Role", "Bowler", "Name", "Player3");
List<Map<String, String>> list = ImmutableList.of(x1, y1, z1);
Map<String, List<String>> z = list.stream()
.flatMap(e -> e.entrySet().stream())
.collect(Collectors.groupingBy(Map.Entry::getKey,
Collectors.mapping(Map.Entry::getValue, Collectors.toList())));

listOfData.stream()
.flatMap(e -> e.entrySet().stream())
.collect(Collectors.groupingBy(Map.Entry::getKey,
Collectors.mapping(Map.Entry::getValue,
Collectors.toList())));
update:
Slightly different variant to user1692342's answer for completeness.
list.stream()
.map(e -> Arrays.asList(e.get("Role"), e.get("Name")))
.collect(Collectors.groupingBy(e -> e.get(0),
Collectors.mapping(e -> e.get(1), Collectors.toList())));

Based on the idea given by Aomine:
list.stream()
.map(e -> new AbstractMap.SimpleEntry<>(e.get("Role"), e.get("Name")))
.collect(Collectors.groupingBy(Map.Entry::getKey,
Collectors.mapping(Map.Entry::getValue, Collectors.toList())));

The Collectors class provides convenience methods in the form of i.e. groupingBy which allow to group similar objects by a certain classifier. The classifier method is the input to that particular grouping function. This function will generate a Map with the respective classifier methods value as key and a list of objects that share the same classifier method value as value.
Therefore a code like
Map<String, List<Person>> roles2Persons =
lis.stream().collect(Collectors.groupingBy(Person::getRole));
will generate a mapping for the respective roles Person objects may fulfill to a list of Person objects that share the same role as the key in the map.
After the above collector was applied the resulting map will contain the desired form of
key1: Batsman, values: List(Player1, Player2)
key2: Bowler, values: List(Player3)

Here is a generic approach to this problem. The groupBy method below takes 2 to 3 arguments.
A collection of type List<E>
A keyFn (key) of type Function<E, K>
(Optional) A valueFn (value) of type Function<E, V> or simply E
I included some unit tests below.
CollectionUtils.java
package org.example.util;
import static java.util.Collections.unmodifiableMap;
import static java.util.stream.Collectors.*;
import java.util.*;
import java.util.AbstractMap.SimpleEntry;
import java.util.Map.Entry;
import java.util.function.Function;
public class CollectionUtils {
public static <E, K, V> Map<K, List<V>> groupBy(
Collection<E> collection, Function<E, K> keyFn, Function<E, V> valueFn) {
return collection.stream()
.map(item -> new SimpleEntry<K, V>(keyFn.apply(item), valueFn.apply(item)))
.collect(groupingBy(Entry::getKey, mapping(Entry::getValue, toList())));
}
public static <E, K> Map<K, List<E>> groupBy(Collection<E> collection, Function<E, K> keyFn) {
return groupBy(collection, keyFn, Function.identity());
}
public static <K, V> Map<K, V> immutableMapOf(K k1, V v1, K k2, V v2) {
Map<K, V> mutableMap = new HashMap<>();
mutableMap.put(k1, v1);
mutableMap.put(k2, v2);
return unmodifiableMap(mutableMap);
}
}
CollectionUtilsTest.java
package org.example.util;
import static java.util.Arrays.asList;
import static org.example.util.CollectionUtils.*;
import static org.junit.Assert.*;
import java.util.*;
import org.junit.Test;
import org.slf4j.*;
public class CollectionUtilsTest {
private static final Logger logger = LoggerFactory.getLogger(CollectionUtilsTest.class);
private static final List<Map<String, String>> data =
asList(
immutableMapOf("Username", "Batman", "Role", "Leader"),
immutableMapOf("Username", "Robin", "Role", "Subordinate"),
immutableMapOf("Username", "Superman", "Role", "Leader"));
#Test
public void testGroupBy() {
logger.info("Test groupBy(Collection<E>, Function<E, K>, Function<E, V>)");
Map<String, List<String>> grouped = groupBy(data, m -> m.get("Role"), m -> m.get("Username"));
logger.info("Checking keys...");
assertNotNull(grouped.get("Leader"));
assertNotNull(grouped.get("Subordinate"));
logger.info("Checking values...");
assertEquals("Batman", grouped.get("Leader").get(0));
assertEquals("Superman", grouped.get("Leader").get(1));
assertEquals("Robin", grouped.get("Subordinate").get(0));
}
#Test
public void testGroupBySimple() {
logger.info("Test groupBy(Collection<E>, Function<E, K>)");
Map<String, List<Map<String, String>>> grouped = groupBy(data, m -> m.get("Role"));
logger.info("Checking keys...");
assertNotNull(grouped.get("Leader"));
assertNotNull(grouped.get("Subordinate"));
logger.info("Checking values...");
assertEquals("Batman", grouped.get("Leader").get(0).get("Username"));
assertEquals("Superman", grouped.get("Leader").get(1).get("Username"));
assertEquals("Robin", grouped.get("Subordinate").get(0).get("Username"));
}
}

Merge list of maps using Java 8 Stream API

I'm trying to learn Java 8 Stream and when I try to convert some function to java8 to practice. I meet a problem.
I'm curious that how can I convert follow code to java stream format.
/*
* input example:
* [
{
"k1": { "kk1": 1, "kk2": 2},
"k2": {"kk1": 3, "kk2": 4}
}
{
"k1": { "kk1": 10, "kk2": 20},
"k2": {"kk1": 30, "kk2": 40}
}
]
* output:
* {
"k1": { "kk1": 11, "kk2": 22},
"k2": {"kk1": 33, "kk2": 44}
}
*
*
*/
private static Map<String, Map<String, Long>> mergeMapsValue(List<Map<String, Map<String, Long>>> valueList) {
Set<String> keys_1 = valueList.get(0).keySet();
Set<String> keys_2 = valueList.get(0).entrySet().iterator().next().getValue().keySet();
Map<String, Map<String, Long>> result = new HashMap<>();
for (String k1: keys_1) {
result.put(k1, new HashMap<>());
for (String k2: keys_2) {
long total = 0;
for (Map<String, Map<String, Long>> mmap: valueList) {
Map<String, Long> m = mmap.get(k1);
if (m != null && m.get(k2) != null) {
total += m.get(k2);
}
}
result.get(k1).put(k2, total);
}
}
return result;
}

The trick here is to collect correctly the inner maps. The workflow would be:
Flat map the list of map List<Map<String, Map<String, Long>>> into a stream of map entries Stream<Map.Entry<String, Map<String, Long>>>.
Group by the key of each of those entry, and for the values mapped to same key, merge the two maps together.
Collecting maps by merging them would ideally warrant a flatMapping collector, which unfortunately doesn't exist in Java 8, although it will exist in Java 9 (see JDK-8071600). For Java 8, it is possible to use the one provided by the StreamEx library (and use MoreCollectors.flatMapping in the following code).
private static Map<String, Map<String, Long>> mergeMapsValue(List<Map<String, Map<String, Long>>> valueList) {
return valueList.stream()
.flatMap(e -> e.entrySet().stream())
.collect(Collectors.groupingBy(
Map.Entry::getKey,
Collectors.flatMapping(
e -> e.getValue().entrySet().stream(),
Collectors.<Map.Entry<String,Long>,String,Long>toMap(Map.Entry::getKey, Map.Entry::getValue, Long::sum)
)
));
}
Without using this convenient collector, we can still build our own with equivalent semantics:
private static Map<String, Map<String, Long>> mergeMapsValue2(List<Map<String, Map<String, Long>>> valueList) {
return valueList.stream()
.flatMap(e -> e.entrySet().stream())
.collect(Collectors.groupingBy(
Map.Entry::getKey,
Collector.of(
HashMap::new,
(r, t) -> t.getValue().forEach((k, v) -> r.merge(k, v, Long::sum)),
(r1, r2) -> { r2.forEach((k, v) -> r1.merge(k, v, Long::sum)); return r1; }
)
));
}

As a starting point, converting to use computeIfAbsent and merge gives us the following:
private static <K1, K2> Map<K1, Map<K2, Long>> mergeMapsValue(List<Map<K1, Map<K2, Long>>> valueList) {
final Map<K1, Map<K2, Long>> result = new HashMap<>();
for (final Map<K1, Map<K2, Long>> map : valueList) {
for (final Map.Entry<K1, Map<K2, Long>> sub : map.entrySet()) {
for (final Map.Entry<K2, Long> subsub : sub.getValue().entrySet()) {
result.computeIfAbsent(sub.getKey(), k1 -> new HashMap<>())
.merge(subsub.getKey(), subsub.getValue(), Long::sum);
}
}
}
return result;
}
This removes much of the logic from your inner loop.
This code below is wrong, I leave it here for reference.
Converting to the Stream API is not going to make it neater, but lets give it a go.
import static java.util.stream.Collectors.collectingAndThen;
import static java.util.stream.Collectors.groupingBy;
import static java.util.stream.Collectors.mapping;
import static java.util.stream.Collectors.toList;
private static <K1, K2> Map<K1, Map<K2, Long>> mergeMapsValue(List<Map<K1, Map<K2, Long>>> valueList) {
return valueList.stream()
.flatMap(v -> v.entrySet().stream())
.collect(groupingBy(Entry::getKey, collectingAndThen(mapping(Entry::getValue, toList()), l -> l.stream()
.reduce(new HashMap<>(), (l2, r2) -> {
r2.forEach((k, v) -> l2.merge(k, v, Long::sum);
return l2;
}))));
}
This is what I've managed to come up with - it's horrible. The problem is that with the foreach approach, you have a reference to each level of the iteration - this makes the logic simple. With the functional approach, you need to consider each folding operation separately.
How does it work?
We first stream() our List<Map<K1, Map<K2, Long>>>, giving a Stream<Map<K1, Map<K2, Long>>>. Next we flatMap each element, giving a Stream<Entry<K1, Map<K2, Long>>> - so we flatten the first dimension. But we cannot flatten further as we need to K1 value.
So we then use collect(groupingBy) on the K1 value giving us a Map<K1, SOMETHING> - what is something?
Well, first we use a mapping(Entry::getValue, toList()) to give us a Map<K1, List<Map<K2, Long>>>. We then use collectingAndThen to take that List<Map<K2, Long>> and reduce it. Note that this means we produce an intermediate List, which is wasteful - you could get around this by using a custom Collector.
For this we use List.stream().reduce(a, b) where a is the initial value and b is the "fold" operation. a is set to new HashMap<>() and b takes two values: either the initial value or the result of the previous application of the function and the current item in the List. So we, for each item in the List use Map.merge to combine the values.
I would say that this approach is more or less illegible - you won't be able to decipher it in a few hours time, let alone a few days.

I took the flatMap(e -> e.entrySet().stream()) part from Tunaki, but used a shorter variant for the collector:
Map<String, Integer> merged = maps.stream()
.flatMap(map -> map.entrySet().stream())
.collect(Collectors.toMap(
Map.Entry::getKey, Map.Entry::getValue, Integer::sum));
More elaborate example:
Map<String, Integer> a = new HashMap<String, Integer>() {{
put("a", 2);
put("b", 5);
}};
Map<String, Integer> b = new HashMap<String, Integer>() {{
put("a", 7);
}};
List<Map<String, Integer>> maps = Arrays.asList(a, b);
Map<String, Integer> merged = maps.stream()
.flatMap(map -> map.entrySet().stream())
.collect(Collectors.toMap(
Map.Entry::getKey, Map.Entry::getValue, Integer::sum));
assert merged.get("a") == 9;
assert merged.get("b") == 5;

Reducing list of two level maps to a single two level map using java 8 stream operators

I have a list of nested maps (List<Map<String, Map<String, Long>>>) and the goal is to reduce the list to a single map, and the merging is to be done as follow: if map1 contains x->{y->10, z->20} and map2 contains x->{y->20, z->20} then these two should be merged to x->{y->30, z->40}.
I tried to do it as follow and this is working fine.
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.function.BinaryOperator;
import java.util.stream.Collectors;
public class Test {
public static void main(String args[]) throws IOException {
Map<String, Map<String, Long>> data1 = new HashMap<>();
Map<String, Long> innerData1 = new HashMap<>();
innerData1.put("a", 10L);
innerData1.put("b", 20L);
data1.put("x", innerData1);
Map<String, Long> innerData2 = new HashMap<>();
innerData2.put("b", 20L);
innerData2.put("a", 10L);
data1.put("x", innerData1);
Map<String, Map<String, Long>> data2 = new HashMap<>();
data2.put("x", innerData2);
List<Map<String, Map<String, Long>>> mapLists = new ArrayList<>();
mapLists.add(data1);
mapLists.add(data2);
Map<String, Map<String, Long>> result = mapLists.stream().flatMap(map -> map.entrySet().stream()).
collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue, new BinaryOperator<Map<String, Long>>() {
#Override
public Map<String, Long> apply(Map<String, Long> t,
Map<String, Long> u) {
Map<String, Long> result = t;
for(Entry<String, Long> entry: u.entrySet()) {
Long val = t.getOrDefault(entry.getKey(), 0L);
result.put(entry.getKey(), val + entry.getValue());
}
return result;
}
}));
}
}
Is there any other better and efficient approach to solve this?
How to do it more cleanly if the nesting level is more than 2? Suppose the List is like List<Map<String, Map<String, Map<String, Long>>>> and we have to reduce it to a single Map<String, Map<String, Map<String, Long>>>, assuming similar merge functionality as above.

You have the general idea, it is just possible to simplify a little the process of merging two maps together. Merging the two maps can be done easily with:
Map<String, Integer> mx = new HashMap<>(m1);
m2.forEach((k, v) -> mx.merge(k, v, Long::sum));
This code creates the merged map mx from m1, then iterates over all entries of the second map m2 and merges each entry into mx with the help of Map.merge(key, value, remappingFunction): this method will add the given key with the given value if no mapping existed for that key, else it will remap the existing value for that key and the given value with the given remapping function. In our case, the remapping function should sum the two values together.
Code:
Map<String, Map<String, Long>> result =
mapLists.stream()
.flatMap(m -> m.entrySet().stream())
.collect(Collectors.toMap(
Map.Entry::getKey,
Map.Entry::getValue,
(m1, m2) -> {
Map<String, Long> mx = new HashMap<>(m1);
m2.forEach((k, v) -> mx.merge(k, v, Long::sum));
return mx;
}
));
If there are more "levels", you could define a merge method:
private static <K, V> Map<K, V> merge(Map<K, V> m1, Map<K, V> m2, BiFunction<? super V, ? super V, ? extends V> remappingFunction) {
Map<K, V> mx = new HashMap<>(m1);
m2.forEach((k, v) -> mx.merge(k, v, remappingFunction));
return mx;
}
and use it recursively. For example, to merge two Map<String, Map<String, Long>> m1 and m2, you could use
merge(m1, m2, (a, b) -> merge(a, b, Long::sum));
as the remapping function is Collectors.toMap.

Using my StreamEx library:
Map<String, Map<String, Long>> result = StreamEx.of(mapLists)
.flatMapToEntry(m -> m)
.toMap((m1, m2) -> EntryStream.of(m1).append(m2).toMap(Long::sum));
The flatMapToEntry intermediate operation flattens maps into EntryStream<String, Map<String, Long>> which extends Stream<Map.Entry<String, Map<String, Long>>>. The toMap terminal operation just creates a map from the stream of entries using the supplied merge function. To merge two maps we use EntryStream again.

In Java 8 how do I transform a Map<K,V> to another Map<K,V> using a lambda?

I've just started looking at Java 8 and to try out lambdas I thought I'd try to rewrite a very simple thing I wrote recently. I need to turn a Map of String to Column into another Map of String to Column where the Column in the new Map is a defensive copy of the Column in the first Map. Column has a copy constructor. The closest I've got so far is:
Map<String, Column> newColumnMap= new HashMap<>();
originalColumnMap.entrySet().stream().forEach(x -> newColumnMap.put(x.getKey(), new Column(x.getValue())));
but I'm sure there must be a nicer way to do it and I'd be grateful for some advice.

You could use a Collector:
import java.util.*;
import java.util.stream.Collectors;
public class Defensive {
public static void main(String[] args) {
Map<String, Column> original = new HashMap<>();
original.put("foo", new Column());
original.put("bar", new Column());
Map<String, Column> copy = original.entrySet()
.stream()
.collect(Collectors.toMap(Map.Entry::getKey,
e -> new Column(e.getValue())));
System.out.println(original);
System.out.println(copy);
}
static class Column {
public Column() {}
public Column(Column c) {}
}
}

Map<String, Integer> map = new HashMap<>();
map.put("test1", 1);
map.put("test2", 2);
Map<String, Integer> map2 = new HashMap<>();
map.forEach(map2::put);
System.out.println("map: " + map);
System.out.println("map2: " + map2);
// Output:
// map: {test2=2, test1=1}
// map2: {test2=2, test1=1}
You can use the forEach method to do what you want.
What you're doing there is:
map.forEach(new BiConsumer<String, Integer>() {
#Override
public void accept(String s, Integer integer) {
map2.put(s, integer);
}
});
Which we can simplify into a lambda:
map.forEach((s, integer) -> map2.put(s, integer));
And because we're just calling an existing method we can use a method reference, which gives us:
map.forEach(map2::put);

Keep it Simple and use Java 8:-
Map<String, AccountGroupMappingModel> mapAccountGroup=CustomerDAO.getAccountGroupMapping();
Map<String, AccountGroupMappingModel> mapH2ToBydAccountGroups =
mapAccountGroup.entrySet().stream()
.collect(Collectors.toMap(e->e.getValue().getH2AccountGroup(),
e ->e.getValue())
);

The way without re-inserting all entries into the new map should be the fastest it won't because HashMap.clone internally performs rehash as well.
Map<String, Column> newColumnMap = originalColumnMap.clone();
newColumnMap.replaceAll((s, c) -> new Column(c));

If you use Guava (v11 minimum) in your project you can use Maps::transformValues.
Map<String, Column> newColumnMap = Maps.transformValues(
originalColumnMap,
Column::new // equivalent to: x -> new Column(x)
)
Note: The values of this map are evaluated lazily. If the transformation is expensive you can copy the result to a new map like suggested in the Guava docs.
To avoid lazy evaluation when the returned map doesn't need to be a view, copy the returned map into a new map of your choosing.

Here is another way that gives you access to the key and the value at the same time, in case you have to do some kind of transformation.
Map<String, Integer> pointsByName = new HashMap<>();
Map<String, Integer> maxPointsByName = new HashMap<>();
Map<String, Double> gradesByName = pointsByName.entrySet().stream()
.map(entry -> new AbstractMap.SimpleImmutableEntry<>(
entry.getKey(), ((double) entry.getValue() /
maxPointsByName.get(entry.getKey())) * 100d))
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));

If you don't mind using 3rd party libraries, my cyclops-react lib has extensions for all JDK Collection types, including Map. You can directly use the map or bimap methods to transform your Map. A MapX can be constructed from an existing Map eg.
MapX<String, Column> y = MapX.fromMap(orgColumnMap)
.map(c->new Column(c.getValue());
If you also wish to change the key you can write
MapX<String, Column> y = MapX.fromMap(orgColumnMap)
.bimap(this::newKey,c->new Column(c.getValue());
bimap can be used to transform the keys and values at the same time.
As MapX extends Map the generated map can also be defined as
Map<String, Column> y

From Java 9 onwards it is even easier to do the transformation within the map part of the stream. Is was already possible to use a new AbstractMap.SimpleImmutableEntry but the Map interface has an additional static method Map.entry which can also create an entry which can be used for this usecase.
Java 9+
import java.util.Map;
import java.util.Map.Entry;
import java.util.stream.Collectors;
public class App {
public static void main(String[] args) {
Map<String, Column> x;
Map<String, Column> y = x.entrySet().stream()
.map(entry -> Map.entry((entry.getKey(), new Column(entry.getValue())))
.collect(Collectors.toMap(Entry::getKey, Entry::getValue));
}
}