Develop Reference

Function to return a difference between two complex nested json with json array<Unordered>

The objective here is to compare two json files and print the difference between two of the files.
Its a nested JSON file with nested JSON array. The idea is to keep the reference file constant and compare the newly created file with reference.
Issue here is since the order of JSON parameter is varying every time and the below code is comparing line for line the validation Its flattening the JSON and making that as key value pair and comparing.Its failing because the order is not maintained. please note that the JSON file has 300k lines and ordering using a particular field doesn't work.
Gson gson = new Gson();
Type type = new TypeToken<Map<String, Object>>(){}.getType();
Map<String, Object> expMap = gson.fromJson(expectedJson, type);//expectedJson- json file parsed as a string
Map<String, Object> actMap = gson.fromJson(foundJson, type);
List<String> expMsgList=new ArrayList<>();
List<String> actMsgList=new ArrayList<>();
logger.info("Flatten the map inorder to compare the Nested JSON objects");
Map<String, Object> expFlatMap = FlatMapUtil.flatten(expMap);
Map<String, Object> actFlatMap = FlatMapUtil.flatten(actMap);
MapDifference<String, Object> difference = Maps.difference(expFlatMap, actFlatMap);
if(expMap.keySet().equals(actMap.keySet())) {
if(!difference.entriesDiffering().isEmpty()) {
difference.entriesDiffering().forEach((key, value) ->{
String keyString=null;
String keyStringarr[]=key.split("/");;
keyString=keyStringarr[keyStringarr.length -1];
if(!(fieldSkip.contains(keyString))){ //tp skip few parameters
String valueExp=value.toString().split(",")[0].split("\\(")[1];
String valueAct=value.toString().split(",")[1].split("\\)")[0];
String valueforKey="EXPECTED_VALUE="+valueExp+" and ACTUAL_VALUE="+ valueAct;
failedItems.put(key, valueforKey);
logger.info(key + " -->"+valueforKey );
}
}
);
Utility file for flattening the json file
package com.cisco.cnp.commonUtils;
import java.util.AbstractMap.SimpleEntry;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.stream.IntStream;
import java.util.stream.Stream;
public final class FlatMapUtil {
private FlatMapUtil() {
throw new AssertionError("No instances");
}
public static Map<String, Object> flatten(Map<String, Object> map) {
return map.entrySet()
.stream()
.flatMap(FlatMapUtil::flatten)
.collect(LinkedHashMap::new, (m, e) -> m.put("/" + e.getKey(), e.getValue()), LinkedHashMap::putAll);
}
private static Stream<Entry<String, Object>> flatten(Entry<String, Object> entry) {
if (entry == null) {
return Stream.empty();
}
if (entry.getValue() instanceof Map<?, ?>) {
Map<?, ?> properties = (Map<?, ?>) entry.getValue();
return properties.entrySet()
.stream()
.flatMap(e -> flatten(new SimpleEntry<>(entry.getKey() + "/" + e.getKey(), e.getValue())));
}
if (entry.getValue() instanceof List<?>) {
List<?> list = (List<?>) entry.getValue();
return IntStream.range(0, list.size())
.mapToObj(i -> new SimpleEntry<String, Object>(entry.getKey() + "/" + i, list.get(i)))
.flatMap(FlatMapUtil::flatten);
}
return Stream.of(entry);
}
}

Java guava Splitter failing when there are duplicate keys

How we can handle duplicate keys while using java guava Splitter function. Here is the sample code which is encountering the following issue. Is there a better way to handle this issue.
String fieldSplit = " ";
String valueSplit = "=";
String message = "ip=1.2.9.0 error=NA ip=1.2.9.0";
Map<String, String> parserMap = Splitter.on(fieldSplit).omitEmptyStrings().withKeyValueSeparator(valueSplit).split(message);
Exception in thread "kv-plugin-ac801a38-66f1-4ffe-86ca-f9eb6c823842-StreamThread-1" org.apache.kafka.streams.errors.StreamsException: Exception caught in process. taskId=0_0, processor=KSTREAM-SOURCE-0000000000, topic=kv-input, partition=0, offset=22, stacktrace=java.lang.IllegalArgumentException: Duplicate key [ip] found.
Im getting the above error. Can somebody suggest a better way to handle this. Since im new to java.

Depends on what you want to do with the duplicate keys.
Map<String, String> is a key value storage that can have only unique keys and only one value.
If you want to store all those values you would need something like Map<String, List<String> or Guava Multimap.
In this case you cannot do this with the Splitter as it cannot handle duplicate keys. You would need to write the logic by yourself.
String fieldSplit = " ";
String valueSplit = "=";
String message = "ip=1.2.9.0 error=NA ip=1.2.9.0";
Map<String, List<String>> parserMap = new HashMap<>();
for (String part : message.split(" ")) {
String[] subparts = part.split("=", 2);
if (!parserMap.contains(subparts[0])) {
parserMap.put(subparts[0], new ArrayList<>());
}
parserMap.get(subparts[0]).add(subparts[1]);
}
If you want to omit those duplicate entries you can still use the Map<String, String> with something like this.
Map<String, String> parserMap = new HashMap<>();
for (String part : message.split(" ")) {
String[] subparts = part.split("=", 2);
if (!parserMap.contains(subparts[0])) {
parserMap.put(subparts[0], subparts[1]);
}
}

Throwing on a duplicate key is a documented behavior of MapSplitter#split, so depending on what you want, you have to write your own "key-value" spliter consisting of two splitters. Please look at examples below, you can collect results to map with desired behavior (overwrite or discard) or even try out collecting to ListMultimap, but it makes result's values being stored in lists, even if there's only one value.
public class SO66139006 {
private static final Splitter PAIRS_SPLITTER = Splitter.on(' '); // .trimResults().omitEmptyStrings() if necessary
private static final Splitter KEY_VALUE_SPLITTER = Splitter.on('=').limit(2);
#Test
public void shouldOverwriteValuesOnDuplicateKey() {
//given
String message = "ip=42.42.42.0 error=NA ip=1.2.9.0";
//when
Map<String, String> result = parseOverwritingValues(PAIRS_SPLITTER, KEY_VALUE_SPLITTER, message);
//then
assertThat(result) // {ip=1.2.9.0, error=NA}
.containsExactly(entry("ip", "1.2.9.0"), entry("error", "NA"));
}
private Map<String, String> parseOverwritingValues(Splitter pairsSplitter, Splitter keyValueSplitter, String message) {
return Streams.stream(pairsSplitter.split(message))
.map(keyValueSplitter::splitToList)
.collect(toImmutableMap(
list -> list.get(0),
list -> list.get(1),
(oldValue, newValue) -> newValue
));
}
#Test
public void shouldDiscardValuesOnDuplicateKey() {
//given
String message = "ip=42.42.42.0 error=NA ip=1.2.9.0";
//when
Map<String, String> result = parseDiscardingValues(PAIRS_SPLITTER, KEY_VALUE_SPLITTER, message);
//then
assertThat(result) // {ip=42.42.42.0, error=NA}
.containsExactly(entry("ip", "42.42.42.0"), entry("error", "NA"));
}
private Map<String, String> parseDiscardingValues(Splitter pairsSplitter, Splitter keyValueSplitter, String message) {
return Streams.stream(pairsSplitter.split(message))
.map(keyValueSplitter::splitToList)
.collect(toImmutableMap(
list -> list.get(0),
list -> list.get(1),
(oldValue, newValue) -> oldValue
));
}
#Test
public void shouldAppendValuesOnDuplicateKey() {
//given
String message = "ip=42.42.42.0 error=NA ip=1.2.9.0";
//when
ListMultimap<String, String> result = parseMultipleValues(PAIRS_SPLITTER, KEY_VALUE_SPLITTER, message);
//then
assertThat(result.asMap()) // {ip=[42.42.42.0, 1.2.9.0], error=[NA]}
.containsExactly(entry("ip", ImmutableList.of("42.42.42.0", "1.2.9.0")), entry("error", ImmutableList.of("NA")));
}
private ListMultimap<String, String> parseMultipleValues(Splitter pairsSplitter, Splitter keyValueSplitter, String message) {
return Streams.stream(pairsSplitter.split(message))
.map(keyValueSplitter::splitToList)
.collect(toImmutableListMultimap(
list -> list.get(0),
list -> list.get(1)
));
}
#Test
public void shouldThrowByDefault() {
//given
String fieldSplit = " ";
String valueSplit = "=";
String message = "ip=1.2.9.0 error=NA ip=1.2.9.0";
//when
final Throwable throwable = catchThrowable(() -> Splitter.on(fieldSplit).omitEmptyStrings().withKeyValueSeparator(valueSplit).split(message));
//then
assertThat(throwable)
.isInstanceOf(IllegalArgumentException.class)
.hasMessage("Duplicate key [ip] found.");
}
}

Convert nested for loop into java8 stream

I am trying to convert the below code into Java 8 Stream for nested for loop.
I have tried to take stream for outer loop but not sure how to write condition and assign a variable there.
final Map<String, String> events = new HashMap<>();
for (final Event s : result.getEvents()) {
String eventDetail = "";
for (final Data d : s.getData()) {
if (StringUtils.isNotEmpty(d.getValue()) && StringUtils.isNotEmpty(eventDetail)) {
eventDetail = eventDetail + "-" + d.getValue();
} else {
eventDetail = eventDetail + d.getValue();
}
}
events.put(s.getReferenceID(), eventDetail);
}
Result should be map value.

It looks like your goal is to concatenate the value members of the Data instances of each
Event into a "-" separated String, and map this String to the Event's reference ID.
This can be done with Collectors.joining():
Map<String, String>
events = result.getEvents()
.stream()
.map(s -> new SimpleEntry<>(s.getReferenceID(),s.getData().stream().map(Data::getValue).collect(Collectors.joining("-"))))
.collect(Collectors.toMap(Map.Entry::getKey,Map.Entry::getValue));
or, if you wish to eliminate empty values:
Map<String, String>
events = result.getEvents()
.stream()
.map(s -> new SimpleEntry<>(s.getReferenceID(),s.getData().stream().map(Data::getValue).filter(StringUtils::isNotEmpty).collect(Collectors.joining("-"))))
.collect(Collectors.toMap(Map.Entry::getKey,Map.Entry::getValue));

Is this what you want?
Function<Event, String> mapper = event -> event.getData().stream()
.map(Data::getValue)
.filter(StringUtils::isNotEmpty)
.reduce("", (value1, value2) -> value1 + "-" + value2);
final Map<String, String> events = result.getEvents().stream()
.collect(Collectors.toUnmodifiableMap(Event::getReferenceID, mapper));

Usng StringJoiner in complex HashMaps

I have a list of Maps as below:
List<Map<String,Object>> someObjectsList = new ArrayList<Map<String,Object>>();
I am storing the following data in each HashMap
key value
2017-07-21 2017-07-21-07.33.28.429340
2017-07-24 2017-07-24-01.23.33.591340
2017-07-24 2017-07-24-01.23.33.492340
2017-07-21 2017-07-21-07.33.28.429540
I want to iterate through the list of HashMaps and check if the key matches with the first 10 characters of any of the HashMap value, then I want to store those keys and values in the following format. i.e. by using the telemeter 'comma'. The ultimate aim is to group the unique keys of the HashMaps and their relative values (if the key matches with the first 10 characters of any of the HashMap value) in a new HashMap.
key value
2017-07-21 2017-07-21-07.33.28.429340,2017-07-21-07.33.28.429540
2017-07-24 2017-07-24-01.23.33.591340,2017-07-24-01.23.33.492340
I am trying with following java code using StringJoiner, but not getting the results as expected. Any clue on how to frame the logic here?
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.StringJoiner;
public class SampleOne {
public static void main(String[] args) {
// TODO Auto-generated method stub
List<Map<String, Object>> someObjectsList = new ArrayList<Map<String, Object>>();
Map<String, Object> mapOne = new HashMap<String, Object>();
mapOne.put("2017-07-21", "2017-07-21-07.33.28.429340");
Map<String, Object> mapTwo = new HashMap<String, Object>();
mapTwo.put("2017-07-24", "2017-07-24-01.23.33.591340");
Map<String, Object> mapThree = new HashMap<String, Object>();
mapThree.put("2017-07-24", "2017-07-24-01.23.33.492340");
Map<String, Object> mapFour = new HashMap<String, Object>();
mapFour.put("2017-07-21", "2017-07-21-07.33.28.429540");
someObjectsList.add(mapOne);
someObjectsList.add(mapTwo);
someObjectsList.add(mapThree);
someObjectsList.add(mapFour);
for (Map map : someObjectsList) {
StringJoiner sj = new StringJoiner(",");
for (Object key : map.keySet()) {
String value = ((String) map.get(key));
String date = value.substring(0, Math.min(value.length(), 10));
//System.out.println(str);
//System.out.println(value);
if(key.equals(date)) {
sj.add(value);
System.out.println(sj.toString());
}
}
}
}
}
output:
2017-07-21-07.33.28.429340
2017-07-24-01.23.33.591340
2017-07-24-01.23.33.492340
2017-07-21-07.33.28.429540

Make use of the .merge function:
Map<String, Object> finalMap = new HashMap<String, Object>();
for (Map map : someObjectsList) {
for (Object key : map.keySet()) {
String value = ((String) map.get(key));
finalMap.merge((String) key, value, (k, v) -> k + "," + v);
}
}
which outputs:
{2017-07-21=2017-07-21-07.33.28.429340,2017-07-21-07.33.28.429540,
2017-07-24=2017-07-24-01.23.33.591340,2017-07-24-01.23.33.492340}
The same can be achieved by the following one-liner:
someObjectsList.stream()
.flatMap(i -> i.entrySet().stream())
.collect(Collectors.toMap(Entry::getKey, Entry::getValue,
(k, v) -> k + "," + v));

On your code, you are using different StringJoiner on each map. So, it's creating a new instance of it.
You can save your keys on a map. An example code:
(Edit: I did not remove your StringJoiner part.)
public static void main(String[] args) {
// TODO Auto-generated method stub
List<Map<String, Object>> someObjectsList = new ArrayList<Map<String, Object>>();
Map<String, Object> mapOne = new HashMap<String, Object>();
mapOne.put("2017-07-21", "2017-07-21-07.33.28.429340");
Map<String, Object> mapTwo = new HashMap<String, Object>();
mapTwo.put("2017-07-24", "2017-07-24-01.23.33.591340");
Map<String, Object> mapThree = new HashMap<String, Object>();
mapThree.put("2017-07-24", "2017-07-24-01.23.33.492340");
Map<String, Object> mapFour = new HashMap<String, Object>();
mapFour.put("2017-07-21", "2017-07-21-07.33.28.429540");
someObjectsList.add(mapOne);
someObjectsList.add(mapTwo);
someObjectsList.add(mapThree);
someObjectsList.add(mapFour);
Map<String, Object> outputMap = new HashMap<String, Object>();
for (Map map : someObjectsList) {
StringJoiner sj = new StringJoiner(",");
for (Object key : map.keySet()) {
String value = ((String) map.get(key));
String date = value.substring(0, Math.min(value.length(), 10));
//System.out.println(str);
//System.out.println(value);
if(key.equals(date)) {
sj.add(value);
System.out.println(sj.toString());
if(outputMap.containsKey(key)) {
String str = (String) map.get(key);
str = str + "," + value;
outputMap.put((String)key, str);
} else {
outputMap.put((String)key, value);
}
}
}
}
for (String map : outputMap.keySet()) {
System.out.println(map + " " + outputMap.get(map));
}
}

You are looking for the grouping behavior of processing a List. You can use the advantage of java-stream since java-8. In any case, you need a new Map to store the values in order to print them. :
someObjectsList.stream()
.flatMap(i -> i.entrySet().stream()) // flatmapping to entries
.collect(Collectors.groupingBy(Entry::getKey)) // grouping them using the key
In case you want to use for-loops. In this case it is harder since the more entries might appear in each List item:
final Map<String, List<Object>> map = new HashMap<>();
for (Map<String, Object> m: someObjectsList) { // iterate List<Map>
for (Entry<String, Object> entry: m.entrySet()) { // iterate entries of each Map
List<Object> list;
final String key = entry.getKey(); // key of the entry
final Object value = entry.getValue(); // value of the entry
if (map.containsKey(key)) { // if the key exists
list = map.get(key); // ... use it
} else {
list = new ArrayList<>(); // ... or else create a new one
}
list.add(value); // add the new value
map.put(key, list); // and add/update the entry
}
}
Printing out of Map<String, List<Object>> map in both cased will produce the following output:
2017-07-21=[2017-07-21-07.33.28.429340, 2017-07-21-07.33.28.429540],
2017-07-24=[2017-07-24-01.23.33.591340, 2017-07-24-01.23.33.492340]

Any reason you're using Object over String and avoiding safety checks? That said, it's not "the first 10 characters", you want to see if value starts with key full-stop (all your keys are 10 characters). So in that case you can just do if (value.startsWith(key)) { ... }. Don't forget your newlines if the stringjoiner wasn't full. Lastly, you don't need a List, a Map can hold multiple keys at once. An alternative way of doing it:
//LinkedHashMap will preserve our insertion order
Map<String, String> map = new LinkedHashMap<>();
map.put("2017-07-21", "2017-07-21-07.33.28.429340");
map.put("2017-07-24", "2017-07-24-01.23.33.591340");
//note duplicates are overwritten, but no value change here
map.put("2017-07-24", "2017-07-24-01.23.33.492340");
map.put("2017-07-21", "2017-07-21-07.33.28.429540");
// You can also use Java 8 streams for the concatenation
// but I left it simple
List<String> matches = map.entrySet()
.filter(e -> e.getValue().startsWith(e.getKey())
.collect(Collectors.toList());
String concatenated = String.join("\n", matches);
If you wanted to generate that string without streams, it would look like this (again, not using #entrySet for simplicity, but it would be more efficient here):
List<String> matches = new ArrayList<>();
StringJoiner joiner = new StringJoiner("\n");
for (String key : map.keySet()) {
String value = map.get(key);
if (value.startsWith(key)) {
joiner.add(value);
}
}
//joiner#toString will give the expected result

Convert functional Java 8 stream function to Java 7

My project uses Java 7 and can not be upgraded to Java 8, but according to time I will need a class to authenticate a method. That class contains a method that uses Java 8 Stream API to join a Map into a String by using Collectors.joining(), which is all functionality of Java 8.
I have searched a lot in order to transform it into valid Java 7 code but did not manage it and need help with it.
private static String getSignatureBaseString(String url, String method, Map<String, String> parameters) {
String requestURL = urlEncode(url);
// 1. Percent encode every key and value that will be signed.
Map<String, String> encodedParameters = percentEncodeParameters(parameters);
// 2. Sort the list of parameters alphabetically by encoded key.
encodedParameters = getSortedParameters(encodedParameters);
String paramsString = **mapToString(encodedParameters, SpecialSymbol.EQUAL.getEncoded(), SpecialSymbol.AMP.getEncoded());
return String.format(BASE_SIGNATURE_FORMAT, method, requestURL, paramsString);
}
//========
private static String mapToString(Map<String, String> paramsMap, String keyValueDelimiter, String paramsDelimiter) {
//**this return
return paramsMap.entrySet().stream()
.map(entry -> entry.getKey() + keyValueDelimiter + entry.getValue())
.collect(Collectors.joining(paramsDelimiter));
}

Just run over the entries and add the key/value pair into a StringBuilder. Collectors.joining() prevents that paramsDelimiter is inserted at the end. This is catched in the for-loop by the check of the firstLoop variable.
private static String mapToString(Map<String, String> paramsMap, String keyValueDelimiter, String paramsDelimiter)
{
boolean firstLoop = true;
StringBuilder builder = new StringBuilder();
for (Map.Entry<String, String> entry : paramsMap.entrySet())
{
if (!firstLoop)
{
builder.append(paramsDelimiter);
}
builder.append(entry.getKey());
builder.append(keyValueDelimiter);
builder.append(entry.getValue());
firstLoop = false;
}
return builder.toString();
}