Develop Reference

How to safely extract nested values from YAML file using SnakeYaml?

I am working on extracting values from YAML Files in Java using the Snakeyaml library. Unfortunately, I am having a hard time extracting values from these files when I do not know the contents of the file in advance.
As such I am looking for a safe why to extract nested values from a given YAML File.
Here my approach:
Map<String, Object> dataMap = yaml.load(FileUtils.readFileToString(file, StrandardCharsets.UTF_8));
for(Map.Entry<String, Object> entry: dataMap.entrySet()) {
if(entry.getValue() instanceof Map<String, Object>) {
// Do something. Potentially loop again, because I do not know the depth of the File
} else {
// Get actual Value
}
}

I wrote this function that recursively reads all the fields in the yml file.
I have not tested it thoroughly, but I would say that it works as expected.
public static void readMapRec(Map<String,Object> map){
if(Objects.isNull(map) || map.entrySet().size()==0){
return;
}
for(Map.Entry<String,Object> entry: map.entrySet()){
try {
if (entry.getValue() instanceof List) {
for(int i = 0 ; i< ((List<?>) entry.getValue()).size();i++) {
Map<String, Object> casted = (Map<String, Object>) ((List<?>) entry.getValue()).get(i);
readMapRec(casted);
}
}
else {
System.out.println(entry.getKey() +" "+entry.getValue());
}
}catch (Exception ex){
System.out.println(ex.getMessage());
}
}
}
public static void main(String[] args) throws IOException {
InputStream inputStream = new FileInputStream(new File("myFile.yml"));
Yaml yaml = new Yaml();
Map<String, Object> data = yaml.load(inputStream);
readYml(data);
}
So, If you have yml file like this:
id: 20
name: Bruce
year: 2020
address: Gotham City
department: Computer Science
courses:
- name: Algorithms
credits: 6
- name: Data Structures
credits: 5
- name: Design Patterns
credits: 3
books:
- ds:
- a: test
- b: test
- c:
- e: test
- f: test
You should see output like this:
id 20
name Bruce
year 2020
address Gotham City
department Computer Science
name Algorithms
credits 6
name Data Structures
credits 5
name Design Patterns
credits 3
a test
b test
e test
f test

I slightly adapted the function you proposed up top to catch indiscriminate cases:
private #NotNull Map<String[], Object> parseMap(#NotNull Map<String, Object> map, String[] previousRoot) {
Map<String[], Object> values = new HashMap<>();
if(map.entrySet().size()==0){
return values;
}
for(Map.Entry<String, Object> entry: map.entrySet()){
try {
LinkedList<String> list = new LinkedList<>(Arrays.asList(previousRoot));
list.add(entry.getKey());
String[] key = list.toArray(previousRoot);
if (entry.getValue() instanceof List) {
List<Map<String, Object>> objectItems = new ArrayList<>();
for(int i = 0 ; i< ((List<?>) entry.getValue()).size();i++) {
Map<String, Object> casted = (Map<String, Object>) ((List<?>) entry.getValue()).get(i);
if(casted.size() > 1) {
objectItems.add(casted);
} else {
values.putAll(parseMap(casted, key));
}
}
if(!objectItems.isEmpty()) {
System.out.println("Putting Key: " + Arrays.toString(key) + " with Value: " + objectItems);
values.put(key, objectItems);
}
}
else {
System.out.println("Putting Key: " + Arrays.toString(key) + " with Value: " + entry.getValue());
values.put(key, entry.getValue());
}
}catch (Exception ex){
System.out.println("Exception: " + ex.getMessage());
}
}
return values;
}
What do you think?
If you have any suggestions as to how it could be refined, I would be happy to hear them. :)

Java - SnakeYaml | Get all keys of a file

First of all, I have been reading a few posts about keys, but none of them asks my question on how to get ALL keys of a yaml file, only on how to get an specific key.
Now, I want to create a file updater, it works, but it only updates the first keys, without the "sub-keys", here is the code:
InputStream resource = getClass().getClassLoader().getResourceAsStream(dir);
Map<String, Object> data = new Yaml().load(resource);
for(String str : data.keySet()) {
DBot.getConsole().log(str);
if(!contains(str)) {
set(str, data.get(str));
}
}
The file looks like this:
Features.Example.StringA
Features.Example.StringB
With points being spaces to make them sub-keys (stack overflow puts them on a single line, sorry)
Now the thing is, the updater will only work if "Features" is deleted, also, the debug will only print "Features", meaning that only the first key is on the key set, how can I get all keys?

I have finally found how to return a Set with every key separated by a ".", Bukkit/Spigot developers might be familiar with this. First of all, you have to create a class like this:
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
public class YamlKeys {
private static Set<String> keys = new HashSet<String>();
private static String path = "";
YamlKeys(Map<?, ?> data) {
getKeysRecursive(data);
}
private void getKeysRecursive(final Map<?, ?> data) {
for(Object key : data.keySet()) {
final Object value = data.get(key);
if(key instanceof String) {
if(path.length() == 0) {
path = (String)key; // If the key is the first on the path, don't include separator.
} else {
path = path+"."+(String)key; // Here is the separator, you can change it.
}
}
if(value instanceof Map) {
getKeysRecursive((Map<?, ?>) value); // A value map has been found, recursing with that value.
} else {
keys.add(path); // No more maps have been found, we can add the path and stop recursing.
if(path.contains(".")) {
path = path.substring(0, path.lastIndexOf(".")); // Removing last key, so if a value contains more than one key, it won't appear again.
}
}
}
path = ""; // This is important, reset the path.
}
Set<String> getKeys() {
return keys;
}
}
Then, to call it and select if you want to get deep keys or "normal" keys, you can create a method like this:
public Set<String> getKeys(boolean deep) {
Map<String, Object> data = new Yaml().load(inStream);
if(!deep) {
return data.keySet();
} else {
return new YamlKeys(data).getKeys();
}
}
To test it, we can use the following code:
new YamlKeys(data).getKeys().stream().forEach(key -> System.out.println(key));
With this file:
FirstKey:
SecondKey:
Enabled: true
Text: "Some text"
AnotherKey:
AValue: true
AnotherTest:
Enabled: false
Value: true
It returns this output:
FirstKey.SecondKey.AnotherKey.AValue
FirstKey.SecondKey.Enabled
FirstKey.SecondKey.Text
Value
AnotherTest.Enabled
Thanks to roby for telling me about recursion.

SnakeYAML is decoding the yaml into a recursive data structure. For example:
public static void main(String[] args) {
String yaml = "a:\n b: \n c: \"string\"";
Map<String, Object> data = new Yaml().load(yaml);
System.out.println(data);
}
prints out:
{a={b={c=string}}}
Which is a Map<String, Map<String, Map<String, String>>>.
To show how you can work with it recursively, here's how you can print out some of that detail.
private static void printMapRecursive(final Map<?, ?> data) {
for(Object key : data.keySet()) {
System.out.println("key " + key + " is type " + key.getClass().getSimpleName());
final Object value = data.get(key);
if(value instanceof Map){
System.out.println("value for " + key + " is a Map - recursing");
printMapRecursive((Map<?, ?>) value);
} else {
System.out.println("value " + value + " for " + key + " is type " + value.getClass());
}
}
}
Which you can call with printMapRecursive(data); and see output:
key a is type String
value for a is a Map - recursing
key b is type String
value for b is a Map - recursing
key c is type String
value string for c is type class java.lang.String
and an example of recursively transforming the keys:
private static Map<?, ?> mutateMapRecursive(final Map<?, ?> data,
Function<String, String> keyFunction) {
Map<Object, Object> result = new HashMap<>();
for (Object key : data.keySet()) {
final Object value = data.get(key);
if(key instanceof String){
key = keyFunction.apply((String) key);
}
if (value instanceof Map) {
result.put(key, mutateMapRecursive((Map<?, ?>) value, keyFunction));
}
else {
result.put(key, value);
}
}
return result;
}
called like:
final Map<?, ?> transformed = mutateMapRecursive(data, (key) -> "prefix_" + key);
System.out.println(transformed);
emits:
{prefix_a={prefix_b={prefix_c=string}}}

Removing duplicate key-value pairs in a map with values being in a list

Below is my code to detect abbreviations and their long forms. The code loops over a line in a document, loops over each word of that line and identifies an acronym candidate. It then again loops over each line of the document to find an appropriate long form for the abbreviation. My issue is if an acronym occurs multiple times in a document my output contains multiple instances of it. I just want to print an acronym only once with all its possible long forms. Here's my code:
public static void main(String[] args) throws FileNotFoundException
{
BufferedReader in = new BufferedReader(new FileReader("D:\\Workspace\\resource\\SampleSentences.txt"));
String str=null;
ArrayList<String> lines = new ArrayList<String>();
String matchingLongForm;
List <String> matchingLongForms = new ArrayList<String>() ;
List <String> shortForm = new ArrayList<String>() ;
Map<String, List<String>> abbreviationPairs = new HashMap<String, List<String>>();
try
{
while((str = in.readLine()) != null){
lines.add(str);
}
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
String[] linesArray = lines.toArray(new String[lines.size()]);
// document wide search for abbreviation long form and identifying several appropriate matches
for (String line : linesArray){
for (String word : (Tokenizer.getTokenizer().tokenize(line))){
if (isValidShortForm(word)){
for (int i = 0; i < linesArray.length; i++){
matchingLongForm = extractBestLongForm(word, linesArray[i]);
//shortForm.add(word);
if (matchingLongForm != null && !(matchingLongForms.contains(matchingLongForm))){
matchingLongForms.add(matchingLongForm);
//System.out.println(matchingLongForm);
abbreviationPairs.put(word, matchingLongForms);
//matchingLongForms.clear();
}
}
if (abbreviationPairs != null){
//for(abbreviationPairs.)
System.out.println("Abbreviation Pair:" + "\t" + abbreviationPairs);
abbreviationPairs.clear();
matchingLongForms.clear();
//System.out.println("Abbreviation Pair:" + "\t" + abbreviationPairsNew);
}
else
continue;
}
}
}
}
Here's the current output:
Abbreviation Pair: {GLBA=[Gramm Leach Bliley act]}
Abbreviation Pair: {NCUA=[National credit union administration]}
Abbreviation Pair: {FFIEC=[Federal Financial Institutions Examination Council]}
Abbreviation Pair: {CFR=[comments for the Report]}
Abbreviation Pair: {CFR=[comments for the Report]}
Abbreviation Pair: {CFR=[comments for the Report]}
Abbreviation Pair: {CFR=[comments for the Report]}
Abbreviation Pair: {OFAC=[Office of Foreign Assets Control]}

Try to use java.util.Set to store your matching short forms and long forms. From the javadoc of the class:
... If this set already contains the element, the call leaves the set unchanged and returns false. In combination with the restriction on constructors, this ensures that sets never contain duplicate elements...

You want a key value pair for abbreviation and text. So you should use Map.
A map cannot contain duplicate keys; each key can map to at most one value.
The Problem is in the position of the output and not in the map.
You try to output in the loop, so the Map is shown multiple time.
Move the code outside the loop:
if (abbreviationPairs != null){
//for(abbreviationPairs.)
System.out.println("Abbreviation Pair:" + "\t" + abbreviationPairs);
abbreviationPairs.clear();
matchingLongForms.clear();
//System.out.println("Abbreviation Pair:" + "\t" + abbreviationPairsNew);
}

Here's the solution
Thanks to code_angel and Holger
Move the printing code outside the loop and create a new list for every matchingLongForm.
for (String line : linesArray){
for (String word : (Tokenizer.getTokenizer().tokenize(line))){
if (isValidShortForm(word)){
for (int i = 0; i < linesArray.length; i++){
matchingLongForm = extractBestLongForm(word, linesArray[i]);
List <String> matchingLongForms = new ArrayList<String>() ;
if (matchingLongForm != null && !(matchingLongForms.contains(matchingLongForm))&& !(abbreviationPairs.containsKey(word))){
matchingLongForms.add(matchingLongForm);
//System.out.println(matchingLongForm);
abbreviationPairs.put(word, matchingLongForms);
//matchingLongForms.clear();
}
}
}
}
}
if (abbreviationPairs != null){
System.out.println("Abbreviation Pair:" + "\t" + abbreviationPairs);
//abbreviationPairs.clear();
//matchingLongForms.clear();
}
}
The new output:
Abbreviation Pair: {NCUA=[National credit union administration], FFIEC=[Federal Financial Institutions Examination Council], OFAC=[Office of Foreign Assets Control], MSSP=[Managed Security Service Providers], IS=[Information Systems], SLA=[Service level agreements], CFR=[comments for the Report], MIS=[Management Information Systems], IDS=[Intrusion detection systems], TSP=[Technology Service Providers], RFI=[risk that FIs], EIC=[Examples of in the cloud], TIER=[The institution should ensure], BCP=[Business continuity planning], GLBA=[Gramm Leach Bliley act], III=[It is important], FI=[Financial Institutions], RFP=[Request for proposal]}

Adding entries to LinkedList inside ConcurrentHashMap not working

I have a ConcurrentHashMap that contains a string as a key and LinkedList as a value. The size of the list should not be more than 5. I am trying to add some elements to the list but when I print out the Map I see only the last added element. Here is my code:
private ConcurrentHashMap<String, LinkedList<Date>> userDisconnectLogs = new ConcurrentHashMap<String, LinkedList<Date>>();
public void addNewDateEntry(String userId, LinkedList<Date> timeStamps) {
if (timeStamps.size() >= 5) {
timeStamps.poll();
timeStamps.add(new Date());
userDisconnectLogs.put(userId, timeStamps);
} else {
timeStamps.add(new Date());
userDisconnectLogs.put(userId, timeStamps);
}
for (Entry<String, LinkedList<Date>> entry : userDisconnectLogs
.entrySet()) {
String key = entry.getKey().toString();
;
LinkedList<Date> value = entry.getValue();
System.out.println("key: " + key + " value: " + value.size());
}
}
Thank you!

Here for hashMap key must be unique. And from this code it seems key is always same so all the time it will overrite the data.

hasNext skips first value in HashMap

I'm trying to implement an expandable listview with data from a remote server. I've already have the JSON part covered. My sample has three value sets returned (confirmed by checking logcat on the original JSON response). My problem now is while dividing the JSON return into header and child datas, the first value set is skipped. My code is as follows:
int lisDataHeaderCounter = 0;
String searchKey;
for (int i = 0; i < components.length(); i++) {
List<String> component_value = new ArrayList<String>();
searchKey = main_components.get(i);
if (!listDataHeader.contains(searchKey)) {
listDataHeader.add(searchKey);
Iterator<Map.Entry<String, String>> entries = sub_components.entrySet().iterator();
while (entries.hasNext()) {
Map.Entry<String, String> entry = entries.next();
Log.d("getValue() ", "+ " + entry.getValue());
if (searchKey == entry.getKey())
component_value.add(entry.getValue());
}
listDataChild.put(listDataHeader.get(lisDataHeaderCounter), component_value);
lisDataHeaderCounter++;
}
}
I've also tried the code below and it still has the same result.
for (Map.Entry<String, String> entry : sub_components.entrySet()) {
if (searchKey == entry.getKey())
component_value.add(entry.getValue());
}
Here is a sample of the JSON response that is being process by the above codes:
[{"activity_code":"1","activity_name":"Midterm Exam"},
{"activity_code":"1","activity_name":"Final Exam"},
{"activity_code":"2","activity_name":"Project"}]
With the current codes, in the for loop, the first value of searchKey is '1'. When I placed a Log.d(); in the while loop to check what the first value is read, I found that it is "Final Exam" and not "Midterm Exam". Is there a way for me to get the value of the first data set before it goes into the while loop?
Here is a workaround I've made to ensure that the first value would be included to the sub_components. But I guess it doesn't look neat. If anyone has a better solution, please feel free to share.
for (int i = 0; i < components.length(); i++) {
JSONObject c = components.getJSONObject(i);
String formula_code = c.getString(TAG_FORMULA_CODE);
String component_name = c.getString(TAG_ACTIVITY_NAME);
main_components.add(formula_code);
sub_components.put(formula_code, component_name);
if (!listDataHeader.contains(formula_code))
listDataHeader.add(formula_code);
if (i == 0) {
component_value.add(component_name);
}
}
for (int i = 0; i < listDataHeader.size(); i++) {
for (Map.Entry<String, String> entry : sub_components.entrySet()) {
if (listDataHeader.get(i) == entry.getKey())
component_value.add(entry.getValue());
}
listDataChild.put(listDataHeader.get(i), component_value);
component_value = new ArrayList<String>();
}

I can't see any off by one error on the two, but perhaps it's the searchKey == entry.getKey(), that might have to be searchKey.equals(entry.getKey()), but I would need to see more code to know for sure.