I am working on extracting values from YAML Files in Java using the Snakeyaml library. Unfortunately, I am having a hard time extracting values from these files when I do not know the contents of the file in advance.
As such I am looking for a safe why to extract nested values from a given YAML File.
Here my approach:
Map<String, Object> dataMap = yaml.load(FileUtils.readFileToString(file, StrandardCharsets.UTF_8));
for(Map.Entry<String, Object> entry: dataMap.entrySet()) {
if(entry.getValue() instanceof Map<String, Object>) {
// Do something. Potentially loop again, because I do not know the depth of the File
} else {
// Get actual Value
}
}
I wrote this function that recursively reads all the fields in the yml file.
I have not tested it thoroughly, but I would say that it works as expected.
public static void readMapRec(Map<String,Object> map){
if(Objects.isNull(map) || map.entrySet().size()==0){
return;
}
for(Map.Entry<String,Object> entry: map.entrySet()){
try {
if (entry.getValue() instanceof List) {
for(int i = 0 ; i< ((List<?>) entry.getValue()).size();i++) {
Map<String, Object> casted = (Map<String, Object>) ((List<?>) entry.getValue()).get(i);
readMapRec(casted);
}
}
else {
System.out.println(entry.getKey() +" "+entry.getValue());
}
}catch (Exception ex){
System.out.println(ex.getMessage());
}
}
}
public static void main(String[] args) throws IOException {
InputStream inputStream = new FileInputStream(new File("myFile.yml"));
Yaml yaml = new Yaml();
Map<String, Object> data = yaml.load(inputStream);
readYml(data);
}
So, If you have yml file like this:
id: 20
name: Bruce
year: 2020
address: Gotham City
department: Computer Science
courses:
- name: Algorithms
credits: 6
- name: Data Structures
credits: 5
- name: Design Patterns
credits: 3
books:
- ds:
- a: test
- b: test
- c:
- e: test
- f: test
You should see output like this:
id 20
name Bruce
year 2020
address Gotham City
department Computer Science
name Algorithms
credits 6
name Data Structures
credits 5
name Design Patterns
credits 3
a test
b test
e test
f test
I slightly adapted the function you proposed up top to catch indiscriminate cases:
private #NotNull Map<String[], Object> parseMap(#NotNull Map<String, Object> map, String[] previousRoot) {
Map<String[], Object> values = new HashMap<>();
if(map.entrySet().size()==0){
return values;
}
for(Map.Entry<String, Object> entry: map.entrySet()){
try {
LinkedList<String> list = new LinkedList<>(Arrays.asList(previousRoot));
list.add(entry.getKey());
String[] key = list.toArray(previousRoot);
if (entry.getValue() instanceof List) {
List<Map<String, Object>> objectItems = new ArrayList<>();
for(int i = 0 ; i< ((List<?>) entry.getValue()).size();i++) {
Map<String, Object> casted = (Map<String, Object>) ((List<?>) entry.getValue()).get(i);
if(casted.size() > 1) {
objectItems.add(casted);
} else {
values.putAll(parseMap(casted, key));
}
}
if(!objectItems.isEmpty()) {
System.out.println("Putting Key: " + Arrays.toString(key) + " with Value: " + objectItems);
values.put(key, objectItems);
}
}
else {
System.out.println("Putting Key: " + Arrays.toString(key) + " with Value: " + entry.getValue());
values.put(key, entry.getValue());
}
}catch (Exception ex){
System.out.println("Exception: " + ex.getMessage());
}
}
return values;
}
What do you think?
If you have any suggestions as to how it could be refined, I would be happy to hear them. :)
Related
First of all, I have been reading a few posts about keys, but none of them asks my question on how to get ALL keys of a yaml file, only on how to get an specific key.
Now, I want to create a file updater, it works, but it only updates the first keys, without the "sub-keys", here is the code:
InputStream resource = getClass().getClassLoader().getResourceAsStream(dir);
Map<String, Object> data = new Yaml().load(resource);
for(String str : data.keySet()) {
DBot.getConsole().log(str);
if(!contains(str)) {
set(str, data.get(str));
}
}
The file looks like this:
Features.Example.StringA
Features.Example.StringB
With points being spaces to make them sub-keys (stack overflow puts them on a single line, sorry)
Now the thing is, the updater will only work if "Features" is deleted, also, the debug will only print "Features", meaning that only the first key is on the key set, how can I get all keys?
I have finally found how to return a Set with every key separated by a ".", Bukkit/Spigot developers might be familiar with this. First of all, you have to create a class like this:
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
public class YamlKeys {
private static Set<String> keys = new HashSet<String>();
private static String path = "";
YamlKeys(Map<?, ?> data) {
getKeysRecursive(data);
}
private void getKeysRecursive(final Map<?, ?> data) {
for(Object key : data.keySet()) {
final Object value = data.get(key);
if(key instanceof String) {
if(path.length() == 0) {
path = (String)key; // If the key is the first on the path, don't include separator.
} else {
path = path+"."+(String)key; // Here is the separator, you can change it.
}
}
if(value instanceof Map) {
getKeysRecursive((Map<?, ?>) value); // A value map has been found, recursing with that value.
} else {
keys.add(path); // No more maps have been found, we can add the path and stop recursing.
if(path.contains(".")) {
path = path.substring(0, path.lastIndexOf(".")); // Removing last key, so if a value contains more than one key, it won't appear again.
}
}
}
path = ""; // This is important, reset the path.
}
Set<String> getKeys() {
return keys;
}
}
Then, to call it and select if you want to get deep keys or "normal" keys, you can create a method like this:
public Set<String> getKeys(boolean deep) {
Map<String, Object> data = new Yaml().load(inStream);
if(!deep) {
return data.keySet();
} else {
return new YamlKeys(data).getKeys();
}
}
To test it, we can use the following code:
new YamlKeys(data).getKeys().stream().forEach(key -> System.out.println(key));
With this file:
FirstKey:
SecondKey:
Enabled: true
Text: "Some text"
AnotherKey:
AValue: true
AnotherTest:
Enabled: false
Value: true
It returns this output:
FirstKey.SecondKey.AnotherKey.AValue
FirstKey.SecondKey.Enabled
FirstKey.SecondKey.Text
Value
AnotherTest.Enabled
Thanks to roby for telling me about recursion.
SnakeYAML is decoding the yaml into a recursive data structure. For example:
public static void main(String[] args) {
String yaml = "a:\n b: \n c: \"string\"";
Map<String, Object> data = new Yaml().load(yaml);
System.out.println(data);
}
prints out:
{a={b={c=string}}}
Which is a Map<String, Map<String, Map<String, String>>>.
To show how you can work with it recursively, here's how you can print out some of that detail.
private static void printMapRecursive(final Map<?, ?> data) {
for(Object key : data.keySet()) {
System.out.println("key " + key + " is type " + key.getClass().getSimpleName());
final Object value = data.get(key);
if(value instanceof Map){
System.out.println("value for " + key + " is a Map - recursing");
printMapRecursive((Map<?, ?>) value);
} else {
System.out.println("value " + value + " for " + key + " is type " + value.getClass());
}
}
}
Which you can call with printMapRecursive(data); and see output:
key a is type String
value for a is a Map - recursing
key b is type String
value for b is a Map - recursing
key c is type String
value string for c is type class java.lang.String
and an example of recursively transforming the keys:
private static Map<?, ?> mutateMapRecursive(final Map<?, ?> data,
Function<String, String> keyFunction) {
Map<Object, Object> result = new HashMap<>();
for (Object key : data.keySet()) {
final Object value = data.get(key);
if(key instanceof String){
key = keyFunction.apply((String) key);
}
if (value instanceof Map) {
result.put(key, mutateMapRecursive((Map<?, ?>) value, keyFunction));
}
else {
result.put(key, value);
}
}
return result;
}
called like:
final Map<?, ?> transformed = mutateMapRecursive(data, (key) -> "prefix_" + key);
System.out.println(transformed);
emits:
{prefix_a={prefix_b={prefix_c=string}}}
In my SQL Server I have the following result sets after all the condition filtering and sum query execution.
I would like to be shown like this in my page (refer to the screenshot below).
I have tried the below JAVA code that gave me the results that I appended into my datatables.
<%
ArrayList<String[]> rows = sqlq.querySQL();
String rowsetdate = new String();
String rowres1 = new String();
for(String[] rowset : rows) {
rowsetdate = rowset[0];
rowres1 = rowres1 + rowset[1]+ ",";
for(String rowres2 : rowset) {
rowres1 = rowres1 + rowres2 + ",";
}
rowres1 = rowres1.substring(0, rowres1.length()-1);
rowres1 = rowres1 + "|";
}
rowres1 = rowres1.substring(0, rowres1.length()-1);
%>
<tr>
<td><%if (rowres1 == null) out.print(""); else out.print(rowres1);%></td>
</tr>
sqlq.querySQL() is used to send my SQL query to JDBC in order for me to send query to my DB.
Photo below is the appended data in my datatables after the code execution, on the left is the Date and on the right is the data.
I tried some different code,
<%
ArrayList<String[]> rows = sqlq.querySQL();
for(String[] rowset : rows) {
<tr>
<td><%if (rowset[0] == null) out.print(""); else out.print(rowset[0]);%></td>
<td><%if (rowset[1] == null) out.print(""); else out.print(rowset[1]);%></td>
</tr>
}
%>
which did not achieve my expected results also, it returns the data like how I see it in my SSMS (check screenshot below)
What did I do wrong and how should I do it to get my expected outcome? (screenshot below)
Appreciate the help from all of you.
You can use a Map that its key is date and its value again is a Map. Inner Map uses trans as key and sumtot as value.
Map<String, Map<String, String>> mapByDate = new HashMap<>();
TreeSet<String> allTrans = new TreeSet<>();
for (String[] row : rows) {
Map<String, String> mapByDateAndTrans = mapByDate.get(row[0]);
if (mapByDateAndTrans == null) {
mapByDateAndTrans = new HashMap<>();
}
mapByDateAndTrans.put(row[1], row[2]);
mapByDate.put(row[0], mapByDateAndTrans);
allTrans.add(row[1]);
}
Here is a sample code to print data as you might expect:
System.out.println("Date/Trans: " + allTrans);
for (Map.Entry<String, Map<String, String>> mapByDateEntry : mapByDate.entrySet()) {
System.out.print(mapByDateEntry.getKey() + ": ");
Map<String, String> mapByTrans = mapByDateEntry.getValue();
for (String trans : allTrans) {
String sumtot = mapByTrans.get(trans);
if (sumtot != null) {
System.out.print("[ " + sumtot + " ]");
} else {
System.out.print("[ ]");
}
}
System.out.println();
}
The output:
Date/Trans: [11200, 11201, 11202]
2019-07-02: [ 136 ][ 18 ][ 14 ]
2019-07-03: [ 164 ][ 10 ][ 8 ]
Or we can generate an HTML table content:
StringBuilder tableBuilder = new StringBuilder("<table border = 1>");
// table header
tableBuilder.append("<tr>");
tableBuilder.append("<th>date/trans</th>");
for (String trans : allTrans) {
tableBuilder.append("<th>").append(trans).append("</th>");
}
tableBuilder.append("</tr>");
// table rows
for (Map.Entry<String, Map<String, String>> mapByDateEntry : mapByDate.entrySet()) {
tableBuilder.append("<tr>");
tableBuilder.append("<td>").append(mapByDateEntry.getKey()).append("</td>");
Map<String, String> mapByTrans = mapByDateEntry.getValue();
for (String trans : allTrans) {
String sumtot = mapByTrans.get(trans);
if (sumtot != null) {
tableBuilder.append("<td>").append(sumtot).append("</td>");
} else {
tableBuilder.append("<td></td>");
}
}
tableBuilder.append("<tr>");
}
tableBuilder.append("</table>");
System.out.println(tableBuilder.toString());
The Output:
<table border = 1><tr><th>date/trans</th><th>11200</th><th>11201</th><th>11202</th></tr><tr><td>2019-07-02</td><td>136</td><td>18</td><td>14</td><tr><tr><td>2019-07-03</td><td>164</td><td>10</td><td>8</td><tr></table>
If we save generated output as an HTML file, It maybe your desired result (screenshot below). Also you can change the code to be used in JSP:
To have an ordered Map by natural order of its keys, a TreeMap can be used. So to print the data ordered by date, we can construct a new TreeMap containing the mapByDate data:
TreeMap<String, Map<String, String>> sortedMapByDate = new TreeMap<>(mapByDate);
// table rows
for (Map.Entry<String, Map<String, String>> mapByDateEntry : sortedMapByDate.entrySet()) {
tableBuilder.append("<tr>");
tableBuilder.append("<td>").append(mapByDateEntry.getKey()).append("</td>");
Map<String, String> mapByTrans = mapByDateEntry.getValue();
for (String trans : allTrans) {
String sumtot = mapByTrans.get(trans);
if (sumtot != null) {
tableBuilder.append("<td>").append(sumtot).append("</td>");
} else {
tableBuilder.append("<td></td>");
}
}
tableBuilder.append("<tr>");
}
It's wasteful to do this in Java code, that's what the window functions are for in SQL. If you have a query like SELECT datet, trans, sumtout FROM ... you can use SUM with OVER:
SELECT DISTINCT datet, SUM(sumtout) OVER (PARTITION BY datet)
FROM ...
ORDER BY datet;
I have a Map with different values:
props = new HashMap<String, Object>();
props.put("cmis:objectTypeId", "D:ruc:PLICO");
props.put("cmis:name", "PLICO_1.pdf");
props.put("cmis:description", "Descr");
props.put("ruc:doc_surname", "Rossi");
props.put("ruc:doc_name", "Mario");
I want to do a query (QueryStatement or other) that dynamically reads this parameters (some of them can be missing) and build QueryStatement.
Does it exist an easy way to generate the query String for QueryStatement? Or do I should iterate my Map to build a String containing all the parameters and values in my query?
My solution, but maybe somebody know how to improve it without dynamically build the query string:
StringBuilder query = new StringBuilder("SELECT * FROM ? where ");
String folder = null;
if (path!=null)
{
folder = findPath(path);
if (folder==null)
{
return null;
}
query.append("IN_FOLDER(?) AND ");
}
ArrayList <String> values = new ArrayList<String>();
Map<String, Object> properties = loadAnnotationAndData(doc);
String objectType = properties.remove(MyEnum.cmis_object_type_id.getValue()).toString();
for (Map.Entry<String, Object> entry : properties.entrySet())
{
System.out.println(entry.getKey() + " - " + entry.getValue());
query.append(entry.getKey() + "=? AND ");
values.add(entry.getValue().toString());
}
query.delete(query.length()-4, query.length());
query.append(" ORDER BY cmis:creationDate");
System.out.println(query.toString());
Session cmisSession = getCmisSession();
QueryStatement qs=
cmisSession.createQueryStatement(query.toString());
int offset = 1;
qs.setType(offset++, objectType);
if (path!=null)
{
qs.setString(offset++, folder);
}
for (int i=0; i<values.size(); i++)
{
System.out.println(values.get(i).toString());
qs.setString(i+offset, values.get(i).toString());
}
I am trying to iterate two hash maps and print the keys that are matching in both of them.Although both of the hash maps have got matching elements it is consistently saying "no match found".
Below posted is my code.
try {
String s = new String(data);
String string = new String(input_bytes);
StringTokenizer stringTokenizer = new StringTokenizer(s);
StringTokenizer input_stringTokenizer = new StringTokenizer(string);
while(stringTokenizer.hasMoreTokens())
{
map.put(stringTokenizer.nextToken(), stringTokenizer.nextToken());
}
while(input_stringTokenizer.hasMoreTokens())
{
input_map.put(input_stringTokenizer.nextToken(),
input_stringTokenizer.nextToken());
}}
catch (Exception e1) {
e1.printStackTrace();
}}
Iterator input1 = map.entrySet().iterator();
Iterator input_2 = input_map.entrySet().iterator();
while (input_2.hasNext() && input1.hasNext()) {
Map.Entry input_val1 = (Map.Entry) input1.next();
Map.Entry input_val2 = (Map.Entry) input_2.next();
String temp = input_val1.getKey().toString().substring(input_val1.getKey().toString().lastIndexOf("\\") + 1);
String temp_2 = input_val2.getKey().toString().substring(input_val2.getKey().toString().lastIndexOf("\\") + 1);
if(temp.equals(temp_2))
{
System.out.println("element matched");
}
else
{
System.out.println("no match found!");
}
}
My input files are "data" and "input_bytes"
The path in these files is the "key" and the hash is the "value" of the hashmap.
For effective matching i have trimmed the path such that it gives me only the element after the last slash.
"temp" variable in the code will print in the following way :
com.example.android.notepad_4.4.2-eng.build.20150616.1901504.apk ﹕
com.facebook.orca_34.0.0.22.2114.apk
com.android.contacts_4.4.2-eng.build.20150616.1901504.apk
com.amazon.venezia_release-13.0003.844.1C_6430003104.apk
com.android.deskclock_3.0.04.apk
com.google.android.apps.photos_1.0.0.943910814.apk
apuslauncher-2.apk
com.android.vending-v5.8.11-80381100-Android-2.3.apk
net.sylark.apkextractor_1.0.54.apk
Here is how my "data" file look like:
C:\Users\rishii\Desktop\input_3\com.amazon.venezia_release-
13.0003.844.1C_6430003104.apk
266796d1b8e2e016753ee3bf1b50e591
C:\Users\rishii\Desktop\input_3\com.android.browser_4.4.2-
eng.build.20150616.1901504.apk
4aa2091b0e21fc655e19d07e2ae20982
C:\Users\rishii\Desktop\input_3\com.android.calculator2_4.4.2-
eng.build.20150616.1901504.apk
85313ccbd39a43952906b70b941d321b
C:\Users\rishii\Desktop\input_3\com.android.calendar_4.4.2-
eng.build.20150616.1901504.apk
3c85cb87f2e134a4157e5f3747e4df1b
Here is my "input_bytes" file looks like:
C:\Users\rishii\Desktop\baal\com.amazon.venezia_release-
13.0003.844.1C_6430003104.apk
266796d1b8e2e016753ee3bf1b50e591
C:\Users\rishii\Desktop\baal\com.android.browser_4.4.2-
eng.build.20150616.1901504.apk
4aa2091b0e21fc655e19d07e2ae20982
C:\Users\rishii\Desktop\baal\com.android.calculator2_4.4.2-
eng.build.20150616.1901504.apk
85313ccbd39a43952906b70b941d321b
C:\Users\rishii\Desktop\baal\com.android.calendar_4.4.2-
eng.build.20150616.1901504.apk
3c85cb87f2e134a4157e5f3747e4df1b
C:\Users\rishii\Desktop\baal\com.android.camera2_2.0.002-
eng.build.ef73894.060315_142358-704.apk
482205cda6991f89fb35311dea668013
If you can see there are some matches in both the files.Any help would be highly appreciated.
Here's a much simpler way to check if they contain the same keys:
public void findSameKeys(Map<String, String> map1, Map<String, String> map2) {
for (String key : map1.keySet()) {
if (map2.containsKey(key)) {
System.out.println("Matching key: " + key);
}
}
}
The containsKey() method is very useful here.
I have a ConcurrentHashMap that contains a string as a key and LinkedList as a value. The size of the list should not be more than 5. I am trying to add some elements to the list but when I print out the Map I see only the last added element. Here is my code:
private ConcurrentHashMap<String, LinkedList<Date>> userDisconnectLogs = new ConcurrentHashMap<String, LinkedList<Date>>();
public void addNewDateEntry(String userId, LinkedList<Date> timeStamps) {
if (timeStamps.size() >= 5) {
timeStamps.poll();
timeStamps.add(new Date());
userDisconnectLogs.put(userId, timeStamps);
} else {
timeStamps.add(new Date());
userDisconnectLogs.put(userId, timeStamps);
}
for (Entry<String, LinkedList<Date>> entry : userDisconnectLogs
.entrySet()) {
String key = entry.getKey().toString();
;
LinkedList<Date> value = entry.getValue();
System.out.println("key: " + key + " value: " + value.size());
}
}
Thank you!
Here for hashMap key must be unique. And from this code it seems key is always same so all the time it will overrite the data.