I need your help because I don't find the solution in Java for my problem.
I stored in object LinkedHashMap<String, String> this content:
TAG1.TAG2.TAG11 : value1
TAG1.TAG2.TAG12 : value2
TAG1.TAG2.TAG3.TAG131 : value3
TAG1.TAG2.TAG3.TAG132 : value4
TAG1.TAG2.TAG3.TAG133 : value5
TAG1.TAG2.TAG3.TAG134 : value6
TAG1.TAG4.TAG5.TAG21 : value7
TAG1.TAG4.TAG5.TAG22 : value8
TAG1.TAG4.TAG5.TAG23 : value9
TAG6 : value10
I need to display if a tag has 2 or more children, the list of child.
Here is the expected result:
TAG1.TAG2
TAG11 : value1
TAG12 : value2
TAG1.TAG2.TAG3
TAG131 : value3
TAG132 : value4
TAG133 : value5
TAG134 : value6
TAG1.TAG4.TAG5
TAG21 : value7
TAG22 : value8
TAG23 : value9
TAG6 : value10
EDIT 14/06/2022 :
In fact, my original analyse is bad because initialy I have a XML file :
<TAG1>
<TAG2>
<TAG11>value1</TAG11>
<TAG12>value2</TAG12>
<TAG3>
<TAG131>value3</TAG131>
<TAG132>value4</TAG132>
<TAG133>value5</TAG133>
<TAG134>value6</TAG134>
</TAG3>
</TAG2>
<TAG4>
<TAG5>
<TAG21>value7</TAG21>
<TAG22>value8</TAG22>
<TAG23>value9</TAG23>
</TAG5>
</TAG4>
</TAG1>
<TAG6>value10</TAG6>
And I created a map to store it :
TAG1.TAG2.TAG11 : value1
TAG1.TAG2.TAG12 : value2
TAG1.TAG2.TAG3.TAG131 : value3
TAG1.TAG2.TAG3.TAG132 : value4
TAG1.TAG2.TAG3.TAG133 : value5
TAG1.TAG2.TAG3.TAG134 : value6
TAG1.TAG4.TAG5.TAG21 : value7
TAG1.TAG4.TAG5.TAG22 : value8
TAG1.TAG4.TAG5.TAG23 : value9
TAG6 : value10
But, today I have a this case :
<TAG1>
<TAG2>
<TAG11>value1</TAG11>
<TAG12>value2</TAG12>
<TAG3>
<TAG131>value3</TAG131>
<TAG132>value4</TAG132>
<TAG133>value5</TAG133>
<TAG134>value6</TAG134>
</TAG3>
<TAG3>
<TAG131>value11</TAG131>
<TAG132>value12</TAG132>
<TAG133>value13</TAG133>
<TAG134>value14</TAG134>
</TAG3>
</TAG2>
<TAG4>
<TAG5>
<TAG21>value7</TAG21>
<TAG22>value8</TAG22>
<TAG23>value9</TAG23>
</TAG5>
</TAG4>
</TAG1>
<TAG6>value10</TAG6>
But the Map object does not allow to store many keys (in the example many TAG3). Have you got an idea how I can resolve this problem ?
EDIT 15/06/2022 :
In fact the expected result needs to keep the original XML structure.
Here the result of last sample :
TAG1.TAG2
TAG11 : value1
TAG12 : value2
TAG1.TAG2.TAG3
TAG131 : value3
TAG132 : value4
TAG133 : value5
TAG134 : value6
TAG1.TAG2.TAG3
TAG131 : value11
TAG132 : value12
TAG133 : value13
TAG134 : value14
TAG1.TAG4.TAG5
TAG21 : value7
TAG22 : value8
TAG23 : value9
TAG6 : value10
It's to display xml more human reader.
EDIT 04/07/2022 :
I detect a problem of inconsistent with "new TreeMap<>(Comparator.comparingInt(MyTag::getAppearanceOrder)". Indeed, some MyTag object are the same AppearanceOrder, so there is a problem of inconsistent ordering. Some value in Map are so removed.
To resolve I used :
map.entrySet().stream().sorted(Map.Entry.comparingByKey(. . .))
And I store the result in Map with collect().
Below the working code :
public class Main {
public static void main(String[] args) throws ParserConfigurationException, IOException, SAXException {
//Accessing the xml file
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document document = builder.parse(new File("data.xml"));
document.getDocumentElement().normalize();
Element root = document.getDocumentElement();
//Retrieving a List of records where each record contains: the original chain of tags, the numbered chain of tags and the value
String tagSep = ".";
List<Record> listRecords = new ArrayList<>();
visitXMLFile(listRecords, root.getChildNodes(), tagSep, "", "", new HashMap<>());
//Queue sorted by the numbered tag's length in descending order (from the longest to the shortest)
PriorityQueue<Record> queue = new PriorityQueue<>(Comparator.comparing(Record::getTagNumberedLen).reversed());
queue.addAll(listRecords);
//Using a set to have unique numbered tags (no duplicates) to group by in the resulting map
Set<MyTag> setMyTags = new HashSet<>();
//Checking for each numbered tag if its largest substring is equal to any other numbered tag's beginning:
// - if it does, then the substring is collected as a key to group by within the final map
//
// - if it doesn't, then another substring is generated from the previous substring until a matching value is found.
// If no value is found, then the numbered tag is collected entirely as a key for the resulting map.
while (!queue.isEmpty()) {
Record rec = queue.poll();
//This loop keeps creating substrings of the current numbered tag until:
// - the substring matches another numbered tag's beginning
// - or no more substrings can be generated
int lastIndexTagNum = rec.getTagNumbered().lastIndexOf(tagSep);
int lastIndexTag = rec.getTag().lastIndexOf(tagSep);
while (lastIndexTagNum > 0) {
//Checking if the substring matches the beginning of any numbered tag except the current one
String subStrTagNum = rec.getTagNumbered().substring(0, lastIndexTagNum);
if (listRecords.stream().anyMatch(r -> !r.getTagNumbered().equals(rec.getTagNumbered()) && r.getTagNumbered().startsWith(subStrTagNum + tagSep))) {
String subStrTag = rec.getTag().substring(0, lastIndexTag);
int appearanceOrder = listRecords.stream().filter(r -> r.getTagNumbered().startsWith(subStrTagNum + tagSep)).map(r -> r.getAppearanceOrder()).min(Comparator.naturalOrder()).orElse(0);
//If a match is found then the current substring is added to the set and the substring iteration is interrupted
setMyTags.add(new MyTag(subStrTag, subStrTagNum + tagSep, appearanceOrder));
break;
}
//Creating a new substring from the previous substring if no match has been found
lastIndexTagNum = rec.getTagNumbered().substring(0, lastIndexTagNum).lastIndexOf(tagSep);
lastIndexTag = rec.getTag().substring(0, lastIndexTag).lastIndexOf(tagSep);
}
//If no substrings of the current numbered tag matches the beginning of any other numbered tag,
//then the current numbered tag is collected as a key for the resulting map
if (lastIndexTagNum < 0) {
int appearanceOrder = listRecords.stream().filter(r -> r.getTagNumbered().startsWith(rec.getTagNumbered())).map(r -> r.getAppearanceOrder()).min(Comparator.naturalOrder()).orElse(0);
setMyTags.add(new MyTag(rec.getTag(), rec.getTagNumbered(), appearanceOrder));
}
}
//Creating a temporary resulting map (not sorted as the input)
Map<MyTag, List<String>> mapTemp = listRecords.stream()
.collect(Collectors.toMap(
rec -> {
//Looking for the longest numbered tag which matches the beginning of the current record's numbered tag.
//The reason why we need the longest match (i.e. the most accurate) is because some elements
//may share the same parents but be on different levels, for example the values 3, 4, 5 and 6
//have a key whose beginning matches both "TAG1.TAG2" and "TAG1.TAG2.TAG3", but only the longest
//match is actually the right one.
return setMyTags.stream().filter(mt -> rec.getTagNumbered().startsWith(mt.getTagNumbered())).max(Comparator.comparingInt(MyTag::getTagNumberedLen)).orElseThrow(() -> new RuntimeException("No key found"));
},
rec -> {
//Retrieving, like above, the numbered tag that will be used to map the current value
MyTag myTag = setMyTags.stream().filter(mt -> rec.getTagNumbered().startsWith(mt.getTagNumbered())).max(Comparator.comparingInt(MyTag::getTagNumberedLen)).orElseThrow(() -> new RuntimeException("No key found"));
//If the new numbered tag and the record's numbered tag are equal then a List with the current value is returned
if (myTag.getTagNumbered().equals(rec.getTagNumbered())) {
return new ArrayList<>(List.of(rec.getValue()));
} else { //If the new numbered tag is a substring of the record's numbered tag then the rest of the current (non-numbered) tag is added to the value
return new ArrayList<>(List.of(rec.getTag().substring(myTag.getTag().length() + 1) + " : " + rec.getValue()));
}
},
//Handling colliding cases by merging the lists together
(list1, list2) -> {
list1.addAll(list2);
return list1;
}
)
);
//Creating a TreeMap whose ordering is based on the insertion order of the input
Map<MyTag, List<String>> mapRes =
mapTemp.entrySet().stream()
.sorted(Map.Entry.comparingByKey(Comparator.comparingInt(MyTag::getAppearanceOrder)))
.collect(Collectors.toMap(
Map.Entry::getKey, Map.Entry::getValue, (e1, e2) -> e1, LinkedHashMap::new));
//Printing the resulting map
for (Map.Entry<MyTag, List<String>> entry : mapRes.entrySet()) {
System.out.println(entry.getKey());
for (String value : entry.getValue()) {
System.out.println("\t" + value);
}
}
}
private static void visitXMLFile(List<Record> listInput, NodeList nodeList, String tagSep, String tag, String tagNumbered, Map<String, Integer> mapTagOccurrence) {
for (int i = 0; i < nodeList.getLength(); i++) {
Node node = nodeList.item(i);
if (node.hasChildNodes()) {
String newTag = tag.isEmpty() ? node.getNodeName() : tag + tagSep + node.getNodeName();
//Setting or incrementing the number of appearances of a tag chain
//(sometimes a same chain of tags can be repeated, ex: TAG1.TAG2.TAG3)
if (!mapTagOccurrence.containsKey(newTag)) {
mapTagOccurrence.put(newTag, 1);
} else {
mapTagOccurrence.computeIfPresent(newTag, (key, val) -> val + 1);
}
//Creating a numbered version of the tag where its number of appearances is added at the end.
//This is done to uniquely identify different groups of tag chain when these are repeated (ex: TAG1.TAG2.TAG3)
String newTagNum = tagNumbered.isEmpty() ? node.getNodeName() + mapTagOccurrence.get(newTag) : tagNumbered + tagSep + node.getNodeName() + mapTagOccurrence.get(newTag);
visitXMLFile(listInput, node.getChildNodes(), tagSep, newTag, newTagNum, mapTagOccurrence);
} else {
if (!node.getTextContent().trim().equals("")) {
int appearanceOrder = listInput.size() + 1;
listInput.add(new Record(tag, tagNumbered, node.getTextContent().trim(), appearanceOrder));
}
}
}
}
}
class MyTag {
//Tag chain for the user
private String tag;
//Unique tag chain for identification
private String tagNumbered;
private int appearanceOrder;
public MyTag(String tag, String tagNumbered, int appearanceOrder) {
this.tag = tag;
this.tagNumbered = tagNumbered;
this.appearanceOrder = appearanceOrder;
}
public String getTag() {
return tag;
}
public String getTagNumbered() {
return tagNumbered;
}
public int getTagNumberedLen() {
return tagNumbered == null ? 0 : tagNumbered.length();
}
public int getAppearanceOrder() {
return appearanceOrder;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
MyTag tagPair = (MyTag) o;
return Objects.equals(tagNumbered, tagPair.tagNumbered);
}
#Override
public int hashCode() {
return Objects.hash(tagNumbered);
}
#Override
public String toString() {
return tag;
}
}
class Record {
//Tag chain for the user
private String tag;
//Unique tag chain for identification
private String tagNumbered;
private String value;
private int appearanceOrder;
public Record(String tag, String tagNumbered, String value, int appearanceOrder) {
this.tag = tag;
this.tagNumbered = tagNumbered;
this.value = value;
this.appearanceOrder = appearanceOrder;
}
public String getTag() {
return tag;
}
public String getTagNumbered() {
return tagNumbered;
}
public int getTagNumberedLen() {
return tagNumbered == null ? 0 : tagNumbered.length();
}
public String getValue() {
return value;
}
public int getAppearanceOrder() {
return appearanceOrder;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Record record = (Record) o;
return Objects.equals(tagNumbered, record.tagNumbered);
}
#Override
public int hashCode() {
return Objects.hash(tagNumbered);
}
#Override
public String toString() {
return tag + " - " + tagNumbered + " - " + value;
}
}
Edit
At this point to answer your edited question, I had to use a List instead of a Map to store the input since multiple values share the same key and a Map<String, List<String>> wouldn't maintain the insertion order. In fact, the values from 3 to 6 would be alternated with the values from 11 to 14.
Besides, since the same chain of tags can appear several times (ex: TAG1.TAG2.TAG3), I had to implement two custom classes: MyTag and Record.
The first class represents a custom tag made of two fields: tag and tagNumbered. The first field holds the tag chain that must be shown to the user, while the second is used as the actual identifier to group by in the stream operation. tagNumbered is basically a copy of tag where at the end of each nested tag is added its number of appearances.
Instead, the class Record is used to represent a value accompanied by its tag chain and numbered tag chain.
So, the following XML is represented as follows by the respective classes:
<x>
<y>
<z>value1</z>
</y>
<y>
<z>value2</z>
</y>
</x>
Record:
Record1:
- tag: x.y.z
- tagNumbered: x1.y1.z1
- value: value1
Record2:
- tag: x.y.z
- tagNumbered: x1.y2.z1 //because y appears twice within x
- value: value2
MyTag (MyTag is created from Record):
MyTag1:
- tag: x.y.z
- tagNumbered: x1.y1.z1
MyTag2:
- tag: x.y.z
- tagNumbered: x1.y2.z1 //because y appears twice within x
Here is an XML sample based on your question's input, that I've used for the code below.
<root>
<TAG1>
<TAG2>
<TAG11>value1</TAG11>
<TAG12>value2</TAG12>
<TAG3>
<TAG131>value3</TAG131>
<TAG132>value4</TAG132>
<TAG133>value5</TAG133>
<TAG134>value6</TAG134>
</TAG3>
<TAG3>
<TAG131>value11</TAG131>
<TAG132>value12</TAG132>
<TAG133>value13</TAG133>
<TAG134>value14</TAG134>
</TAG3>
</TAG2>
<TAG4>
<TAG5>
<TAG21>value7</TAG21>
<TAG22>value8</TAG22>
<TAG23>value9</TAG23>
</TAG5>
</TAG4>
</TAG1>
<TAG6>value10</TAG6>
</root>
Original Answer Updated
The first part of the problem consists in creating a List<Record> while reading from the XML file which is achieved with the visitXMLFile method.
After reading the records from the file, we need to create a Set of unique numbered tag chains to identify each group of values. This is actually done with a Set<MyTag>; however MyTag's equals() and hashCode() are based exclusively on tagNumbered.
After creating the Set of unique numbered tags, we need to stream the input list of entries with a single operation: collect(Collectors.toMap()). In this operation, each record is mapped to a MyTag (i.e., a numbered tag) of the Set previously created.
Finally, to maintain the original insertion order, the resulting Map has been implemented as a TreeMap initialized with a Comparator defined on the order of the input list's records.
Here is an implementation with detailed comments explaining the whole logic step by step:
public class Main {
public static void main(String[] args) throws ParserConfigurationException, IOException, SAXException {
//Accessing the xml file
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document document = builder.parse(new File("data.xml"));
document.getDocumentElement().normalize();
Element root = document.getDocumentElement();
//Retrieving a List of records where each record contains: the original chain of tags, the numbered chain of tags and the value
String tagSep = ".";
List<Record> listRecords = new ArrayList<>();
visitXMLFile(listRecords, root.getChildNodes(), tagSep, "", "", new HashMap<>());
//Queue sorted by the numbered tag's length in descending order (from the longest to the shortest)
PriorityQueue<Record> queue = new PriorityQueue<>(Comparator.comparing(Record::getTagNumberedLen).reversed());
queue.addAll(listRecords);
//Using a set to have unique numbered tags (no duplicates) to group by in the resulting map
Set<MyTag> setMyTags = new HashSet<>();
//Checking for each numbered tag if its largest substring is equal to any other numbered tag's beginning:
// - if it does, then the substring is collected as a key to group by within the final map
//
// - if it doesn't, then another substring is generated from the previous substring until a matching value is found.
// If no value is found, then the numbered tag is collected entirely as a key for the resulting map.
while (!queue.isEmpty()) {
Record rec = queue.poll();
//This loop keeps creating substrings of the current numbered tag until:
// - the substring matches another numbered tag's beginning
// - or no more substrings can be generated
int lastIndexTagNum = rec.getTagNumbered().lastIndexOf(tagSep);
int lastIndexTag = rec.getTag().lastIndexOf(tagSep);
while (lastIndexTagNum > 0) {
//Checking if the substring matches the beginning of any numbered tag except the current one
String subStrTagNum = rec.getTagNumbered().substring(0, lastIndexTagNum);
if (listRecords.stream().anyMatch(r -> !r.getTagNumbered().equals(rec.getTagNumbered()) && r.getTagNumbered().startsWith(subStrTagNum + tagSep))) {
String subStrTag = rec.getTag().substring(0, lastIndexTag);
int appearanceOrder = listRecords.stream().filter(r -> r.getTagNumbered().startsWith(subStrTagNum + tagSep)).map(r -> r.getAppearanceOrder()).min(Comparator.naturalOrder()).orElse(0);
//If a match is found then the current substring is added to the set and the substring iteration is interrupted
setMyTags.add(new MyTag(subStrTag, subStrTagNum + tagSep, appearanceOrder));
break;
}
//Creating a new substring from the previous substring if no match has been found
lastIndexTagNum = rec.getTagNumbered().substring(0, lastIndexTagNum).lastIndexOf(tagSep);
lastIndexTag = rec.getTag().substring(0, lastIndexTag).lastIndexOf(tagSep);
}
//If no substrings of the current numbered tag matches the beginning of any other numbered tag,
//then the current numbered tag is collected as a key for the resulting map
if (lastIndexTagNum < 0) {
int appearanceOrder = listRecords.stream().filter(r -> r.getTagNumbered().startsWith(rec.getTagNumbered())).map(r -> r.getAppearanceOrder()).min(Comparator.naturalOrder()).orElse(0);
setMyTags.add(new MyTag(rec.getTag(), rec.getTagNumbered(), appearanceOrder));
}
}
//Creating a temporary resulting map (not sorted as the input)
Map<MyTag, List<String>> mapTemp = listRecords.stream()
.collect(Collectors.toMap(
rec -> {
//Looking for the longest numbered tag which matches the beginning of the current record's numbered tag.
//The reason why we need the longest match (i.e. the most accurate) is because some elements
//may share the same parents but be on different levels, for example the values 3, 4, 5 and 6
//have a key whose beginning matches both "TAG1.TAG2" and "TAG1.TAG2.TAG3", but only the longest
//match is actually the right one.
return setMyTags.stream().filter(mt -> rec.getTagNumbered().startsWith(mt.getTagNumbered())).max(Comparator.comparingInt(MyTag::getTagNumberedLen)).orElseThrow(() -> new RuntimeException("No key found"));
},
rec -> {
//Retrieving, like above, the numbered tag that will be used to map the current value
MyTag myTag = setMyTags.stream().filter(mt -> rec.getTagNumbered().startsWith(mt.getTagNumbered())).max(Comparator.comparingInt(MyTag::getTagNumberedLen)).orElseThrow(() -> new RuntimeException("No key found"));
//If the new numbered tag and the record's numbered tag are equal then a List with the current value is returned
if (myTag.getTagNumbered().equals(rec.getTagNumbered())) {
return new ArrayList<>(List.of(rec.getValue()));
} else { //If the new numbered tag is a substring of the record's numbered tag then the rest of the current (non-numbered) tag is added to the value
return new ArrayList<>(List.of(rec.getTag().substring(myTag.getTag().length() + 1) + " : " + rec.getValue()));
}
},
//Handling colliding cases by merging the lists together
(list1, list2) -> {
list1.addAll(list2);
return list1;
}
)
);
//Creating a TreeMap whose ordering is based on the insertion order of the input
Map<MyTag, List<String>> mapRes = new TreeMap<>(Comparator.comparingInt(MyTag::getAppearanceOrder));
mapRes.putAll(mapTemp);
//Printing the resulting map
for (Map.Entry<MyTag, List<String>> entry : mapRes.entrySet()) {
System.out.println(entry.getKey());
for (String value : entry.getValue()) {
System.out.println("\t" + value);
}
}
}
private static void visitXMLFile(List<Record> listInput, NodeList nodeList, String tagSep, String tag, String tagNumbered, Map<String, Integer> mapTagOccurrence) {
for (int i = 0; i < nodeList.getLength(); i++) {
Node node = nodeList.item(i);
if (node.hasChildNodes()) {
String newTag = tag.isEmpty() ? node.getNodeName() : tag + tagSep + node.getNodeName();
//Setting or incrementing the number of appearances of a tag chain
//(sometimes a same chain of tags can be repeated, ex: TAG1.TAG2.TAG3)
if (!mapTagOccurrence.containsKey(newTag)) {
mapTagOccurrence.put(newTag, 1);
} else {
mapTagOccurrence.computeIfPresent(newTag, (key, val) -> val + 1);
}
//Creating a numbered version of the tag where its number of appearances is added at the end.
//This is done to uniquely identify different groups of tag chain when these are repeated (ex: TAG1.TAG2.TAG3)
String newTagNum = tagNumbered.isEmpty() ? node.getNodeName() + mapTagOccurrence.get(newTag) : tagNumbered + tagSep + node.getNodeName() + mapTagOccurrence.get(newTag);
visitXMLFile(listInput, node.getChildNodes(), tagSep, newTag, newTagNum, mapTagOccurrence);
} else {
if (!node.getTextContent().trim().equals("")) {
int appearanceOrder = listInput.size() + 1;
listInput.add(new Record(tag, tagNumbered, node.getTextContent().trim(), appearanceOrder));
}
}
}
}
}
class MyTag {
//Tag chain for the user
private String tag;
//Unique tag chain for identification
private String tagNumbered;
private int appearanceOrder;
public MyTag(String tag, String tagNumbered, int appearanceOrder) {
this.tag = tag;
this.tagNumbered = tagNumbered;
this.appearanceOrder = appearanceOrder;
}
public String getTag() {
return tag;
}
public String getTagNumbered() {
return tagNumbered;
}
public int getTagNumberedLen() {
return tagNumbered == null ? 0 : tagNumbered.length();
}
public int getAppearanceOrder() {
return appearanceOrder;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
MyTag tagPair = (MyTag) o;
return Objects.equals(tagNumbered, tagPair.tagNumbered);
}
#Override
public int hashCode() {
return Objects.hash(tagNumbered);
}
#Override
public String toString() {
return tag;
}
}
class Record {
//Tag chain for the user
private String tag;
//Unique tag chain for identification
private String tagNumbered;
private String value;
private int appearanceOrder;
public Record(String tag, String tagNumbered, String value, int appearanceOrder) {
this.tag = tag;
this.tagNumbered = tagNumbered;
this.value = value;
this.appearanceOrder = appearanceOrder;
}
public String getTag() {
return tag;
}
public String getTagNumbered() {
return tagNumbered;
}
public int getTagNumberedLen() {
return tagNumbered == null ? 0 : tagNumbered.length();
}
public String getValue() {
return value;
}
public int getAppearanceOrder() {
return appearanceOrder;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Record record = (Record) o;
return Objects.equals(tagNumbered, record.tagNumbered);
}
#Override
public int hashCode() {
return Objects.hash(tagNumbered);
}
#Override
public String toString() {
return tag + " - " + tagNumbered + " - " + value;
}
}
Output
TAG1.TAG2
TAG11 : value1
TAG12 : value2
TAG1.TAG2.TAG3
TAG131 : value3
TAG132 : value4
TAG133 : value5
TAG134 : value6
TAG1.TAG2.TAG3
TAG131 : value11
TAG132 : value12
TAG133 : value13
TAG134 : value14
TAG1.TAG4.TAG5
TAG21 : value7
TAG22 : value8
TAG23 : value9
TAG6
value10
Related
First of all, I have been reading a few posts about keys, but none of them asks my question on how to get ALL keys of a yaml file, only on how to get an specific key.
Now, I want to create a file updater, it works, but it only updates the first keys, without the "sub-keys", here is the code:
InputStream resource = getClass().getClassLoader().getResourceAsStream(dir);
Map<String, Object> data = new Yaml().load(resource);
for(String str : data.keySet()) {
DBot.getConsole().log(str);
if(!contains(str)) {
set(str, data.get(str));
}
}
The file looks like this:
Features.Example.StringA
Features.Example.StringB
With points being spaces to make them sub-keys (stack overflow puts them on a single line, sorry)
Now the thing is, the updater will only work if "Features" is deleted, also, the debug will only print "Features", meaning that only the first key is on the key set, how can I get all keys?
I have finally found how to return a Set with every key separated by a ".", Bukkit/Spigot developers might be familiar with this. First of all, you have to create a class like this:
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
public class YamlKeys {
private static Set<String> keys = new HashSet<String>();
private static String path = "";
YamlKeys(Map<?, ?> data) {
getKeysRecursive(data);
}
private void getKeysRecursive(final Map<?, ?> data) {
for(Object key : data.keySet()) {
final Object value = data.get(key);
if(key instanceof String) {
if(path.length() == 0) {
path = (String)key; // If the key is the first on the path, don't include separator.
} else {
path = path+"."+(String)key; // Here is the separator, you can change it.
}
}
if(value instanceof Map) {
getKeysRecursive((Map<?, ?>) value); // A value map has been found, recursing with that value.
} else {
keys.add(path); // No more maps have been found, we can add the path and stop recursing.
if(path.contains(".")) {
path = path.substring(0, path.lastIndexOf(".")); // Removing last key, so if a value contains more than one key, it won't appear again.
}
}
}
path = ""; // This is important, reset the path.
}
Set<String> getKeys() {
return keys;
}
}
Then, to call it and select if you want to get deep keys or "normal" keys, you can create a method like this:
public Set<String> getKeys(boolean deep) {
Map<String, Object> data = new Yaml().load(inStream);
if(!deep) {
return data.keySet();
} else {
return new YamlKeys(data).getKeys();
}
}
To test it, we can use the following code:
new YamlKeys(data).getKeys().stream().forEach(key -> System.out.println(key));
With this file:
FirstKey:
SecondKey:
Enabled: true
Text: "Some text"
AnotherKey:
AValue: true
AnotherTest:
Enabled: false
Value: true
It returns this output:
FirstKey.SecondKey.AnotherKey.AValue
FirstKey.SecondKey.Enabled
FirstKey.SecondKey.Text
Value
AnotherTest.Enabled
Thanks to roby for telling me about recursion.
SnakeYAML is decoding the yaml into a recursive data structure. For example:
public static void main(String[] args) {
String yaml = "a:\n b: \n c: \"string\"";
Map<String, Object> data = new Yaml().load(yaml);
System.out.println(data);
}
prints out:
{a={b={c=string}}}
Which is a Map<String, Map<String, Map<String, String>>>.
To show how you can work with it recursively, here's how you can print out some of that detail.
private static void printMapRecursive(final Map<?, ?> data) {
for(Object key : data.keySet()) {
System.out.println("key " + key + " is type " + key.getClass().getSimpleName());
final Object value = data.get(key);
if(value instanceof Map){
System.out.println("value for " + key + " is a Map - recursing");
printMapRecursive((Map<?, ?>) value);
} else {
System.out.println("value " + value + " for " + key + " is type " + value.getClass());
}
}
}
Which you can call with printMapRecursive(data); and see output:
key a is type String
value for a is a Map - recursing
key b is type String
value for b is a Map - recursing
key c is type String
value string for c is type class java.lang.String
and an example of recursively transforming the keys:
private static Map<?, ?> mutateMapRecursive(final Map<?, ?> data,
Function<String, String> keyFunction) {
Map<Object, Object> result = new HashMap<>();
for (Object key : data.keySet()) {
final Object value = data.get(key);
if(key instanceof String){
key = keyFunction.apply((String) key);
}
if (value instanceof Map) {
result.put(key, mutateMapRecursive((Map<?, ?>) value, keyFunction));
}
else {
result.put(key, value);
}
}
return result;
}
called like:
final Map<?, ?> transformed = mutateMapRecursive(data, (key) -> "prefix_" + key);
System.out.println(transformed);
emits:
{prefix_a={prefix_b={prefix_c=string}}}
Attribute class that contains key and value:
public class Attribute {
private String key;
private String value;
//setter and getter method
}
Main class
public class DroolsMain {
public static HashMap<String , Attribute> hashMapAttribute = new HashMap<String , Attribute>();
public static void main(String[] args) throws DroolsParserException, IOException {
String Keys[] = {"From" , "To" , "Subject" };
String attributesValues[] = { "Sathish" ,"aranjan" , "email" };
for(int i =0;i<Keys.length;i++)
{
Attribute attribute = new Attribute();
attribute.setKey(Keys[i]);
attribute.setValue(attributesValues[i]);
hashMapAttribute.put("" + i, attribute);
}
//...
workingMemory.insert(hashMapAttribute);
}
}
In rule.drl file, I want to compare key-value of an object with another similar type of object which present in HashMap.
rule.drl
rule "Rule 1:"
when
$mapAttribute : Map()
$entry : Entry() from $mapAttribute.entrySet()
$attribute : Attribute() from $entry.getValue()
$attribute1 : Attribute(this != $attribute) from $entry.getValue()
Boolean(booleanValue == true) from (($attribute.getKey contains("From") && $attribute.getValue contains("Sathish")) && ($attribute1.getKey contains("Subject") && $attribute1.getValue contains("email")))
then
System.out.println("Rule 1 run successfully ");
end
How I compare two object value. Please Help me to find solution.
I get a solution from changing rule.drl file:
rule "Rule 1:"
when
$mapAttribute : Map()
$entry : Entry() from $mapAttribute.entrySet()
$attribute : Attribute() from $entry.getValue()
$mapAttribute1 : Map()
$entry1 : Entry() from $mapAttribute1.entrySet()
$attribute1 : Attribute() from $entry1.getValue()
Boolean(booleanValue == true) from (($attribute.getKey contains("From") && $attribute.getValue contains("Sathish")) && ($attribute1.getKey contains("Subject") && $attribute1.getValue contains("email")))
then
System.out.println("Rule 1 run successfully ");
end
I know it may be a very silly question, but I am new to DynamoDB.
My doubt is is it possible to update the value of a Range Key in DynamoDB.
Suppose My Table is "TEST"
{
ID : PK/HK
Date : RK
Name : GSI
Add : LSI
}
I want to modify Date Attribute.
Initial Values in Table was:
{
ID = "344"
Date = "5656"
Name = "ABC"
}
Running this code below. I am able to change the Name Attribute which is GSI.
Map<String,AttributeValue> item = new HashMap<String,AttributeValue>();
item.put("ID", new AttributeValue("344"));
item.put("Date", new AttributeValue("5656"));
Map<String,AttributeValueUpdate> item1 = new HashMap<String,AttributeValueUpdate>();
AttributeValueUpdate update = new AttributeValueUpdate().withValue(new AttributeValue("AMIT")).withAction("PUT");
item1.put("Name", update);
UpdateItemRequest updateItemreq = new UpdateItemRequest("Test",item,item1);
UpdateItemResult updateItemres = dynamoDBUSEast.updateItem(updateItemreq);
But When I change this line
item1.put("Name", update);
with
item1.put("Date", update);
I am getting some error as
Exception in thread "main" com.amazonaws.AmazonServiceException: One or more parameter values were invalid: Cannot update attribute Date. This attribute is part of the key (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: HRRP24Q7C48AMD8ASAI992L6MBVV4KQNSO5AEMVJF66Q9ASUAAJG)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:820)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:439)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:245)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:2908)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.updateItem(AmazonDynamoDBClient.java:1256)
So Is it possible to change the range Key value?
No, like the exception message states, you Cannot update attribute Date. This attribute is part of the key.
You can also see this under the AttributeUpdates documentation:
The names of attributes to be modified, the action to perform on each,
and the new value for each. If you are updating an attribute that is
an index key attribute for any indexes on that table, the attribute
type must match the index key type defined in the AttributesDefinition
of the table description. You can use UpdateItem to update any nonkey
attributes.
The documentation states that you can update any attribute for "an attribute that is an index key attribute for any indexes on that table", which means that when you update an attribute that is projected onto an index, even it is is part of that indexes key, that index will also be updated to reflect the original item.
From the docs of AttributeValueUpdate
You cannot use UpdateItem to update any primary key attributes.
Instead, you will need to delete the item, and then use PutItem to
create a new item with new attributes.
It's a little buried but in docs for UpdateItem it says:
"You can use UpdateItem to update any nonkey attributes."
So, currently the only way to update the primary key of an item is to delete the old item and write a new one.
Here is my implementation of updating id in .net by deleting the item and then recreating it with the new id. I assume java is very similar:
// Based on https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/LowLevelDotNetItemsExample.html
public class UpdateId
{
private static string tableName = "MyTableName";
private static AmazonDynamoDBClient client = new AmazonDynamoDBClient();
private static bool isVerbose = false;
public static void ChangeId(string currentId, string newId)
{
try
{
var deletedItem = DeleteItem(currentId);
if (deletedItem.Count == 0)
{
Console.WriteLine($"ERROR: Item to delete not found: {currentId}");
return;
}
deletedItem["Id"] = new AttributeValue
{
S = newId
};
CreateItem(deletedItem);
var updatedItem = RetrieveItem(newId);
if (updatedItem.Count > 0 && updatedItem["Id"].S == newId)
{
Console.WriteLine($"Item id successfully changed from ({currentId}) to ({newId})");
}
else
{
Console.WriteLine($"ERROR: Item id didn't change from ({currentId}) to ({newId})");
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
Console.WriteLine("To continue, press Enter");
Console.ReadLine();
}
}
private static void CreateItem(Dictionary<string, AttributeValue> item)
{
var request = new PutItemRequest
{
TableName = tableName,
Item = item
};
client.PutItem(request);
}
private static Dictionary<string, AttributeValue> RetrieveItem(string id)
{
var request = new GetItemRequest
{
TableName = tableName,
Key = new Dictionary<string, AttributeValue>()
{
{ "Id", new AttributeValue {
S = id
} }
},
ConsistentRead = true
};
var response = client.GetItem(request);
// Check the response.
var attributeList = response.Item; // attribute list in the response.
if (isVerbose)
{
Console.WriteLine("\nPrinting item after retrieving it ............");
PrintItem(attributeList);
}
return attributeList;
}
private static Dictionary<string, AttributeValue> DeleteItem(string id)
{
var request = new DeleteItemRequest
{
TableName = tableName,
Key = new Dictionary<string, AttributeValue>()
{
{ "Id", new AttributeValue {
S = id
} }
},
// Return the entire item as it appeared before the update.
ReturnValues = "ALL_OLD",
// ExpressionAttributeNames = new Dictionary<string, string>()
// {
// {"#IP", "InPublication"}
// },
// ExpressionAttributeValues = new Dictionary<string, AttributeValue>()
// {
// {":inpub",new AttributeValue {
// BOOL = false
// }}
// },
// ConditionExpression = "#IP = :inpub"
};
var response = client.DeleteItem(request);
// Check the response.
var attributeList = response.Attributes; // Attribute list in the response.
// Print item.
if (isVerbose)
{
Console.WriteLine("\nPrinting item that was just deleted ............");
PrintItem(attributeList);
}
return attributeList;
}
private static void PrintItem(Dictionary<string, AttributeValue> attributeList)
{
foreach (KeyValuePair<string, AttributeValue> kvp in attributeList)
{
string attributeName = kvp.Key;
AttributeValue value = kvp.Value;
Console.WriteLine(
attributeName + " " +
(value.S == null ? "" : "S=[" + value.S + "]") +
(value.N == null ? "" : "N=[" + value.N + "]") +
(value.SS == null ? "" : "SS=[" + string.Join(",", value.SS.ToArray()) + "]") +
(value.NS == null ? "" : "NS=[" + string.Join(",", value.NS.ToArray()) + "]")
);
}
Console.WriteLine("************************************************");
}
}
To call it just do this:
UpdateId.ChangeId("OriginalId", "NewId");
I got a question regarding XML and parsing it. I use JDOM to parse my XML-File, but I got a little Problem.
A sample of my XML-File looks like this:
<IO name="Bus" type="Class">
<ResourceAttribute name="Bandwidth" type="KiloBitPerSecond" value="50" />
</IO>
Bus is a object instance of the class IO. The object got the name and type properties. Additional it has some attributes, like in the sample, the Attribute Bandwidth with the value of 50 and the datatype KiloBitPerSecond.
So when I want to loop over the file with:
for(Element packages : listPackages)
{
Map<String, Values> valueMap = new HashMap<String, Values>();
List<Element> objectInstanceList = packages.getChildren();
for(Element objects : objectInstanceList)
{
List<Element> listObjectClasses = objects.getChildren();
for(Element classes : listObjectClasses)
{
List<Element> listObjectAttributes = classes.getChildren();
for(Element objectAttributes : listObjectAttributes)
{
List<Attribute> listAttributes = objectAttributes.getAttributes();
for(Attribute attributes : listAttributes)
{
String name = attributes.getName();
String value = attributes.getValue();
AttributeType datatype = attributes.getAttributeType();
Values v = new Values(name, datatype, value);
valueMap.put(classes.getName(), v);
System.out.println(name + ":" + value);
}
}
}
}
//System.out.println(valueMap);
}
values is a class which defines the object attribute:
public class Values{
private String name;
//private AttributeType datatype;
private String value;
Thats the rest of the Code. I got two question relating that. The first one got more priority at the moment.
How do I get the values of the object(Attribute.Name = Bandwidth; Attribute.Value = 50) ? Istead that I get
name:Bus
type:Class
I thought about an additional for-loop, but the JDOM class attribute dont have a method called getAttributes().
Thats just second priority because without question 1 I cannot go further. As you see in the sample, an Attribute got 3 properties, name, type and value. How can I extract that triple put of the sample. JDOM seems just to know 2 properties for an Attribute, name and value.
thanks a lot in advance and hopefully I managed to express my self.
Edit: Added an additional for-loop in it, so the output now is:
name:Bandwidth
type:KiloBitPerSecond
value:50
That means name is the name of that property and value is the value of name. Didnt know that. At least question one is clearer now and I can try working on 2, but the new information makes 2 clearer to me.
In xml the opening tag of elements are encosoed between < and > (or />) , after the < comes the name of the element, then comes a list of attributes in the format name="value". An element can be closed inline with /> or with a closing tag </[element name]>
It would be preferable to use recursion to parse your xml instead of badly readable/maintainable nested for loops.
Here is how it could look like:
#Test
public void parseXmlRec() throws JDOMException, IOException {
String xml = "<root>"
+ "<Package>"
+ "<IO name=\"Bus\" type=\"Class\">\r\n" +
" <ResourceAttribute name=\"Bandwidth\" type=\"KiloBitPerSecond\" value=\"50\" />\r\n" +
" </IO>"
+ "</Package>"
+ "</root>";
InputStream is = new ByteArrayInputStream(xml.getBytes());
SAXBuilder sb = new SAXBuilder();
Document document = sb.build(is);
is.close();
Element root = document.getRootElement();
List<Element> children = root.getChildren();
for(Element element : children) {
parseelement(element);
}
}
private void parseelement(Element element) {
System.out.println("Element:" + element.getName());
String name = element.getAttributeValue("name");
if(name != null) {
System.out.println("name: " + name);
}
String type = element.getAttributeValue("type");
if(type != null) {
System.out.println("type: " + type);
}
String value = element.getAttributeValue("value");
if(value != null) {
System.out.println("value: " + value);
}
List<Element> children = element.getChildren();
if(children != null) {
for(Element child : children) {
parseelement(child);
}
}
}
This outputs:
Element: Package
Element: IO
name: Bus
type: Class
Element: ResourceAttribute
name: Bandwidth
type: KiloBitPerSecond
value: 50
While parsing, check the name of each element and instanciate the coresponding objects. For that I would suggest to write a separate method to handle each element. For example:
void parsePackage(Element packageElement) { ... }
parseIO(Element ioElement) { ... }
void parseResourceAttribute(Element resourceAttributeElement) { ... }
I am using Jericho HTML Parser to parse some malformed html. In particular I am trying to get all text nodes, process the text and then replace it.
I want to skip specific elements from processing. For example I want to skip all elements, and any element that has attribute class="noProcess". So, if a div has class="noProcess" then I want to skip this div and all children from processing. However, I do want these skipped elements to return back to the output after processing.
Jericho provides an Iterator for all nodes but I am not sure how to skip complete elements from the Iterator. Here is my code:
private String doProcessHtml(String html) {
Source source = new Source(html);
OutputDocument outputDocument = new OutputDocument(source);
for (Segment segment : source) {
if (segment instanceof Tag) {
Tag tag = (Tag) segment;
System.out.println("FOUND TAG: " + tag.getName());
// DO SOMETHING HERE TO SKIP ENTIRE ELEMENT IF IS <A> OR CLASS="noProcess"
} else if (segment instanceof CharacterReference) {
CharacterReference characterReference = (CharacterReference) segment;
System.out.println("FOUND CHARACTERREFERENCE: " + characterReference.getCharacterReferenceString());
} else {
System.out.println("FOUND PLAIN TEXT: " + segment.toString());
outputDocument.replace(segment, doProcessText(segment.toString()));
}
}
return outputDocument.toString();
}
It doesn't look like using the ignoreWhenParsing() method works for me as the parser just treats the "ignored" element as text.
I was thinking that if I could convert the Iterator loop to a for (int i = 0;...) loop I could probably be able to skip the element and all its children by modifying i to point to the EndTag and then continue the loop.... but not sure.
I think you might want to consider a redesign of the way your segments are built. Is there a way to parse the html in such a way that each segment is a parent element that contains a nested list of child elements? That way you could do something like:
for (Segment segment : source) {
if (segment instanceof Tag) {
Tag tag = (Tag) segment;
System.out.println("FOUND TAG: " + tag.getName());
// DO SOMETHING HERE TO SKIP ENTIRE ELEMENT IF IS <A> OR CLASS="noProcess"
continue;
} else if (segment instanceof CharacterReference) {
CharacterReference characterReference = (CharacterReference) segment;
System.out.println("FOUND CHARACTERREFERENCE: " + characterReference.getCharacterReferenceString());
for(Segment child : segment.childNodes()) {
//Use recursion to process child elements
//You will want to put your for loop in a separate method so it can be called recursively.
}
} else {
System.out.println("FOUND PLAIN TEXT: " + segment.toString());
outputDocument.replace(segment, doProcessText(segment.toString()));
}
}
Without more code to inspect its hard to determine if restructuring the segment element is even possible or worth the effort.
Managed to have a working solution by using the getEnd() method of the Element object of the Tag. The idea is to skip elements if their end position is less than a position you set. So you find the end position of the element you want to exclude and you do not process anything else before that position:
final ArrayList<String> excludeTags = new ArrayList<String>(Arrays.asList(new String[] {"head", "script", "a"}));
final ArrayList<String> excludeClasses = new ArrayList<String>(Arrays.asList(new String[] {"noProcess"}));
Source.LegacyIteratorCompatabilityMode = true;
Source source = new Source(htmlToProcess);
OutputDocument outputDocument = new OutputDocument(source);
int skipToPos = 0;
for (Segment segment : source) {
if (segment.getBegin() >= skipToPos) {
if (segment instanceof Tag) {
Tag tag = (Tag) segment;
Element element = tag.getElement();
// check excludeTags
if (excludeTags.contains(tag.getName().toLowerCase())) {
skipToPos = element.getEnd();
}
// check excludeClasses
String classes = element.getAttributeValue("class");
if (classes != null) {
for (String theClass : classes.split(" ")) {
if (excludeClasses.contains(theClass.toLowerCase())) {
skipToPos = element.getEnd();
}
}
}
} else if (segment instanceof CharacterReference) { // for future use. Source.LegacyIteratorCompatabilityMode = true;
CharacterReference characterReference = (CharacterReference) segment;
} else {
outputDocument.replace(segment, doProcessText(segment.toString()));
}
}
}
return outputDocument.toString();
This should work.
String skipTag = null;
for (Segment segment : source) {
if (skipTag != null) { // is skipping ON?
if (segment instanceof EndTag && // if EndTag found for the
skipTag.equals(((EndTag) segment).getName())) { // tag we're skipping
skipTag = null; // set skipping OFF
}
continue; // continue skipping (or skip the EndTag)
} else if (segment instanceof Tag) { // is tag?
Tag tag = (Tag) segment;
System.out.println("FOUND TAG: " + tag.getName());
if (HTMLElementName.A.equals(tag.getName()) { // if <a> ?
skipTag = tag.getName(); // set
continue; // skipping ON
} else if (tag instanceof StartTag) {
if ("noProcess".equals( // if <tag class="noProcess" ..> ?
((StartTag) tag).getAttributeValue("class"))) {
skipTag = tag.getName(); // set
continue; // skipping ON
}
}
} // ...
}