The idea of the function is that it will build a map that contains a key (path) and the data corresponding to that path. This is why I keep passing the map back as it's being built.
The problem seems to be that it gets to the point where there are no children, yet still appends the next path onto the current path:
Input for the path will always start with "/". We get the children for this, which may be level_1A, level_1C, level_1B. Then I recurse on each of these to see if they have children.
Assume that level_1A has children level_2A, level_2B. Somehow, the algorithm gets caught up and is appending like so:
/level_1A/level_2B/level_2A
Whereas, it should be treating these seperately, like this:
/level_1A/level_2A
/level_1A/level_2B
Here's what the entire structure would look like:
/level_1A data_1A
/level_1A/level_2A data_2A
/level_1A/level_2B data_2B
/level_1B (empty)
/level_1C (empty)
Here's the recursive method:
public Map<String, String> getAll(String path, Map<String, String> all) throws Exception {
List<String> children = client.getChildren().forPath(path);
if(children != null && children.size() > 0) {
for(String child: children) {
System.out.println("child: " + child);
if(!path.equals("/")) {
path = path + "/" + child;
} else {
path = path + child;
}
Stat stat = client.checkExists().watched().forPath(path);
if(stat != null && stat.getDataLength() > 0) {
all.put(path, new String(client.getData().forPath(path)));
}
getAll(path, all);
}
}
return all;
}
The error is here:
for(String child: children) {
if(!path.equals("/")) {
path = path + "/" + child;
} else {
path = path + child;
}
...
}
path variable is out of for loop scope, so in the first iteration you've modified path variable and in the second iteration that modified value is been concatenated with second child and then been passed deeper.
So Just provide for-loop scoped variable and use it in iteration:
for(String child: children) {
String pathChild = path;
if(!path.equals("/")) {
pathChild = path + "/" + child;
} else {
pathChild = path + child;
}
//pathChild is used below
...
}
Related
I have a Map that has been alphabetically sorted by converting it using TreeMap.
The Map contains both a String (installer file name) and Path (installer path on file system) for instance
Map installers;
I need to obtain the most recent installer file name. However, regex seems like it'd be too complicated.
The code I have currently to display the installers and their paths is this:
Map<String, Path> installers = findInstallers();
Set s = installers.entrySet();
Iterator it = s.iterator();
while(it.hasNext()) {
Map.Entry entry = (Map.Entry) it.next();
String installerFile = (String) entry.getKey();
Path installerPath = (Path) entry.getValue();
System.out.println(installerFile + " ==> " + installerPath.toString());
}
System.out.println("================================");
private Map<String, Path> findInstallers() {
HashMap<String, Path> installerPathMap = new HashMap<>();
try {
Path productReleasePath = Paths.get("C:", "test");
List<Path> allPaths = Files.walk(productReleasePath)
.filter(Files::isRegularFile)
.collect(Collectors.toList());
allPaths.forEach(path -> {
if (!path.toFile().getName().toLowerCase().endsWith(".log")) {
String installerFiileName = path.toFile().getName();
installerPathMap.put(installerFiileName, path);
}
});
} catch (IOException e) {
e.printStackTrace();
}
return new TreeMap<>(installerPathMap);
}
This is a sample output:
Client_1.exe ==> C:\test\build_1\Win32\Client_1.exe
Client_5.exe ==> C:\test\build_5\Win32\Client_5.exe
Client_6.exe ==> C:\test\build_6\Win32\Client_6.exe
Server_1.exe ==> C:\test\build_1\Win64\Server_1.exe
Server_2.exe ==> C:\test\build_2\Win64\Server_2.exe
Server_Linux_1.tar.gz ==> C:\test\build_1\Linux32\Server_Linux_1.tar.gz
Server_Linux_2.tar.gz ==> C:\test\build_2\Linux32\Server_Linux_1.tar.gz
================================
I need to shorten my Map to only contain the highest key and it's value pair, so the output is similar to this:
Client_6.exe ==> C:\test\build_6\Win32\Client_6.exe
Server_2.exe ==> C:\test\build_2\Win64\Server_2.exe
Server_Linux_2.tar.gz ==> C:\test\build_2\Linux32\Server_Linux_1.tar.gz
================================
Any help would be greatly appreciated.
If you add the paths to a map using the root of the installer name as a key (i.e. the part before the underscore), and discard the lowest version when there is a key collision, you'll get what you want.
Note that sorting the names alphabetically won't work because version 9 will sort after 10, so you'll have to extract the version and do a numeric comparison.
I'm not certain of your naming convention, but the helper functions in the following example should be easy enough to modify if my assumptions aren't correct.
public class InstallerList {
public static void main(String[] args) throws IOException {
Path productReleasePath = Paths.get("C:", "test");
Collection<Path> installers = Files.walk(productReleasePath)
.filter(Files::isRegularFile)
.filter(p -> !p.getFileName().toString().endsWith(".log"))
// Collect files with the highest version
.collect(Collectors.toMap(
// Key is installer name *without* version
InstallerList::extractName,
// Value mapper; identity mapping to the path
p -> p,
// Select newest version when there is a collision
InstallerList::newer
))
.values();
for (Path path : installers) {
System.out.println(path.getFileName() + " ==> " + path);
}
}
// Extract the root name of an installer from a path (up to but not including the last '_')
public static String extractName(Path path) {
String fileName = path.getFileName().toString();
int i = fileName.lastIndexOf('_');
if (i < 0) {
throw new IllegalArgumentException(fileName);
}
return fileName.substring(0, i);
}
// Return the path with the highest version number
public static Path newer(Path p1, Path p2) {
return extractVersion(p1) > extractVersion(p2) ? p1 : p2;
}
// Extract a version number from a path (number between the last '_' and the following '.')
private static int extractVersion(Path path) {
String fileName = path.getFileName().toString();
int i = fileName.lastIndexOf('_');
if (i < 0) {
throw new IllegalArgumentException(fileName);
}
int j = fileName.indexOf('.', i);
if (j < 0) {
throw new IllegalArgumentException(fileName);
}
return Integer.parseInt(fileName.substring(i + 1, j));
}
}
I'm working on a procedure that should find the lowest-weighted path between two nodes using Dijkstra's algorithm. The procedure should only return paths whose all nodes match specific criteria (i.e. all nodes should have properties with specific values). If at least one node in a path doesn't match the criteria, then the path becomes invalid, and the algorithm should look for the next lowest-weighted path.
In order to achieve this, I'm using a PathExpanderBuilder with node filters, but they don't seem to filter anything.
Here is my code:
public class Demo {
#Procedure
#Description("apoc.algo.dijkstraWithFilters(startNode, endNode, " +
"'distance', 10, 'prop1', 2, 'prop2', [100, 200], 'prop3') " +
" YIELD path, weight - run dijkstra with relationship property name as cost function" +
" and a default weight if the property does not exist")
public Stream<WeightedPathResult> dijkstraWithFilters(
#Name("startNode") Node startNode,
#Name("endNode") Node endNode,
#Name("weightPropertyName") String weightPropertyName,
#Name("defaultWeight") double defaultWeight,
#Name("longPropName") String longPropName,
#Name("longPropValue") long longPropValue,
#Name("listPropName") String listPropName,
#Name("listPropValues") List<Long> listPropValues,
#Name("boolPropName") String boolPropName) {
PathFinder<WeightedPath> algo = GraphAlgoFactory.dijkstra(
buildPathExpanderByPermissions(longPropName, longPropValue, listPropName, listPropValues, boolPropName),
(relationship, direction) -> convertToDouble(relationship.getProperty(weightPropertyName, defaultWeight))
);
return WeightedPathResult.streamWeightedPathResult(startNode, endNode, algo);
}
private double convertToDouble(Object property) {
if (property instanceof Double)
return (double) property;
else if (property instanceof Long)
return ((Long) property).doubleValue();
else if (property instanceof Integer)
return ((Integer) property).doubleValue();
return 1;
}
private PathExpander<Object> buildPathExpanderByPermissions(
String longPropName,
long longPropValue,
String listPropName,
List<Long> listPropValue,
String boolPropName
) {
PathExpanderBuilder builder = PathExpanderBuilder.allTypesAndDirections();
builder.addNodeFilter(
node -> !node.hasProperty(longPropName) ||
node.getProperty(longPropName) instanceof Long &&
(long) node.getProperty(longPropName) < longPropValue
);
builder.addNodeFilter(
node -> {
try {
return !node.hasProperty(listPropName) ||
(boolean) node.getProperty(boolPropName, false) ||
!Collections.disjoint((List<Long>) node.getProperty(listPropName), listPropValue);
}
catch (Exception e){
return false;
}
}
);
return builder.build();
}
}
What am I missing here? Am I making a wrong use of PathExpanderBuilder?
PathExpanderBuilder's are immutable and so calling e.g. addNodeFilter returns a new PathExpanderBuilder with the added filter and so you need to re-assign your builder with that returned instance.
How do I use SOOT to build at Call graph? Or are there any better programs for this? I have been sent around the same five pages looking for answers and I can't find what I am looking for. There are also a problem with the plugin version to Eclipse. It is installed correct but I cant choose it when I want to run the code.
Small modification to previous answer
private static void visit(CallGraph cg, SootMethod method) {
String identifier = method.getSignature();
visited.put(method.getSignature(), true);
dot.drawNode(identifier);
// iterate over unvisited parents
Iterator<MethodOrMethodContext> ptargets = new Sources(cg.edgesInto(method));
if (ptargets != null) {
while (ptargets.hasNext()) {
SootMethod parent = (SootMethod) ptargets.next();
if (!visited.containsKey(parent.getSignature())) visit(cg, parent);
}
}
Here are some examples include call graph for Java. http://www.brics.dk/SootGuide/
And call graph for apk.
https://github.com/secure-software-engineering/soot-infoflow/issues/38
If you want to get the dot file, you can iterate over the callgraph and write the contents out in dot format like this.
private static void visit(CallGraph cg, SootMethod method) {
String identifier = method.getSignature();
visited.put(method.getSignature(), true);
dot.drawNode(identifier);
// iterate over unvisited parents
Iterator<MethodOrMethodContext> ptargets = new Targets(cg.edgesInto(method));
if (ptargets != null) {
while (ptargets.hasNext()) {
SootMethod parent = (SootMethod) ptargets.next();
if (!visited.containsKey(parent.getSignature())) visit(cg, parent);
}
}
// iterate over unvisited children
Iterator<MethodOrMethodContext> ctargets = new Targets(cg.edgesOutOf(method));
if (ctargets != null) {
while (ctargets.hasNext()) {
SootMethod child = (SootMethod) ctargets.next();
dot.drawEdge(identifier, child.getSignature());
System.out.println(method + " may call " + child);
if (!visited.containsKey(child.getSignature())) visit(cg, child);
}
}
}
I got a question regarding XML and parsing it. I use JDOM to parse my XML-File, but I got a little Problem.
A sample of my XML-File looks like this:
<IO name="Bus" type="Class">
<ResourceAttribute name="Bandwidth" type="KiloBitPerSecond" value="50" />
</IO>
Bus is a object instance of the class IO. The object got the name and type properties. Additional it has some attributes, like in the sample, the Attribute Bandwidth with the value of 50 and the datatype KiloBitPerSecond.
So when I want to loop over the file with:
for(Element packages : listPackages)
{
Map<String, Values> valueMap = new HashMap<String, Values>();
List<Element> objectInstanceList = packages.getChildren();
for(Element objects : objectInstanceList)
{
List<Element> listObjectClasses = objects.getChildren();
for(Element classes : listObjectClasses)
{
List<Element> listObjectAttributes = classes.getChildren();
for(Element objectAttributes : listObjectAttributes)
{
List<Attribute> listAttributes = objectAttributes.getAttributes();
for(Attribute attributes : listAttributes)
{
String name = attributes.getName();
String value = attributes.getValue();
AttributeType datatype = attributes.getAttributeType();
Values v = new Values(name, datatype, value);
valueMap.put(classes.getName(), v);
System.out.println(name + ":" + value);
}
}
}
}
//System.out.println(valueMap);
}
values is a class which defines the object attribute:
public class Values{
private String name;
//private AttributeType datatype;
private String value;
Thats the rest of the Code. I got two question relating that. The first one got more priority at the moment.
How do I get the values of the object(Attribute.Name = Bandwidth; Attribute.Value = 50) ? Istead that I get
name:Bus
type:Class
I thought about an additional for-loop, but the JDOM class attribute dont have a method called getAttributes().
Thats just second priority because without question 1 I cannot go further. As you see in the sample, an Attribute got 3 properties, name, type and value. How can I extract that triple put of the sample. JDOM seems just to know 2 properties for an Attribute, name and value.
thanks a lot in advance and hopefully I managed to express my self.
Edit: Added an additional for-loop in it, so the output now is:
name:Bandwidth
type:KiloBitPerSecond
value:50
That means name is the name of that property and value is the value of name. Didnt know that. At least question one is clearer now and I can try working on 2, but the new information makes 2 clearer to me.
In xml the opening tag of elements are encosoed between < and > (or />) , after the < comes the name of the element, then comes a list of attributes in the format name="value". An element can be closed inline with /> or with a closing tag </[element name]>
It would be preferable to use recursion to parse your xml instead of badly readable/maintainable nested for loops.
Here is how it could look like:
#Test
public void parseXmlRec() throws JDOMException, IOException {
String xml = "<root>"
+ "<Package>"
+ "<IO name=\"Bus\" type=\"Class\">\r\n" +
" <ResourceAttribute name=\"Bandwidth\" type=\"KiloBitPerSecond\" value=\"50\" />\r\n" +
" </IO>"
+ "</Package>"
+ "</root>";
InputStream is = new ByteArrayInputStream(xml.getBytes());
SAXBuilder sb = new SAXBuilder();
Document document = sb.build(is);
is.close();
Element root = document.getRootElement();
List<Element> children = root.getChildren();
for(Element element : children) {
parseelement(element);
}
}
private void parseelement(Element element) {
System.out.println("Element:" + element.getName());
String name = element.getAttributeValue("name");
if(name != null) {
System.out.println("name: " + name);
}
String type = element.getAttributeValue("type");
if(type != null) {
System.out.println("type: " + type);
}
String value = element.getAttributeValue("value");
if(value != null) {
System.out.println("value: " + value);
}
List<Element> children = element.getChildren();
if(children != null) {
for(Element child : children) {
parseelement(child);
}
}
}
This outputs:
Element: Package
Element: IO
name: Bus
type: Class
Element: ResourceAttribute
name: Bandwidth
type: KiloBitPerSecond
value: 50
While parsing, check the name of each element and instanciate the coresponding objects. For that I would suggest to write a separate method to handle each element. For example:
void parsePackage(Element packageElement) { ... }
parseIO(Element ioElement) { ... }
void parseResourceAttribute(Element resourceAttributeElement) { ... }
I am using Jericho HTML Parser to parse some malformed html. In particular I am trying to get all text nodes, process the text and then replace it.
I want to skip specific elements from processing. For example I want to skip all elements, and any element that has attribute class="noProcess". So, if a div has class="noProcess" then I want to skip this div and all children from processing. However, I do want these skipped elements to return back to the output after processing.
Jericho provides an Iterator for all nodes but I am not sure how to skip complete elements from the Iterator. Here is my code:
private String doProcessHtml(String html) {
Source source = new Source(html);
OutputDocument outputDocument = new OutputDocument(source);
for (Segment segment : source) {
if (segment instanceof Tag) {
Tag tag = (Tag) segment;
System.out.println("FOUND TAG: " + tag.getName());
// DO SOMETHING HERE TO SKIP ENTIRE ELEMENT IF IS <A> OR CLASS="noProcess"
} else if (segment instanceof CharacterReference) {
CharacterReference characterReference = (CharacterReference) segment;
System.out.println("FOUND CHARACTERREFERENCE: " + characterReference.getCharacterReferenceString());
} else {
System.out.println("FOUND PLAIN TEXT: " + segment.toString());
outputDocument.replace(segment, doProcessText(segment.toString()));
}
}
return outputDocument.toString();
}
It doesn't look like using the ignoreWhenParsing() method works for me as the parser just treats the "ignored" element as text.
I was thinking that if I could convert the Iterator loop to a for (int i = 0;...) loop I could probably be able to skip the element and all its children by modifying i to point to the EndTag and then continue the loop.... but not sure.
I think you might want to consider a redesign of the way your segments are built. Is there a way to parse the html in such a way that each segment is a parent element that contains a nested list of child elements? That way you could do something like:
for (Segment segment : source) {
if (segment instanceof Tag) {
Tag tag = (Tag) segment;
System.out.println("FOUND TAG: " + tag.getName());
// DO SOMETHING HERE TO SKIP ENTIRE ELEMENT IF IS <A> OR CLASS="noProcess"
continue;
} else if (segment instanceof CharacterReference) {
CharacterReference characterReference = (CharacterReference) segment;
System.out.println("FOUND CHARACTERREFERENCE: " + characterReference.getCharacterReferenceString());
for(Segment child : segment.childNodes()) {
//Use recursion to process child elements
//You will want to put your for loop in a separate method so it can be called recursively.
}
} else {
System.out.println("FOUND PLAIN TEXT: " + segment.toString());
outputDocument.replace(segment, doProcessText(segment.toString()));
}
}
Without more code to inspect its hard to determine if restructuring the segment element is even possible or worth the effort.
Managed to have a working solution by using the getEnd() method of the Element object of the Tag. The idea is to skip elements if their end position is less than a position you set. So you find the end position of the element you want to exclude and you do not process anything else before that position:
final ArrayList<String> excludeTags = new ArrayList<String>(Arrays.asList(new String[] {"head", "script", "a"}));
final ArrayList<String> excludeClasses = new ArrayList<String>(Arrays.asList(new String[] {"noProcess"}));
Source.LegacyIteratorCompatabilityMode = true;
Source source = new Source(htmlToProcess);
OutputDocument outputDocument = new OutputDocument(source);
int skipToPos = 0;
for (Segment segment : source) {
if (segment.getBegin() >= skipToPos) {
if (segment instanceof Tag) {
Tag tag = (Tag) segment;
Element element = tag.getElement();
// check excludeTags
if (excludeTags.contains(tag.getName().toLowerCase())) {
skipToPos = element.getEnd();
}
// check excludeClasses
String classes = element.getAttributeValue("class");
if (classes != null) {
for (String theClass : classes.split(" ")) {
if (excludeClasses.contains(theClass.toLowerCase())) {
skipToPos = element.getEnd();
}
}
}
} else if (segment instanceof CharacterReference) { // for future use. Source.LegacyIteratorCompatabilityMode = true;
CharacterReference characterReference = (CharacterReference) segment;
} else {
outputDocument.replace(segment, doProcessText(segment.toString()));
}
}
}
return outputDocument.toString();
This should work.
String skipTag = null;
for (Segment segment : source) {
if (skipTag != null) { // is skipping ON?
if (segment instanceof EndTag && // if EndTag found for the
skipTag.equals(((EndTag) segment).getName())) { // tag we're skipping
skipTag = null; // set skipping OFF
}
continue; // continue skipping (or skip the EndTag)
} else if (segment instanceof Tag) { // is tag?
Tag tag = (Tag) segment;
System.out.println("FOUND TAG: " + tag.getName());
if (HTMLElementName.A.equals(tag.getName()) { // if <a> ?
skipTag = tag.getName(); // set
continue; // skipping ON
} else if (tag instanceof StartTag) {
if ("noProcess".equals( // if <tag class="noProcess" ..> ?
((StartTag) tag).getAttributeValue("class"))) {
skipTag = tag.getName(); // set
continue; // skipping ON
}
}
} // ...
}