I am working with code which makes use of java.nio.file.Path. I have a path like
/tmp/something/first/second/third/last
which I see only as
{parent.dir}/first/second/third/{path.end}
In this example /tmp/something or {parent.dir} is a path which can be different during runtime and which is irrelevant for me. The same applies for the last element in the path {path.end}
What I would need is to iterate over elements between {parent.dir} and {path.end}. Basically to test each element in the path if it exists and if it is a file or
a directory or something else (it doesn't matter).
I am therefore looking for some elegant/simple and proper way to iterate over instance of Pathwhere I can access paths like these:
/tmp
/tmp/something/
/tmp/something/first
...
/tmp/something/first/second/third/last
ideally in this case without the first 2 and the last iteration.
I am looking for a good solution using Path and java.nio and not the old way. I know I can achieve this using old io API but I am interested in the proper way of using nio.
Here I print all directories child of your parent dir:
Files.walk(Paths.get(${parent.dir})).filter(path -> Files.isDirectory(path, LinkOption.NOFOLLOW_LINKS)).forEach(System.out::println);
You can pass another lambda to the forEach method for your own purpose.
Also replace ${parent.dir} with your correct value as a String.
(The code above may throw IOException).
Assuming that the base and end parts are parameters, while the in-between part is fixed, a solution can look like this:
static void iterate(Path base, Path end) {
if(!base.isAbsolute() || end.isAbsolute()) throw new
IllegalArgumentException("base must be absolute, end must be relative");
// test the fixed in-between paths
Path fixed=Paths.get("first", "second", "third");
for(Path part: fixed) {
base=base.resolve(part);
System.out.print(base);
if(Files.isDirectory(base)) {
System.out.println(" is a directory");
}
else {
System.out.println(Files.exists(part)?" is not a directory":" does not exist");
return;
}
}
// test the end path
end=base.resolve(end);
System.out.print(end+(
Files.isDirectory(end)? " is a directory":
Files.exists(end)? " is not a directory": " does not exist"));
}
It stops the iteration once it encounters a non-directory path component. You will have to adapt this behavior if you want to enforce a specific policy regarding following symbolic links…
Related
picoCLI's #-file mechanism is almost what I need, but not exactly. The reason is that I want to control the exact location of additional files parsed -- depending on previous option values.
Example: When called with the options
srcfolder=/a/b optionfile=of.txt, my program should see the additional options read from /a/b/of.txt, but when called with srcfolder=../c optionfile=of.txt, it should see those from ../c/of.txt.
The #-file mechanism can't do that, because it expands ALL the option files (always relative to the current folder, if they're relative) prior to processing ANY option values.
So I'd like to have picoCLI...
process options "from left to right",
recursively parse an option file when it's mentioned in an optionfile option,
and after that continue with the following options.
I might be able to solve this by recursively starting to parse from within the annotated setter method:
...
Config cfg = new Config();
CommandLine cmd = new CommandLine(cfg);
cmd.parseArgs(a);
...
public class Config {
#Option(names="srcfolder")
public void setSrcfolder(String path) {
this.srcfolder=path;
}
#Option(names="optionfile")
public void parseOptionFile(String pathAndName) {
// validate path, do some other housekeeping...
CommandLine cmd = new CommandLine(this /* same Config instance! */ );
cmd.parseArgs(new String[] { "#"+this.srcfolder + pathAndName });
}
...
This way several CommandLine instances would call setter methods on the same Config instance, recursively "interrupting" each other. Now comes the actual question: Is that a problem?
Of course my Config class has state. But do CommandLine instances also have state that might get messed up if other CommandLine instances also modify cfg "in between options"?
Thanks for any insights!
Edited to add: I tried, and I'm getting an UnmatchedArgumentException on the #-file option:
Exception in thread "main" picocli.CommandLine$UnmatchedArgumentException: Unmatched argument at index 0: '#/path/to/configfile'
at picocli.CommandLine$Interpreter.validateConstraints(CommandLine.java:13490)
...
So first I have to get around this: Obviously picoCLI doesn't expand the #-file option unless it's coming directly from the command line.
I did get it to work: several CommandLine instance can indeed work on the same instance of an annotated class, without interfering with each other.
There are some catches and I had to work around a strange picoCLI quirk, but that's not exactly part of an answer to this question, so I explain them in this other question.
I am working on fixing Veracode issues in my application. Veracode has highlighted the flaw "External Control of File Name or Path (CWE ID 73) " in below code.
Thread.currentThread().getContextClassLoader().getResourceAsStream(lookupName)
How do I validate the parameter? If I need to use below ESAPI validation, then what is the exact parameter I should be passing in getValidFileName() method. Currently I am passing the parameters as below.
ESAPI.validator().getValidFileName(lookupName, lookupName,
ESAPI.securityConfiguration().getAllowedFileExtensions(), false);
Correct me whether I am following the right approach for fixing this issue.
There are several suggestions at: https://community.veracode.com/s/article/how-do-i-fix-cwe-73-external-control-of-file-name-or-path-in-java
You can use hardcoded values, if these files are stored in the server side.
(i.e.: in a HashMap).
Another solution is to use a custom validator (from veracode page) :
// GOOD Code
String extension = request.getParameter("extension");
File f = new File(buildValidAvatarPath(extension))
#FilePathCleanser
public String buildValidAvatarPath(extension) {
String[] allowedExtensions = new String[]{"jpg","gif","png"};
String extension = "png"; // Default extension
for (String allowedExtension: allowedExtensions) {
if (allowedExtension.equals(request.getParameter("extension"))) {
extension = request.getParameter("extension");
}
}
// See "Note on authorization"
User user = getCurrentUser();
if (!userMayAccessFile(user, path)) {
throw new AuthorizationException("User may not access this file", user);
}
File(configPath + "avatar." + extension)
return path;
}
Okay, so the problem is that you are allowing user-control of that file path. Imagine its on a UNIX box and they enter:
../../../../../../../etc/shadow
Whatever user privileges are granted to the user running that java Thread is possible to expose to the user in question. I don't know what processing is going on in your application, but the danger is that you need to prevent user control of that lookup variable.
The call you're making is consistent with the single test in ValidatorTest.java, which is definitely a deficiency in code coverage on our behalf.
Now, there's an excellent chance that even if you use this call that Veracode might still flag it: the default file list in ESAPI.properties will need to be either truncated for your use case, or you'll have to create your own Validator rule for legal file extensions for your specific use case.
Which brings up the next bit: There's a lot of mischief that can happen in regards to file uploads.
In short, to be actually secure about file uploads will require more than what ESAPI currently offers, which is unfortunately, only an extension check. In your particular case, make sure you try some directory traversal attacks. And use that OWASP link to help analyze your application.
Given that the OP wants to clear the issue in Veracode, you would want to chain a couple calls:
ESAPI.validator().getValidDirectoryPath() and ESAPI.Validator.getValidFileName()
But be sure you've properly truncated the extension list in HttpUtilities.ApprovedUploadExtensions in validator.properties as the default list is too permissive, at least until we release 2.1.0.2.
I have to stress however that even with this particular combination there is absolutely nothing ESAPI does to prevent a user from renaming "netcat.exe" to "puppies.xlsx" and bypassing your validation check, that's why the rant on the first part of this answer.
ESAPI's file validation is NOT secure, it's quite simply better than nothing at all.
Doing this correctly requires more work than just using 1-2 calls to ESAPI.
DISCLAIMER: as of this writing I am the project co-lead for ESAPI.
You can change file name by sanitizing it as below code snippet:
private static String sanitizeFileName(String name) {
return name
.chars()
.mapToObj(i -> (char) i)
.map(c -> Character.isWhitespace(c) ? '_' : c)
.filter(c -> Character.isLetterOrDigit(c) || c == '-' || c == '_' || c == ':')
.map(String::valueOf)
.collect(Collectors.joining());
}
For some methods I get the warning. This is what it says when expanded.
The following code (mkDirs()) gives the warning
if (!myDir.exists()) {
myDir.mkdirs();
}
Reports any calls to specific methods where the result of that call is ignored. Both methods specified in the inspection's settings and methods annotated with org.jetbrains.annotations.Contract(pure=true) are checked. For many methods, ignoring the result is perfectly legitimate, but for some methods it is almost certainly an error. Examples of methods where ignoring the result of a call is likely to be an error include java.io.inputStream.read(), which returns the number of bytes actually read, any method on java.lang.String or java.math.BigInteger, as all of those methods are side-effect free and thus pointless if ignored.
What does it mean? How to to avoid it? How should it be addressed?
It is possible to omit the warning using annotation
#SuppressWarnings("ResultOfMethodCallIgnored")
public void someMethod() {
...
myDir.mkdirs();
...
}
If the directory exists, the mkdir() operation will return false, if it does not exist, it will be created (if you have the appropriate rights, of course), therefore, IMHO, the check via isExists() can be omitted.
However, as indicated above, if you are working with this directory, it is a good idea to make sure that it exists.
This method suggests that you are creating directory with your specified path if directory is not already created then this function will create a new one for you.It is returning true/false weather directory exists or created.Perhaps in some situation due to low storage it not created at specified path and you are trying to write contents into file within that directory it will throw ioException.So you should utilise if condition
if(myDir.mkdirs())
{
//execute what ever you want to do
}
else
{
// show error message for any failure.
}
In a DirectoryWalker class I want to find out if a File instance is actually a symbolic link to a directory (assuming, the walker walks on UNIX systems). Given, I already know the instance is a directory, would the following be a reliable condition to determine the symbolic link?
File file;
// ...
if (file.getAbsolutePath().equals(file.getCanonicalPath())) {
// real directory ---> do normal stuff
}
else {
// possible symbolic link ---> do link stuff
}
The technique used in Apache Commons uses the canonical path to the parent directory, not the file itself. I don't think that you can guarantee that a mismatch is due to a symbolic link, but it's a good indication that the file needs special treatment.
This is Apache code (subject to their license), modified for compactness.
public static boolean isSymlink(File file) throws IOException {
if (file == null)
throw new NullPointerException("File must not be null");
File canon;
if (file.getParent() == null) {
canon = file;
} else {
File canonDir = file.getParentFile().getCanonicalFile();
canon = new File(canonDir, file.getName());
}
return !canon.getCanonicalFile().equals(canon.getAbsoluteFile());
}
Java 1.6 does not provide such low level access to the file system. Looks like NIO 2, which should be included in Java 1.7, will have support for symbolic links. A draft of the new API is available. Symbolic links are mentioned there, creating and following them is possible. I'm not exactly sure that which method should be used to find out whether a file is a symbolic link. There's a mailing list for discussing NIO 2 - maybe they will know.
Also, watch out for file.isFile() and file.isDirectory() both returning results based on the resolved file and therefore both returning false when file refers to a symlink where the target doesn't exist.
(I know this isn't a useful answer in itself but it tripped me up a couple of times so thought I should share)
It looks like getCanonicalPath() can do other things that might make it different from the absolute path.
This method first converts this pathname to absolute form if necessary, as if by invoking the getAbsolutePath() method, and then maps it to its unique form in a system-dependent way. This typically involves removing redundant names such as "." and ".." from the pathname, resolving symbolic links (on UNIX platforms), and converting drive letters to a standard case (on Microsoft Windows platforms).
But it might work for the vast majority of your use cases; your mileage may vary.
If you are already coding something specifically for *nix, then you could do a shell command from Java like this:
Process p = Runtime.getRuntime().exec(new String[]{"test", "-h", yourFileName});
p.waitFor();
if (p.exitValue() == 0)
System.out.println("This file is a symbolic link");
else
System.out.println("This file is not a symbolic link");
That's very specific to *nix, but it does at least work.
Sorry to reply to such an old post, but I was looking for a solution for Windows systems some time back, and some of the previous answers didn't work out for me. If you're not concerned with cross platform compatibility and only need a solution for Windows, the following technique worked well for my purposes.
File f = new File("whatever file or folder");
if (f instanceof ShellFolder) {
ShellFolder sf = (ShellFolder)f;
if (sf.isLink()) {
// Your code when it's a link
}
}
I thought I would share some good fortune I had in dealing with this issue. I am using JDK 1.6.0_23 and so I cannot benefit from NIO2. I am building and running on Windows 7 /x64 ONLY so mileage may vary in other environments. Unfortunately, other solutions here did not work for me in avoiding NullPointerExceptions caused when attempting to traverse a junction (probably because junction != symlink....). While I am not constrained by JDK version, I decided to keep at the problem for a bit longer.
I had this code which would cause a NullPointerException if used on a symbolic link or when encountering the 'System Volume Information' directory. (Note, traverseItem.f() returns an object of type java.io.File)
if (traverseItem.f().isDirectory) {
for (File item : traverseItem.f().listFiles()) {
So, it is supposedly a directory but calling listFiles() on it causes an NPE. What to do? I spied the list() method and wondered if it would exhibit the same behavior. What I discovered was the following:
Calling list() on a File describing an empty folder returns a String[] array of length zero. However, calling list() on a File describing a junction which would otherwise crash from listFiles() returns null
I was able to avoid the NullPointerExceptions by adding the following test before calling listFiles()
String[] contents = traverseItem.f().list();
if (contents != null) { //Non-traversible if null, possibly junction or ???
It remains to exhaustively test all cases of junction, symbolic link, hard link, and dare I mention it, shortcut, but this may help some.
In a DirectoryWalker class I want to find out if a File instance is actually a symbolic link to a directory (assuming, the walker walks on UNIX systems). Given, I already know the instance is a directory, would the following be a reliable condition to determine the symbolic link?
File file;
// ...
if (file.getAbsolutePath().equals(file.getCanonicalPath())) {
// real directory ---> do normal stuff
}
else {
// possible symbolic link ---> do link stuff
}
The technique used in Apache Commons uses the canonical path to the parent directory, not the file itself. I don't think that you can guarantee that a mismatch is due to a symbolic link, but it's a good indication that the file needs special treatment.
This is Apache code (subject to their license), modified for compactness.
public static boolean isSymlink(File file) throws IOException {
if (file == null)
throw new NullPointerException("File must not be null");
File canon;
if (file.getParent() == null) {
canon = file;
} else {
File canonDir = file.getParentFile().getCanonicalFile();
canon = new File(canonDir, file.getName());
}
return !canon.getCanonicalFile().equals(canon.getAbsoluteFile());
}
Java 1.6 does not provide such low level access to the file system. Looks like NIO 2, which should be included in Java 1.7, will have support for symbolic links. A draft of the new API is available. Symbolic links are mentioned there, creating and following them is possible. I'm not exactly sure that which method should be used to find out whether a file is a symbolic link. There's a mailing list for discussing NIO 2 - maybe they will know.
Also, watch out for file.isFile() and file.isDirectory() both returning results based on the resolved file and therefore both returning false when file refers to a symlink where the target doesn't exist.
(I know this isn't a useful answer in itself but it tripped me up a couple of times so thought I should share)
It looks like getCanonicalPath() can do other things that might make it different from the absolute path.
This method first converts this pathname to absolute form if necessary, as if by invoking the getAbsolutePath() method, and then maps it to its unique form in a system-dependent way. This typically involves removing redundant names such as "." and ".." from the pathname, resolving symbolic links (on UNIX platforms), and converting drive letters to a standard case (on Microsoft Windows platforms).
But it might work for the vast majority of your use cases; your mileage may vary.
If you are already coding something specifically for *nix, then you could do a shell command from Java like this:
Process p = Runtime.getRuntime().exec(new String[]{"test", "-h", yourFileName});
p.waitFor();
if (p.exitValue() == 0)
System.out.println("This file is a symbolic link");
else
System.out.println("This file is not a symbolic link");
That's very specific to *nix, but it does at least work.
Sorry to reply to such an old post, but I was looking for a solution for Windows systems some time back, and some of the previous answers didn't work out for me. If you're not concerned with cross platform compatibility and only need a solution for Windows, the following technique worked well for my purposes.
File f = new File("whatever file or folder");
if (f instanceof ShellFolder) {
ShellFolder sf = (ShellFolder)f;
if (sf.isLink()) {
// Your code when it's a link
}
}
I thought I would share some good fortune I had in dealing with this issue. I am using JDK 1.6.0_23 and so I cannot benefit from NIO2. I am building and running on Windows 7 /x64 ONLY so mileage may vary in other environments. Unfortunately, other solutions here did not work for me in avoiding NullPointerExceptions caused when attempting to traverse a junction (probably because junction != symlink....). While I am not constrained by JDK version, I decided to keep at the problem for a bit longer.
I had this code which would cause a NullPointerException if used on a symbolic link or when encountering the 'System Volume Information' directory. (Note, traverseItem.f() returns an object of type java.io.File)
if (traverseItem.f().isDirectory) {
for (File item : traverseItem.f().listFiles()) {
So, it is supposedly a directory but calling listFiles() on it causes an NPE. What to do? I spied the list() method and wondered if it would exhibit the same behavior. What I discovered was the following:
Calling list() on a File describing an empty folder returns a String[] array of length zero. However, calling list() on a File describing a junction which would otherwise crash from listFiles() returns null
I was able to avoid the NullPointerExceptions by adding the following test before calling listFiles()
String[] contents = traverseItem.f().list();
if (contents != null) { //Non-traversible if null, possibly junction or ???
It remains to exhaustively test all cases of junction, symbolic link, hard link, and dare I mention it, shortcut, but this may help some.