I want remove the extension of a file. For example:
ActualFile = image.png
ExpectedFile = image
Which method is the most efficient to use?
removeExtension() method provided by org.apache.commons.io
fileName.substring(0, fileName.lastIndexOf('.'))
No difference, except one. Do you really want to add whole Apache lib to use only one method? If you use Apache in your application - then use it as well. If not - do create your custom implementation - this is not a rocket-science.
Looking at removeExtension from commons-io you can see that substring is also used underwater in that method in a similar way you describe:
public static String removeExtension(final String fileName) {
if (fileName == null) {
return null;
}
failIfNullBytePresent(fileName);
final int index = indexOfExtension(fileName);
if (index == NOT_FOUND) {
return fileName;
}
return fileName.substring(0, index);
}
Your method is faster since there are less operations being done, but the removeExtension method has the failIfNullBytePresent which states:
Check the input for null bytes, a sign of unsanitized data being
passed to to file level functions.
This may be used for poison byte attacks.
and indexOfExtension to get the index of the extension which has more checks (as you can see in the javadoc of that method here).
Conclusion
Your method is faster but I'd say using the commons-io is safer/more consistent in various situations, but what to use depends on how complex your situation is whether it's a critical feature of an application or just a home made project for yourself. removeExtension is not that complex or slow that you shouldn't use it perse.
Related
We need a method to sign messages using signatures that are as short as possible and came across the BLS scheme, which promises rather short-ish signatures. Trying the JPBC implementation, the examples are easy to set up and run, but they lack a rather crucial part: storing and loading the private keys.
The example from the current JPBC BLS website 1 does not contain any storage whatsoever, it just verifies a message using the instances in RAM.
An older example from the same website 2 which is no longer linked on the website but can be found using search engines refers to a store method which seems to have since been removed from the library in favour of an implementation that does not contain any storage capabilities.
The AsymmetricCipherKeyPair instances (which are what I get from the keygen) are not serializable by themselves, neither are instances of BLS01PublicKeyParameters or BLS01PrivateKeyParameters, with the fields that contain the keys (sk and pk) being private and typed only to the Element interface that doesn't really say much about the contents.
As a workaround, I have implemented a store method, that (stripped of all exception handling) roughly looks like this:
public static void storePrivateKey(AsymmetricCipherKeyPair key, String filename)
throws FileNotFoundException, IOException {
Field f = null;
f = key.getPrivate().getClass().getDeclaredField("sk");
if (f != null) {
f.setAccessible(true);
Object fieldContent = null;
fieldContent = f.get(key.getPrivate());
if (fieldContent != null) {
byte[] data = null;
if (fieldContent instanceof ImmutableZrElement) {
ImmutableZrElement izr = (ImmutableZrElement)fieldContent;
data = izr.toBytes();
}
try (FileOutputStream fos = new FileOutputStream(filename)) {
fos.write(data);
}
}
}
}
With a similar approach for public keys. That means, I'm now down to using reflection to retrieve the contents of a private field in order to store it somewhere. That solution is obviously a hackish collection of all sorts of bad smells, but it's so far the best that I've come up with. I know that writing some bytes to disk shouldn't really be that hard, but I really can't seem to find the proper way to do this. Also, to be blunt, I'm not into crypto: I want to apply this scheme to sign and verify some messages, that is all. I understand that I should dig deeper into the math of the whole approach, but time is limited - which is why I picked a library in the first place.
I am working on fixing Veracode issues in my application. Veracode has highlighted the flaw "External Control of File Name or Path (CWE ID 73) " in below code.
Thread.currentThread().getContextClassLoader().getResourceAsStream(lookupName)
How do I validate the parameter? If I need to use below ESAPI validation, then what is the exact parameter I should be passing in getValidFileName() method. Currently I am passing the parameters as below.
ESAPI.validator().getValidFileName(lookupName, lookupName,
ESAPI.securityConfiguration().getAllowedFileExtensions(), false);
Correct me whether I am following the right approach for fixing this issue.
There are several suggestions at: https://community.veracode.com/s/article/how-do-i-fix-cwe-73-external-control-of-file-name-or-path-in-java
You can use hardcoded values, if these files are stored in the server side.
(i.e.: in a HashMap).
Another solution is to use a custom validator (from veracode page) :
// GOOD Code
String extension = request.getParameter("extension");
File f = new File(buildValidAvatarPath(extension))
#FilePathCleanser
public String buildValidAvatarPath(extension) {
String[] allowedExtensions = new String[]{"jpg","gif","png"};
String extension = "png"; // Default extension
for (String allowedExtension: allowedExtensions) {
if (allowedExtension.equals(request.getParameter("extension"))) {
extension = request.getParameter("extension");
}
}
// See "Note on authorization"
User user = getCurrentUser();
if (!userMayAccessFile(user, path)) {
throw new AuthorizationException("User may not access this file", user);
}
File(configPath + "avatar." + extension)
return path;
}
Okay, so the problem is that you are allowing user-control of that file path. Imagine its on a UNIX box and they enter:
../../../../../../../etc/shadow
Whatever user privileges are granted to the user running that java Thread is possible to expose to the user in question. I don't know what processing is going on in your application, but the danger is that you need to prevent user control of that lookup variable.
The call you're making is consistent with the single test in ValidatorTest.java, which is definitely a deficiency in code coverage on our behalf.
Now, there's an excellent chance that even if you use this call that Veracode might still flag it: the default file list in ESAPI.properties will need to be either truncated for your use case, or you'll have to create your own Validator rule for legal file extensions for your specific use case.
Which brings up the next bit: There's a lot of mischief that can happen in regards to file uploads.
In short, to be actually secure about file uploads will require more than what ESAPI currently offers, which is unfortunately, only an extension check. In your particular case, make sure you try some directory traversal attacks. And use that OWASP link to help analyze your application.
Given that the OP wants to clear the issue in Veracode, you would want to chain a couple calls:
ESAPI.validator().getValidDirectoryPath() and ESAPI.Validator.getValidFileName()
But be sure you've properly truncated the extension list in HttpUtilities.ApprovedUploadExtensions in validator.properties as the default list is too permissive, at least until we release 2.1.0.2.
I have to stress however that even with this particular combination there is absolutely nothing ESAPI does to prevent a user from renaming "netcat.exe" to "puppies.xlsx" and bypassing your validation check, that's why the rant on the first part of this answer.
ESAPI's file validation is NOT secure, it's quite simply better than nothing at all.
Doing this correctly requires more work than just using 1-2 calls to ESAPI.
DISCLAIMER: as of this writing I am the project co-lead for ESAPI.
You can change file name by sanitizing it as below code snippet:
private static String sanitizeFileName(String name) {
return name
.chars()
.mapToObj(i -> (char) i)
.map(c -> Character.isWhitespace(c) ? '_' : c)
.filter(c -> Character.isLetterOrDigit(c) || c == '-' || c == '_' || c == ':')
.map(String::valueOf)
.collect(Collectors.joining());
}
In a DirectoryWalker class I want to find out if a File instance is actually a symbolic link to a directory (assuming, the walker walks on UNIX systems). Given, I already know the instance is a directory, would the following be a reliable condition to determine the symbolic link?
File file;
// ...
if (file.getAbsolutePath().equals(file.getCanonicalPath())) {
// real directory ---> do normal stuff
}
else {
// possible symbolic link ---> do link stuff
}
The technique used in Apache Commons uses the canonical path to the parent directory, not the file itself. I don't think that you can guarantee that a mismatch is due to a symbolic link, but it's a good indication that the file needs special treatment.
This is Apache code (subject to their license), modified for compactness.
public static boolean isSymlink(File file) throws IOException {
if (file == null)
throw new NullPointerException("File must not be null");
File canon;
if (file.getParent() == null) {
canon = file;
} else {
File canonDir = file.getParentFile().getCanonicalFile();
canon = new File(canonDir, file.getName());
}
return !canon.getCanonicalFile().equals(canon.getAbsoluteFile());
}
Java 1.6 does not provide such low level access to the file system. Looks like NIO 2, which should be included in Java 1.7, will have support for symbolic links. A draft of the new API is available. Symbolic links are mentioned there, creating and following them is possible. I'm not exactly sure that which method should be used to find out whether a file is a symbolic link. There's a mailing list for discussing NIO 2 - maybe they will know.
Also, watch out for file.isFile() and file.isDirectory() both returning results based on the resolved file and therefore both returning false when file refers to a symlink where the target doesn't exist.
(I know this isn't a useful answer in itself but it tripped me up a couple of times so thought I should share)
It looks like getCanonicalPath() can do other things that might make it different from the absolute path.
This method first converts this pathname to absolute form if necessary, as if by invoking the getAbsolutePath() method, and then maps it to its unique form in a system-dependent way. This typically involves removing redundant names such as "." and ".." from the pathname, resolving symbolic links (on UNIX platforms), and converting drive letters to a standard case (on Microsoft Windows platforms).
But it might work for the vast majority of your use cases; your mileage may vary.
If you are already coding something specifically for *nix, then you could do a shell command from Java like this:
Process p = Runtime.getRuntime().exec(new String[]{"test", "-h", yourFileName});
p.waitFor();
if (p.exitValue() == 0)
System.out.println("This file is a symbolic link");
else
System.out.println("This file is not a symbolic link");
That's very specific to *nix, but it does at least work.
Sorry to reply to such an old post, but I was looking for a solution for Windows systems some time back, and some of the previous answers didn't work out for me. If you're not concerned with cross platform compatibility and only need a solution for Windows, the following technique worked well for my purposes.
File f = new File("whatever file or folder");
if (f instanceof ShellFolder) {
ShellFolder sf = (ShellFolder)f;
if (sf.isLink()) {
// Your code when it's a link
}
}
I thought I would share some good fortune I had in dealing with this issue. I am using JDK 1.6.0_23 and so I cannot benefit from NIO2. I am building and running on Windows 7 /x64 ONLY so mileage may vary in other environments. Unfortunately, other solutions here did not work for me in avoiding NullPointerExceptions caused when attempting to traverse a junction (probably because junction != symlink....). While I am not constrained by JDK version, I decided to keep at the problem for a bit longer.
I had this code which would cause a NullPointerException if used on a symbolic link or when encountering the 'System Volume Information' directory. (Note, traverseItem.f() returns an object of type java.io.File)
if (traverseItem.f().isDirectory) {
for (File item : traverseItem.f().listFiles()) {
So, it is supposedly a directory but calling listFiles() on it causes an NPE. What to do? I spied the list() method and wondered if it would exhibit the same behavior. What I discovered was the following:
Calling list() on a File describing an empty folder returns a String[] array of length zero. However, calling list() on a File describing a junction which would otherwise crash from listFiles() returns null
I was able to avoid the NullPointerExceptions by adding the following test before calling listFiles()
String[] contents = traverseItem.f().list();
if (contents != null) { //Non-traversible if null, possibly junction or ???
It remains to exhaustively test all cases of junction, symbolic link, hard link, and dare I mention it, shortcut, but this may help some.
guys
I am implementing a simple example of 2 level cache in java:
1st level is memeory
2nd - filesystem
I am new in java and I do this just for understanding caching in java.
And sorry for my English, this language is not native for me :)
I have completed 1st level by using LinkedHashMap class and removeEldestEntry method and it is looks like this:
import java.util.*;
public class level1 {
private static final int max_cache = 50;
private Map cache = new LinkedHashMap(max_cache, .75F, true) {
protected boolean removeEldestEntry(Map.Entry eldest) {
return size() > max_cache;
}
};
public level1() {
for (int i = 1; i < 52; i++) {
String string = String.valueOf(i);
cache.put(string, string);
System.out.println("\rCache size = " + cache.size() +
"\tRecent value = " + i +
" \tLast value = " +
cache.get(string) + "\tValues in cache=" +
cache.values());
}
}
Now, I am going to code my 2nd level. What code, methods I should write to implement this tasks:
1) When the 1st level cache is full, the value shouldn't be removed by removeEldestEntry but it should be moved to 2nd level (to file)
2) When the new values are added to 1st level, firstly this value should be checked in file (2nd level) and if it exists it should be moved from 2nd to 1st level.
And I tried to use LRUMap to upgrade my 1st level but the compiler couldn't find class LRUMap in library, what's the problem? Maybe special syntax needed?
You can either use the built in java serialization mechanism and just send your stuff to file by wrapping FileOutputStrem with DataOutputStream and then calling writeObjet().
This method is simple but not flexible enough. for example you will fail to read old cache from file if your classes changed.
You can use serialization to xml, e.g. JaxB or XStream. I used XStream in past and it worked just fine. You can easily store any collection in file and the restore it.
Obviously you can store stuff in DB but it is more complicated.
A remark is that you are not getting thread safety under consideration for your cache! By default LinkedHashMap is not thread-safe and you would need to synchronize your access to it. Even better you could use ConcurrentHashMap which deals with synchronization internally being able to handle by default 16 separate threads (you can increase this number via one of its constructors).
I don't know your exact requirements or how complicated you want this to be but have you looked at existing cache implementations like the ehcache library?
In a DirectoryWalker class I want to find out if a File instance is actually a symbolic link to a directory (assuming, the walker walks on UNIX systems). Given, I already know the instance is a directory, would the following be a reliable condition to determine the symbolic link?
File file;
// ...
if (file.getAbsolutePath().equals(file.getCanonicalPath())) {
// real directory ---> do normal stuff
}
else {
// possible symbolic link ---> do link stuff
}
The technique used in Apache Commons uses the canonical path to the parent directory, not the file itself. I don't think that you can guarantee that a mismatch is due to a symbolic link, but it's a good indication that the file needs special treatment.
This is Apache code (subject to their license), modified for compactness.
public static boolean isSymlink(File file) throws IOException {
if (file == null)
throw new NullPointerException("File must not be null");
File canon;
if (file.getParent() == null) {
canon = file;
} else {
File canonDir = file.getParentFile().getCanonicalFile();
canon = new File(canonDir, file.getName());
}
return !canon.getCanonicalFile().equals(canon.getAbsoluteFile());
}
Java 1.6 does not provide such low level access to the file system. Looks like NIO 2, which should be included in Java 1.7, will have support for symbolic links. A draft of the new API is available. Symbolic links are mentioned there, creating and following them is possible. I'm not exactly sure that which method should be used to find out whether a file is a symbolic link. There's a mailing list for discussing NIO 2 - maybe they will know.
Also, watch out for file.isFile() and file.isDirectory() both returning results based on the resolved file and therefore both returning false when file refers to a symlink where the target doesn't exist.
(I know this isn't a useful answer in itself but it tripped me up a couple of times so thought I should share)
It looks like getCanonicalPath() can do other things that might make it different from the absolute path.
This method first converts this pathname to absolute form if necessary, as if by invoking the getAbsolutePath() method, and then maps it to its unique form in a system-dependent way. This typically involves removing redundant names such as "." and ".." from the pathname, resolving symbolic links (on UNIX platforms), and converting drive letters to a standard case (on Microsoft Windows platforms).
But it might work for the vast majority of your use cases; your mileage may vary.
If you are already coding something specifically for *nix, then you could do a shell command from Java like this:
Process p = Runtime.getRuntime().exec(new String[]{"test", "-h", yourFileName});
p.waitFor();
if (p.exitValue() == 0)
System.out.println("This file is a symbolic link");
else
System.out.println("This file is not a symbolic link");
That's very specific to *nix, but it does at least work.
Sorry to reply to such an old post, but I was looking for a solution for Windows systems some time back, and some of the previous answers didn't work out for me. If you're not concerned with cross platform compatibility and only need a solution for Windows, the following technique worked well for my purposes.
File f = new File("whatever file or folder");
if (f instanceof ShellFolder) {
ShellFolder sf = (ShellFolder)f;
if (sf.isLink()) {
// Your code when it's a link
}
}
I thought I would share some good fortune I had in dealing with this issue. I am using JDK 1.6.0_23 and so I cannot benefit from NIO2. I am building and running on Windows 7 /x64 ONLY so mileage may vary in other environments. Unfortunately, other solutions here did not work for me in avoiding NullPointerExceptions caused when attempting to traverse a junction (probably because junction != symlink....). While I am not constrained by JDK version, I decided to keep at the problem for a bit longer.
I had this code which would cause a NullPointerException if used on a symbolic link or when encountering the 'System Volume Information' directory. (Note, traverseItem.f() returns an object of type java.io.File)
if (traverseItem.f().isDirectory) {
for (File item : traverseItem.f().listFiles()) {
So, it is supposedly a directory but calling listFiles() on it causes an NPE. What to do? I spied the list() method and wondered if it would exhibit the same behavior. What I discovered was the following:
Calling list() on a File describing an empty folder returns a String[] array of length zero. However, calling list() on a File describing a junction which would otherwise crash from listFiles() returns null
I was able to avoid the NullPointerExceptions by adding the following test before calling listFiles()
String[] contents = traverseItem.f().list();
if (contents != null) { //Non-traversible if null, possibly junction or ???
It remains to exhaustively test all cases of junction, symbolic link, hard link, and dare I mention it, shortcut, but this may help some.