The class BasicFileAttributes, for examining the properties of a file in the file system, has the method isRegularFile(). Unfortunately, the Javadoc description is rather lacking:
Tells whether the file is a regular file with opaque content.
What does this mean? What exactly is a regular file with opaque content? I can tell from the other methods in the class that it's not a directory or symbolic link, so I'm inclined to think that it's everything else. However, there apparently is some type of "irregular file" because a method exists called isOther(), which returns true if it's not a directory, symbolic link, or "regular file".
So what exactly is an regular file in Java?
For example in UNIX, a regular file is one that is not special in some way. Special files include symbolic links and directories. A regular file is a sequence of bytes stored permanently in a file system.
Read this answer # UNIX & Linux stackexchange: What is a regular file?
I figure rm -i is an alias, possibly rm -i. The "regular" part doesn't
mean anything in particular, it only means that it's not a pipe,
device, socket or anything other "special".
it means the file is not a symlink, pipe, rand, null, cpu, etc.
Perhaps you have heard the linux philosophy everything is a text. This
isn't literally true, but it suggests a dominant operational context
where string processing tools can be applied to filesystem elements
directly. In this case, it means that in a more literal fashion. To
see the detection step in isolation, try the command file, as in file
/etc/passwd or file /dev/null.
From Files Reference - AIX IBM
A file is a collection of data that can be read from or written to. A file can be a program you create, text you write, data you acquire, or a device you use. Commands, printers, terminals, and application programs are all stored in files. This allows users to access diverse elements of the system in a uniform way and gives the operating system great flexibility. No format is implied when a file is created.
There are three types of files
Regular - Stores data (text, binary, and executable).
Directory - Contains information used to access other files.
Special - Defines a FIFO (first-in, first-out) file or a physical device.
Regular files are the most common. When a word processing program is used to create a document, both the program and the document are contained in regular files.
Regular files contain either text or binary information. Text files are readable by the user. Binary files are readable by the computer. Binary files can be executable files that instruct the system to accomplish a job. Commands, shell scripts, and other programs are stored in executable files.
Directories contain information the system needs to access all types of files, but they do not contain the actual file data. As a result, directories occupy less space than a regular file and give the file-system structure flexibility and depth. Each directory entry represents either a file or subdirectory and contains the name of a file and the file's i-node (index node reference) number. The i-node number represents the unique i-node that describes the location of the data associated with the file. Directories are created and controlled by a separate set of commands. See "Directories" in Operating system and device management for more information.
Special files define devices for the system or temporary files created by processes. There are three basic types of special files: FIFO (first-in, first-out), block, and character. FIFO files are also called pipes. Pipes are created by one process to temporarily allow communication with another process. These files cease to exist when the first process finishes. Block and character files define devices.
All this above is from the first link. I've checked in many other sources regarding Operational Systems differences and it seems this one is the most common definition on all sources i've found.
I am not an expert on this but at the first look BasicFileAttributes is not a class but an interface. So whatever a regular file is depends on the implementation of this interface. I can see that there is e.g. the class WindowsFileAttributs that implements this interface.
If you have a look at the OpenJDK version of this class you will find that it is
!isSymbolicLink() && !isDirectory() && !isOther();
Get all other information from the code ;-)
Related
Linux Api has O_TMPFILE flag to be specified with open system call creating unnamed temporary file which cannot be opened by any path. So we can use this to write data to the file "atmoically" and the linkat the given file to the real path. According to the open man page it can be implemented as simple as
char path[1000];
int fd = open("/tmp", O_TMPFILE | O_WRONLY, S_IWUSR);
write(fd, "123456", sizeof("123456"));
sprintf(path, "/proc/self/fd/%d", fd);
linkat(AT_FDCWD, path, AT_FDCWD, "/tmp/1111111", AT_SYMLINK_FOLLOW);
Is there a Java alternative (probably non crossplatform) to do atomic write to a file without writing Linux-specific JNI function? Files.createTempFile does completely different thing.
By atomic write I mean that either it cannot be opened and be read from or it contains all the data required to be writted.
I don't believe Java has an API for this, and it seems to depend on both the OS and filesystem having support, so JNI might be the only way, and even then only on Linux.
I did a quick search for what Cygwin does, seems to be a bit of a hack just to make software work, creating a file with a random name then excluding it only from their own directory listing.
I believe the closest you can get in plain Java is to create a file in some other location (kinda like a /proc/self/fd/... equivalent), and then when you are done writing it, either move it or symbolic link it from the final location. To move the file, you want it on the same filesystem partition so the file contents don't actually need to be copied. Programs watching for the file in say /tmp/ wouldn't see it until the move or sym link creation.
You could possibly play around with user accounts and filesystem permissions to ensure that no other (non SYSTEM/root) program can see the file initially even if they tried to look wherever you hid it.
What is the difference between symbolic links and shortcuts?
As I was reading Oracle's Documentation on java.nio.file package, there I came to know about symbolic links. At first, I thought they are talking about shortcuts because at first they both appeared the same to me. But later I realised that I am kind of assuming this without a proof so i googled it and found some links about the differences between the two. But I couldn't get a clear view of what these actually are as the answers were not that good. I hope I can get an explanation here!
A symbolic link is a special file which references another path on disk. If an application opens this file (or includes it in a path), the operating system will silently follow the link.
For instance, assume you have a symbolic link l which points to a directory d with a file file in it. Observe:
$ ls d/
file
$ ls l/
file
$ touch l/file2 # Create a new file in the path under l
$ ls d/
file file2
Symbolic links (symlinks) are a feature of the filesystem part of the operating system. They can be used by any application; applications that do want to treat symlinks differently (such as backup programs, which must not blindly follow symlinks lest the number of files becomes infinite) need to take special precaution to do so.
In contrast, a shortcut is a regular file which is rendered differently by the user interface(UI). A shortcut file also contains the desired target path, but can contain additional information, such as an icon, command-line arguments, instructions to the UI to run the destination as another user, categorization of the destination, translation information and the like. For an application, a shortcut is a regular file whose contents describe the shortcut. For instance, if you open a .desktop file with an editor, you'll see information like
Icon=libreoffice-calc
Exec=libreoffice --calc %U
GenericName=Spreadsheet
GenericName[da]=Regneark
You should generally use shortcuts if you want to add an entry to the user interface (typically the graphical parts). Shortcuts allow you extended customization - for instance, Danish users will see Regneark, and English users Spreadsheet in the above example.
If, on the other hand, you want the alias to be visible to all applications, go for a symlink instead. For instance, you could have a legacy name of a file that some applications expect, or one application that can do multiple things, or the current version of a project directory.
Note that some systems do not support symlinks, and some have no use for shortcuts. If there is no graphical interface present (for instance a home router without a monitor attached), shortcuts would not be useful. On the other hand, symlinks require support by the filesystem (FAT has none) as well as the operating system (DOS/old versions of Windows lack support).
How can we identify whether given File is a named pipe? Any way to use same code for both Linux and Windows.
You may have some success with isOther() in the NIO2 Files API. This tells you that it is not a file, not a directory, and not a link. That leaves a named pipe, a unix socket, or a device..... perhaps that's enough for you?
Is there any way in Java to write out to a temporary file securely?
As far as I can tell, the only way to create a temporary file (createTempFile) does't actually open it at the same time, so there's a race condition between file open & file write. Am I missing something? I couldn't find the C source code behind createFileExclusively(String) in UnixFileSystem.java, but I doubt it can really do anything since the file open occurs in the Java code after the temp file is created (unless it tries to do something with file locks?).
The problem
Between when the temporary file is created & you open it, a malicious attacker could unlink that temporary file & put malicious stuff there. For example, an attacker could create a named pipe to read sensitive data. Or similarly if you eventually copy the file by reading it, then the named pipe could just ignore everything written & supply malicious content to be read.
I remember reading of numerous examples of temporary file attacks in the past 10+ years that exploit the race condition between when the name appears in the namespace and when the file is actually opened.
Hopefully a mitigating factor is that Java set's the umask correctly so a less-privileged user can't read/write to the file and typically the /tmp directory restricts permissions properly so that you can't perform an unlink attack.
Of course if you pass a custom directory for the temporary file that's owned by a less-privileged user who's compromised, the user could do an unlink attack against you. Hell, with inotify, it's probably even easier to exploit the race condition than just a brute force loop that does a directory listing.
http://kurt.seifried.org/2012/03/14/creating-temporary-files-securely/
Java
use java.io.File.createTempFile() – some interesting info at http://www.veracode.com/blog/2009/01/how-boring-flaws-become-interesting/
for directories there is a helpful posting at How to create a temporary directory/folder in Java?
Java 7
for files use java.io.File.createTempFile()
for directories use createTempDirectory()
http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html
Since Java 7 we have OpenOption.
An object that configures how to open or create a file.
Objects of this type are used by methods such as newOutputStream, newByteChannel, FileChannel.open, and AsynchronousFileChannel.open when opening or creating a file.
Of particular interest is StandardOpenOptions.CREATE_NEW.
Create a new file, failing if the file already exists. The check for the existence of the file and the creation of the file if it does not exist is atomic with respect to other file system operations.
So, you can do something like this:
FileChannel mkstemp() {
Path path = Files.createTempFile(null, null);
Files.delete(path);
return FileChannel.open(path, WRITE, CREATE_NEW);
}
Implementing the same template behaviour is left as exercise to the reader.
Keep in mind that on many systems, just because a file doesn't have a name doesn't at all mean it's inaccessible. For example, on Linux open file descriptors are available in /proc/<pid>/fd/<fdno>. So you should make sure that your use of temporary files is secure even if someone knows / has a reference to the open file.
You might get a more useful answer if you specify exactly what classes of attacks you are trying to prevent.
Secure against other ordinary userid's? Yes, on any properly functioning multi-user system.
Secure against the your own userid or the superuser? No.
Is there a way to search an entire computer for a String value representing a file name in Java? This would be without knowing the names of the drives on the computer.
You can iterate through the file system by looking at file names, getting the contents if it's a directory, etc. recursively.
If the sticking point is how to get the drives on the computer, look at the File.listRoots() function to get a list of the drive letters.
ETA:
To be absolutely safe, you'll want to include some limits on recursive processing. It's possible to have loops in the file system with symbolic links and such (especially in LINUX/UNIX, but third party tools can enable this in Windows as well).
To make sure you don't get into a loop when dealing with symbolic links, use the File.getCanonicalPath methods to get the "real" path for the directory and keep track of all visited canonical paths. You could also use getCanonicalFile and keep track of all the files, but that's probably not needed unless you really want to avoid the occasional instance where you'll process the same file twice.
You can use the File object to determine whether you are looking at a file or a directory:
File file = new File("/"); //root
Then as you are recursing (or iterating depending on your preference) you have a simple check:
if(tempFile.isDirectory())
//do recursive call on that directory
else
//perform check on file name
Also not forget exceptions in recursive processing. Some folders may not be accessible due to access right restrictions. Also, the Windows system folder "System Volume Information" cannot be entered in Windows Explorer, so I suppose it will throw an exception if you try to get inside programmaticaly.
You can use a recursive call through the entire file system: You can use the following methods of java.io.File:
listRoots(): list the root files (In Windows it's the drives, on Mac OS X and linux it's '/').
listFiles(): list the enclosing files of a directory