I'd like to traverse a directory structure in Java 8 using the Stream API in depth-first order. The reason for this is that I want to sort the contents in the files according to a timestamp present in each file on a per directory basis. Essentially I'm looking for something similar to Files#walk but for directories. How can I achieve this?
Using StreamEx is a breeze:
File root = new File("someFilePath");
StreamEx.ofTree(root, x -> StreamEx.of(x.listFiles(File::isDirectory)))
.map(File::getAbsolutePath) // or whathever you need to do with the folder
.forEach(System.out::println); // the same as previous line
It’s hard to understand your problem description, most notably, what you actually want to do when you say you want to “sort the contents in the files according to a timestamp present in each file on a per directory basis”.
And when you say you want “something similar to Files#walk but for directories”, it implies that you have a problem with using Files.walk with directories, but it’s not clear, what problem you have.
I.e., you can simply list all subdirectories of the current directory like
Files.walk(Paths.get("")).filter(Files::isDirectory).forEach(System.out::println);
So if that’s not matching your goal, you have to spent more time elaborating your goal.
Related
I have two directories with identical folder tree structure and file names. These are the data of current week and last week. I want to combine (append the new data to the old ones) each pair of them (name to name) into and save it under another directory with the original file tree structure.
How do I do it in Java? Please help.
What is the best way to read and write a file?
I think I would use a recursive approach, iterating over the current week's structure. For each file found, locate the corresponding files in last week's structure and append it to the current week's file. If the structures are not guaranteed to be identical, you might also need to recurse over last week's files to pick up anything that does not exist in this week's directory tree.
As for the actual Java part of the question, i.e. the implementation of those operations, the following may be helpful:
java.io.File
java.io.FileReader
java.io.FileWriter
Apache Commons IO
I'm new to MarkLogic and trying to implement following scenario with its Java API:
For each user I'll have two directories, something like:
1.1. user1/xmls/recent/
1.2. user1/xmls/archived/
When user is doing something with his xml - it's put to the "recent" directory;
When user is doing something with his next xml and "recent" directory is full (e.g. has some amount of documents, let's say 20) - the oldest document is moved to the "archived" directory;
User can request all documents from the "recent" directory and should get no more than 20 records;
User can remove something from the "recent" directory manually; In this case, if it had 20 documents, after deleting one it must have 19;
User can do something with his xmls simultaneously and "recent" directory should never become bigger than 20 entries.
Questions are:
In order to properly handle simultaneous adding of xmls to the "recent" directory, should I block whole "recent" directory when adding new entry (to actually add it, check if there are more than 20 records after adding, select the oldest 21st one and move it to the "archived" directory and do all these steps atomically)? How can I do it?
Any suggestions on how to implement this via Java API?
Is it possible to change document's URI (e.g. replace "recent" with "archived" in my case)?
Should I consider using MarkLogic's collections here?
I'm open to any suggestions and comments (as I said I'm new to MarkLogic and maybe my thoughts on how to handle described scenario are completely wrong).
You can achieve atomicity of a sequence of transactions using Multi-Statement Transactions (MST)
It is possible to MST from the Java API: http://docs.marklogic.com/guide/java/transactions#id_79848
It's not possible to change a URI. However, it is possible to use an MST to delete the old document and reinsert a new one using the new URI in one an atomic step. This would have the same effect.
Possibly, and judging from your use case, unless you must have the recent/archived information as part of the URI, it may be simpler to store this information in collections. However, you should read the documentation and evaluate for yourself: http://docs.marklogic.com/guide/search-dev/collections#chapter
Personally I would skip all the hassle with separate directories as well as collections. You would endlessly have to move files around, or changes their properties. It would be much easier to not calculate anything up front, and simply use lastModified property, or something alike, to determine most recent items at run-time.
HTH!
For those unfamiliar with JNotify, this is a library which provides an easy way to monitor events in a directory.
For instance, when a file gets deleted in the selected folder, the method "fileDeleted" gets called, along with a few parameters. Here's an example of the fileDeleted method:
public void fileDeleted(int wd, String rootPath, String name) {
print("deleted " + rootPath + " : " + name);
}
Now, I would like to know if the deleted file was a file or directory. My usual approach is to create a new File object with the given path, and use the methods isFile() and isDirectory()
However, since this file is already deleted, these methods always return false.
So here's my concrete question: I have the path to a deleted file or directory, how can I tell wether it was a file or a directory? Is there a workaround to this? What's the best practice to do here?
Thank you in advance.
I suggest using a better API for this, like Commons IO. It has this distinction in its interface org.apache.commons.io.monitor.FileAlterationListener and its methods onFile...(), onDirectory...(). Alternatively, and this is probably the best approach, use the new standard feature for this that comes with Java 7, WatchService, as discussed here.
How big is the directory structure you're looking at?
My first instinct is to build an internal representation of the directory structure, using some simple graph traversal algorithm, and then do a lookup every time something is removed to figure out what it was.
<edit>
If you know your directory structure is a strict tree you can use a simple recursion to traverse the file system, and create a map of Files or Strings to boolean, so you can do an easy lookup. Then, once you've got the map built it should be easy to maintain using the JNotify events.
<edit/>
even for medium-sized directories I would think this could be made pretty quick. What is this for? Might there be another way of going about achieving the same goal?
I am facing the same problem. Yet as far as I understand it, Java's WatchService does not allow monitoring of subdirectories, so I cannot use it (task is to monitor changes to a structure containing ~40K folders). I will try and go ahead using the simple (and fallible) heuristic
If it contains a dot ('.'), it's a file.
I will post updates if I come across something more sophisticated...
Why doesn't File.renameTo(...) create sub-directories contained in the destination file path?
For instance,
File source = new File(System.getProperty("user.dir") +
"/src/MyFolder/MyZipFolder.zip");
File dest = new File(System.getProperty("user.dir") +
"/src/MyOtherFolder/MyZipFolder.zip");
System.out.println(source.renameTo(dest));
Since MyOtherFolder does not exist, this will always return false. In order for this to work, I have to ensure that all sub-directories exist either by creating them programmatically(i.e. mkdirs()), or manually. Is there a reason why this functionality was not included in this method?
Why?
Possibly for consistency / compatibility with the APIs that typical operating systems and other programming language runtime libraries provide.
Possibly because it would be a bad idea to create the intermediate directories if the user didn't really mean this to happen; e.g. if he / she simply mistyped one of the directory names in the path.
But it is not really relevant. The bottom line is that this is the way that the renameTo method behaves.
The current File API isn't very well implemented in Java. There is a lot of functionality that would be desirable in a File API that isn't currently present such as move, copy and retrieving file metadata.
I don't think anyone will be able to give you an answer as to why the API is written as is. Probably a poor first draft that went live and couldn't be changed due to backwards compatibility issues.
These issue have been addressed in the upcoming Java 7. A entirely new API has been created to deal with files java.nio.file.Files.
Creating sub-directories may be considered as unexpected side effect from other point of view. Are you sure everyone needs it implicitly?
You have answers but I was thinking along the lines:
A feature request to add a new method File.renameTo(File src, File destination, int makeDirs)
with three constants for makeDirs:
1) do not make sub folder(s)/ dirs
2) only make the final folder if it does not exist meaning if you specify /r1/r2/r3/file.extn then only make r3 if it does not exist, if r2 or any other does not exist then return false.
3) make all possible sub dirs
if its a OS that does not have sub folders then do as you do now
the old method would remain as is
I'm working on a small application in Java that takes a directory structure and renames the files according to a certain format, after parsing the original name.
What is the best Java class / methodology to use in order to facilitate these file operations?
Edit: the question is only regarding the file operations part, I got the "getting the formatted name" down :)
Edit 2: Also, how do I list files recursively?
Use java.io.File
Listing all files in a directory
http://www.javaprogrammingforums.com/java-code-snippets-tutorials/3-how-list-all-files-directory.html
File folder = new File(path);
File[] listOfFiles = folder.listFiles();
for (int i = 0; i < listOfFiles.length; i++) {
// Do something with "listOfFiles[i]"
}
UPDATE
To list the files recursively, your best approach is fairly easy:
Create a queue of directories. Initially add the first directory to the queue
Pop the first directory element off the queue.
List all files in that directory, same as above
Iterate over all the files in that directory
If a file is a directory (use isDirectory() method), add it to the back of the queue.
Else, process this next file as needed (e.g. print)
Stop when the queue is empty.
An example (I think a bit different from my approach above) is http://www.javapractices.com/topic/TopicAction.do?Id=68
Renaming a file
http://www.roseindia.net/java/example/java/io/RenameFileOrDir.shtml
boolean Rename = oldfile.renameTo(newfile);
Finding a new name to rename to
I'm not sure what you want the formatting rules to be - when I implemented the same utility in Perl for my own use I used Regular Expressions. For Java, that'd be java.util.regex
This Sun Totorial could be a good start. If I where you I would basically retrieve all the files in the directory and then loop through them, as shown here. You might have to use regular expressions as well, a basic tutorial can be found here
You can always use the standard java.io.File class, but it's primitive and not very useful on its own.
For complex file-I/O operations, I recommend using Apache Commons IO, which provides a rich class library for (among other things) file operations. See classes like FileUtils and FilesystemUtils
There's the class File, that does all you need:
listFiles()
renameTo()