Develop Reference

Return files values of recursive method

I would like to return values of a recursive method which list all files in directory and subdirectories. The goal is to create a HashSet of md5 file values.
For the moment the code works fine but just in root dir, not in recursive.
static Set<String> localMd5Listing() throws Exception {
List<String> localMd5List = new ArrayList<String>();
if(!baseModDirectoryFile.exists()){
System.out.println("baseModDirectory doesn't exist, check your baseModDirectory variable.");
}
else{
if(baseModDirectoryFile.isDirectory()){
File[] paths = baseModDirectoryFile.listFiles();
if(paths != null){
for(File path:paths){
if(path.isFile()){
FileInputStream fis = new FileInputStream(path);
String md5 = org.apache.commons.codec.digest.DigestUtils.md5Hex(fis);
fis.close();
localMd5List.add(md5);
}
else if(path.isDirectory()){
listChildren(path);
//Check md5 for children files as path.isFile condition
}
}
}
}
}
Set<String> localSet = new HashSet<String>();
localSet.addAll(localMd5List);
localMd5List.clear();
localMd5List.addAll(localSet);
return localSet;
}
listChildren method for recursive result :
public static void listChildren(File dir) throws IOException{
File[] files = dir.listFiles();
for(File file:files){
if(file.isDirectory()){
listChildren(file);
}
else{
//Return file variable to use them in localMd5Listing()
}
}
}
Unfortuntely I didn't know how to link the 2 methods to return values of listChildren() in localMd5Listing(). I think it's not the good way to have a listFile() in the first method too.
Thank you !

Extract the if statement in localMD5listing into a method recurseMD5 that takes a File argument and the list of hashes for update. Then start off the process by calling
recurseMD5(baseModDirectoryFile, localmd5List);
and in recurseMD5 you just recurse on all listFiles() when the parameter is a directory. If, OTOH, it is a regular file, you add the md5.
void recurseMD5(File it, List<String> hashes) {
if (it.isDirectory) {
for (File f : it.listFiles()) recurseMD5(f, hahses);
}
else {
// process MD5 hash of file
}
}

the basic setup you want is something like the following pseudocode
public List<string> getAllHashes() {
List<String> hashes = new List<String>();
//calculate all the hashes
foreach(directory d in directories) {
hashes.add(directory.getAllHashes());
}
return hashes;
}
I know that this is no full code whatsoever, but with this you should be able to make the recursive loop. Don't forget to check if there actually ARE directories in there, or you might get a nullpointer!

Filtering files in a directory in Java

public class Sorter {
String dir1 = ("C:/Users/Drew/Desktop/test");
String dir2 = ("C:/Users/Drew/Desktop/");
public void SortingAlgo() throws IOException {
// Declare files for moving
File sourceDir = new File(dir1);
File destDir = new File(dir2);
//Get files, list them, grab only mp3 out of the pack, and sort
File[] listOfFiles = sourceDir.listFiles();
if(sourceDir.isDirectory()) {
for(int i = 0; i < listOfFiles.length; i++) {
//list Files
System.out.println(listOfFiles[i]);
String ext = FilenameUtils.getExtension(dir1);
System.out.println(ext);
}
}
}
}
I am trying to filter out only .mp3's in my program. I'm obviously a beginner and tried copying some things off of Google and this website. How can I set a directory (sourceDir) and move those filtered files to it's own folder?

File provides an ability to filter the file list as it's begin generated.
File[] listOfFiles = sourceDir.listFiles(new FileFilter() {
#Override
public boolean accept(File pathname) {
return pathname.getName().toLowerCase().endsWith(".mp3");
}
});
Now, this has a number of benefits, the chief among which is you don't need to post-process the list, again, or have two lists in memory at the same time.
It also provides pluggable capabilities. You could create a MP3FileFilter class for instance and re-use it.

I find the NIO.2 approach using GLOBs or custom filter the cleanest solution. Check out this example on how to use GLOB or filter example in the attached link:
Path directoryPath = Paths.get("C:", "Program Files/Java/jdk1.7.0_40/src/java/nio/file");
if (Files.isDirectory(directoryPath)) {
try (DirectoryStream<Path> stream = Files.newDirectoryStream(directoryPath, "*.mp3")) {
for (Path path : stream) {
System.out.println(path);
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
For more information about content listing and directory filtering visit Listing and filtering directory contents in NIO.2

if(ext.endWith(".mp3")){
//do what ever you want
}

Odd Bug , using File.RenameTo()

I have a program in which i must rename a set of folders. they are all in "ID [Name]" format, and I want to rename them to "Name [ID]". (Its more of a training for me, for learning java))
the problem is, if the number of folders it must rename go beyond 20-24 . the program won't work, and will give the files faulty names. (the renaming process succeeds, but names are wrong)
but if they are below 20 folders, it works perfectly.(tested with the same folders)
here's the whole code:
public class DirRename {
private String parentDir;
private DirectoryStream<Path> fileList;
public DirRename(final Path dir)
{
parentDir = dir.toString();
if(!Files.exists(dir) || !Files.isDirectory(dir) || !Files.isReadable(dir))
System.out.println("Directory Read Error!!!");
//filter to return only directories in parent folder
DirectoryStream.Filter<Path> dirOnlyFilter =
new DirectoryStream.Filter<Path>() {
public boolean accept(Path file) throws IOException {
return (Files.isDirectory(file));
}
};
try
{
fileList = Files.newDirectoryStream(dir,dirOnlyFilter);
}
catch(IOException | DirectoryIteratorException x)
{
System.err.println(x);
}
}
public void rename()
{
for(Path filepath : fileList)
{
String name = filepath.getFileName().toString();
File inFile = filepath.toFile();
if(!inFile.exists() || !inFile.isDirectory() || !inFile.canWrite())
{
System.out.println("Directory is not writeable");
return;
}
Pattern regex = Pattern.compile("((?:[\\w\\d]*(?:\\s|-){0,2}[\\w\\d]+)*)\\s*-*\\s*(?:\\[|\\Q(\\E)(.+)(?:\\]|\\Q)\\E)$");
Matcher match = regex.matcher(name);
while(match.find())
{
String gameID = match.group(1);
String gameName = match.group(2);
String rename = parentDir+File.separator+gameName+" ["+gameID+"]";
File toFile = new File(rename);
if(!Paths.get(rename).isAbsolute())
{
System.out.println("Cannot rename "+name+"to "+rename);
return;
}
if(inFile.renameTo(toFile))
System.out.println("Success!");
else
System.out.println("Renaming Failed!!! for "+rename);
}
}
}
}
I tried checking the names with "system.out.println(toFile.getName())" while deleting the line "inFile.renameTo(toFile)". all names were correct.
but when i added that line back, the same names were printed incorrectly.(although some that were printed correctly were renamed incorrectly)
I'm completely confused. and I'm new to java, and generally less than a noob programmer. can someone please tell me what's going on?
Many thanks
EDIT: I found the problem.the loop:
for(Path filepath : fileList){}
runs 116 times while i only have 64 folders. I can't find any explanation as to why this happens, I use the same loop to print folder names only in the following function and it runs exactly 64 times.( exactly the number of folders I have)
public void printFolders()
{
for(Path filepath : fileList)
System.out.println(filepath.getFileName());
}

okay I finally Fixed my own problem. here's my guess on why this happened (I don't know the inner working of DirectoryStream so its just a guess).
when the folders were more than a few, the stream would read the previously renamed folders and add them as new folders, thus they were getting renamed twice. either changing the name back to original, or deforming it (the renaming wasn't designed to be 100% re-applicable).
In case of a few folders, the loop would be over before the stream had the chance to refresh, thus no problems.
so here's how i fixed it. by adding the following method, and iterating through an array of paths instead of the stream.
private Path[] getVerifiedPaths()
{
ArrayList<Path> verifiedFilePaths= new ArrayList<>();
for(Path filepath : fileList)
verifiedFilePaths.add(filepath);
return verifiedFilePaths.toArray(new Path[0]);
}
Path[] filePaths = getVerifiedPaths();
for(Path filePath : filePaths) { ...rename...}
instead of:
for(Path filepath : fileList){...rename...}
thanks to "JB Nizet" for his suggestion (comment above).

Delete all files in directory (but not directory) - one liner solution

I want to delete all files inside ABC directory.
When I tried with FileUtils.deleteDirectory(new File("C:/test/ABC/")); it also deletes folder ABC.
Is there a one liner solution where I can delete files inside directory but not directory?

import org.apache.commons.io.FileUtils;
FileUtils.cleanDirectory(directory);
There is this method available in the same file. This will also recursively deletes all sub-folders and files under them.
Docs: org.apache.commons.io.FileUtils.cleanDirectory

Do you mean like?
for(File file: dir.listFiles())
if (!file.isDirectory())
file.delete();
This will only delete files, not directories.

Peter Lawrey's answer is great because it is simple and not depending on anything special, and it's the way you should do it. If you need something that removes subdirectories and their contents as well, use recursion:
void purgeDirectory(File dir) {
for (File file: dir.listFiles()) {
if (file.isDirectory())
purgeDirectory(file);
file.delete();
}
}
To spare subdirectories and their contents (part of your question), modify as follows:
void purgeDirectoryButKeepSubDirectories(File dir) {
for (File file: dir.listFiles()) {
if (!file.isDirectory())
file.delete();
}
}
Or, since you wanted a one-line solution:
for (File file: dir.listFiles())
if (!file.isDirectory())
file.delete();
Using an external library for such a trivial task is not a good idea unless you need this library for something else anyway, in which case it is preferrable to use existing code. You appear to be using the Apache library anyway so use its FileUtils.cleanDirectory() method.

Java 8 Stream
This deletes only files from ABC (sub-directories are untouched):
Arrays.stream(new File("C:/test/ABC/").listFiles()).forEach(File::delete);
This deletes only files from ABC (and sub-directories):
Files.walk(Paths.get("C:/test/ABC/"))
.filter(Files::isRegularFile)
.map(Path::toFile)
.forEach(File::delete);
^ This version requires handling the IOException

Or to use this in Java 8:
try {
Files.newDirectoryStream( directory ).forEach( file -> {
try { Files.delete( file ); }
catch ( IOException e ) { throw new UncheckedIOException(e); }
} );
}
catch ( IOException e ) {
e.printStackTrace();
}
It's a pity the exception handling is so bulky, otherwise it would be a one-liner ...

public class DeleteFile {
public static void main(String[] args) {
String path="D:\test";
File file = new File(path);
File[] files = file.listFiles();
for (File f:files)
{if (f.isFile() && f.exists)
{ f.delete();
system.out.println("successfully deleted");
}else{
system.out.println("cant delete a file due to open or error");
} } }}

rm -rf was much more performant than FileUtils.cleanDirectory.
Not a one-liner solution but after extensive benchmarking, we found that using rm -rf was multiple times faster than using FileUtils.cleanDirectory.
Of course, if you have a small or simple directory, it won't matter but in our case we had multiple gigabytes and deeply nested sub directories where it would take over 10 minutes with FileUtils.cleanDirectory and only 1 minute with rm -rf.
Here's our rough Java implementation to do that:
// Delete directory given and all subdirectories and files (i.e. recursively).
//
static public boolean clearDirectory( File file ) throws IOException, InterruptedException {
if ( file.exists() ) {
String deleteCommand = "rm -rf " + file.getAbsolutePath();
Runtime runtime = Runtime.getRuntime();
Process process = runtime.exec( deleteCommand );
process.waitFor();
file.mkdirs(); // Since we only want to clear the directory and not delete it, we need to re-create the directory.
return true;
}
return false;
}
Worth trying if you're dealing with large or complex directories.

For deleting all files from directory say "C:\Example"
File file = new File("C:\\Example");
String[] myFiles;
if (file.isDirectory()) {
myFiles = file.list();
for (int i = 0; i < myFiles.length; i++) {
File myFile = new File(file, myFiles[i]);
myFile.delete();
}
}

Another Java 8 Stream solution to delete all the content of a folder, sub directories included, but not the folder itself.
Usage:
Path folder = Paths.get("/tmp/folder");
CleanFolder.clean(folder);
and the code:
public interface CleanFolder {
static void clean(Path folder) throws IOException {
Function<Path, Stream<Path>> walk = p -> {
try { return Files.walk(p);
} catch (IOException e) {
return Stream.empty();
}};
Consumer<Path> delete = p -> {
try {
Files.delete(p);
} catch (IOException e) {
}
};
Files.list(folder)
.flatMap(walk)
.sorted(Comparator.reverseOrder())
.forEach(delete);
}
}
The problem with every stream solution involving Files.walk or Files.delete is that these methods throws IOException which are a pain to handle in streams.
I tried to create a solution which is more concise as possible.

I think this will work (based on NonlinearFruit previous answer):
Files.walk(Paths.get("C:/test/ABC/"))
.sorted(Comparator.reverseOrder())
.map(Path::toFile)
.filter(item -> !item.getPath().equals("C:/test/ABC/"))
.forEach(File::delete);
Cheers!

package com;
import java.io.File;
public class Delete {
public static void main(String[] args) {
String files;
File file = new File("D:\\del\\yc\\gh");
File[] listOfFiles = file.listFiles();
for (int i = 0; i < listOfFiles.length; i++)
{
if (listOfFiles[i].isFile())
{
files = listOfFiles[i].getName();
System.out.println(files);
if(!files.equalsIgnoreCase("Scan.pdf"))
{
boolean issuccess=new File(listOfFiles[i].toString()).delete();
System.err.println("Deletion Success "+issuccess);
}
}
}
}
}
If you want to delete all files remove
if(!files.equalsIgnoreCase("Scan.pdf"))
statement it will work.

Java to traverse all directories and find a file

I have to search for a file which can be in any directory or drive. It should be compatible with any operating system. When I googled, most of the code iterates through a particular directory but not full file system. Is there any way to do it efficiently ? Any help or suggestion will be really appreciated.
Below code where i got from http://www.mkyong.com/java/how-to-traverse-a-directory-structure-in-java/, but we have to pass some directory as parameter. Is there a way to generalize to get all the locations ?
public static void main (String args[]) {
displayIt(new File("C:\\"));
}
public static void displayIt(File node){
System.out.println(node.getAbsoluteFile());
if(node.isDirectory()){
String[] subNote = node.list();
for(String filename : subNote){
displayIt(new File(node, filename));
}
}

Apache Commons-IO is a good API for this kind of Operation. For Unix system you could just use root "/" however this will not do for windows, hence you will have to ask for all roots and iterate over them:
File[] roots = File.listRoots();
Collection<File> files = new ArrayList<File>();
for(File root : roots) {
files.addAll(FileUtils.listFiles(
root,
new RegexFileFilter(<your regex filter>),
DirectoryFileFilter.DIRECTORY
));
}

This sort of code snippet will list all the files in a directory and sub directories. You do not have to add any of the files to allFiles and you could do your check there. As you havent supplied any code yet (so I assume you havent tried anything) I'll let you update it ;)
private void addFiles(File file, Collection<File> allFiles) {
File[] files = file.listFiles();
if (files != null) {
for (File f : files) {
allFiles.add(f);
addFiles(f, allFiles);
}
}
}

if you want to do this by recursion, here is code for DFS, code might not working (i never test it), and it is not optimized, but it might give you some ideas how to solve your problem
File find(String directoryName, String pattern)
{
File currentDirectory = loadFile(directoryName);
for (String name: currentDirectory .list())
{
File children = loadFile(name)
if (children.isDirectory())
{
File file = find(name, pattern)
if (file !=null)
{
return file;
}
}
else
{
if (match(name,pattern)
{
return children;
}
}
}
return null;
}