Develop Reference

File. ListFiles with FileNameFilter returning null instead of list of files with matching file name

I have written a find function which is like this :
public static List<File> find ( String path, String fName) {
List<File> list = new ArrayList<>() ;
File dir = new File(path) ;
if( dir. isDirectory() ) {
for( String aChild : dir. list()) {
list = find(path + File.separator + aChild, fName) ;
}
}
else {
File[] files = dir. listFiles ( (d, name) -> name. startsWith(fName) && name. endsWith(".txt")) ;
for(File fl : files)
list. add(fl) ;
}
return list;
}
The Directory structure on my Local machine is like C:\Salary with sub directories like January, February etc. Each of the sub directory contains files like 601246_jan_sal.txt or 601246_ feb_sal.txt.
I am calling the find function like
List<File> filePath = Utils. find("C:\\Salary\\", "601246") ;
And then performing operation on each individual file.
The problem is that in the find method dir.listFiles(FileNameFilter) is returning null value.
What am I doing wrong?

Below is basically the same method with the exception that is uses regex along with the String#matches() method to determine a file name match. I used regex so that the ? and * wildcard characters can be used within your file name search criteria, for example:
"601246*.txt"
You may find this useful for other searches you might like to carry out.
There is no returned object with this method, you just need to pass the List to it. Here is an example of how you might use it:
List<File> fileList = new ArrayList<>();
String searchCriteria = "601246*.txt";
searchFolder(new File("C:\\Salary"), searchCriteria, fileList);
// Display found files within the Console Window:
if (!fileList.isEmpty()) {
for (File file : fileList) {
System.out.println(file.getAbsolutePath());
}
}
else {
System.out.println("File name (" + searchCriteria + ") can not be found!");
}
This will search the directory (and all its sub-directories) located at C:\Salary within the local file system for all files that start with 601246 and ends with .txt. Since your files are in the following format:
601246_jan_sal.txt or 601246_ feb_sal.txt
and you happen to want all the files for February Sales, your search criteria might be: *feb?sal.txt.
Here is the searchFolder() method:
/**
* This method navigates through the supplied directory and any
* sub-directories contained within it for the supplied file name search
* criteria. Anything found is placed within the supplied List object.<br>
*
* #param file (File) The starting point directory (folder) in
* the local file system where the file(s) search
* should begin.<br>
*
* #param searchCriteria (String) The name of the file to search for. The
* wildcard characters '?' and '*' can also be used
* within the search criteria. Using an asterisk (*)
* allows you to replace a string of text. This is
* often useful if you know what kind of file you’re
* looking for but don’t know where it is or what
* certain name parts might be. <br><br>
*
* The wildcard '?' lets you use it to replace any character in a search.
* This means that if you’re looking for a file and you’re not sure how it
* is spelled, you can simply substitute '?' for the characters you don’t
* know. In the following example, we search for files that start with
* “img_2” and ends with “.jpg”:<pre>
*
* img_2???.jpg</pre><br>
*
* #param list (List Interface of type File: {#code List<File>})
* This would be the List of File you pass to this
* method. It will be this list that is filled with
* found files objects.<br>
*
* #param ignoreLetterCase (Optional - Boolean) Default is true. The search
* is not letter case sensitive. If false is
* supplied then the search is letter case
* sensitive.
*/
public static void searchFolder(File file, String searchCriteria, List<File> list, boolean... ignoreLetterCase) {
boolean ignoreCase = true;
if (ignoreLetterCase.length > 0) {
ignoreCase = ignoreLetterCase[0];
}
// Convert the supplied criteria string to a Regular Expression
// for the String#matches() method
String regEx = searchCriteria.replace("?", ".").replace("-", ".").replace("*", ".*?");
if (ignoreCase) {
regEx = "(?i)" + regEx;
}
if (file.isDirectory()) {
//System.out.println("Searching directory ... " + file.getAbsoluteFile());
//do you have permission to read this directory?
if (file.canRead()) {
for (File temp : file.listFiles()) {
if (temp.isDirectory()) {
searchFolder(temp, searchCriteria, list, ignoreCase);
}
else {
if (temp.getName().matches(regEx)) {
list.add(temp);
}
}
}
}
else {
System.err.println(file.getAbsoluteFile() + " - PERMISSION DENIED!");
}
}
}

Since you are using recursion, you should pass the list (of files) as a parameter to method find and not create a new list on each invocation. Hence the method find does not need to return a value.
public static void find(String path, String fName, List<File> fileList) {
File dir = new File(path);
if (dir.isDirectory()) {
for (File aChild : dir.listFiles()) {
if (aChild.isDirectory()) {
find(aChild.getAbsolutePath(), fName, fileList);
}
else {
String name = aChild.getName();
if (name.startsWith(fName) && name.endsWith(".txt")) {
fileList.add(aChild);
}
}
}
}
else {
String name = aChild.getName();
if (name.startsWith(fName) && name.endsWith(".txt")) {
fileList.add(aChild);
}
}
}
If method parameter path indicates a directory, then list the files in that directory. For each file in the directory, check whether the name of the file matches your search criteria and if it does then add it to the list. If it doesn't then check if it is itself a directory and if it is then recursively call method find with the new directory.
Initially call method find like so
List<File> list = new ArrayList<File>();
find("C:\\Salary\\", "601246", list);
Now list should contain the list of relevant files. So the following line will print the contents of list
System.out.println(list);

I see a fundamental problem in your logic that answers your question. Here are the key lines along with an explanation:
if( dir.isDirectory() ) {
....
}
else {
File[] files = dir.listFiles(...); // <- "dir" is a file, not a directory
for(File fl : files)
list.add(fl) ;
}
You check if File object dir represents a directory. If that test fails (ie: it's not a directory), then you call dir.listFiles on it and assign the result to files. But if it's not a directory, then by the definition of that function, it will return null. It seems that if the check for a directory fails, you should just add the object to list instead of performing another operation on it.
I think you want this:
if( dir.isDirectory() ) {
....
}
else {
list.add(dir) ;
}
I guess dir isn't really the right name for the variable here, as it isn't always a directory.

Note that the Files.walk() method already does what you're trying to do.
public static List<File> find ( String path, String fName) {
List<File> result = new ArrayList<>();
FileVisitor<Path> visitor = new SimpleFileVisitor() {
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
if (attrs.isRegularFile()) {
String name = file.getFileName();
if (name.startsWith(fName) && name.endsWith(".txt")) {
result.add(file.toFile());
}
}
return FileVisitResult.CONTINUE;
}
};
Files.walk(Paths.get(path), visitor);
return result;
}

Fastest way to search a file in many folders

I need to search for a file based on a list of filenames inside a directory that contains 3100+ files and 14 folders but it takes hours to complete the search. Furthermore, I am only talking about 1 list of filenames yet, I still have other list of filenames to search.
After a locating the file to be searched, I need to access it and search for words inside it. And lastly proceed to the next file.
What I currently doing right now is that I used the concept of Breadth-First Search but it also takes hours to complete the search.
Are there any other ways to complete this task much faster?

See comments in the code
public class FileFinderApp {
// Create a list of file names that you want to process
private List<String> fileNames = new ArrayList<String>(Arrays.asList(new String[] {"test.txt","test1.txt","test2.txt"}));
// Create a FolderFilter, this just allows us to find files that are actually folders
private FolderFilter folderFilter = new FolderFilter();
public FileFinderApp() {
// Let the user know we are doing something in case there is no other output
System.out.println("Finding Files");
// Create a File to represent our starting folder
File startFolder = new File("F:\\FileTest");
// Get the list of Files that match our criteria
List<File> foundFiles = findFiles(startFolder, new FileNameFilter());
// loop through our files and do something with them
for (File file : foundFiles) {
// do something with each file
System.out.println(file.toString());
}
// Let the user know we are done.
System.out.println("Done Finding Files");
}
// This filter returns true if the File is a file (not folder, etc.) and matches one of our file names.
class FileNameFilter implements FileFilter {
public boolean accept(File file) {
return fileNames.contains(file.getName()) && file.isFile();
}
}
// This filter only returns folders
class FolderFilter implements FileFilter {
public boolean accept(File file) {
return file.isDirectory();
}
}
// Here's the meat and potatoes
private List<File> findFiles(File folder, FileFilter filter) {
// Create an empty list of Files
List<File> foundFiles = new ArrayList<File>();
// Find all sub-folders
File[] folders = folder.listFiles(folderFilter);
// Find the folders that pass our filter
File[] files = folder.listFiles(filter);
// add the files to our found files
foundFiles.addAll(Arrays.asList(files));
// for (File file : files) {
// System.out.println(file.getAbsolutePath());
// }
// loop through our sub-folders and get any files that match our filter there and add them to our list
// This is recursive and will execute as many levels as there are nested folders
for(File subFolder : folders) {
foundFiles.addAll(findFiles(subFolder, filter));
}
return foundFiles;
}
public static void main(String[] args) {
// don't block the UI
SwingUtilities.invokeLater(new Runnable() {
public void run() {
new FileFinderApp();
}
});
}
}

See comments :
/**
* #param searchIn
* a valid path to any non null file or directory
* #param searchFor
* any non null name of a file or directory
*/
public static File findFile(String searchIn, String searchFor){
return findFile(new File(searchIn), searchFor);
}
/**
* #param searchIn
* not null file or directory
* #param searchFor
* any non null name of a file or directory
*/
public static File findFile(File searchIn, String searchFor){
if(searchIn.getName().equalsIgnoreCase(searchFor)) {
return searchIn;
}
if(searchIn.isDirectory() && (searchIn.listFiles() != null)){
for (File file : searchIn.listFiles() ){
File f = findFile(file, searchFor);
if(f != null) {
return f;
}
}
}
return null;
}
Tested with :
String sourcePath = "C:\\";
String searchFor = "Google Docs.ico";
File f = findFile(sourcePath, searchFor);
Performance measured :
time:13.596946 seconds, directories:17985 files:116837

read single file type from directory containing multiple file types java

Is it possible to read single type of file (say like CSV or txt) from the directory containing multiple file types (say like cvs , txt , doc, xml, html, etc..)
My problem is I have to provide path of the mainDirectory as an input , which further has layers of subdirectories inside it. I specifically need to read and process CSV files within these directories as I further dive in.
I am done with multiple layers of folder traversing using recursion courtesy which I have the names and total count of the files within the mainDirectory. I am also done with logic to read and CSV files. All I need to do is to get path only of CSV files and process them.
i am using below mentioned code for traversing multiple folders and getting the name :-
package org.ashu.input;
import java.io.File;
/**
*
* #author ashutosh
*/
public class MultiDir {
private int fileCount;
public void readFile(File f){
System.out.println(f.getPath());
fileCount++;
}
public void readDir(File f){
File subdir[] = f.listFiles();
for(File f_arr : subdir){
if(f_arr.isFile()){
this.readFile(f_arr);
}
if(f_arr.isDirectory()){
this.readDir(f_arr);
}
}
}
public static void main(String[] args){
MultiDir md = new MultiDir();
File mainDir = new File("/Users/ashutosh/Downloads/main_directory");
md.readDir(mainDir);
System.out.println(md.fileCount);//all file count = 1576, need to specifically get CSV
}
}
Any suggestion please.

This code will return every file matching the given extension (in your case .csv):
public static List<File> getFiles(String extension, final File folder)
{
extension = extension.toUpperCase();
final List<File> files = new ArrayList<File>();
for (final File file : folder.listFiles())
{
if (file.isDirectory())
files.addAll(getFiles(extension, file));
else if (file.getName().toUpperCase().endsWith(extension))
files.add(file);
}
return files;
}

You can simply check the extension of the file in your readDir() method. The below looks for jpg, you can use your desired extension
public void readDir(File f){
File subdir[] = f.listFiles();
for(File f_arr : subdir){
if(f_arr.isFile() && f_arr.getName().endsWith(".jpg")){
this.readFile(f_arr);
}
if(f_arr.isDirectory()){
this.readDir(f_arr);
}
}
}

List files from directories and sub directories in java including only partial file paths

I need to get the paths of files and their parent directories in java from a given directory but not including it.
So for example, If my method was given the path: /home/user/test as a path it would return the paths of all files in that directory and under it.
So if /home/user/test had the sub folders: /subdir1 and /subdir2 each containing file1.txt and file2.txt then the result of the method would be 2 strings containing /subdir1/file1.txt and /subdir2/file2.txt
And if subdir1 had a directory inside it called subsubdir and inside that file3.txt, then the string created for that file would be /subdir1/subsubdir/file3.txt, and if there are further sub directories that would continue.
The idea is I just want the directory paths above the file but not the absolute path so only the directories AFTER the initial given path.
I know its a little confusing but I'm sure someone can make sense of it. Right now all I have is a recursive function that prints out file names and their absolute paths.
Any assistance on this?

What would have been nice if you had tried something and asked questions about that...
However...
public class TestFileSearch {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
new TestFileSearch();
}
public TestFileSearch() {
File parentPath = new File("C:/Users/shane/Documents");
List<String> files = list(parentPath);
for (String file : files) {
System.out.println(file);
}
}
protected List<String> list(File parent) {
return listFiles(parent, parent);
}
protected List<String> listFiles(File parent, File folder) {
List<String> lstFiles = new ArrayList<String>(25);
if (folder.isDirectory()) {
File[] files = folder.listFiles();
if (files != null) {
for (File file : files) {
if (file.isDirectory()) {
lstFiles.addAll(listFiles(parent, file));
} else {
String path = file.getPath();
String offset = parent.getPath();
path = path.substring(offset.length());
lstFiles.add(path);
}
}
}
}
return lstFiles;
}
}
You could simply do a normal folder recursion, returning a list of files and THEN strip of the prefix, but that's up to you

What about using the absolute path you currently have but removing the prefix from it using String.replace
You said you had the full, absolute path, say in full
then just do
String relative = full.replace(prefix, "");

If you have the input "/home/user/text", all absolute paths to files will start with /home/user/text/. If you're already able to print a list of all files under text/, then all you need to do is take the suitable substring.
The following function should visit all files under pathToDir. In the printFileName function, you can remove the /home/user/text part and print the file names
public static void gotoAllFiles(File pathToDir) {
if (pathToDir.isDirectory()) {
String[] subdirs = pathToDir.list();
for (int i=0; i<subdirs.length; i++) {
gotoAllFiles(new File(pathToDir, subdirs[i]));
}
} else {
printFileName(pathToDir);
}
}

For each file found, print the file.getAbsolutePath().substring(rootPath.length());

Get file case-sensitive name, with case-insensitive spelling

I am making an application where the user picks a file from:
FilePicker.PickFile(filename)
where filename is a string.
In the method, it will translate into:
File file = new File(filename);
and nothing is wrong with that. Next, I do,
if(file.exists()){
System.out.println(file.getName());
}
else{
System.out.println("Fail.");
}
and this is where the problem begins. I want to get the name of the file, say "HELLO.txt," but if filename is "hello.txt," it still passes the file.exists() check, and file.getName() returns as "hello.txt," not "HELLO.txt". Is there a way, to return file.getName() as the case-sensitive version as "HELLO.txt?" Thanks!
An example:
HELLO.txt is the real file
FilePicker.PickFile("hello.txt");
OUTPUT:
hello.txt

When you are using Windows, which is case preserving (FAT32/NTFS/..), you can use file.getCanonicalFile().getName() to get the canonical name of the selected file.
When you are using Linux or Android and you want to select a file based on a file name that does not necessarily match case, iterate through all files in the file's directory (file.getParent()), and pick the one that .equalsIgnoreCase the filename. Or see Case-insensitive File.equals on case-sensitive file system

/**
* Maps lower case strings to their case insensitive File
*/
private static final Map<String, File> insensitiveFileHandlerCache = new HashMap<String, File>();
/**
* Case insensitive file handler. Cannot return <code>null</code>
*/
public static File newFile(String path) {
if (path == null)
return new File(path);
path = path.toLowerCase();
// First see if it is cached
if (insensitiveFileHandlerCache.containsKey(path)) {
return insensitiveFileHandlerCache.get(path);
} else {
// If it is not cached, cache it (the path is lower case)
File file = new File(path);
insensitiveFileHandlerCache.put(path, file);
// If the file does not exist, look for the real path
if (!file.exists()) {
// get the directory
String parentPath = file.getParent();
if (parentPath == null) {
// No parent directory? -> Just return the file since we can't find the real path
return file;
}
// Find the real path of the parent directory recursively
File dir = Util.newFile(parentPath);
File[] files = dir.listFiles();
if (files == null) {
// If it is not a directory
insensitiveFileHandlerCache.put(path, file);
return file;
}
// Loop through the directory and put everything you find into the cache
for (File otherFile : files) {
// the path of our file will be updated at this point
insensitiveFileHandlerCache.put(otherFile.getPath().toLowerCase(), otherFile);
}
// if you found what was needed, return it
if (insensitiveFileHandlerCache.containsKey(path)) {
return insensitiveFileHandlerCache.get(path);
}
}
// Did not find it? Return the file with the original path
return file;
}
}
Use
File file = newFile(path);
instead of
File file = new File(path);
It's backed by a cache so it shouldn't be too slow. Did a few test runs and it seems to work. It recursively checks the the parent directories to see if they do have the correct letter cases. Then it lists for each directory all files and caches their correct letter casing. In the end it checks if the file with the path has been found and returns the file with the case sensitive path.

Looks like in Java 7 and above on Windows, you can use Path#toRealPath(NOFOLLOW_LINKS) and it would be more correct than getCanonicalFile() in the presence of symlinks.