I have an android app that needs to be able to delete large directories quickly. I know that when an app has a lot of data, the OS is able to clear the data app instantly (from app settings). However, when I manually try to delete a directory with a background task, looping through the folders and deleting the files takes a long time (several minutes) to complete. How does the OS delete data so quickly, but when it is done in code, it takes such a long time? Is there a faster way to do it?
private boolean deleteDirectory(String dir) {
Log.d(TAG, "deleteDirectory");
// If dir is not separated from the end of the character to the file, the file is automatically added delimiter
if (!dir.endsWith(File.separator))
dir = dir + File.separator;
File dirFile = new File(dir);
// If dir corresponding file does not exist or is not a directory, then exit
if ((!dirFile.exists()) || (!dirFile.isDirectory())) {
return false;
}
boolean flag = true;
// delete all the files in the folder, including subdirectories
File[] files = dirFile.listFiles();
if (files != null) {
for (int i = 0; i < files.length; i++) {
final int count = i;
// delete subfolders
if (files[i].isFile()) {
try {
FileUtils.forceDelete(files[count]);
} catch (IOException e) {
Log.e(TAG, e.getLocalizedMessage(), e);
}
} else if (files[i].isDirectory()) {
flag = deleteDirectory(files[i]
.getAbsolutePath());
}
if (dir == path) {
Log.d(TAG, "delete dir: " + dirFile);
curPosi++;
publishProgress(curPosi);
}
}
}
if (!flag) {
return false;
}
// delete the current directory
if (dirFile.delete()) {
return true;
} else {
return false;
}
}
I'm pretty sure that the OS deletes it at a lower level, by accessing the actual filesystem. Either way, you should try using the Apache Commons FileUtils library -- it has a deleteDirectory() method, which will save you a lot of recursion.
Alternatively, you can actually try to run "rm -rf" directly, using runtime.exec(), but I'm not sure that permissions it would require. Here's an example
Related
I am creating an application that automatically sorts and organizes files into a database. I have written my code to read files within the imported folder one at a time, and process them into the DB. However, I am having trouble looping this process, so that I can process files that are nested in any amount of folders within the original folder that the user wants to input.
I simply need to instruct my program to go back to a specific part of my code and start running from there again.
Another possible way to solve this issue would be to create a way to list out all of the individual files within folder (including all the files within subfolders), and I could easily fit that into my program too.
I tried using labeled continue, return, and break keywords based off of an answer I got online, but I never expected those to succeed in looping my code back to a specific spot.
JFileChooser chooser = new JFileChooser();
chooser.setSelectedFiles(null);
chooser.setFileSelectionMode(JFileChooser.FILES_AND_DIRECTORIES);
chooser.showOpenDialog(null);
//Getting file paths from within folder
File f = chooser.getSelectedFile();
String file = f.getAbsolutePath();
if (f.isDirectory()) {
//Need to loop back to here
File folder = new File(file);
File[] listOfFiles = folder.listFiles();
for (int i = 0; i < listOfFiles.length; i++) {
if (listOfFiles[i].isDirectory()) {
//Code here is run if there is a folder within a folder. I tested it too
//I want the code here to loop back above where it says "Need to loop back to here"
}
if (listOfFiles[i].isFile()) { //Once I list the files from within the folder, their information gets assigned variable here, and the rest of my program sorts it and saves it to DB accordingly.
//Everything below here is not important, but it might be helpful to see what happens each file with the folders.
System.out.println(listOfFiles[i]);
String filename = (listOfFiles[i].getName()); //For Files
Long filemodified = (listOfFiles[i].lastModified());
String filepath = (listOfFiles[i].getAbsolutePath());
Long filesizeraw = (listOfFiles[i].length());
long filehashcode = (listOfFiles[i].hashCode());
String fileparent = (listOfFiles[i].getParent());
Currently, there is no error message. It would process any individual files directly in the imported file (not nested in any folder within the folder), but wouldn't get to any of the files that are in folders within folders.
Another possible way to solve this issue would be to create a way to list out all of the individual files within folder (including all the files within subfolders), and I could easily fit that into my program too
Although this doesn't do the SQLite inserts, the following class extracts a list (of File objects) the files (thus file name and path are available via the File object).
public class FTS {
private ArrayList<File> mFileList; //Resultant list of Files extracted
private String mBaseDirectory; // The Directory to search
private long mSubDirectoryCount; // The count of the subdirectories
//Constructor
public FTS(String directory) {
this.mBaseDirectory = directory;
this.mSubDirectoryCount = 0;
buildFileListing(this.mBaseDirectory);
}
//
private void buildFileListing(String directory) {
// Initialise the ArrayList for the result
if (mFileList == null) {
mFileList = new ArrayList(){};
}
//Get the File (directory to process)
File dir = new File(directory);
// Get the List of the Directories contents
String[] filelist = dir.list();
// If empty (null) then return
if (filelist == null) {
return;
}
// Loop through the directory list
for (String s: filelist) {
//get the current list item as a file
File f = new File(dir.getAbsolutePath() + File.separator + s);
// is it a file or directory?
if (f.isFile() && !f.isDirectory()) {
this.mFileList.add(f); // If a file then add the file to the extracted list
} else {
// If a directory then increment the count of the subdirectories processed
mSubDirectoryCount++;
// and then recursively call this method to process the directory
buildFileListing(f.getAbsolutePath());
}
}
}
// return the list of extracted files
public ArrayList<File> getFileList() {
return this.mFileList;
}
// return the number of sub-directories processed
public long getSubDirectoryCount() {
return this.mSubDirectoryCount;
}
}
An example usage of the above is :-
public class Main {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
// TODO code application logic here
FTS fileTreeSearch;
String BaseDirectory = "E:" + File.separator;
List<File> files = (fileTreeSearch = new FTS(BaseDirectory)).getFileList();
System.out.println("Extracted " + String.valueOf(files.size()) + " files, from " + String.valueOf(fileTreeSearch.getSubDirectoryCount()) + " sub-directories of " + BaseDirectory);
/* this commented out code would process all the extracted files
for (File f: files) {
System.out.println("File is " + f.getName() + "\t\t path " + f.getAbsolutePath());
}
*/
}
}
Example output from running the above :-
Extracted 186893 files, from 54006 sub-directories of E:\
I am trying to move files from one directory to another delete that file from source directory after moving.
for (File file : files) {
if (file != null) {
boolean status = moveFile(file, filePath, name, docGroupId);
if (status) {
//some operations....
}
}
}
public static boolean moveFile(final File file, final String filePath, final String groupName, Integer docGroupId) {
// TODO Auto-generated method stub
String selectedDirectory = filePath + File.separator + groupName;
InputStream in = null;
OutputStream out = null;
try {
if (!file.isDirectory()) {
File dir = new File(selectedDirectory);
if (!dir.exists()) {
dir.mkdirs();
}
String newFilString = dir.getAbsolutePath() +
File.separator + file.getName();
File newFile = new File(newFilString);
in = new FileInputStream(file);
out = new FileOutputStream(newFile);
byte[] moveBuff = new byte[1024];
int butesRead;
while ((butesRead = in.read(moveBuff)) > 0) {
out.write(moveBuff, 0, butesRead);
}
}
in.close();
out.close();
if(file.delete())
return true;
} catch (Exception e) {
return false;
}
}
The program works on Linux-Ubuntu and all files are moved to another directory and deleted from source directory, but in Windows system all files are moved but failed to delete one or two files from source directory. Please note that while debugging the program is working fine.
Consider using Files.delete instead of File.delete. The javadoc says:
Note that the Files class defines the delete method to throw an IOException when a file cannot be deleted. This is useful for error reporting and to diagnose why a file cannot be deleted.
This should provide the information necessary to diagnose the problem.
So, if problem comes with delete, possible explanations:
you do file.delete() on every files and directories. How do you know the directory is empty ? If not, it will fail, then what happen to next instructions ?
file deletion is OS-dependant. On Windows, you can have many security issues, depending on which user, which rights, which location. You should check with a file-delete-alone program;
last: files can be locked by other programs (even explorer), it is also OS-dependant.
You don't need any of this if the source and target are in the same file system. Just use File.renameTo().
I am writing a code to count file with specific extension from folder and it subfolder. this code work well when i count from small and medium large folder.
the problem start if i count from root, example from D:// . it will count for some time but stop at the middle. i assume the code is not efficient to read large number of files and folder.
public static int countFiles(File directory, String ext) {
int count = 0;
for(File file : directory.listFiles()) {
System.out.println(file);
if(file.isDirectory()) {
count += countFiles(file, ext);
}
String textFile = file.toString();
if(textFile.endsWith(ext)){
count++;
}
}
error appear
screenshot line error
here is my full code CLICK HERE
What I think is happening is that directory.listFiles() is returning null.
Despite not being mentioned in the documentation, this can happen if you don't have sufficient access rights to list the directory contents. You can pretty easily fix this as shown below.
Also, there is a slight logic error in that (for example) a directory named Files ending with .xls can also get counted as a file, when it clearly isn't. I've fixed this too.
public int countFiles(File directory, String... exts) {
int count = 0;
File[] files = directory.listFiles();
if (files != null) {
for (File file : files) {
if (file.isDirectory()) {
count += countFiles(file, exts);
} else {
String textFile = file.toString();
for (String ext : exts) {
if (textFile.endsWith(ext)) {
count++;
}
}
}
}
}
return count;
}
I would simply check for nulls at the start of countFiles.
public int countFiles(File directory, String... exts) {
if (directory == null) return 0;
If you're doing a large recursive scan of the entire file system -- it makes sense that a directory could no longer exist between the time countFiles(file, exts); is called and when directory.listFiles() is called.
When I start my application I create a temp folder:
public static File createTempDir(String name) throws IOException {
File tempDir = File.createTempFile(name, "");
if (!(tempDir.delete())) {
throw new IOException("could not delete" + tempDir.getAbsolutePath());
}
if (!(tempDir.mkdir())) {
throw new IOException("could not create" + tempDir.getAbsolutePath());
}
tempDir.deleteOnExit();
return tempDir;
}
During a session a user might load a file. As a result the old temp dir is deleted and a new is created based on the ID of the file loaded.
During load where the old temp dir is deleted I sometimes get a:
java.io.IOException: Unable to delete file:
Here is how the old temp folder is deleted:
public void cleanup(String tmpPath) {
File tmpFolder = new File(tmpPath);
if (tmpFolder != null && tmpFolder.isDirectory()) {
try {
FileUtils.deleteDirectory(file);
} catch (IOException e) {
e.printStackTrace();
}
}
}
where FileUtils is: org.apache.commons.io.FileUtils. Typically the content of the temp folder is:
mytempfolder_uuid
|-> mysubfolder
|-> myImage.jpg
And the error is:
java.io.IOException: Unable to delete file: C:\Users\xxx\AppData\Local\Temp\mytempfolder_uuid\mysubfolder\myImage.jpg
I have tried to debug the application and before the delete operation is executed verified that the above image is actually located in the specified folder.
The nasty thing is that it only happens sometimes. I have made sure not to have the folder/files in the temp folder open in any other applications. Any ideas/suggestions?
You cannot delete files which are open and you can't delete a directory which contains a file. You have to ensure all files in the directory are closed.
I'd suggest you use the Guava library. It has a method Files.createTempDir() that does exactly what you seem to need:
Atomically creates a new directory somewhere beneath the system's
temporary directory (as defined by the java.io.tmpdir system
property), and returns its name. Use this method instead of
File.createTempFile(String, String) when you wish to create a
directory, not a regular file. A common pitfall is to call
createTempFile, delete the file and create a directory in its place,
but this leads a race condition which can be exploited to create
security vulnerabilities, especially when executable files are to be
written into the directory. This method assumes that the temporary
volume is writable, has free inodes and free blocks, and that it will
not be called thousands of times per second.
try deleting the files in the temp folder before deleting it. Try somethng like
private boolean deleteFolder(File path) {
if (path.exists()) {
File[] files = path.listFiles();
for (File f : files) {
if (f.isDirectory()) {
deleteFolder(f);
} else {
f.delete();
}
}
}
return path.delete();
}
also using deleteOnExit is not a very good idea...
cheers!
public static boolean deleteDir(String path)
{
java.io.File dir = new java.io.File(path);
if (dir.isDirectory())
{
String[] filesList = dir.list();
for(String s : filesList)
{
boolean success = new java.io.File(dir, s).delete();
if(!success)
{
return false;
}
}
}
return dir.delete();
}
and then you can use it like: deleteDir("C:\\MyFolder\\subFolder\\")
I'm currently using a Java FTP library (ftp4j) to access a FTP server. I want to do a file count and directory count for the server, but this means I would need to list files within directories within directories within directories, etc.
How is this achievable? Any hints would be greatly appreciated.
Extract from the code:
client = new FTPClient();
try {
client.connect("");
client.login("", "");
client.changeDirectory("/");
FTPFile[] list = client.list();
int totalDIRS = 0;
int totalFILES = 0;
for (FTPFile ftpFile : list) {
if (ftpFile.getType() == FTPFile.TYPE_DIRECTORY) {
totalDIRS++;
}
}
message =
"There are currently " + totalDIRS + " directories within the ROOT directory";
client.disconnect(true);
} catch (Exception e) {
System.out.println(e.toString());
}
Make a recursive function that given a file that might be a directory returns the number of files and directories in it. Use isDir and listFiles.
Try using recursive functions.
This is could be a function that checks a directory for files, then you can do a check if a file has a child, that would be a directory.
If it has a child you can call that same function again for that directory, etc.
Like this pseudo java here:
void Function(String directory){
... run through files here
if (file.hasChild())
{
Function(file.getString());
}
}
I'm sure you can use that kind of coding in counting the files as well...
Just use a recursive function like below.
Note that my code uses Apache Commons Net, not ftp4j, what the question is about. But the API is pretty much the same and ftp4j seems to be an abandoned project now anyway.
private static void listFolder(FTPClient ftpClient, String remotePath) throws IOException
{
System.out.println("Listing folder " + remotePath);
FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);
for (FTPFile remoteFile : remoteFiles)
{
if (!remoteFile.getName().equals(".") && !remoteFile.getName().equals(".."))
{
String remoteFilePath = remotePath + "/" + remoteFile.getName();
if (remoteFile.isDirectory())
{
listFolder(ftpClient, remoteFilePath);
}
else
{
System.out.println("Foud remote file " + remoteFilePath);
}
}
}
}