Recursively copy one directory to another, if files don't match - java

I have a service that will be populating a directory that I need to copy to another directory periodically. The source will be populated periodically.
When I copy the directory, it will be quite large, so I only want to add files in the destination, or overwrite files that aren't the same file (e.g. file size mismatch or modification date).
Is there a simple way to do this? I'm aware of FileUtils, but it's unclear to me if it will always ovewrite all the files, and what "merge" means here, specifically if it will not copy files that already match.

Files.walkFileTree and the other methods of Files can do it:
public void copyTree(Path source,
Path destination)
throws IOException {
Files.walkFileTree(source,
new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir,
BasicFileAttributes attr)
throws IOException {
Path destPath = destination.resolve(source.relativize(dir));
Files.createDirectories(destPath);
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFile(Path file,
BasicFileAttributes attr)
throws IOException {
Path destPath = destination.resolve(source.relativize(file));
FileTime sourceTime = Files.getLastModifiedTime(file);
FileTime destinationTime = Files.getLastModifiedTime(destPath);
if (!Files.exists(destPath) ||
sourceTime.compareTo(destinationTime) > 0) {
Files.copy(file, destPath,
StandardCopyOption.COPY_ATTRIBUTES,
StandardCopyOption.REPLACE_EXISTING);
}
return FileVisitResult.CONTINUE;
}
});
}

Related

Delete Files and Folders inside Folder if is empty (Java)

String dirWay = "C:\\Project";
int daysBack = 7;
File directory = new File(dirWay);
if(directory.exists()){
File[] listFiles = directory.listFiles();
long purgeTime = System.currentTimeMillis() - (daysBack * 24 * 60 * 60 * 1000);
for(File listFile : listFiles) {
if(listFile.lastModified() < purgeTime) {
if(!listFile.delete()) {
System.err.println("Unable to delete file: " + listFile);
} else {
System.out.println(listFile);
}
}
}
}
This is working only for Project folder files. But i have some folders and files in the Project folder, and each folder has some folders and files.
How i can check inside all folder and check files last modified date and delete it if more than 7 days?
For example i have directory: C:/Project/JavaIdea/...
If files in the JavaIdea folder older than 7 days, i need delete all files and JavaIdea folder too.
The easiest way is probably to use Files.walkFileTree. Files.walk gives you a Stream<Path>, but folders come first, not after. The FileVisitor on the other hand uses two events for folders - before visiting, and after.
Something that could work (untested):
Deque<LongAdder> counts = new ArrayDeque<>(); // counting # of remaining files per level
Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(T dir, BasicFileAttributes attrs) {
counts.push(new LongAdder());
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult postVisitDirectory(T dir, BasicFileAttributes attrs) throws IOException {
long remaining = counts.pop().sum();
if (remaining == 0) {
// the directory is empty now, delete it
}
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFile(T file, BasicFileAttributes attrs) throws IOException {
if (is old enough to delete) {
// delete
} else {
counts.peek().increment();
}
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFileFailed(T file, IOException exc) {
counts.peek().increment();
return FileVisitResult.CONTINUE;
}
});

Not able to replace existing directory

I am trying to move folder 114229494 from one path to another and I want to replace the existing folder at the destination path (D:\SampleP2) but I am getting DirectoryNotEmptyException with the code mentioned below.
Since I don't want to change the name of the folder I mentioned D:\SampleP2\114229494 this as destination path
There are some images inside the folder.
Please help me to figure out what is wrong with this code.
public class MoveFiles {
private String source_path= "D:\\sampleP1\\114229494";
private String destination_path= "D:\\SampleP2\\114229494";
public void movefolder() {
File source = new File(source_path);
File destination = new File(destination_path);
Path path1 = FileSystems.getDefault().getPath(source_path);
Path path2 = FileSystems.getDefault().getPath(destination_path);
System.out.println(source);
System.out.println(destination);
try {
Files.move(path1, path2, StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
e.printStackTrace();
System.out.println("there is file in uploads");
}
}
}
If path1 is a directory and path2 exists already you cannot use Files.move(path1, path2) nor Files.move(path1, path2, StandardCopyOption.REPLACE_EXISTING) as these give rise to either FileAlreadyExistsException or DirectoryNotEmptyException.
A reliable move operation for occasions where the destination directory exists already needs to traverse the source directory tree and move each file to the same location under the destination, cleaning up folders as they are emptied.
The Files.walkFileTree method handles traversals, add a suitable FileVisitor action to move / merge the path1=>path2 files:
public static void move(final Path source, final Path dest) throws IOException {
FileVisitor<Path> visitor = new FileVisitor<>() {
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
final Path target = dest.resolve(source.relativize(dir));
System.out.println("Files.createDirectories("+target+")");
Files.createDirectories(target);
return FileVisitResult.CONTINUE;
}
public FileVisitResult visitFile(Path p, BasicFileAttributes attrs) throws IOException {
final Path targ = dest.resolve(source.relativize(p));
System.out.println("Files.move("+p+", "+targ+", StandardCopyOption.REPLACE_EXISTING)");
Files.move(p, targ, StandardCopyOption.REPLACE_EXISTING);
return FileVisitResult.CONTINUE;
}
public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
// Dir should be empty by now or there is coding error:
System.out.println("Files.deleteIfExists("+dir+")");
if (!Files.deleteIfExists(dir))
throw new IOException("Failed to delete src dir: "+dir);
return FileVisitResult.CONTINUE;
}
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
throw exc;
}
};
Files.walkFileTree(source, visitor);
}
Note that the above works for files or folders - Files.walkFileTree handles calling the appropriate callbacks.
The logic of the above move could be made significantly quicker if you use File.move(subdir,targsubdir) if detecting that a subdir is not found in the target - replacing createDirectories / deleteIfExists. That would avoid need to move every file underneath that tree. However I will leave that as an exercise for the reader.

visiting first level of directory and get Size of each directory Java

I need the list of the directories in the C: And get size for each one.
I am trying with this code:
int[] count = {0};
try {
Files.walkFileTree(Paths.get(dir.getPath()), new HashSet<FileVisitOption>(Arrays.asList(FileVisitOption.FOLLOW_LINKS)),
Integer.MAX_VALUE, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult visitFile(Path file , BasicFileAttributes attrs) throws IOException {
System.out.printf("Visiting file %s\n", file);
++count[0];
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFileFailed(Path file , IOException e) throws IOException {
System.err.printf("Visiting failed for %s\n", file);
return FileVisitResult.SKIP_SUBTREE;
}
#Override
public FileVisitResult preVisitDirectory(Path dir , BasicFileAttributes attrs) throws IOException {
System.out.printf("About to visit directory %s\n", dir);
return FileVisitResult.CONTINUE;
}
});
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
The problem I have is that if I use exactly this, Takes a lot of time because it visits each file! of the disk. I tried with differents options for FileVisitResult but I can't get the desired result. I need only first level with space in the disk.
Thanks in advance for any help.
Folders do not have a (significant) size of their own. Usually when a folder size is discussed, the meaning is the sum of sizes of all files under that folder and its sub-folders. This operation takes a "very long" time for C:\ (try right-clicking your Program Files folder and hitting properties). Remove all logs and try again. Compare the time it took your code, with the time it takes Windows Explorer to calculate that size (again, by right-clicking your Program Files folder and hitting properties).
In any case, Apache's commons-io has a one-liner:
long size = FileUtils.sizeOfDirectory(folder);
And Java-8 has another "pure" one-liner:
long size = Files.walk(Paths.get("C:\\"))
.filter(p -> p.toFile().isFile())
.mapToLong(p -> p.toFile().length())
.sum();

Working around access denied in a FileWalking Tree in Java7

The following is some simple code just to test the Files.walkFileTree() method. However, the folder /etc/ssl/private, which has these permissions (rwx--x---), throws an exception, even when I thought I guarded it with an if statement (if (permissions.equals("rwx--x---")).
What am I doing wrong? Thanks in advance.
public static void main (String []args) throws IOException, InterruptedException
{
Files.walkFileTree(Paths.get("/"), new WalkingTheThing2());
}
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException
{
PosixFileAttributeView posixView = Files.getFileAttributeView(dir, PosixFileAttributeView.class);
PosixFileAttributes posixAttr = posixView.readAttributes();
String permissions =PosixFilePermissions.toString(posixAttr.permissions());
if (permissions.equals("rwx--x---"))
{
return FileVisitResult.SKIP_SUBTREE;
}
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
throws IOException {
try{
System.out.println(file.getFileName()+" " +Files.size(file));
return FileVisitResult.CONTINUE;
}
catch(IOException io){return FileVisitResult.CONTINUE;}
}
The exception I get is: java.nio.file.AccessDeniedException: /etc/ssl/private
EDIT: Solved by overriding visitFileFailed:
public FileVisitResult visitFileFailed(Path file, IOException io)
{
return FileVisitResult.SKIP_SUBTREE;
}
If you are traversing the whole directory System and there is a situation where you got some type of Exception like AccessDeniedException and you want skip that file so that you could check the other file you need to override the visitFileFailed and skip that file or directory.
#Override
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
System.err.printf("Visiting failed for %s\n", file);
return FileVisitResult.SKIP_SUBTREE;
}
This is the work around I have found when Walking through the directory system.
Although overriding visitFileFailed solved your problem, it might be hiding the fact you are still doing several things wrong:
Files.getFileAttributeView can return null (e.g. if the file system does not support POSIX file permissions) making posixView.readAttributes() fail on NPE
posixView.readAttributes() can itself throw exceptions (e.g. if you don't have the required permissions to read the file's permissions) - this might be the cause of the AccessDeniedException you got
Not entirely wrong, but comparing the string representation of the permissions might be inappropriate for this use case, unless you want to explicitly check that the file has the given permissions - and does not have the the others; a different approach would be to check just for the required permissions:
Set<PosixFilePermission> perm = posixAttr.permissions();
if (perm.contains(OWNER_READ) || perm.contains(GROUP_READ) || perm.contains(OTHERS_READ)) {
return FileVisitResult.SKIP_SUBTREE;
}

Java nio - cannot delete directory which was emptied

I am trying to walk the file tree and delet all files/directories. The code is below:
Files.walkFileTree(metricPath, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult visitFile(Path file,
BasicFileAttributes attrs) throws IOException {
Files.delete(file);
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult postVisitDirectory(Path dir,
IOException exc) throws IOException {
if (exc == null) {
Files.delete(dir);
return FileVisitResult.CONTINUE;
} else {
throw exc;
}
}
});
}
This code is run in between unit tests, each of which is generating a separate file, in the form folder1/folder2/file. When I try to walk that tree, The DirectoryNotEmptyException is thrown when folder1 attempts to be deleted, although it is clearly empty...
Have you checked that dir for hidden files? On Windows it could be that some process have opened this directory and opened file HANDLE still exists in system HANDLE table. In that case directory is locked and java could throw that exception.
As I can see it on your code, there should not be a problem, not unless one file/folder is in read only mode. you may want to explore changing the file permission first before deleting.
can you also try to put Files.delete() method on the following override
public FileVisitResult visitFileFailed(Path file, IOException exc)
Reference:
JAVA NIO Directory Delete
Use Apache Commons FileUtils.deleteDirectory() or FileUtils.deleteQuietly()
Alternatively, you can import Apache Commons IO and use its FileUtils.deleteDirectory(File directory) method. Just one line is enough as it deletes all files and subdirectories recursively:
FileUtils.deleteDirectory(dirToBeDeleted);

Categories

Resources