Develop Reference

How to copy specific files from one directory to another in Java

I have a directory with a bunch of files that i need to copy to another directory , Using Java i would only like to copy the files that end with a ".txt" extension. I'm familiar with doing it for one file as shown below , please could you help me do this as a loop to see which files in the source directory match with a "txt" extension and then copy them all over to a new directory.
File sourceFileLocation = new File(
"C:\\Users\\mike\\data\\assets.txt");
File newFileLocation = new File(
"C:\\Users\\mike\\destination\\newFile.txt");
try {
Files.copy(sourceFileLocation.toPath(), newFileLocation.toPath());
} catch (Exception e) {
e.printStackTrace();
}

You can use Files#list(Path) to obtain a stream and use stream operations to filter and collect only those file names that contain the extension txt. For example:
List<Path> paths = Files.list(Paths.get("C:/Users/hecto/Documents")).filter(path -> path.toString().endsWith(".txt")).collect(Collectors.toList());
for (Path path : paths) {
System.out.println(path.toString());
}
For me, this prints out:
C:\Users\hecto\Documents\file1.txt
C:\Users\hecto\Documents\file2.txt
C:\Users\hecto\Documents\file3.txt
Even though I have other files and folders in that directory
Using that, I came up with this solution to copy over those filtered files from their current location to a new destination and preserving the original names (Using Java 8 or above):
try (Stream<Path> stream = Files.list(Paths.get("C:/Users/hecto/Documents"))) {
List<Path> paths = stream.filter(path -> path.toString().endsWith(".txt")).collect(Collectors.toList());
for (Path source : paths) {
Path destination = Paths.get("C:/Users/hecto/Desktop/target" + File.separator + source.getFileName());
Files.copy(source, destination, StandardCopyOption.REPLACE_EXISTING);
}
}
(Answer updated to use try-with-resources to close the stream)

Java error when adding and creating zip files: The process cannot access the file

I made a Java program using Java ZipFileSystem (Java 8) to archive files into zip files. I thought it'd be simple and straightforward, but after running for a while, it craps out with "java.nio.file.FileSystemException". Code snippet below:
......
fileName = fileName + "_";
int zipFileNameMidPart;
try (DirectoryStream<Path> dirStream = Files.newDirectoryStream(sourceDir, getFilter(sourceDir)))
{
for(Path path : dirStream)
{
lastModified.setTimeInMillis(Files.getLastModifiedTime(path, LinkOption.NOFOLLOW_LINKS).toMillis());
zipFileNameMidPart = fileName.equalsIgnoreCase("salesfaxes") ? lastModified.get(Calendar.MONTH)+1 : (lastModified.get(Calendar.MONTH)/3)+1;
zipFileName = fileName+lastModified.get(Calendar.YEAR)+"_"+zipFileNameMidPart+".zip";
zipFile = Paths.get(srcDirTmp, zipFileName);
addToZipArchive(path, zipFile, !Files.exists(zipFile), tmpMoveDir);
//Move archived file to the temp directory.
currentBackupFile = tmpMoveDir.resolve(path.getFileName());
logger.info("Moving "+path+" to "+currentBackupFile);
Files.move(path, currentBackupFile, StandardCopyOption.REPLACE_EXISTING);
}
}//try
protected void addToZipArchive(Path fileToBeArchived, Path zipArchive, boolean create, Path tmpBackupDir) throws Exception
{
/* Define ZIP File System Properies in HashMap */
Map<String, String> zip_properties = new HashMap<>();
/* set create to true if you want to create a new ZIP file */
zip_properties.put("create", Boolean.toString(create));
/* specify encoding to UTF-8 */
zip_properties.put("encoding", "UTF-8");
/* Locate File on disk for creation */
URI zip_disk = URI.create("jar:file:/" + zipArchive.toString().replace("\\", "/"));
totalCnt++;
/* Create ZIP file System */
try (FileSystem zipfs = FileSystems.newFileSystem(zip_disk, zip_properties))
{
/* Path inside ZIP File */
Path pathInZipfile = zipfs.getPath(fileToBeArchived.getFileName().toString());
boolean copyOk = true;
try
{
logger.info("Adding "+fileToBeArchived.toString()+" to "+(create?"new ":"existing ")+"zip file "+zipArchive.toString());
/* Add file to archive */
Files.copy(fileToBeArchived, pathInZipfile, StandardCopyOption.COPY_ATTRIBUTES, StandardCopyOption.REPLACE_EXISTING);
}
catch(Exception e)
{
copyOk = false;
logger.log(Level.SEVERE, e.getMessage(), e);
}
if(copyOk)
{
if(create)
createCnt++;
else
addCnt++;
}
else
{
logger.severe("Skip file "+fileToBeArchived+"! Error encounted while archiving it.");
skippedCnt++;
}
}
}
Stacktrace:
SEVERE: Error occured!
java.nio.file.FileSystemException: C:\temp\bols\bols_2017_4.zip: The process cannot access the file because it is being used by another process.
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at com.sun.nio.zipfs.ZipFileSystem.sync(ZipFileSystem.java:1301)
at com.sun.nio.zipfs.ZipFileSystem.close(ZipFileSystem.java:277)
at ZipArchiveUtil.addToZipArchive(ZipArchiveUtil.java:177)
at ZipArchiveUtil.processDirs(ZipArchiveUtil.java:282)
at ZipArchiveUtil.main(ZipArchiveUtil.java:392)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:61)
The directories are quite large containing tens of thousands tif files. And zip files can also grow very large (exceed gigabyte in file sizes).
I've tried to run the program at night and weekend, still got same error after running for a while.
Also replicated the env on a different pc, same result.
Seems like the program is still holding on the zip file while trying to add new file into it... What am I missing? Did some googling, haven't found much info.
Another weird thing, from time to time the program "pauses", doing nothing. If I do CTRL Z, then it starts processing again. I checked in Task Manager, no CPU and disk usages while it's on "pause".
Any helps/suggestions are greatly appreciated.

File.createNewFile() failing in java (Ubuntu 12.04)

I am trying to createNewFile() in java.I have written down the following example.I have compiled it but am getting a run time error.
import java.io.File;
import java.io.IOException;
public class CreateFileExample
{
public static void main(String [] args)
{
try
{
File file = new File("home/karthik/newfile.txt");
if(file.createNewFile())
{
System.out.println("created new fle");
}else
{
System.out.println("could not create a new file");
}
}catch(IOException e )
{
e.printStackTrace();
}
}
}
It is compiling OK.The run time error that I am getting is
java.io.IOException: No such file or directory
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:947)
at CreateFileExample.main(CreateFileExample.java:16)

some points here
1- as Victor said you are missing the leading slash
2- if your file is created, then every time you invoke this method "File.createNewFile()" will return false
3- your class is very platform dependent (one of the main reasons why Java is powerful programming language is that it is a NON-PLATFORM dependent), instead you can detect a relative location throw using the System.getProperties() :
// get System properties :
java.util.Properties properties = System.getProperties();
// to print all the keys in the properties map <for testing>
properties.list(System.out);
// get Operating System home directory
String home = properties.get("user.home").toString();
// get Operating System separator
String separator = properties.get("file.separator").toString();
// your directory name
String directoryName = "karthik";
// your file name
String fileName = "newfile.txt";
// create your directory Object (wont harm if it is already there ...
// just an additional object on the heap that will cost you some bytes
File dir = new File(home+separator+directoryName);
// create a new directory, will do nothing if directory exists
dir.mkdir();
// create your file Object
File file = new File(dir,fileName);
// the rest of your code
try {
if (file.createNewFile()) {
System.out.println("created new fle");
} else {
System.out.println("could not create a new file");
}
} catch (IOException e) {
e.printStackTrace();
}
this way you will create your file in any home directory on any platform, this worked for my windows operating system, and is expected to work for your Linux or Ubuntu as well

You're missing the leading slash in the file path.
Try this:
File file = new File("/home/karthik/newfile.txt");
That should work!

Actually this error comes when there is no directory "karthik" as in above example and createNewFile() is only to create file not for directory use mkdir() for directory and then createNewFile() for file.

Add file to a folder that is inside a zip file java

I am trying to add a txt file into a folder which is inside a zip file.
First, I was extracting all the contents of zip file then adding the txt file and then zipping back.
Then I read about the nio method which I can modify the zip without extracting it. Using this method I can add the txt file to the main folder of zip but I can't go deeper.
testing.zip file has res folder in it.
Here is my code:
Path txtFilePath = Paths.get("\\test\\prefs.txt");
Path zipFilePath = Paths.get("\\test\\testing.zip");
FileSystem fs;
try {
fs = FileSystems.newFileSystem(zipFilePath, null);
Path fileInsideZipPath = fs.getPath("res/prefs.txt"); //when I remover "res/" code works.
Files.copy(txtFilePath, fileInsideZipPath);
fs.close();
} catch (IOException e) {
e.printStackTrace();
}
I get the following exception:
java.nio.file.NoSuchFileException: res/

(edit to give the actual answer)
Do:
fs.getPath("res").resolve("prefs.txt")
instead of:
fs.getPath("res/prefs.txt")
The .resolve() method will do the correct thing with regards to file separators etc.

The fs.getPath("res/prefs.txt") should certainly work and you don't need to split it to fs.getPath("res").resolve("prefs.txt") as the approved answer says.
The exception java.nio.file.NoSuchFileException: res/ is slightly confusing because it mentions file but in fact directory is missing.
I had a similar problem and all I had to do was:
if (fileInsideZipPath.getParent() != null)
Files.createDirectories(fileInsideZipPath.getParent());
See full example:
#Test
public void testAddFileToArchive() throws Exception {
Path fileToAdd1 = rootTestFolder.resolve("notes1.txt");
addFileToArchive(archiveFile, "notes1.txt", fileToAdd1);
Path fileToAdd2 = rootTestFolder.resolve("notes2.txt");
addFileToArchive(archiveFile, "foo/bar/notes2.txt", fileToAdd2);
. . .
}
public void addFileToArchive(Path archiveFile, String pathInArchive, Path srcFile) throws Exception {
FileSystem fs = FileSystems.newFileSystem(archiveFile, null);
Path fileInsideZipPath = fs.getPath(pathInArchive);
if (fileInsideZipPath.getParent() != null) Files.createDirectories(fileInsideZipPath.getParent());
Files.copy(srcFile, fileInsideZipPath, StandardCopyOption.REPLACE_EXISTING);
fs.close();
}
If I remove Files.createDirectories() bit, and ensure clear start with clear test directory, I get:
java.nio.file.NoSuchFileException: foo/bar/
at com.sun.nio.zipfs.ZipFileSystem.checkParents(ZipFileSystem.java:863)
at com.sun.nio.zipfs.ZipFileSystem.newOutputStream(ZipFileSystem.java:528)
at com.sun.nio.zipfs.ZipPath.newOutputStream(ZipPath.java:792)
at com.sun.nio.zipfs.ZipFileSystemProvider.newOutputStream(ZipFileSystemProvider.java:285)
at java.nio.file.Files.newOutputStream(Files.java:216)
at java.nio.file.Files.copy(Files.java:3016)
at java.nio.file.CopyMoveHelper.copyToForeignTarget(CopyMoveHelper.java:126)
at java.nio.file.Files.copy(Files.java:1277)
at my.home.test.zipfs.TestBasicOperations.addFileToArchive(TestBasicOperations.java:111)
at my.home.test.zipfs.TestBasicOperations.testAddFileToArchive(TestBasicOperations.java:51)

How can I access a folder inside of a resource folder from inside my jar File?

I have a resources folder/package in the root of my project, I "don't" want to load a certain File. If I wanted to load a certain File, I would use class.getResourceAsStream and I would be fine!! What I actually want to do is to load a "Folder" within the resources folder, loop on the Files inside that Folder and get a Stream to each file and read in the content... Assume that the File names are not determined before runtime... What should I do? Is there a way to get a list of the files inside a Folder in your jar File?
Notice that the Jar file with the resources is the same jar file from which the code is being run...

Finally, I found the solution:
final String path = "sample/folder";
final File jarFile = new File(getClass().getProtectionDomain().getCodeSource().getLocation().getPath());
if(jarFile.isFile()) { // Run with JAR file
final JarFile jar = new JarFile(jarFile);
final Enumeration<JarEntry> entries = jar.entries(); //gives ALL entries in jar
while(entries.hasMoreElements()) {
final String name = entries.nextElement().getName();
if (name.startsWith(path + "/")) { //filter according to the path
System.out.println(name);
}
}
jar.close();
} else { // Run with IDE
final URL url = Launcher.class.getResource("/" + path);
if (url != null) {
try {
final File apps = new File(url.toURI());
for (File app : apps.listFiles()) {
System.out.println(app);
}
} catch (URISyntaxException ex) {
// never happens
}
}
}
The second block just work when you run the application on IDE (not with jar file), You can remove it if you don't like that.

Try the following.
Make the resource path "<PathRelativeToThisClassFile>/<ResourceDirectory>" E.g. if your class path is com.abc.package.MyClass and your resoure files are within src/com/abc/package/resources/:
URL url = MyClass.class.getResource("resources/");
if (url == null) {
// error - missing folder
} else {
File dir = new File(url.toURI());
for (File nextFile : dir.listFiles()) {
// Do something with nextFile
}
}
You can also use
URL url = MyClass.class.getResource("/com/abc/package/resources/");

The following code returns the wanted "folder" as Path regardless of if it is inside a jar or not.
private Path getFolderPath() throws URISyntaxException, IOException {
URI uri = getClass().getClassLoader().getResource("folder").toURI();
if ("jar".equals(uri.getScheme())) {
FileSystem fileSystem = FileSystems.newFileSystem(uri, Collections.emptyMap(), null);
return fileSystem.getPath("path/to/folder/inside/jar");
} else {
return Paths.get(uri);
}
}
Requires java 7+.

I know this is many years ago . But just for other people come across this topic.
What you could do is to use getResourceAsStream() method with the directory path, and the input Stream will have all the files name from that dir. After that you can concat the dir path with each file name and call getResourceAsStream for each file in a loop.

I had the same problem at hands while i was attempting to load some hadoop configurations from resources packed in the jar... on both the IDE and on jar (release version).
I found java.nio.file.DirectoryStream to work the best to iterate over directory contents over both local filesystem and jar.
String fooFolder = "/foo/folder";
....
ClassLoader classLoader = foofClass.class.getClassLoader();
try {
uri = classLoader.getResource(fooFolder).toURI();
} catch (URISyntaxException e) {
throw new FooException(e.getMessage());
} catch (NullPointerException e){
throw new FooException(e.getMessage());
}
if(uri == null){
throw new FooException("something is wrong directory or files missing");
}
/** i want to know if i am inside the jar or working on the IDE*/
if(uri.getScheme().contains("jar")){
/** jar case */
try{
URL jar = FooClass.class.getProtectionDomain().getCodeSource().getLocation();
//jar.toString() begins with file:
//i want to trim it out...
Path jarFile = Paths.get(jar.toString().substring("file:".length()));
FileSystem fs = FileSystems.newFileSystem(jarFile, null);
DirectoryStream<Path> directoryStream = Files.newDirectoryStream(fs.getPath(fooFolder));
for(Path p: directoryStream){
InputStream is = FooClass.class.getResourceAsStream(p.toString()) ;
performFooOverInputStream(is);
/** your logic here **/
}
}catch(IOException e) {
throw new FooException(e.getMessage());
}
}
else{
/** IDE case */
Path path = Paths.get(uri);
try {
DirectoryStream<Path> directoryStream = Files.newDirectoryStream(path);
for(Path p : directoryStream){
InputStream is = new FileInputStream(p.toFile());
performFooOverInputStream(is);
}
} catch (IOException _e) {
throw new FooException(_e.getMessage());
}
}

Another solution, you can do it using ResourceLoader like this:
import org.springframework.core.io.Resource;
import org.apache.commons.io.FileUtils;
#Autowire
private ResourceLoader resourceLoader;
...
Resource resource = resourceLoader.getResource("classpath:/path/to/you/dir");
File file = resource.getFile();
Iterator<File> fi = FileUtils.iterateFiles(file, null, true);
while(fi.hasNext()) {
load(fi.next())
}

If you are using Spring you can use org.springframework.core.io.support.PathMatchingResourcePatternResolver and deal with Resource objects rather than files. This works when running inside and outside of a Jar file.
PathMatchingResourcePatternResolver r = new PathMatchingResourcePatternResolver();
Resource[] resources = r.getResources("/myfolder/*");
Then you can access the data using getInputStream and the filename from getFilename.
Note that it will still fail if you try to use the getFile while running from a Jar.

As the other answers point out, once the resources are inside a jar file, things get really ugly. In our case, this solution:
https://stackoverflow.com/a/13227570/516188
works very well in the tests (since when the tests are run the code is not packed in a jar file), but doesn't work when the app actually runs normally. So what I've done is... I hardcode the list of the files in the app, but I have a test which reads the actual list from disk (can do it since that works in tests) and fails if the actual list doesn't match with the list the app returns.
That way I have simple code in my app (no tricks), and I'm sure I didn't forget to add a new entry in the list thanks to the test.

Below code gets .yaml files from a custom resource directory.
ClassLoader classLoader = this.getClass().getClassLoader();
URI uri = classLoader.getResource(directoryPath).toURI();
if("jar".equalsIgnoreCase(uri.getScheme())){
Pattern pattern = Pattern.compile("^.+" +"/classes/" + directoryPath + "/.+.yaml$");
log.debug("pattern {} ", pattern.pattern());
ApplicationHome home = new ApplicationHome(SomeApplication.class);
JarFile file = new JarFile(home.getSource());
Enumeration<JarEntry> jarEntries = file.entries() ;
while(jarEntries.hasMoreElements()){
JarEntry entry = jarEntries.nextElement();
Matcher matcher = pattern.matcher(entry.getName());
if(matcher.find()){
InputStream in =
file.getInputStream(entry);
//work on the stream
}
}
}else{
//When Spring boot application executed through Non-Jar strategy like through IDE or as a War.
String path = uri.getPath();
File[] files = new File(path).listFiles();
for(File file: files){
if(file != null){
try {
InputStream is = new FileInputStream(file);
//work on stream
} catch (Exception e) {
log.error("Exception while parsing file yaml file {} : {} " , file.getAbsolutePath(), e.getMessage());
}
}else{
log.warn("File Object is null while parsing yaml file");
}
}
}

Took me 2-3 days to get this working, in order to have the same url that work for both Jar or in local, the url (or path) needs to be a relative path from the repository root.
..meaning, the location of your file or folder from your src folder.
could be "/main/resources/your-folder/" or "/client/notes/somefile.md"
Whatever it is, in order for your JAR file to find it, the url must be a relative path from the repository root.
it must be "src/main/resources/your-folder/" or "src/client/notes/somefile.md"
Now you get the drill, and luckily for Intellij Idea users, you can get the correct path with a right-click on the folder or file -> copy Path/Reference.. -> Path From Repository Root (this is it)
Last, paste it and do your thing.

Simple ... use OSGi. In OSGi you can iterate over your Bundle's entries with findEntries and findPaths.

Inside my jar file I had a folder called Upload, this folder had three other text files inside it and I needed to have an exactly the same folder and files outside of the jar file, I used the code below:
URL inputUrl = getClass().getResource("/upload/blabla1.txt");
File dest1 = new File("upload/blabla1.txt");
FileUtils.copyURLToFile(inputUrl, dest1);
URL inputUrl2 = getClass().getResource("/upload/blabla2.txt");
File dest2 = new File("upload/blabla2.txt");
FileUtils.copyURLToFile(inputUrl2, dest2);
URL inputUrl3 = getClass().getResource("/upload/blabla3.txt");
File dest3 = new File("upload/Bblabla3.txt");
FileUtils.copyURLToFile(inputUrl3, dest3);