Develop Reference

Packaged Jar unable to read files from mapped network drive (Windows)

I have a Spring Boot application which reads files in folders on a mapped network drive, i.e. m:/PRODUCTION
The problem is, when I execute the jar file, my debugging output shows that no files exist in the folder, even though the folder is full of a files.
I have IntelliJ installed on the same machine, and if I run the application from it's source code, it works absolutely fine.
The method I have that reads filename to an array is;
private File[] getFilesInPath(String path) {
File folder = new File(path);
File[] listOfFiles = folder.listFiles();
if (listOfFiles != null) {
Arrays.sort(listOfFiles, Comparator.comparingLong(File::lastModified));
}
return listOfFiles;
}
Then, I call this function from a number of places in the application, here's an example;
public void manuallyProcessAttritionData(List<Line> lines) {
Handler handler = new AttritionHandler().setLines(lines).setService(this);
String pathToProcess = dataFolder + attritionFolder;
log.debug("Processing path: " + pathToProcess);
File[] listOfFiles = getFilesInPath(pathToProcess);
if (listOfFiles != null) {
log.debug("Number of files to process: " + listOfFiles.length);
for (File file : listOfFiles) {
handler.processFile(file);
}
} else {
log.debug("No files to process");
}
}
The output from running the above is;
Processing ATTRITION data...
Processing path: m:/PRODUCTION
No files to process
...Finished processing ATTRITION data
I've confirmed the path is correct, running the following commands from the command line works fine and there are files in the result;
cd m:\PRODUCTION
M:\PRODUCTION>
Does anyone know of a reason why the folder can be read perfectly find from the application running in IntelliJ, but not when packaged as a JAR file?

Looks like your development account Intellij has access to this drive where as the account which started this spring-boot doesn't have access to it.
I would suggest using full path instead of a mapped network path.

There should be no difference in running the code from intelliJ or running it from the built jar. But there may be other factors comming into play. Maybe you run intelliJ with a different user than the jvm that runs the jar? I propose the following steps to resolve the issue:
1. Use the Path Class to avoid platform specific problems (eg. file separators)
Instead of
new File("path/to/my/directory");
you should use
Paths.get("path", "to", "my", "directory").toFile()
or
Paths.get("path/to/my/directory").toFile()
2. Check the file attributes
Use the code below to investigate if the directory exists, if you have the correct permissions etc.
Path directory = Paths.get("e:/TEMP");
System.out.println("Absolute Path of directory: " + directory.toAbsolutePath());
System.out.println("Directory exists: " + directory.toFile().exists());
System.out.println("Directory is a directory: " + directory.toFile().isDirectory());
System.out.println("Directory isReadable: " + directory.toFile().canRead());
System.out.println("Directory isWriteable: " + directory.toFile().canWrite());
this should output something like:
Absolute Path of directory: e:\TEMP
Directory exists: true
Directory is a directory: true
Directory isReadable: true
Directory isWriteable: true

Reading .txt File From Relative Path in Java

I know this question has been asked in a myriad of variations, but today I wish to emphasize one particular scenario when one wishes to read from a .txt file without specifying the absolute path.
Suppose we have the following set up in Eclipse.
projectName/packageName/subPackage/
and we have a class named Read.java inside the subPackage. The class will be attempting to read from the input1.txt.
We also have input1.txt inside the very same subPackage.
If one uses absolute paths, the code inside Read.java will be something of the following (let's assume now that input1.txt is placed on my Desktop for illustration purposes):
// Create a list to store the list of strings from each line of the input1.txt.
LinkedList<String> inputStrings = new LinkedList<String>();
BufferedReader bufferedTextIn = null;
try {
String line;
// Specify the file
String fileName = "C:" + File.separator
+ "Users" + File.separator
+ "Kevin" + File.separator
+ "Desktop" + File.separator
+ "input1.txt";
bufferedTextIn = new BufferedReader(new FileReader(fileName));
while ((line = bufferedTextIn.readLine()) != null) {
inputStrings.add(line);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (bufferedTextIn != null) {
bufferedTextIn.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
The problem with the above is the use of absolute paths to my desktop. If I passed the code to my friend, he would need to manually change the path to his desktop. Even if I put input1.txt in my project folder, my friend would still need to manually change the path to make it work.
Note that using File.separator is a good practice because different OS interprets separators a bit differently, but it is still insufficient.
So what do we do instead?

Here is my solution.
String fileName = Read.class.getResource("input1.txt").getPath();
System.out.println(fileName);
bufferedTextIn = new BufferedReader(new FileReader(fileName));
Let's recap the scenario. We have our input1.txt file placed in the SAME FOLDER as the Read.java. So, the code above attempts to go to where Read.class exists (which is somewhere in the bin folder in Eclipse), and look for input1.txt. This is the path RELATIVE to where Read.class is located (in this case, it is trivially in the same folder, but you could very well specify another folder relative to where Read.class is located). The print statement lets you know exactly where it is located and is a good practice while debugging.
When you build in Eclipse, the .java files in the src folder would be compiled into .class files and be placed in the bin folder. The neat thing is that input1.txt is ALSO copied over the bin folder (and all the package hierarchies are maintained).
An important thing to note is to use getPath() rather than toString(), because the latter will add some extra text to the front of the path (I only knew that because I printed it out) and thus you get a NULL pointer exception because the fileName was not formatted correctly.
Another important thing to note is that I used Read.class.getResource("input1.txt").getPath(); instead of this.getClass().getResource("input1.txt").getPath(); because the code was called in a static context (in my main method). If you create an object, then feel free to use the latter.
If you're interested in more advanced features, you can check out the link below:
What is the difference between Class.getResource() and ClassLoader.getResource()?
I hope this helps!
Edit:
You could use the following to get the directory where Read.class resides in also.
String fileName = Read.class.getResource(".").getPath();
Specifying getResource("..") would go to the parent directory.
String fileName = Read.class.getResource("..").getPath();
The above may be useful if you want to have more control specifying the path (e.g. if you want to create output.txt inside the directory where Read.class resides, use
String fileName = Read.class.getResource(".").getPath() + "output.txt";

If you knew the file was going to be located in the same folder in each system this program was run on, you could use system variables to ensure any path defined would still work with different users. For windows, I've used:
String user = new com.sun.security.auth.module.NTSystem().getName();
to get a user name. This could then be substituted in your example to be:
String fileName = "C:" + File.separator
+ "Users" + File.separator
+ user + File.separator
+ "Desktop" + File.separator
+ "input1.txt";
I'm not sure how this would work outside of Windows however.

Delete files using java

In my java application, I'm using FilenameFilter to get zip files in given directory. My directory structure is looks like below.
D\:\MyFiles\data\dir1
D\:\MyFiles\data\dir2
D\:\MyFiles\data\dir3
D\:\MyFiles\data\dir4
zip files are in dir folders. I'm giving only D\\:\\MyFiles\\data to my program and it find folders start with dir using FilenameFilter and then find files ends with zip on dir folders.
Inside a for loop I'm creating new File objects for each zip files and call delete() to delete them, but they aren't deleted.
I have printed file path using getPath() method; output is looks like below.
D\:\MyFiles\data\dir1\a.zip
D\:\MyFiles\data\dir1\b.zip
D\:\MyFiles\data\dir2\b1.zip
D\:\MyFiles\data\dir3\d.zip
Then I manually created a File object as File f = new File("D/:/MyFiles/data/dir1/a.zip") and try to delete. It succeeded.
How can I delete files? How can I give the correct path?
UPDATES
This is the code what I'm using:
// this contains folders start with 'dir' in 'D:\MyFiles\data\'
Vector<String> dirList = utl.identifyDir(conf);
File dir;
for (int i = 0; i < dirList.size(); i++) {
// in my properties file ITEM_FOLDER is written as ITEM_FOLDER=D\:\\MyFiles\\data
// LOG.fine(conf.readConfig(Configuration.ITEM_FOLDER)); returns D:\MyFiles\data
dir = new File(conf.readConfig(Configuration.ITEM_FOLDER)
+ File.separator + dirList.get(i));
// this contains all the files ends with 'zip' in 'dir' folders in 'D:\MyFiles\data\'
Vector<String> zipFiles = utl.identifyZipFiles(dir);
for (int x = 0; x < zipFiles.size(); x++) {
/* delete */
File sourcePath = new File(
conf.readConfig(Configuration.ITEM_FOLDER)
+ File.separator + dirList.get(i)
+ File.separator + zipFiles.get(x));
boolean sp = sourcePath.delete();
LOG.fine("sourcePath : " + sourcePath.getPath() + " : "
+ sp);
// one of LOG prints is D:\MyFiles\data\dir3\d.zip : false
}
}

After reading your update, I think there are 2 possible things going on here.
You've still got something open in your application. You don't happen to use a FileInputStream or anything?
Another process is keeping the .zip busy. Did you open that file? Try closing the explorer window or something like that.
EDIT: A checklist from an other user:
Check that you've got the path correct, e.g. what does file.exists() return?
Check that you've got permission to delete the file as the user running your application
Check that you haven't got an open handle to the file within your code (e.g. have you just read from it and not closed the input stream?)
Check that you don't have the file opened in a desktop app

When you create a new File-object to test, something is different then when you use getPath. Notice how all the slashes in the pathname are \ instead of /.

How to specify a directory when creating a File object?

This should be a really simple question but Google + my code isn't working out.
In Eclipse on Windows, I want my program to look inside a certain folder. The folder is directly inside the Project folder, on the same level as .settings, bin, src, etc. My folder is called surveys, and that's the one I want my File object to point at.
I don't want to specify the full path because I want this to run on both of my computers. Just the path immediately inside my Project.
I'm trying this code but it isn't working - names[] is coming back null. And yes I have some folders and test junk inside surveys.
File file = new File("/surveys");
String[] names = file.list();
for(String name : names)
{
if (new File("/surveys/" + name).isDirectory())
{
System.out.println(name);
}
}
I'm sure my mistake is within the String I'm passing to File, but I'm not sure what's wrong?

In your question you didn't specify what platform you are running on. On non-Windows, a leading slash signifies an absolute path. Best to remove the leading slash. Try this:
File file = new File("surveys");
System.out.println("user.dir=" + System.getProperty("user.dir"));
System.out.println("file is at: " + file.getCanonicalPath());
String[] names = file.list();
for(String name : names)
{
if (new File(file, name).isDirectory())
{
System.out.println(name);
}
}
Make sure the in your run configuration, the program is running from the projects directory (user.dir = <projects>)

Make sure that your file is a directory before using file.list() on it, otherwise you will get a nasty NullPointerException.
File file = new File("surveys");
if (file.isDirectory()){
...
}
OR
if (names!=null){
...
}

If you checked the full path of your file with
System.out.println(file.getCanonicalPath())
the picture would immediately become clear. File.getCanonicalPath gives you exactly the full path. Note that File normalizes the path, eg on Windows "c:/file" is converted to "C:\file".

Java File.renameTo(File) not working

I'm trying to list a directory's contents, and rename certain files.
public void run(String dirName) {
try {
File parDir = new File(dirName);
File[] dirContents = parDir.listFiles();
// Rename if necessary
for(File f : dirContents) {
System.out.println("f is:\n" + f.toString());
String name = f.getName();
String subbedName = name.replaceAll("\uFFFD", "_");
System.out.println("\n" + "name = " + name + ", subbedName = " + subbedName + "\n");
if(!name.equals(subbedName)) {
File newFile = new File(f.getParentFile(), subbedName);
System.out.println("newFile is:\n" + newFile.toString());
if(!f.renameTo(newFile))
System.out.println("Tried to change file name but couldn't.");
}
}
}
catch(Exception exc1) {
System.out.println("Something happened while listing and renaming directory contents: " + exc1.getMessage());
}
}
When I run this, I get "Tried to change file name but couldn't." I don't believe that Java is considering these files to be "open", so I don't think that's the reason. I've even ran chmod 777 myDir where myDir is the value of the dirName string passed into the run method.
What am I missing here? Why won't Java rename these file(s)? These are CentOS machines.
Edit: Added printouts for both f and newFile, which is as follows:
f is:
/root/path/to/mydir/test�.txt
newFile is:
/root/path/to/mydir/test_.txt

You need to create your new File object with the full pathname of those files. So
String name = f.getName(); // gets the name without the directory
should likely be:
String name = f.getAbsolutePath();
(your search/replace may need to change)

The problem is that f.getName() returns the last name component of the path that is represented by f. You then massage this String and turn it back into a File. But the File now represents a path relative to the current directory, not the directory containing the original path.
As a result your code is actually attempting to rename the files from dirName into the application's current directory. That could fail because files already exist in the current directory with those names, or because the dirName and the current directory are in different file systems. (You cannot rename a file from one filesystem to another ... you have to copy it.)
Please note that a File in Java represents a pathname, not a file or a folder. In your code, the f objects are the pathnames for file system objects (either files or folders) in the directory denoted by the String dirname. Each of these f objects will have a directory part.
There is more than one way to fix your code; for example
change name = f.getName() to name = f.toString()
change new File(subbedName) to new File(f.getParentFile(), subbedName)
I have an alternative / additional theory.
The pathname of the file containing the \uFFFD character is coming out as "mojibake"; i.e. the kind of garbled text that you get when you display encoded text using the wrong encoding. And since we are seeing 3 characters of garbled text, I suspect that it is attempting to display the UTF-8 rendering of \uFFFD as Latin-1.
So my theory is that the same think is happening when the File.renameTo method is converting f to the form that it is going to provide to the system call. For some reason that is no clear to me, Java could be using the wrong encoding, and as a result producing a "name" for the original file that doesn't match the name of the file in the file system. That would be sufficient to cause the rename to fail.
Possibly related questions / links:
File name charset problem in java
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4733494 (Note that Sun decided this was not a Java bug, and most of the "me too" comments on the bug report are from people who do not understand the explanation ...)

f.getName(); only returns the name of the folder, not the full path. So subbedName becomes a relative path file. Try something with f.getCanonicalPath() instead.