Develop Reference

Java read a folder from project base directory

I have a maven project with typical project structure. At the base of the project, I have a directory names res that has some resources (> 500 MB).
I am trying to use
this.class().getClassLoader().getResourceAsStream("res/xxx")
this code fragment to read from that folder, but it returns a null resource stream.
I have a few workarounds below, but none of these are acceptable due to reasons explained below.
I can move the folder to {base}/target/classes, and it will be read, but it will also get cleaned when I do a mvn clean. Hence, this approach doesn't work. For some reason, specifying the path as ../../res/xxx also doesn't work.
I can move the folder to {base}/src/resources, but then it will get copied to target/classes and the jar. Hence this is also not acceptable.
Though I am open to trying some other java APIs, I may have to use the class loader mechanism only as there is some external library component that is also trying to access the res folder using the similar approach.
Is there some way I can read the res folder from projects base directory? Is there some setting in pom.xml file that can help me with that?

Use this.class().getClassLoader().getResourceAsStream("/res/xxx") and then you will be reading from the root of the classpath irrespective of the actual file/folder location in windows or Linux. This is actually one of the safest ways to read files especially when you do not know how your application will eventually be deployed.
It means though that your classpath must include the parent of res and that res/ must be copied over when you deploy your app
Otherwise, if you want to use relative paths, try this snippet (taken from this SO answer):
String filePath = new File("").getAbsolutePath();
filePath.concat("path to the property file");

If you use this.class().getClassLoader().getResourceAsStream("/res/xxx"), it is going to try to load resources from the classpath. If that's not want you want to do, you're going to need to specify an absolute path.
Resource on Classpath
If you don't want the resource built into your JAR, I'd suggest having a different maven project with the file in it, under the src/main/resources directory. This will create a jar file with the file in it, which will be ~500MB.
After this, you can include a dependency on this project in your project containing your application code. This will then be able to reference it from the jar file using getResourceAsStream(...).
If you don't want this large jar file to ship with your application, make sure you mark the dependency with <scope>provided</scope>.
File from Absolute Path
You will need to take the file location as a parameter in your main method, and then use new File("C:\\path\\to\\your\\file.txt") and then use a FileReader to read it in.

Java resources - Use file far off in directory tree

I have a newbie Java question.
I had to make suite of J/DBUnit tests for some stored procedures we use in SQL Server. These tests use some XML files in a couple of sub-directories that I originally had placed in the same directory as my Java project.
Anyway, upon checking these tests in, our SVN manager wanted to keep the .java files in one part of the tree, and resources (like the XML files and required JARs) in another part of the tree.
So, my tests had originally referenced the XML files with a relative path which doesn't work now.
My question is:
Can I make the directories with my XML files available with the CLASSPATH (I hope so).
Assuming that works, how do I reference a file in my code that was included this way?
If I shouldn't be using the CLASSPATH for this, I'm open to other solutions.

Forget calsspath. Provide your tests with a parameter/configuration which defines the root dir for the relative paths of the XML files.

Using the classpath is no problem, the standard maven project layout looks like the following:
src
main
java
resources
test
java
resources
target
classes
test-classes
The compiler compiles src/main/java to target/classes, the resources of src/main/resources are copied to the target/classes folder, similar for the tests. If the tests have a classpath containing classes and test-classes, all works fine.
How is your project layout is, how is it build?

No, you should not use CLASSPATH in this instance since it is used by Java. However, you can use a similar approach by loading a value from an environment variable or configuration file which indicates the directory where the XML files are stored.

You can do this without making any changes to your classpath. The idea is to store the resource files in a separate directory, but have them copied to a directory in your classpath when you run your build process.
Here is an example configuration:
source Directory is ${basedir}/src/main/java
resource directory is ${basedir}/src/main/resources
In your build script, copy both the .java files and the resource files (.xml) to a directory in your classpath, say:
${basedir}/target/classes
Your test code runs against the target dir. The target directory is not checked in to SVN, keeping your SVN admin happy, and you don't have to make changes to your code.

How to access Jar file located within source folders?

in my Java project I am using an H2 in-memory database, for which I have to load the JDBC driver when I initialize my application. I want/need to load the H2 .jar file dynamically, so I do the following:
String classname = "org.h2.Driver";
URL u = new URL("jar:file:libs/h2.jar!/");
URLClassLoader ucl = new URLClassLoader(new URL[] { u });
Driver d = (Driver) Class.forName(classname, true, ucl).newInstance();
DriverManager.registerDriver(new DriverShim(d));
When I put the H2 .jar file into a "libs" folder outside my Java source code folder (that is, in Eclipse, this "libs" directory is on the same level as the "src" folder), then this approach works fine. However, unfortunately I have to put this H2 .jar file into a folder within the source code folder tree, but below the main class folder.
For example, my Java package structure looks like this in Eclipse:
<project>/src/my/app/MyApp.java // main class of my application
<project>/src/my/app/sub/package/h2.jar // how to access this?
<project>/libs/h2.jar // loading from here works
I know this is stupid, but unfortunately I have to work with this strange setup. But what I don't know: how can I edit my Java code (listed above) in order to work with this setup?
EDIT: This has to work outside Eclipse as well, so adding the JAR file to the Java Build Path in Eclipse is no option for me.
EDIT2: I already tried to load "jar:file:my/app/sub/package/h2.jar!/", but that did not work for me.
Thanks in advance for all helpful ideas!
Kind regards, Matthias

In some frameworks referring to files inside JARs can be done using the classpath: prefix. I doubt URLClassLoader supports it natively, but it's worth a try (e.g. classpath:/my/app/sub/package/h2.jar). But since that doesn't work with URLClassLoader, here are other ways:
One way to do it would be to write your own ClassLoader which reads the JAR file from classpath (using getResourceAsStream), uncompresses it (using ZipInputStream) to memory (e.g. a map of byte arrays) and loads the classes from there.
Another, slightly easier way, is to read the JAR file from classpath and write it into a temporary file. Then you can use the plain URLClassLoader to load classes from it. This has the disadvantage that the file must be written to a file and the file probably cannot be removed until the JVM exits (unless using Java 7 or higher).
I'm using the second approach (copying to a temp file) in one project, though I'm using it to launch an external process. I would be curious to hear why you have such a requirement. If it's just a matter of having the whole application in one JAR, there are numerous simpler methods for achieving that (Maven Assembly Plugin, Maven Shade Plugin, Jar Jar Links, One-JAR to name a few).
No it's not a homework, but an online build system that uses my classes under my/app/* and several other classes (not from me) to automatically build the whole solution. Anyway, I can't give you more details on the internals of this system, as I don't know them. As said, I simply have to live with it, and that is why I am asking here...
Sounds like you are working in a WTF environment (does it have a name?), so here are some ways to start hacking around it:
Find out more about your environment, especially absolute file paths of the following: directory where the source files are saved, directory where the generated .class files are saved, and the current working directory when the program is run.
If you can get any kind of output of what your program prints during runtime, you can put into your application some debug code where you use File.listFiles() to crawl the machine's directory trees. If you can get output only from what happens when compiling, it might be possible to execute your own code during compile by creating your own annotation processor (apt is part of javac since Java 6), though I'm not sure whether the annotation processor must be compiled first separately.
The working directory can be read from the user.dir system property and the location of class files can be probably gotten from the java.class.path system property (unless custom class loaders are used). There is no guarantee that a JAR file in the source directory would be copied to the classpath, so you might need to do some looking around.
Then when you know the file path of the JAR file, then you can get an URL to it using new File("path/to/h2.jar").toURI().toURL() which you can then pass to URLClassLoader.
If nothing else works, upload the source code of the libraries and compile them together with your project.
In the long run, try to replace the WTF build environment with one that uses a standard build tool (such as Maven) and a common CI server (such as Jenkins). It's normal for projects to have lots of library dependencies, so you shouldn't need to hack around a build environment to use them.

Load all files from directory using .getClassLoader().getResource()

In one of my JUnit tests, I am trying to load all the files contained in a directory. I used .getClassLoader().getResource("otherresources") to find the directory. I then made a new java.io.File. I then used listFiles() to get all the children files and then used .getClassLoader().getResource() again to load each of those files.
URL url = FileLoadTest.class.getClassLoader().getResource("otherresources");
File directory = new File(url.getPath());
File[] files = directory.listFiles();
Basically, I want to be able to load all the files in a directory without knowing exactly what they are.
I can properly run the test in Eclipse. When I go to build the project with Maven (mvn install) or run the test case by itself using surefire (mvn -Dtest=FileTest test) the test case fails with a NullPointerException. I think the issue has something to do with the File api not working as intended within the JAR file that the resources are deployed to.
Any tips on how to fix this?

Correct, the File API can only read from the file system, and not within JAR files.
Unfortunately, there's not a good way to do exactly what you're trying to do, without using some additional libraries that utilize some hacks in an attempt to accomplish just this. Resources on the classpath are not meant to be enumerable, as there is no guarantee where they are loaded from (could be from disk, in a JAR file, HTTP, a database, or other exotic resources - including ones where the enumeration of available files would not be feasible). The best approach is to include an "index" or other similar file with a well-known name, which can be referenced to find other resources you're interested in.
One of these "hacks" would be, if you know the path to the JAR file(s), you could read from them using JarFile (or even just ZipFile).

.getClassLoader().getResource("otherresources")
Untested. Once you have the 'directory', use it to get back to the archive itself.
Establish a ZipInputStream to the archive.
Call getNextEntry() until null and add the entries matching the required location to an expandable list (e.g. ArrayList).
Construct an URL from the location of the archive and ZipEntry.getName()
Of course, I would normally suggest creating a list of the target resources when making the archive, then including that list at a known location in the archive. But the above might suffice for this use case.

How do you get file path for file when user directory changes?

I have a java app, and the log4j.properties file is in src/com/my/path/props. On compile, it's copied into classes/com/my/path/props
The file is loaded via PropertyConfigurator.configureAndWatch(user.dir + "/classes/com/my/path/props/log4j.properties").
This all works fine normally, though it's not ideal because of using user.dir (but I do not know another way to reference a file relative to the "application's start directory"). The problem manifests when trying to run this application using an NT Service wrapper. When done this way, the user.dir changes from the application's root dir to wherever the NTService wrapper's exe file is.
My question is: What's the appropriate way to get a the String file path representation of the log4j.properties file in my classes/com/my/path/props/ directory? I realize this would completely break down if the props file were in a jar; but in this case, it's not and is simply a file on the file system.
I've tried new File(this.getClass().getClassLoader().getResource("com/my/path/props/log4j.properties").getURI()).getAbsolutePath(), but that fails because on production, the path to the file is actually a UNC path and consequently throws a "URI has an authority component" exception.
How do other people deal with this problem?
Thanks.

OK. So... you asked how other people deal with this problem. First, they do not leave it up to Eclipse for where files get placed. They choose where they want them, how they want to access them, and then have their build tool (which unless they are just playing around, should not be an IDE like Eclipse, but rather a dedicated build tool like Maven or Ant) where to place it.
The choice of where you want the file depends on what you want to do with it. If its simply a config file that will never be edited at runtime, you typically place it inside your JAR (which is another practice - applications are placed in one or more JARs, WARs, or EARs, not a classes directory). If the file is to be edited at runtime, which from your "watching" it appears to be the case, you typically put it in a config directory outside your JAR.
How you access it (from the filepath or the classpath) is another choice. Where possible, I favor accessing files from the classpath because it is more portable - and when in a JAR, pretty much required. If that doesn't make sense in your case, then choose a path other than "user.dir" if that is changing when you deploy. You can hard-code it, use an environment variable, a property, a config file, a command line argument, etc. to set the actual path.
Always choose where things go and how you access them. Don't let your tools choose for you. It will make your life easier :-)

I took singleshot's advice and kept the properties files out of src and instead in a separate directory which I added to the classpath. In retrospect, this was indeed boneheaded to have configured it the way I did originally.
From there, my problem was getting a File from a URL. I ended up finding what I needed in Commons IO FileUtils, with its toFile(URL) method.
The code ended up looking like this:
private URL maintenanceConfigPath = this.getClass().getClassLoader().getResource("MaintenanceConfig.properties");
....
File f = FileUtils.toFile(maintenanceConfigPath);
....
Again, thanks to all for your feedback and for pointing me down a path that got me towards an answer