Develop Reference

"jar:file" vs. only "file:" reference when retrieving resources in Java

This question is related to my question
Jetty 11.0.11 - 404 on html file in \src\main\webapp\static - maven embedded fat jar
What --EXACTLY-- does "jar:file" mean as a Java resource reference, vs. just "file:"?
And how is that influenced by the operating system ran under?
E. g. using this resource reference in Jetty webserver, in Windows with Oracle JDK 17, files are found as resources and parsed by Jetty webserver:
file:///D:/Projects/verdi_2/target/classes/static/,AVAILABLE}{file:/D:/Projects/verdi_2/target/classes/static}
Using this resource reference in Jetty webserver, in Ubuntu Linux 20.04 LTS with Oracle JDK 17, NO files are found and nothing can be parsed by Jetty webserver:
jar:file:/usr/src/verdi/verdi-12-JDK17-jar-with-dependencies.jar!/static
Is there a difference in how a Linux version of JDK interprets "jar:file" vs. how a Windows version of the JDK interprets "jar:file"?
EDIT: The related issue is the Jetty webserver apparently can no longer serve resources directly out of a JAR file it is itself embedded in. This is now a GitHub bug ticket at https://github.com/eclipse/jetty.project/issues/8549

file: is the beginning of a general file url. jar:file: is that for a jar file particularly, with a view to referring (usually) to a particular entry in a jar. Here's an example you can run (obviously with your own jar url) where you can save an entry as a file (given by the parameter to the app)
import java.nio.file.Paths;
import java.nio.file.Files;
import java.net.URL;
public class JarUrl {
public static void main(String[] args) {
try {
URL url = new URL("jar:file:root.jar!/root/a/b.txt");
Files.copy(url.openStream(), Paths.get(args[0]));
}
catch(Throwable t) {
t.printStackTrace();
}
}
}

What --EXACTLY-- does "jar:file" mean as a Java resource reference, vs. just "file:"?
You're mischaracterising the URL a little bit. The string until the first : decides the 'scheme' of a URL, so, the pertinent question is: How does jar: work. The file: part is a smaller aspect of a sub-part of the jar bit.
How does jar: work
The format is jar:(URL-of-jar)!(path-inside-jar)
Where URL-of-jar is itself a valid URL (and file: is just one way to do that. So is http, for that matter), and path-inside-jar is not a URL but a path.
The meaning is: First, resolve the 'URL-of-jar' URL. This gets you a jar file. Then, open the jar file, and retrieve the resource at the stated path.
So, to pull this one apart:
jar:file:/usr/src/verdi/verdi-12-JDK17-jar-with-dependencies.jar!/static
The jar is located at URL file:/usr/src/verdi/verdi-12-JDK17-jar-with-dependencies.jar and the resource it is referring to is the /static resource inside the jar found at the given URL.
How does file: work
That's not java-specific; file: is a generally available URL scheme. You can even type it in a web browser. The more general URL formatting scheme is scheme://server/resource, but with file:, server doesn't apply (it is by definition local to the system you are on), so usually its put as file:///foo, i.e. - an empty 'server' part. Because 3 slashes is a drag to type, I guess, file:/resource is allowed by some 'URL parsers', including java's in this regard, so, file:/usr/... simply maps straight to a local folder: /usr/src/verdi/verdi-12-JDK-etc, as in, if you type ls /usr/src/verdi/verdi-12-JDK17-jar-with-dependencies.jar on the command line on your system, it would show a result (and if it does not, this URL would fail to find anything).
And how is that influenced by the operating system ran under?
It isn't. file URLs are a general concept that work on any platform. Of course, /usr/src/verdi/etc is never going to work correctly on a windows platform. Or on anybody else's machine. The problem isn't "Oh no! This won't run on another OS!". The problem with file URLs, especially absolute ones, is "Oh no! This will not run on any machine other than this one!".
file:///D:/Projects
I've explained the triple slashes earlier. This is the standard windows 'scheme' for how to stick paths in file URLs: Always forward slashes (even though windows normally uses backslashes), and treat the disk letter as if it is a 'drive' in the 'root': /D:/Project is URL-ese for:
D:
cd \Project
There is no difference in OS at all - file: URLs are handled by 'interpret this file URL the way any file URL would be interpreted on this machine'.

The answer to the related question
Jetty 11.0.11 - 404 on html file in \src\main\webapp\static - maven embedded fat jar
which prompted this post is in the long series of posts beneath this GitHub issue for jetty:
https://github.com/eclipse/jetty.project/issues/8549
In essence, eventually I had to first clean up my Maven pom.xml (see this thread for the discussion and for links to a pom.xml example that is compliant with Maven Shade plugin and Jetty 11.0.11 requirements and standards) then at the end of the day hardcode a link to the JAR file to find the HTML, JS, etc. resources Jetty was to serve out as a webpage. Also put in a conditional where, on compiling, I need to specify if the code will run "in-IDE" (in my case, Netbeans 14) or "in-JAR" - e. g. in a detached JRE elsewhere than the Netbeans 14 IDE.
Also dropped using the Jetty WebAppContext class and started rendering web content out of a normal ServletContextHandler.
Hopefully this may help someone upgrading Jetty from Jetty 9.xxx to 11 and finding that it all falls apart.
For details as to why they changed so much, see the GitHub link (the last few entries are apropos.)
The github discussion also contains full working source code (startJettyc method) that solved the issue of getting a 404 in a detached, non-IDE modality where the JAR was being run in an JRE separate from an IDE.
Stefan

Eclipse, Tomcat and MacOS X: user.dir is not correct

I installed tomcat and I am using it with Eclipse Luna Java EE. The problem is that one of my classes reads a text file. When I execute it as a java application, it reads the file correctly. When I execute it on the server ("Run on the server"), it then crashes with a nullpointexception.
I checked the path where it is reading the file. When it executes on the server, it tries to read the file from "Eclipse.app/Contents/MacOS/". Just for the sake of it, I tried to put the text file right there, and it does not say it didn't find it anymore, but whenever I try to use Scanner to read it, it produces this:
java.util.Scanner[delimiters=\p{javaWhitespace}+][position=0][match valid=false][need input=false][source closed=false][skipped=false][group separator=.][decimal separator=\,][positive prefix=][negative prefix=\Q-\E][positive suffix=][negative suffix=][NaN string=\Q?\E][infinity string=\Q?\E]
and never goes beyond a single line (hasNext is always false).
So, my questions are:
1 - How can I get it to look for the .txt file elsewhere (on the root of the project, for example)?
2 - Why scanner only finds content on the file when running the class as a java application, not on the server?

If you're running the app as a web app then you should be looking for either:
an absolute path
a resource (e.g., on the classpath)
a context-relative path
Absolute paths are easy and can be defined using init parameters, system properties, config files, etc.
Classpath resources must live on the app's classpath.
Relative resources are off the ServletContext.
The functionality that uses the file should be usable regardless of where the file comes from, e.g., it should take a stream or something. That way the functionality can be used from both your web app, standalone classes like a static main thing, or a unit test.

To those who had the same problem:
I was printing the Scanner object, that's why I was getting that text.
That understood, Scanner still wasn't reading through the file, and I read elsewhere that this tends to be an encoding issue.
Somehow, Scanner worked properly only when I informed the encoding of the file I was reading. So
Scanner f=new Scanner (fileobject, "UTF-8");
worked for me.
As of the user.dir, I changed the attribute of Apache tomcat on Eclipse, by choosing "Run configurations", selecting Apache, and in Attributes, I selected my current workplace.

Java $APP_PACKAGE Pathname Variable Not Working

According to the Java Info.plist Key Reference for Mac, you're supposed to be able to use the $APP_PACKAGE variable to be able to access the root directory of a Mac Application bundle. So I figure that I can store a file in the Contents/Resources/ folder and access it by calling
new File("$APP_PACKAGE/Contents/Resources/MyFile.txt")
However, this doesn't seem to work and I simply get a file not found error. Moreover, I've tried the following to see if I could narrow down the problem:
new File("$APP_PACKAGE/Contents").exists() //Returns false
System.out.printline(new File("$APP_PACKAGE").getParent()) //Returns an empty string
I did generate the Mac OS X bundle using Eclipse's Export to Mac OS X application bundle, if that matters. Any help (or a suitable workaround) would be greatly appreciated!

I figured it out! Even though the app had a hard time telling me what the working directory is, I managed to figure out that it is the folder that the application is in. Then I was able to solve it by referring to the app as a directory:
new File("MyApp.app/Contents/Resources/MyFile.txt").exists() //Returns true!!!
I'd also like to add that I found the suggestion on this blog post to add the following to my info.plist file:
<key>WorkingDirectory</key>
<string>$APP_PACKAGE/Contents/Resources</string>
Unfortunately, this doesn't seem to work on Mac OS 10.7.5 and the working directory just gets reset to the folder that the application is in.

If the value is set as as an environment variable, you may use System.getEnv() to first get the actual value of APP_PACKAGE then create the File object using it.
String appPackage = System.getEnv("APP_PACKAGE");
new File(appPackage + "/Contents/Resources/MyFile.txt");
That said, from the reading of your given link I wonder if this variable is just expended while read from the plist file, not in your Java process. If the variable is not actually given as an environment variable to your Java program, you can retrieve your file in the Contents directory easily since as I remember the packaging of an app on Mac should have the following architecture
/Contents
/MacOS
YourBinary
/Resources
YourFile.txt
Depending on the current working directory of your app (I think it should default to /Contents/MacOS) you can retrieve the correct path using ../Resources/YourFile.txt. If you don't know the current working directory you can print the value of new File(".").getAbsolutePath()

Java - Find absolute path of file

I need to find the path to the user's vlc.exe file.
How can I do this?
I read this http://docs.oracle.com/javase/tutorial/essential/io/find.html and tried using code like
PathMatcher match = FileSystems.getDefault().getPathMatcher("glob:vjlc.{exe, jpg, png}");
Path filename = FileSystems.getDefault().getPath("vjlc.exe","");
if(match.matches(filename))
{
System.out.println(filename);
}
and
File fil = new File("vlc.exe");
System.out.println( fil.getAbsolutePath() );
neither of which worked

I believe you are trying to do something that is not quite right.
First, you're assuming that vlc.exe exists on the local machine. But what happens if it doesn't?
Second, what happens if VLC decides at some point (new build comes out, or upgrade) to change the exe file name to vlc2.exe?
To deal with this kind of dependency, I suggest you'll pass the vlc file location as a program argument to the main() method.
This way, you can create a batch file that tries to locate the vlc.exe path, and pass it through to the java program.
Another alternative, is to setup an environment variable, that will be set up during the installation of your java application. The installation can search for vlc.exe path, or have the user to set it up. Once the variable is set, the java program can read it from the system arguments (see this example).
A third way is to have a setting files (*.ini like), that will contain the vlc exe path. You can then have the file modified according to the relevant path, and have the java program read from it (as property file). The file can be auto generated too, during the installation process, or manually edited post installation.

You can use getAbsolutePath() function.

I think you're looking for ways of searching for the vlc.exe executable on the PATH. If so, something like the following should help:
String path = System.getenv("PATH");
String pathSeparator = System.getProperty("path.separator");
for (String pathElement : path.split(pathSeparator)) {
File file = new File(pathElement, "vlc.exe");
if (file.isFile()) {
// vlc.exe exists in this location.
}
}

When a user runs VLC installer to install VLC media player under Windows, the installer creates a Windows registry key entry HKLM\SOFTWARE\VideoLAN\VLC\InstallDir. You can retrieve the path stored in the key using Java as follows:
http://www.davidc.net/programming/java/reading-windows-registry-java-without-jni
read/write to Windows Registry using Java
If the HKLM\SOFTWARE\VideoLAN\VLC\InstallDir key is present, you know VLC is installed. If the user decides to install VLC at a different directory than what is suggested by VLC installer as default, the key will be able to tell you that.
This only works when the user installs VLC through its installer. But, it won't work if the user simply extracts VLC from its zip distribution file since this approach won't touch the Windows registry.

loading from JAR files during deployment vs development

when i am loading some data into my java program, i usually use FileInputStream. however i deploy the program as a jar file and webstart, so i have to use getRessource() or getRessourceAsStream() to load the data directly from the jar file.
now it is quite annoying to always switch this code between development and deployment?
is there a way autmate this? i.e. is there a way to know if the code is run from a jar or not?
when i try to load it withoug jar like this:
InputStream is = this.getClass().getResourceAsStream("file.txt");
the returned inputstream is simply null, although the file is definitely in the root directory of the application.
thanks!

Why do you use FileInputStream during development? Why not just use getResourceAsStream from the very start? So long as you place your files in an appropriate place in your classpath, you shouldn't have any problems. It can still be a file in the local filesystem rather than in a jar file.
It's helpful to develop with the final deployment environment in mind.
EDIT: If you want something in the root directory of your classpath, you should either use:
InputStream x = getClass().getResourceAsStream("/file.txt");
or
InputStream x = getClass().getClassLoader().getResourceAsStream("file.txt");
Basically Class.getResourceAsStream will resolve relative resources to the package containing the class; ClassLoader.getResourceAsStream resolves everything relative to the "root" package.

You could read your data always as a ressource. You only have to add the path where the data lies to your classpath.
If your data stays in WEB-INF/somewhere/mydata.txt inside your jar file, you will access it with:
getClass().getResourceAsStream( "/WEB-INF/somewhere/mydata.txt" )
Now, if you create a development directory /devel/WEB-INF/somewhere/mydata.txt and put /devel to your classpath, your code will work in development and production.
EDIT after explanation in question:
In your case this.getClass().getResourceAsStream( "mydata.txt" ) the resource is taken from the same position where the classfile of this is taken from. If you want to keep this, then you have to create a directory /devel/<path of package>/mydata.txt and again add /devel to your classpath.

How about setting a system property in your dev environment, via the -D switch? e.g. java -D:mypropertyname=mypropertyvalue
You could set the property in ant scripts in your dev environment, other environments don't get the property:
e.g.
public static boolean isDevEnvironment(){ return System.getProperty("mypropertyname")!=null;}
You might find a better way to hack it from one of the existing System Properties

If a file is considered part of your deployed application (as opposed to be part of the installation specific files) and can be located through the classpath then consider simply always using getResourceAsStream since it works regardless of the actual deployment scheme as long as it is in the classpath.
You might also find the information available from the JVM relevant (if allowed by the security manager):
// Get the location of this class
Class cls = this.getClass();
ProtectionDomain pDomain = cls.getProtectionDomain();
CodeSource cSource = pDomain.getCodeSource();
URL loc = cSource.getLocation(); // file:/c:/almanac14/examples/
http://www.exampledepot.com/egs/java.lang/ClassOrigin.html?l=rel

There shouldn't be any difference between development vs deployment, IHMO.
Classloader.getResource or getResourceAsStream works well, you can read resources and even write them.You can write your own Protocol handles and access everything as an URL/URI, which allows you to read and write resources and also allows proper identification of who actually provide the resource.
The only problem is if an URLStreamHandlerFactory is already registered(in a J2EE application the container could install a factory and you don't have any way to go over and install your own) and you cannot use your handlers "anywhere".
Knowing that, it is preferred to implement your own "resources". At that time when I need it I couldn't find something like that so I had to implement my own ResourceManager. For me it looks more intuitive to access a resource like
Resource layout = ResourceManager.resolve("view://layout/main.jsp")
instead of
URL layout = Classloader.getResource("some_package/view/layout/main.jsp")