Develop Reference

Java load jar files from directory

I currently develop an open-source project where people may add their own .jar to extend the included features. However, I'm stuck with how to load the classes in the jar.
I have an abstract class which has an abstract method onEnable() and some getter that provides some objects to work with the application. The plugin needs the subclass my plugin-class BasePlugin. The jar should be added to /plugins and thus I want all *.jar files in the /plugins folder to be loaded when the application starts.
The problem I'm running to now is that, of all the approaches I found, I need to declare a classpath of the classes in the jar file, which I do not know. Neither do I know the name of the jar file. Thus, I need to scan the /plugins folder for any *.jar file and load the corresponding class inside the jar which implements BasePlugin and invoke the onEnable() method.

The basic idea is too...
Read all the files in a specific directory
Convert the File reference to a URL for each result
Use a URLClassLoader, seeded with the URL results to load each result
Use URLClassLoader#findResources to find all the match resources with a specific name
Iterate over the matching resources and load each one, which should give, at least, the "entry point" class name
Load the class specified by the "entry point" property
For example...
public List<PluginClass> loadPlugins() throws MalformedURLException, IOException, ClassNotFoundException {
File plugins[] = new File("./Plugins").listFiles(new FileFilter() {
#Override
public boolean accept(File file) {
return file.getName().endsWith(".jar");
}
});
List<URL> plugInURLs = new ArrayList<>(plugins.length);
for (File plugin : plugins) {
plugInURLs.add(plugin.toURI().toURL());
}
URLClassLoader loader = new URLClassLoader(plugInURLs.toArray(new URL[0]));
Enumeration<URL> resources = loader.findResources("/META-INFO/Plugin.properties");
List<PluginClass> classes = new ArrayList<>(plugInURLs.size());
while (resources.hasMoreElements()) {
URL resource = resources.nextElement();
Properties properties = new Properties();
try (InputStream is = resource.openStream()) {
properties.load(is);
String className = properties.getProperty("enrty-point");
PluginClass pluginClass = loader.loadClass(className);
classes.add(pluginClass);
}
}
return classes
}
nb: I've not run this, but this is the "basic

SpigotMC uses JAR files as plugins as well, inside the jar is a plugin.yaml file that stores extra information about the plugin including the classpath. You don't need to use a YAML file, instead you could use something like JSON or even a plain text file.
The YAML file is inside the jar and can be accessed by using some of the methods explained here. You can then get the classpath property and then load the jar using the methods explained here. Extra information can be stored about the plugin such as the name, version, and dependencies.

Java already has a class for this: ServiceLoader.
The ServiceLoader class was introduced with Java 6, but the “SPI jar” concept is actually as old as Java 1.3. The idea is that each jar contains a short text file that describes its implementations of a particular service provider interface.
For instance, if a .jar file contains two subclasses of BasePlugin named FooPlugin and BarPlugin, the .jar file would also contain the following entry:
META-INF/services/com.example.BasePlugin
And that jar entry would be a text file, containing the following lines:
com.myplugins.FooPlugin
com.myplugins.BarPlugin
Your project would scan for the plugins by creating a ClassLoader that reads from the plugins directory:
Collection<URL> urlList = new ArrayList<>();
Path pluginsDir = Paths.get(
System.getProperty("user.home"), "plugins");
try (DirectoryStream<Path> jars =
Files.newDirectoryStream(pluginsDir, "*.jar")) {
for (Path jar : jars) {
urlList.add(jar.toUri().toURL());
}
}
URL[] urls = urlList.toArray(new URL[0]);
ClassLoader pluginClassLoader = new URLClassLoader(urls,
BasePlugin.class.getClassLoader());
ServiceLoader<BasePlugin> plugins =
ServiceLoader.load(BasePlugin.class, pluginClassLoader);
for (BasePlugin plugin : plugins) {
plugin.onEnable();
// etc.
}
An additional advantage of using ServiceLoader is that your code will be capable of working with modules, a more complete form of code encapsulation introduced with Java 9 which offers increased security.

There is an example here it may be helpful. Also, you should take a look at OSGi.

How to Get List of Classpath Resources In Nested Jar?

I have a Maven project, and I am packaging it as a single fat jar using the One-Jar Maven plugin. According to this page on the One-Jar site documentation resource loading must be done in a special manner with One-Jar packaged apps, however that documentation does not cover how to list the resources in a manner which does not depend on their custom class loader.
My question is this: how to I list the contents of the root of my classpath within an inner jar in a packaging agnostic manner, meaning not assuming it will always be in such special packaging? As covered in this SO question, Spring's PathMatchingResourcePatternResolver will not work because Spring's DefaultResourceLoader ignores the classloader.
NOTE: The closest SO question I could find was this question, but this does not address the listing of classpath resources from nested jars.

It can be done in a FAT jar. Your own answer stating it's not possible has a link to a page that shows it's possible! It's not that insane of a solution. Here's a slightly modified version:
private List<String> getResourceFiles(String path) throws IOException
{
List<String> files = new ArrayList<>();
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
URL dirURL = classLoader.getResource(path);
if (dirURL.getProtocol().equals("jar"))
{
String jarPath = dirURL.getPath().substring(5, dirURL.getPath().indexOf("!"));
try (final JarFile jar = new JarFile(URLDecoder.decode(jarPath, "UTF-8")))
{
Enumeration<JarEntry> entries = jar.entries();
while (entries.hasMoreElements())
{
String name = entries.nextElement().getName();
if (name.startsWith(path + "/") && !name.endsWith("/"))
{
files.add(name);
}
}
}
return files;
}
// return some other way when not a JAR
}
This version will return the full resource path of any file under the given resource path, recursively. For example, an input of config/rest would return files under that resource folder as:
config/rest/options.properties
config/rest/samples/data.json
With those values, one can get the resource with ClassLoader::getResource, or get the contents of the resource with ClassLoader::getResourceAsStream.

After reviewing many articles, and blogs posts, web pages, site and the Spring source code, I had concluded this is just a weakness in Java and is not possible in any sane manner.

Executing jar issue - uncaught error fetching image

I've made an audio player and the jar was made with netbeans. To load the images I've used:
ClassLoader cl = this.getClass().getClassLoader();
URL playerIconURL = cl.getResource("tp/audioplayer/Images/icon.png");
if (playerIconURL != null){
ImageIcon playerIcon = new ImageIcon(playerIconURL);
frame.setIconImage(playerIcon.getImage());
}
else{
System.err.println("cannot load player icon");
}
I mention that the folder Images is in the src/tp/audioplayer.
When I'm running the application inside netbeans everything is allright, but when I execute the jar in command prompt,the application starts but it's blank and it blocks and I get:
Can you tell me what I've done wrong or what is the problem? Thanks in advance!

If tp is in your classpath you will have to load it with cl.getResource("/tp/audioplayer/Images/icon.png") if tp is NOT a source folder (but still added to the buildpath.
If you add tp as a sourcefolder then
cl.getResource("/audioplayer/Images/icon.png")
Note that jars are casesensitive, make sure you the case-sensitive file-path.

Try any of these:
// using getResourceAsStream
InputStream is = this.getClass().getResourceAsStream( "picture.gif" );
// or
InputStream is = MyClass.class.getResourceAsStream( "stuff.ser" );
// or
InputStream is = MyApp.class.getClassLoader().getResourceAsStream( "InWords.properties" );
The resource in the jar file must be qualified with same package name as the class you call getResourceAsStream from. Alternatively, you can use an absolute name beginning with a / where dots get mapped to /s. If you don’t have a lead /, you have a relative name, and the name of the package will be prepended. If you use a /, you must include the name of the package yourself, or whatever name the resource is filed under in the jar.
For example you could specify /com/mindprod/mypackage/mystuff.ser or /com.mindprod.mypackage.mystuff.ser or simply mystuff.ser. Don’t use Windows style filenames with . These are not filenames, but Java resources that live along with the class files either in jars or sometimes freestanding on disk, or on the server.
In theory, getResourceAsStream will look in the local classpath, in the jar and in the directory where the class file was loaded from.

Internal JAR uses files on the file system

I have a use case where I need to export this specific piece of code as a java library (which will be a JAR eventually) but the problem is that it needs to use some piece of information stored in physical files on the file system.
I have 2 questions here:
1) Where should I put these files on the filesystem (One option that I could think of was in the resources directory of the Java module containing the library: Have a doubt though that the resources directory also gets compiled into the jar?)
2) When I am using this library from an external Java application, how would the library be able to locate the files? Would they still be in the classpath?

You have two options, first one is to place the files inside the package structure, so that they will be packed inside the jar. You would get them from the code like this:
getClass().getResourceAsStream("/path/to/your/resource/resource.ext");
If you would call it from a static method of class named A then you should write like this:
A.class.getResourceAsStream("/path/to/your/resource/resource.ext");
The "/path" part of the path is the topmost package, and the resource.ext is your file name.
The other option is to put them outside the jar package, but then the jar needs to know their location:
provide it as an argument to the program (java -jar program.jar system/path/to/file)
hardcode the location from which you would read the file with paths
The way I undestood your queastion and answered it, it has nothing to do with classpath:
The CLASSPATH variable is one way to tell applications, including the JDK tools, where to look for user classes. (Classes that are part of the JRE, JDK platform, and extensions should be defined through other means, such as the bootstrap class path or the extensions directory.)
EDIT:
but you can nevertheless, put it there and get it from code like this:
System.getProperty("java.class.path");
It would however require some logic to parse it out.

You can pass the location of the files in a property file or some technique like this.

Where should I put these files on the filesystem
That is up to you to decide, though it would be a good idea to make this configurable. It would also be a good idea to try to fit into the conventions of the host operating system / distro, though these vary ... and depend on the nature of your application.
When I am using this library from an external Java application, how would the library be able to locate the files?
You would typically use a configuration property or initialization parameter to hold/pass the location. If you were writing an application rather that a library, you could use the Java Preferences APIs, though this probably a poor choice for a library.
Would they still be in the classpath?
Only if you put the location on the classpath ... and that is going to make configuration more tricky. Given that these files are required to be stored in the file system, I'd recommend using FileInputStream or similar.

Using Eclipse, I always create a package 'resources' where I put the files the jar needs. I access the files (from pretty much anywhere) through
this.getClass().getClassLoader().getResources("/resources/file.ext");
With export->runnable jar all those files are included in the .jar. I'm not sure this is the correct way of doing it though. Also, I'm not 100% sure about the "/" before resources, maybe it should be omitted.

I found a relevant answer as a part of another question : How to load a folder from a .jar?
I am able to successfully retrieve the files using the following code:
/**
* List directory contents for a resource folder. Not recursive.
* This is basically a brute-force implementation.
* Works for regular files and also JARs.
*
* #author Greg Briggs
* #param clazz Any java class that lives in the same place as the resources you want.
* #param path Should end with "/", but not start with one.
* #return Just the name of each member item, not the full paths.
* #throws URISyntaxException
* #throws IOException
*/
String[] getResourceListing(Class clazz, String path) throws URISyntaxException, IOException {
URL dirURL = clazz.getClassLoader().getResource(path);
if (dirURL != null && dirURL.getProtocol().equals("file")) {
/* A file path: easy enough */
return new File(dirURL.toURI()).list();
}
if (dirURL == null) {
/*
* In case of a jar file, we can't actually find a directory.
* Have to assume the same jar as clazz.
*/
String me = clazz.getName().replace(".", "/")+".class";
dirURL = clazz.getClassLoader().getResource(me);
}
if (dirURL.getProtocol().equals("jar")) {
/* A JAR path */
String jarPath = dirURL.getPath().substring(5, dirURL.getPath().indexOf("!")); //strip out only the JAR file
JarFile jar = new JarFile(URLDecoder.decode(jarPath, "UTF-8"));
Enumeration<JarEntry> entries = jar.entries(); //gives ALL entries in jar
Set<String> result = new HashSet<String>(); //avoid duplicates in case it is a subdirectory
while(entries.hasMoreElements()) {
String name = entries.nextElement().getName();
if (name.startsWith(path)) { //filter according to the path
String entry = name.substring(path.length());
int checkSubdir = entry.indexOf("/");
if (checkSubdir >= 0) {
// if it is a subdirectory, we just return the directory name
entry = entry.substring(0, checkSubdir);
}
result.add(entry);
}
}
return result.toArray(new String[result.size()]);
}
throw new UnsupportedOperationException("Cannot list files for URL "+dirURL);
}

How to walk through Java class resources?

I know we can do something like this:
Class.class.getResourceAsStream("/com/youcompany/yourapp/module/someresource.conf")
to read the files that are packaged within our jar file.
I have googled it a lot and I am surely not using the proper terms; what I want to do is to list the available resources, something like this:
Class.class.listResources("/com/yourcompany/yourapp")
That should return a list of resources that are inside the package com.yourcompany.yourapp.*
Is that possible? Any ideas on how to do it in case it can't be done as easily as I showed?
Note: I know it is possible to know where your jar is and then open it and inspect its contents to achieve it. But, I can't do it in the environment I am working in now.

For resources in a JAR file, something like this works:
URL url = MyClass.class.getResource("MyClass.class");
String scheme = url.getProtocol();
if (!"jar".equals(scheme))
throw new IllegalArgumentException("Unsupported scheme: " + scheme);
JarURLConnection con = (JarURLConnection) url.openConnection();
JarFile archive = con.getJarFile();
/* Search for the entries you care about. */
Enumeration<JarEntry> entries = archive.entries();
while (entries.hasMoreElements()) {
JarEntry entry = entries.nextElement();
if (entry.getName().startsWith("com/y/app/")) {
...
}
}
You can do the same thing with resources "exploded" on the file system, or in many other repositories, but it's not quite as easy. You need specific code for each URL scheme you want to support.

In general can't get a list of resources like this. Some classloaders may not even be able to support this - imagine a classloader which can fetch individual files from a web server, but the web server doesn't have to support listing the contents of a directory.
For a jar file you can load the contents of the jar file explicitly, of course.
(This question is similar, btw.)

The most robust mechanism for listing all resources in the classpath is currently to use this pattern with ClassGraph, because it handles the widest possible array of classpath specification mechanisms, including the new JPMS module system. (I am the author of ClassGraph.)
List<String> resourceNames;
try (ScanResult scanResult = new ClassGraph()
.whitelistPaths("com/yourcompany/yourapp")
.scan()) {
resourceNames = scanResult.getAllResources().getNames();
}

I've been looking for a way to list the contents of a jar file using the classloaders, but unfortunately this seems to be impossible. Instead what you can do is open the jar as a zip file and get the contents this way. You can use standard (here) ways to read the contents of a jar file and then use the classloader to read the contents.

I usually use
getClass().getClassLoader().getResourceAsStream(...)
but I doubt you can list the entries from the classpath, without knowing them a priori.