Develop Reference

Loading external Classes with external dependencies - URLClassLoader ClassNotFoundException

I am loading a Class from an external.JAR file and by means of URLClassLoader, which works as long as the external Class does not reference another JAR. If I do it yields a ClassNotFoundException.
As a workaround I add the other second tier JAR as a dependency, but I would like to load these also dynamically on runtime.
Question: How do I load an external Class which references other external classes? or how do I load external JAR files and classes, in
the correct order so I am not getting an exception?
Should I catch the exception and then "first" load that class that was not yet loaded?

You actually could load them all using a child first class loader. That main jar and all of its dependencies would be apart of that class loader and can be referenced and thrown away if needed.
Main Class Loader -> ChildFirstClassLoader -> Loads jar and dependent jars
This is a good example
Here is another SO similar reference
Semi Example
File file = new File("c:\\free-universe-games-llc\\app.jar");
URL url = file.toURI().toURL();
ChildFirstClassLoader urlClassLoader = new ChildFirstClassLoader(new URL[]{url}, SomeClassLoader.class.getClassLoader());
Class<?> aClass = urlClassLoader.loadClass("com.freeuniversegames.app.App.class");
Object o = aClass.newInstance();

How to detect one resource file is from jar or from file?

I want to load one resource file from classpath. This file may be in a jar file or just a separate file. How can I know that ? Thanks

This should do the trick assuming that MyClass is a class in the same ClassLoader or a child ClassLoader of the one that contains your resource file.
MyClass.class.getResourceAsStream("/path/to/my/resource")
Finds a resource with a given name. The rules for searching resources
associated with a given class are implemented by the defining class
loader of the class. This method delegates to this object's class
loader. If this object was loaded by the bootstrap class loader, the
method delegates to
ClassLoader.getSystemResourceAsStream(java.lang.String).
More details here
The proposal above provide your resource as a InputStream, if you want to have it as an URL use MyClass.class. getResource("/path/to/my/resource") instead

Java resource from class vs Thread

what is the difference between
getClass().getResource("some-resource-file.txt")
vs
Thread.currentThread().getContextClassLoader().getResource("some-resource-file.txt")
I have resources in src/test/resources & I am trying to access them from Unit test. It's a typical maven style directory structure.
I was expecting both to behave identical. But it's not., getClass().getResource() doesn't fetch the resource where as from Thread I am able to fetch the resource.
So how do they differ ?

Let's say you're developing a library and the library jar is placed into a web container's classpath.
Now let's say a webapp, using this library, is deployed in the container.
The webapp will have its own class loader, using WEB-INF/classes and WEB-INF/lib/*.jar as its classpath. And the container, for each request coming to your webapp, will set the current thread classloader to the class loader of the classpath.
When your library code uses getClass().getResource(), it will load the resource using the classloader used to load the library classes. It will thus use the container's class loader, and will thus use the resources in your library's jar and in the other libraries used to start the container.
If your library code uses Thread.currentThread().getContextClassLoader() instead to load the resource, it will use the classloader associated with the current thread, and will thus load the resources from the webapp's class loader, looking for the resource in WEB-INF/classes and in the jars inside WEB-INF/lib.
The latter can be what you want. For example, if you're designing a logging library (please don't), the logger will be able to read a different configuration file for each webapp, instead of having a single config shared by all the webapps.
Regarding the way the two methods look for resources, they all finally delegate to a ClassLoader to load the resource. But loading it via a Class will treat relative paths as relative to the invoked class, whereas loading it via a ClassLoader expects a path starting at the root of the package tree. Suppose your class is in the package com.foo, then
MyClass.class.getResource("hello.txt")
is equivalent to
MyClass.class.getResource("/com/foo/hello.txt")
and is equivalent to
MyClass.class.getClassLoader().getResource("com/foo/hello.txt");

There is a special case getting the first class running (which is why you have to declare the main() method as static with an array of strings as an argument).
Once that class is loaded and is running, future attempts at loading classes are done by the class loader. At its simplest, a class loader creates a flat name space of class bodies that are referenced by a string name. Each class in Java uses own classloader to load other classes. So if ClassA.class references ClassB.class then ClassB needs to be on the classpath of the ClassLoader of ClassA, or its parents.
The thread context ClassLoader is a special one in that it is the current ClassLoader for the currently running thread. This is useful in multi-classloader environments. An object can be created from a class in ClassLoader C and then passed to a thread owned by ClassLoader D. In this case the object needs to use Thread.currentThread().getContextClassLoader() directly if it wants to load resources that are not available on its own ClassLoader .

How Classloader determines which classes it can load?

I'm reading on class loading in Java.
Motivation
Assuming we have a classloader hierarchy that looks like this, I understand that classes loaded by First are not directly accessible by classes loaded by Second (and vice versa).
Bootstrap
|
System
|
Common
/ \
First Second
I also understand that a classloader checks with its parent class loader whether it can load the class and, if that is the case, delegates the loading to its parent.
Question
How do classloaders actually determine whether they can load some given class?

That differs depending on the implementation of the classloader. But all Classes a ClassLoader can load are retrieved by ClassLoader.findClass(String)
There are many implementations but the most common one is the URLClassLoader which loads classes from directories and jar files.

The classloader checks all classes (java class files) within your CLASSPATH path variable. If your class is found there, it exists, otherwise it doesn't.
So practically, your /src directory and all subdirectories (=packages) are scanned.

The classloader transforms the requested class name into a file name and then tries to find a "class file" of that name from a file system. As #poitroae notes, it uses the CLASSPATH variable, if set, as a starting place. Most IDEs and such extend this to include your working directories for the project.

Java Classloader unable to load modules?

I seem to be having a problem with loading classes in a module loader for an application I'm developing. Basically, all classes I'm going to be loading with it extend another class, which is located in a package in the actual application. For our purposes, we'll call it Module. Modules are located in a separate folder outside the actual application.
The loader iterates through a folder and executes the loadFile() method on any file with the extension .class. All classes have the package declaration as the Module class, as well as the extends Module declaration in the class header.
This is the loadFile() method, header and exception clauses excluded:
String fileName = file.getName();
String className = fileName.replace(".class", ""); //Strips extension
Class<?> aClass = Class.forName(className, true, new URLClassLoader(new URL[] { file.toURI().toURL() }));
Class<? extends Module> modClass = aClass.asSubclass(Module.class);
return modClass.getConstructor().newInstance();
I keep getting a ClassNotFoundException on the third line. And past that, if it ClassNotFoundException weren't thown, would all dependencies be resolved?

From the documentation for URLCLassLoader:
This class loader is used to load classes and resources from a search path of URLs referring to
both JAR files and directories. Any URL that ends with a '/' is assumed to refer to a directory.
Otherwise, the URL is assumed to refer to a JAR file which will be opened as needed.
So, you must use URLs for either directories or .jars
Two solutions:
Force your users to give you .jar files, including a manifest of some sort inside with the classname that they wish to be loaded. This approach is used by the Bukkit developers. Having used this method in the past, the dependencies should all be packaged in the .jar file and thus in the URLClassLoader's search path and able to be loaded.
Use the URL of the file's directory, and search that directory for .class files. I'm not sure if dependencies will be loaded using this method.

In the URLClassLoader, do not pass the file, but the parent folder. However, this works correctly if classes are all in the "default package", so the .class files you are loading must not have a package declaration on top.
By default, a class loader will also trigger loading of all classes required to properly build the class: it will try to load the super class, the super super class etc... all the interfaces and super interfaces, the classes needed for static fields and methods, the classes needed for method signatures (return types and parameters). It will not usually try to load classes used internally by methods, not until you execute those methods.
However, usually a class loader does not "contain" all those classes, for example your class will end up inheriting from java.lang.Object, and your URLClassLoader will not contain the Object.class file. So, class loaders delegate to their parent class loaders.
You are currently creating a URLClassLoader without specifying a parent, in Java 7 at least the parent will default to the "system class loader", which is fine as long as you are in a plain java application, and not executing your code itself inside a specific hierarchy of class loaders. If however you are running that code in a web application, or in an OSGI container etc.. the you should give the URLClassLoader a proper parent to delegate to, for example Thread.currentThread().getContextClassLoader() or this.getClass().getClassLoader().
I suppose you need all of this because you need to load those class dynamically at runtime.