Java Urlclass loader cast exception [duplicate] - java

We learned from the release notes of Java 9 that
The application class loader is no longer an instance of java.net.URLClassLoader (an implementation detail that was never specified in previous releases). Code that assumes that ClassLoader::getSytemClassLoader returns a URLClassLoader object will need to be updated.
This breaks old code, which scans the classpath as follows:
Java <= 8
URL[] ressources = ((URLClassLoader) classLoader).getURLs();
which runs into a
java.lang.ClassCastException:
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to
java.base/java.net.URLClassLoader
So for Java 9+ the following workaround was proposed as a PR at the Apache Ignite Project, which works as intended given adjustments in the JVM runtime options: --add-opens java.base/jdk.internal.loader=ALL-UNNAMED. However, as mentioned in the comments below, this PR was never merged into their Master branch.
/*
* Java 9 + Bridge to obtain URLs from classpath...
*/
private static URL[] getURLs(ClassLoader classLoader) {
URL[] urls = new URL[0];
try {
//see https://github.com/apache/ignite/pull/2970
Class builtinClazzLoader = Class.forName("jdk.internal.loader.BuiltinClassLoader");
if (builtinClazzLoader != null) {
Field ucpField = builtinClazzLoader.getDeclaredField("ucp");
ucpField.setAccessible(true);
Object ucpObject = ucpField.get(classLoader);
Class clazz = Class.forName("jdk.internal.loader.URLClassPath");
if (clazz != null && ucpObject != null) {
Method getURLs = clazz.getMethod("getURLs");
if (getURLs != null) {
urls = (URL[]) getURLs.invoke(ucpObject);
}
}
}
} catch (NoSuchMethodException | InvocationTargetException | NoSuchFieldException | IllegalAccessException | ClassNotFoundException e) {
logger.error("Could not obtain classpath URLs in Java 9+ - Exception was:");
logger.error(e.getLocalizedMessage(), e);
}
return urls;
}
However, this causes some severe headache due to the use of Reflection here. This is kind of an anti-pattern and is strictly criticized by the forbidden-apis maven plugin:
Forbidden method invocation: java.lang.reflect.AccessibleObject#setAccessible(boolean) [Reflection usage to work around access flags fails with SecurityManagers and likely will not work anymore on runtime classes in Java 9]
Question
Is there a safe way to access the list of all resource URLs in the class- / module path, which can be accessed by the given classloader, in OpenJDK 9/10 without using sun.misc.* imports (e.g. by using Unsafe)?
UPDATE (related to the comments)
I know, that I can do
String[] pathElements = System.getProperty("java.class.path").split(System.getProperty("path.separator"));
to obtain the elements in the classpath and then parse them to URLs. However - as far as I know - this property only returns the classpath given at the time of the application launch. However, in a container environment this will be the one of the application server and might not be sufficient, e.g. then using EAR bundles.
UPDATE 2
Thank your for all your comments. I will test, if System.getProperty("java.class.path") will work for our purposes and update the question, if this fullfills our needs.
However, it seems that other projects (maybe for other reasons, e.g Apache TomEE 8) suffer the same pain related to the URLClassLoader- for this reason, I think it is a valueable question.
UPDATE 3
Finally, we did switch to classgraph and migrated our code to this library to resolve our use-case to load ML resources bundled as JARs from the classpath.

I think this is an XY problem. Accessing the URLs of all resources on the classpath is not a supported operation in Java and is not a good thing to try to do. As you have already seen in this question, you will be fighting against the framework all the way if you try to do this. There will be a million edge cases that will break your solution (custom classloaders, EE containers, etc. etc.).
Please could you expand on why you want to do this?
If you have some kind of plugin system and are looking for modules that interface with your code which may have been provided at runtime, then you should use the ServiceLoader API, i.e.:
A service provider that is packaged as a JAR file for the class path is identified by placing a provider-configuration file in the resource directory META-INF/services. The name of the provider-configuration file is the fully qualified binary name of the service. The provider-configuration file contains a list of fully qualified binary names of service providers, one per line.
For example, suppose the service provider com.example.impl.StandardCodecs is packaged in a JAR file for the class path. The JAR file will contain a provider-configuration file named:
META-INF/services/com.example.CodecFactory
that contains the line:
com.example.impl.StandardCodecs # Standard codecs

AFAIK you can parse the java.class.path system property to get the urls:
String classpath = System.getProperty("java.class.path");
String[] entries = classpath.split(File.pathSeparator);
URL[] result = new URL[entries.length];
for(int i = 0; i < entries.length; i++) {
result[i] = Paths.get(entries[i]).toAbsolutePath().toUri().toURL();
}
System.out.println(Arrays.toString(result)); // e.g. [file:/J:/WS/Oxygen-Stable/jdk10/bin/]

Related

How to create a custom ClassLoader for nested JAR files

I am working with a Java library that has some nested JAR files in lib package.
I have 2 issues:
I cannot see referenced types in my IDE (I am using JetBrains IntelliJ)
Of course I get class not defined at runtime
I understand that I have to create and use a custom ClassLoader, will it solve both problems?
Is this the recommended way of achieving this result?
The JAR file is an Italian government provided library and I cannot modify it as it will be periodically updated as the regulation changes.
Yes, as far as I know, the standard ClassLoaders do not support nested JARs. Which is sad, since it would be a really nice idea, but Oracle just doesn't give a damn about it. Here is a 18-year old ticket:
https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4735639
If you are getting those JARs from somebody else, the best thing would be to contact the vendor and ask them for a delivery in standards-compatible format. From your answer I realize that this might be difficult to achieve, but I would still try to talk to them, because it's the right thing to do. I'm pretty sure that everybody else in your position has the same issue. According to industry standards, such situation would usually hint your vendor into using Maven repository for their deliverables.
If talking to your vendor fails, you can re-pack the JARs as you get them. I would recommend writing an automated script for that and making sure it gets run on each delivery. You can either put all .class files into one uber-JAR, or just move the nested JARs outside the enclosing JAR. Caveat 1: there can be more than one class with the same name, so you need to make sure to take the correct one. Caveat 2: if the JARs were signed, you will lose the signature (unless you sign them with your own).
Option 3: you can always implement your own ClassLoader to load the classes from anywhere, even from the kitchen sink.
This guy did exactly this: https://www.ibm.com/developerworks/library/j-onejar/index.html
The short summary is that such a ClassLoader has to perform recursive unzipping, which is a bit of a pain-in-the-ass because archives are essentially made for stream access and not for random access, but apart from that it's perfectly doable.
You can use his solution as a "wrapper loader" which will replace your main class.
As far as IntelliJ IDEA goes, I don't believe it supports this functionality out-of-the box. The best thing would be either to re-package JARs as described above and add them as separate classpath entries, or to search if anybody has written a plugin for nested JAR support.
I don't know what you want to do after load jars.
In my case, use jar dynamic loading for Servlet samples.
try{
final URLClassLoader loader = (URLClassLoader)ClassLoader.getSystemClassLoader();
final Method method = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
method.setAccessible(true);
new File(dir).listFiles(new FileFilter() {
#Override
public boolean accept(File jar) {
// load file if it is 'jar' type
if( jar.toString().toLowerCase().contains(".jar") ){
try {
method.invoke(loader, new Object[]{jar.toURI().toURL()});
XMLog.info_arr(logger, jar, " is loaded.");
JarInputStream jarFile = new JarInputStream(new FileInputStream(jar));
JarEntry jarEntry;
while (true) {
// load jar file
jarEntry = jarFile.getNextJarEntry();
if (jarEntry == null) {
break;
}
// load .class file in loaded jar file
if (jarEntry.getName().endsWith(".class")) {
Class loadedClass = Class.forName(jarEntry.getName().replaceAll("/", "\\.").replace(".class",""));
/*
* In my case, I load jar file for Servlet.
* If you want to use it for other case, then change below codes
*/
WebServlet annotaions = (WebServlet) loadedClass.getAnnotation(WebServlet.class);
// load annotation and mapping if it is Servlet
if (annotaions.urlPatterns().length > 0) {
ServletRegistration.Dynamic registration = servletContextEvent.getServletContext().addServlet(annotaions.urlPatterns().toString(), loadedClass);
registration.addMapping(annotaions.urlPatterns());
}
}
}
} catch (Exception e) {
System.err.println("Can't load classes in jar");
}
}
return false;
}
});
} catch(Exception e) {
throw new RuntimeException(e);
}
Interestingly I just solved a version of this problem for JesterJ, though I had the additional requirement of loading dependencies for the code in the jar file as well. JesterJ (as of this evening's commits!) runs from a fat jar and accepts an argument denoting a second fat jar containing the classes, dependencies and configuration for a document ingestion plan (the user's code that I need to run).
The way my solution works is I borrow the knowledge of how to load jars inside of jars from Uno-Jar (the library that produces the fat jar), and stuff my own classloader in above it to control the evaluation order of the class loaders.
The key bit from https://github.com/nsoft/jesterj/blob/jdk11/code/ingest/src/main/java/org/jesterj/ingest/Main.java looks like this:
JesterJLoader jesterJLoader;
File jarfile = new File(javaConfig);
URL planConfigJarURL;
try {
planConfigJarURL = jarfile.toURI().toURL();
} catch (MalformedURLException e) {
throw new RuntimeException(e); // boom
}
jesterJLoader = (JesterJLoader) ClassLoader.getSystemClassLoader();
ClassLoader loader;
if (isUnoJar) {
JarClassLoader jarClassLoader = new JarClassLoader(jesterJLoader, planConfigJarURL.toString());
jarClassLoader.load(null);
loader = jarClassLoader;
} else {
loader = new URLClassLoader(new URL[]{planConfigJarURL}, jesterJLoader);
}
jesterJLoader.addExtLoader(loader);
My JesterJLoader is here:
https://github.com/nsoft/jesterj/blob/jdk11/code/ingest/src/main/java/org/jesterj/ingest/utils/JesterJLoader.java
Though if you are happy to simply delegate up and rely on existing classes on the main class path (rather than loading additional dependencies from the sub-fat-jar like I'm doing) yours could be much simpler. I go to a lot of effort to allow it to check the sub-jar first rather than delegating up to the parent immediately, and then have to keep track of what's been sent to the sub-jar to avoid loops and subsequent StackOverflowError...
Also note that the line where I get the system class loader is going to NOT be what you want, I'm also monkeying with the system loader to work around impolite things that some of my dependencies are doing with class loading.
If you decide to try to check out Uno-Jar pls note that resource loading for this nested scenario may yet be wonky and things definitely won't work before https://github.com/nsoft/uno-jar/commit/cf5af42c447c22edb9bbc6bd08293f0c23db86c2
Also: recently committed thinly tested code warning :)
Disclosure: I maintain both JesterJ and Uno-Jar (a fork of One-JAR the library featured in the link supplied by jurez) and welcome any bug reports or comments or even contributions!

Read the jar version for a class

For a webservice client I'd like to use Implementation-Title and Implementation-Version from the jar file as user-agent string. The question is how to read the jar's manifest.
This question has been asked multiple times, however the answer seems not applicable for me. (e.g. Reading my own Jar's Manifest)
The problem is that simply reading /META-INF/MANIFEST.MF almost always gives wrong results. In my case, it would almost always refer to JBoss.
The solution proposed in https://stackoverflow.com/a/1273196/4222206
is problematic for me as you'd have to hardcode the library name to stop the iteration, and then still it may mean two versions of the same library are on the classpath and you just return the first - not necessarily the right - hit.
The solution in https://stackoverflow.com/a/1273432/4222206
seems to work with jar:// urls only which completely fails within JBoss where the application classloader produces vfs:// urls.
Is there a way for code in a class to find it's own manifest?
I tried the abovementioned items which seem to run well in small applications run from the java command line but then I'd like to have a portable solution as I cannot predict where my library would be used later.
public static Manifest getManifest() {
log.debug("getManifest()");
synchronized(Version.class) {
if(manifest==null) {
try {
// this works wrongly in JBoss
//ClassLoader cl = Version.class.getProtectionDomain().getClassLoader();
//log.debug("found classloader={}", cl);
//URL manifesturl = cl.getResource("/META-INF/MANIFEST.MF");
URL jar = Version.class.getProtectionDomain().getCodeSource().getLocation();
log.debug("Class loaded from {}", jar);
URL manifesturl = null;
switch(jar.getProtocol()) {
case "file":
manifesturl = new URL(jar.toString()+"META-INF/MANIFEST.MF");
break;
default:
manifesturl = new URL(jar.toString()+"!/META-INF/MANIFEST.MF");
}
log.debug("Expecting manifest at {}", manifesturl);
manifest = new Manifest(manifesturl.openStream());
}
catch(Exception e) {
log.info("Could not read version", e);
}
}
}
The code will detect the correct jar path. I assumed by modifying the url to point to the manifest would give the required result however I get this:
Class loaded from vfs:/C:/Users/user/Documents/JavaLibs/wildfly-18.0.0.Final/bin/content/webapp.war/WEB-INF/lib/library-1.0-18.jar
Expecting manifest at vfs:/C:/Users/user/Documents/JavaLibs/wildfly-18.0.0.Final/bin/content/webapp.war/WEB-INF/lib/library-1.0-18.jar!/META-INF/MANIFEST.MF
Could not read version: java.io.FileNotFoundException: C:\Users\hiran\Documents\JavaLibs\wildfly-18.0.0.Final\standalone\tmp\vfs\temp\tempfc75b13f07296e98\content-e4d5ca96cbe6b35e\WEB-INF\lib\library-1.0-18.jar!\META-INF\MANIFEST.MF (The system cannot find the path specified)
I checked that path and it seems even the first URL to the jar (obtained via Version.class.getProtectionDomain().getCodeSource().getLocation() ) was wrong already. It should have been C:\Users\user\Documents\JavaLibs\wildfly-18.0.0.Final\standalone\tmp\vfs\temp\tempfc75b13f07296e98\content-e4d5ca96cbe6b35e\WEB-INF\lib\library-1.0.18.jar.
So this could even point to a problem in Wildfly?
It seems I found some suitable solution here:
https://stackoverflow.com/a/37325538/4222206
So in the end this code can display the correct version of the jar (at least) in JBoss:
this.getClass().getPackage().getImplementationTitle();
this.getClass().getPackage().getImplementationVersion();
Hopefully I will find this answer when I search next time...

How to safely access the URLs of all resource files in the classpath in Java 9+?

We learned from the release notes of Java 9 that
The application class loader is no longer an instance of java.net.URLClassLoader (an implementation detail that was never specified in previous releases). Code that assumes that ClassLoader::getSytemClassLoader returns a URLClassLoader object will need to be updated.
This breaks old code, which scans the classpath as follows:
Java <= 8
URL[] ressources = ((URLClassLoader) classLoader).getURLs();
which runs into a
java.lang.ClassCastException:
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to
java.base/java.net.URLClassLoader
So for Java 9+ the following workaround was proposed as a PR at the Apache Ignite Project, which works as intended given adjustments in the JVM runtime options: --add-opens java.base/jdk.internal.loader=ALL-UNNAMED. However, as mentioned in the comments below, this PR was never merged into their Master branch.
/*
* Java 9 + Bridge to obtain URLs from classpath...
*/
private static URL[] getURLs(ClassLoader classLoader) {
URL[] urls = new URL[0];
try {
//see https://github.com/apache/ignite/pull/2970
Class builtinClazzLoader = Class.forName("jdk.internal.loader.BuiltinClassLoader");
if (builtinClazzLoader != null) {
Field ucpField = builtinClazzLoader.getDeclaredField("ucp");
ucpField.setAccessible(true);
Object ucpObject = ucpField.get(classLoader);
Class clazz = Class.forName("jdk.internal.loader.URLClassPath");
if (clazz != null && ucpObject != null) {
Method getURLs = clazz.getMethod("getURLs");
if (getURLs != null) {
urls = (URL[]) getURLs.invoke(ucpObject);
}
}
}
} catch (NoSuchMethodException | InvocationTargetException | NoSuchFieldException | IllegalAccessException | ClassNotFoundException e) {
logger.error("Could not obtain classpath URLs in Java 9+ - Exception was:");
logger.error(e.getLocalizedMessage(), e);
}
return urls;
}
However, this causes some severe headache due to the use of Reflection here. This is kind of an anti-pattern and is strictly criticized by the forbidden-apis maven plugin:
Forbidden method invocation: java.lang.reflect.AccessibleObject#setAccessible(boolean) [Reflection usage to work around access flags fails with SecurityManagers and likely will not work anymore on runtime classes in Java 9]
Question
Is there a safe way to access the list of all resource URLs in the class- / module path, which can be accessed by the given classloader, in OpenJDK 9/10 without using sun.misc.* imports (e.g. by using Unsafe)?
UPDATE (related to the comments)
I know, that I can do
String[] pathElements = System.getProperty("java.class.path").split(System.getProperty("path.separator"));
to obtain the elements in the classpath and then parse them to URLs. However - as far as I know - this property only returns the classpath given at the time of the application launch. However, in a container environment this will be the one of the application server and might not be sufficient, e.g. then using EAR bundles.
UPDATE 2
Thank your for all your comments. I will test, if System.getProperty("java.class.path") will work for our purposes and update the question, if this fullfills our needs.
However, it seems that other projects (maybe for other reasons, e.g Apache TomEE 8) suffer the same pain related to the URLClassLoader- for this reason, I think it is a valueable question.
UPDATE 3
Finally, we did switch to classgraph and migrated our code to this library to resolve our use-case to load ML resources bundled as JARs from the classpath.
I think this is an XY problem. Accessing the URLs of all resources on the classpath is not a supported operation in Java and is not a good thing to try to do. As you have already seen in this question, you will be fighting against the framework all the way if you try to do this. There will be a million edge cases that will break your solution (custom classloaders, EE containers, etc. etc.).
Please could you expand on why you want to do this?
If you have some kind of plugin system and are looking for modules that interface with your code which may have been provided at runtime, then you should use the ServiceLoader API, i.e.:
A service provider that is packaged as a JAR file for the class path is identified by placing a provider-configuration file in the resource directory META-INF/services. The name of the provider-configuration file is the fully qualified binary name of the service. The provider-configuration file contains a list of fully qualified binary names of service providers, one per line.
For example, suppose the service provider com.example.impl.StandardCodecs is packaged in a JAR file for the class path. The JAR file will contain a provider-configuration file named:
META-INF/services/com.example.CodecFactory
that contains the line:
com.example.impl.StandardCodecs # Standard codecs
AFAIK you can parse the java.class.path system property to get the urls:
String classpath = System.getProperty("java.class.path");
String[] entries = classpath.split(File.pathSeparator);
URL[] result = new URL[entries.length];
for(int i = 0; i < entries.length; i++) {
result[i] = Paths.get(entries[i]).toAbsolutePath().toUri().toURL();
}
System.out.println(Arrays.toString(result)); // e.g. [file:/J:/WS/Oxygen-Stable/jdk10/bin/]

Problem reloading a jar using URLClassLoader

I need to add plugin functionality to an existing application for certain parts of the application. I want to be able to add a jar at runtime and the application should be able to load a class from the jar without restarting the app. So far so good. I found some samples online using URLClassLoader and it works fine.
I also wanted the ability to reload the same class when an updated version of the jar is available. I again found some samples and the key to achieving this as I understand is that I need to use a new classloader instance for each new load.
I wrote some sample code but hit a NullPointerException. First let me show you guys the code:
package test.misc;
import java.io.File;
import java.net.URL;
import java.net.URLClassLoader;
import plugin.misc.IPlugin;
public class TestJarLoading {
public static void main(String[] args) {
IPlugin plugin = null;
while(true) {
try {
File file = new File("C:\\plugins\\test.jar");
String classToLoad = "jartest.TestPlugin";
URL jarUrl = new URL("jar", "","file:" + file.getAbsolutePath()+"!/");
URLClassLoader cl = new URLClassLoader(new URL[] {jarUrl}, TestJarLoading.class.getClassLoader());
Class loadedClass = cl.loadClass(classToLoad);
plugin = (IPlugin) loadedClass.newInstance();
plugin.doProc();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
Thread.sleep(30000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
IPlugin is a simple interface with just one method doProc:
public interface IPlugin {
void doProc();
}
and jartest.TestPlugin is an implementation of this interface where doProc just prints out some statements.
Now, I package the jartest.TestPlugin class into a jar called test.jar and place it under C:\plugins and run this code. The first iteration runs smoothly and the class loads without issues.
When the program is executing the sleep statement, I replace C:\plugins\test.jar with a new jar containing an updated version of the same class and wait for the next iteration of while. Now here's what I don't understand. Sometimes the updated class gets reloaded without issues i.e. the next iteration runs fine. But sometimes, I see an exception thrown:
java.lang.NullPointerException
at java.io.FilterInputStream.close(FilterInputStream.java:155)
at sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream.close(JarURLConnection.java:90)
at sun.misc.Resource.getBytes(Resource.java:137)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:256)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at test.misc.TestJarLoading.main(TestJarLoading.java:22)
I have searched on the net and scratched my head but can't really arrive at any conclusion as to why this exception is thrown and that too - only sometimes, not always.
I need your experience and expertise to understand this. What's wrong with this code? Please help!!
Let me know if you need any more info. Thanks for looking!
For everyone's benefit, let me summarize the real problem and the solution that worked for me.
As Ryan pointed out, there is a bug in JVM, which affects Windows Platform. URLClassLoader does not close the open jar files after it opens them for loading classes, effectively locking the jar files. The jar files can't be deleted or replaced.
The solution is simple: close the open jar files after they've been read. However, to get a handle to the open jar files, we need to use reflection since the properties we need to traverse down are not public. So we traverse down this path
URLClassLoader -> URLClassPath ucp -> ArrayList<Loader> loaders
JarLoader -> JarFile jar -> jar.close()
The code to close the open jar files can be added to a close() method in a class extending URLClassLoader:
public class MyURLClassLoader extends URLClassLoader {
public PluginClassLoader(URL[] urls, ClassLoader parent) {
super(urls, parent);
}
/**
* Closes all open jar files
*/
public void close() {
try {
Class clazz = java.net.URLClassLoader.class;
Field ucp = clazz.getDeclaredField("ucp");
ucp.setAccessible(true);
Object sunMiscURLClassPath = ucp.get(this);
Field loaders = sunMiscURLClassPath.getClass().getDeclaredField("loaders");
loaders.setAccessible(true);
Object collection = loaders.get(sunMiscURLClassPath);
for (Object sunMiscURLClassPathJarLoader : ((Collection) collection).toArray()) {
try {
Field loader = sunMiscURLClassPathJarLoader.getClass().getDeclaredField("jar");
loader.setAccessible(true);
Object jarFile = loader.get(sunMiscURLClassPathJarLoader);
((JarFile) jarFile).close();
} catch (Throwable t) {
// if we got this far, this is probably not a JAR loader so skip it
}
}
} catch (Throwable t) {
// probably not a SUN VM
}
return;
}
}
(This code was taken from the second link that Ryan posted. This code is also posted on the bug report page.)
However, there's a catch: For this code to work and be able to get a handle to the open jar files to close them, the loader used to load the classes from the file by URLClassLoader implementation has to be a JarLoader. Looking at the source code of URLClassPath (method getLoader(URL url)), I noticed that it uses a JARLoader only if the file string used to create the URL does not end in "/". So, the URL must be defined like this:
URL jarUrl = new URL("file:" + file.getAbsolutePath());
The overall class loading code should look something like this:
void loadAndInstantiate() {
MyURLClassLoader cl = null;
try {
File file = new File("C:\\jars\\sample.jar");
String classToLoad = "com.abc.ClassToLoad";
URL jarUrl = new URL("file:" + file.getAbsolutePath());
cl = new MyURLClassLoader(new URL[] {jarUrl}, getClass().getClassLoader());
Class loadedClass = cl.loadClass(classToLoad);
Object o = loadedClass.getConstructor().newInstance();
} finally {
if(cl != null)
cl.close();
}
}
Update: JRE 7 has introduced a close() method in the class URLClassLoader which may have solved this issue. I haven't verified it.
This behaviour is related to a bug in the jvm
2 workarounds are documented here
Starting from Java 7, you indeed have a close() method in URLClassLoader but it is not enough to release completely the jar files if you call directly or indirectly methods of type ClassLoader#getResource(String), ClassLoader#getResourceAsStream(String) or ClassLoader#getResources(String). Indeed by default, the JarFile instances are automatically stored into the cache of JarFileFactory in case we call directly or indirectly one of the previous methods and those instances are not released even if we call java.net.URLClassLoader#close().
So a hack is still needed in this particular case even with Java 1.8.0_74, here is my hack https://github.com/essobedo/application-manager/blob/master/src/main/java/com/github/essobedo/appma/core/util/Classpath.java#L83 that I use here https://github.com/essobedo/application-manager/blob/master/src/main/java/com/github/essobedo/appma/core/DefaultApplicationManager.java#L388. Even with this hack, I still had to call the GC explicitly to fully release the jar files as you can see here https://github.com/essobedo/application-manager/blob/master/src/main/java/com/github/essobedo/appma/core/DefaultApplicationManager.java#L419
This is an update tested on java 7 with success. Now the URLClassLoader works fine for me
MyReloader
class MyReloaderMain {
...
//assuming ___BASE_DIRECTORY__/lib for jar and ___BASE_DIRECTORY__/conf for configuration
String dirBase = ___BASE_DIRECTORY__;
File file = new File(dirBase, "lib");
String[] jars = file.list();
URL[] jarUrls = new URL[jars.length + 1];
int i = 0;
for (String jar : jars) {
File fileJar = new File(file, jar);
jarUrls[i++] = fileJar.toURI().toURL();
System.out.println(fileJar);
}
jarUrls[i] = new File(dirBase, "conf").toURI().toURL();
URLClassLoader classLoader = new URLClassLoader(jarUrls, MyReloaderMain.class.getClassLoader());
// this is required to load file (such as spring/context.xml) into the jar
Thread.currentThread().setContextClassLoader(classLoader);
Class classToLoad = Class.forName("my.app.Main", true, classLoader);
instance = classToLoad.newInstance();
Method method = classToLoad.getDeclaredMethod("start", args.getClass());
Object result = method.invoke(instance, args);
...
}
Close and Restart the ClassReloader
then update your jar and call
classLoader.close();
then you can restart the app with the new version.
Do not include your jar into your base class loader
Do not include your jar into your base class loader "MyReloaderMain.class.getClassLoader()" of the "MyReloaderMain", in other words develop 2 project with 2 jars one for "MyReloaderMain" and the other one for your real application without dependency between the two, or you will not able to understand who i loading what.
The error is still present in jdk1.8.0_25 on Windows. Although #Nicolas' answer helps, I hit a ClassNotFound for sun.net.www.protocol.jar.JarFileFactory when running it on WildFly, and several vm crashes while debugging some box tests...
Therefore I ended up extracting the part of the code which deals with loading and unloading, to an external jar. From the main code I just call this with java -jar.... all looks fine for now.
NOTE: Windows does release the locks on the loaded jar files when the jvm exits, that is why this works.
In principle, a class that has already been loaded cannot be reloaded with the same classloader.
For a new load, it is necessary to create a new classloader and thus load the class.
Using URLClassLoader has one problem and that is that the jar file remains open.
If you have multiple classes loaded from one jar file by different instances of URLClassLoader and you change the jar file at runtime, you will usually get this error: java.util.zip.ZipException: ZipFile invalid LOC header (bad signature). The error may be different.
In order for the above errors not to occur, it is necessary to use the close method on all URLClassLoaders using the given jar file. But this is a solution that actually leads to a restart of the entire application.
A better solution is to modify the URLClassLoader so that the contents of the jar file are loaded into the RAM cache. This no longer affects other URLClassloaders that read data from the same jar file. The jar file can then be freely changed while the application is running. For example, you can use this modification of URLClassLoader for this purpose: in-memory URLClassLoader

Launching Java Subprocess using parent process Classpath

I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process. The following is not enough, because it doesn't include any dynamically loaded classes:
String classpath = System.getProperty("java.class.path");
Currently I'm searching for each needed class with the code below. However, on some machines this fails for some classes/libs, the source variable is null. Is there a more reliable and simpler way to get the location of libs that are used by the current jvm process?
String stax = ClassFinder.classPath("javax.xml.stream.Location");
public static String classPath(String qualifiedClassName) throws NotFoundException {
try {
Class qc = Class.forName( qualifiedClassName );
CodeSource source = qc.getProtectionDomain().getCodeSource();
if ( source != null ) {
URL location = source.getLocation();
String f = location.getPath();
f = URLDecoder.decode(f, "UTF-8"); // decode URL to avoid spaces being replaced by %20
return f.substring(1);
} else {
throw new ClassFinder().new NotFoundException(qualifiedClassName+" (unknown source, likely rt.jar)");
}
} catch ( Exception e ) {
throw new ClassFinder().new NotFoundException(qualifiedClassName);
}
}
See my previous question which covers getting the classpath as well as how to launch a sub-process.
I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process.
You mean invoke a new JVM?
Given that...
it is possible to plug in all sorts of agents and instrumentation into a JVM that can transform classes at load time
it is possible to take a byte array and turn it into a class
it is possible to have complex class loader hierarchies with varying visibility between classes and have the same classes loaded multiple times
...there is no general, magic, catch-all and foolproof way to do this. You should design your application and its class loading mechanisms to achieve this goal. If you allow 3rd party plug-ins, you'll have to document how this works and how they have to register their libraries.
If you look at the javadoc for Class.getClassLoader, you'll see that the "bootstrap" classloader is typically represented as the null. "String.class.getClassLoader()" will return null on the normal sun jvm implementations. i think this implementation detail carries over into the CodeSource stuff. As such, I wouldn't imagine you would need to worry about any class which comes from the bootstrap classloader as long as your sub-process uses the same jvm impl as the current process.

Categories

Resources