I'm dynamically loading a class and calling a method on it. This class does JNI. When I call the class, java attempts to load the library. This causes an error because the library is not on the libpath. I'm calling from instead a jar so I can't easily change the libpath (especially since the library is not in the same directory or a sub directory of the jar). I do know the path of the library, but how can I load it before I load the class.
Current code:
public Class<?> loadClass(String name) throws ClassNotFoundException {
if(!CLASS_NAME.equals(name))
return super.loadClass(name);
try {
URL myUrl = new URL(classFileUrl);
URLConnection connection = myUrl.openConnection();
InputStream input = connection.getInputStream();
byte[] classData = readConnectionToArray(input);
return defineClass(CLASS_NAME,
classData, 0, classData.length);
} catch (MalformedURLException e) {
throw new UndeclaredThrowableException(e);
} catch (IOException e) {
throw new UndeclaredThrowableException(e);
}
}
Exception:
Can't find library libvcommon.so
java.lang.UnsatisfiedLinkError: vcommon (A file or directory in the path name does not exist.)
at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:998)
at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:962)
at java.lang.System.loadLibrary(System.java:465)
at vcommon.(vcommon.java:103)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:200)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:599)
at com.fortune500.fin.v.vunit.reflection.ReflectionvProcessor.calculateV(ReflectionvProcessor.java:36)
at com.fortune500.fin.v.vunit.UTLTestCase.execute(UTLTestCase.java:42)
at com.fortune500.fin.v.vunit.TestSuite.execute(TestSuite.java:15)
at com.fortune500.fin.v.vunit.batch.Testvendor.execute(Testvendor.java:101)
at com.fortune500.fin.v.vunit.batch.Testvendor.main(Testvendor.java:58)
Edit: I'm having a 64bit vs 32bit issue right now. I'll come back to this when I've sorted that out.
Related: Dynamic loading a class in java with a different package name
If you know the path to the library, you could add the path to the java.library.path environment variable in your custom class loader. A simpler approach is to compute the path and use it in your call to Runtime.loadLibrary.
The code below outlines two approaches, using loadLibrary and setting the java.library.path system property.
if(CLASS_NAME.equals(name)) {
// two ways of doing this - either load the library explicitly from the full path
if (useFullPath) {
Runtime.getRuntime().loadLibrary("/full/path/to/mylibrary");
}
else { // or tweaking the library path
System.setProperty("java.library.path",
System.getProperty("java.library.path")
+ System.getProperty("file.separator")
+ "/path/to/lib");
}
}
return super.loadClass(name);
You mention your code is being called from a jar - using a custom class loader is going to be difficult if the ClassLoader is part of the jar as well. Have you verified that your class loader is indeed being used?
A simpler approach is to change the current call to loadLibrary in your native class to use the full path. E.g. fetch from system properties, or compute it, if you know in advance where to find it. This is only an option of course if you have the source to the native class. If you can't modify the native class, then use the loadLibrary call in the classloader.
It's my understanding that calls to load library with the same library name (regardless of path) load the same library. (At least, that's the behaviour on Windows - I haven't verified on Linux.) So, even though the classloader loads the library using the full path, and the native class loads the library using it's simple name, both should resolve to the same library.
(Just for completeness, resolving equivalent libraries happens in the kernel, again, speaking from Win32 experience. Each library internally has a name, and windows only loads one instance of the library with the same internal name per process.)
Related
I am working with a Java library that has some nested JAR files in lib package.
I have 2 issues:
I cannot see referenced types in my IDE (I am using JetBrains IntelliJ)
Of course I get class not defined at runtime
I understand that I have to create and use a custom ClassLoader, will it solve both problems?
Is this the recommended way of achieving this result?
The JAR file is an Italian government provided library and I cannot modify it as it will be periodically updated as the regulation changes.
Yes, as far as I know, the standard ClassLoaders do not support nested JARs. Which is sad, since it would be a really nice idea, but Oracle just doesn't give a damn about it. Here is a 18-year old ticket:
https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4735639
If you are getting those JARs from somebody else, the best thing would be to contact the vendor and ask them for a delivery in standards-compatible format. From your answer I realize that this might be difficult to achieve, but I would still try to talk to them, because it's the right thing to do. I'm pretty sure that everybody else in your position has the same issue. According to industry standards, such situation would usually hint your vendor into using Maven repository for their deliverables.
If talking to your vendor fails, you can re-pack the JARs as you get them. I would recommend writing an automated script for that and making sure it gets run on each delivery. You can either put all .class files into one uber-JAR, or just move the nested JARs outside the enclosing JAR. Caveat 1: there can be more than one class with the same name, so you need to make sure to take the correct one. Caveat 2: if the JARs were signed, you will lose the signature (unless you sign them with your own).
Option 3: you can always implement your own ClassLoader to load the classes from anywhere, even from the kitchen sink.
This guy did exactly this: https://www.ibm.com/developerworks/library/j-onejar/index.html
The short summary is that such a ClassLoader has to perform recursive unzipping, which is a bit of a pain-in-the-ass because archives are essentially made for stream access and not for random access, but apart from that it's perfectly doable.
You can use his solution as a "wrapper loader" which will replace your main class.
As far as IntelliJ IDEA goes, I don't believe it supports this functionality out-of-the box. The best thing would be either to re-package JARs as described above and add them as separate classpath entries, or to search if anybody has written a plugin for nested JAR support.
I don't know what you want to do after load jars.
In my case, use jar dynamic loading for Servlet samples.
try{
final URLClassLoader loader = (URLClassLoader)ClassLoader.getSystemClassLoader();
final Method method = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
method.setAccessible(true);
new File(dir).listFiles(new FileFilter() {
#Override
public boolean accept(File jar) {
// load file if it is 'jar' type
if( jar.toString().toLowerCase().contains(".jar") ){
try {
method.invoke(loader, new Object[]{jar.toURI().toURL()});
XMLog.info_arr(logger, jar, " is loaded.");
JarInputStream jarFile = new JarInputStream(new FileInputStream(jar));
JarEntry jarEntry;
while (true) {
// load jar file
jarEntry = jarFile.getNextJarEntry();
if (jarEntry == null) {
break;
}
// load .class file in loaded jar file
if (jarEntry.getName().endsWith(".class")) {
Class loadedClass = Class.forName(jarEntry.getName().replaceAll("/", "\\.").replace(".class",""));
/*
* In my case, I load jar file for Servlet.
* If you want to use it for other case, then change below codes
*/
WebServlet annotaions = (WebServlet) loadedClass.getAnnotation(WebServlet.class);
// load annotation and mapping if it is Servlet
if (annotaions.urlPatterns().length > 0) {
ServletRegistration.Dynamic registration = servletContextEvent.getServletContext().addServlet(annotaions.urlPatterns().toString(), loadedClass);
registration.addMapping(annotaions.urlPatterns());
}
}
}
} catch (Exception e) {
System.err.println("Can't load classes in jar");
}
}
return false;
}
});
} catch(Exception e) {
throw new RuntimeException(e);
}
Interestingly I just solved a version of this problem for JesterJ, though I had the additional requirement of loading dependencies for the code in the jar file as well. JesterJ (as of this evening's commits!) runs from a fat jar and accepts an argument denoting a second fat jar containing the classes, dependencies and configuration for a document ingestion plan (the user's code that I need to run).
The way my solution works is I borrow the knowledge of how to load jars inside of jars from Uno-Jar (the library that produces the fat jar), and stuff my own classloader in above it to control the evaluation order of the class loaders.
The key bit from https://github.com/nsoft/jesterj/blob/jdk11/code/ingest/src/main/java/org/jesterj/ingest/Main.java looks like this:
JesterJLoader jesterJLoader;
File jarfile = new File(javaConfig);
URL planConfigJarURL;
try {
planConfigJarURL = jarfile.toURI().toURL();
} catch (MalformedURLException e) {
throw new RuntimeException(e); // boom
}
jesterJLoader = (JesterJLoader) ClassLoader.getSystemClassLoader();
ClassLoader loader;
if (isUnoJar) {
JarClassLoader jarClassLoader = new JarClassLoader(jesterJLoader, planConfigJarURL.toString());
jarClassLoader.load(null);
loader = jarClassLoader;
} else {
loader = new URLClassLoader(new URL[]{planConfigJarURL}, jesterJLoader);
}
jesterJLoader.addExtLoader(loader);
My JesterJLoader is here:
https://github.com/nsoft/jesterj/blob/jdk11/code/ingest/src/main/java/org/jesterj/ingest/utils/JesterJLoader.java
Though if you are happy to simply delegate up and rely on existing classes on the main class path (rather than loading additional dependencies from the sub-fat-jar like I'm doing) yours could be much simpler. I go to a lot of effort to allow it to check the sub-jar first rather than delegating up to the parent immediately, and then have to keep track of what's been sent to the sub-jar to avoid loops and subsequent StackOverflowError...
Also note that the line where I get the system class loader is going to NOT be what you want, I'm also monkeying with the system loader to work around impolite things that some of my dependencies are doing with class loading.
If you decide to try to check out Uno-Jar pls note that resource loading for this nested scenario may yet be wonky and things definitely won't work before https://github.com/nsoft/uno-jar/commit/cf5af42c447c22edb9bbc6bd08293f0c23db86c2
Also: recently committed thinly tested code warning :)
Disclosure: I maintain both JesterJ and Uno-Jar (a fork of One-JAR the library featured in the link supplied by jurez) and welcome any bug reports or comments or even contributions!
I'm tackling to use kyotocabinet in Playframework.
and following error occurred.
I'm using Eclipse and playframework-1.2.3.
and kyotocabinet is native library so I'm using its Java-Binding.
the reproduce code is simple.
in controller:
public static void somePage() {
DB db = new DB();//error occurred
render();
}
Internal Server Error (500) for request GET /
Execution exception (In /app/controllers/TestApp.java around line 45)
NoClassDefFoundError occured : Could not initialize class kyotocabinet.DB
play.exceptions.JavaExecutionException: Could not initialize class kyotocabinet.DB
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:229)
at Invocation.HTTP Request(Play!)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class kyotocabinet.DB
at controllers.TestApp.somePage(TestApp.java:45)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:546)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:500)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:476)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:471)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:159)
... 1 more
build in Eclipse was completed but error occured at runtime.
I guess it is because kyotocabinet.dll is missing.(only jkyotocabinet.jar was found)
I configured the location of kyotocabinet.dll to Java Build Path > Source > Native Library Location of my playframework project.
and it was good in other projects.
How to use native library in playframework?
any example or tutorial?
Play.getFile and System.load didn't work.
package controllers;
import play.Play;
import play.jobs.*;
#OnApplicationStart
public class Bootstrap extends Job {
public void doJob() {
String path = "D:/MyProject/lib/jkyotocabinet.dll";
Play.getFile(path);
//System.load(path); if this was enabled, following error occurred: "Native Library D:\MyProject\lib\jkyotocabinet.dll already loaded in another classloader". so I guess the dll was loaded.
System.out.println("bootstrap loaded");//this is displayed.
}
}
UnsatisfiedLinkError occured : no jkyotocabinet in java.library.path
this Japanese blog tells Play!Framework cannot load native library.
http://d.hatena.ne.jp/hjym_u/20110702/1309609277
I already tried these: Absolute path, Relative path, System.load, System.loadLibrary, Play.getFile.
as conclusive approach, I put jkyotocabinet.dll to current directory(D:/MyProejct/), and wrote this code.
public static void somePage(){
File f = Play.getFile("jkyotocabinet.dll");
if(f != null && f.isFile() && f.canRead() && f.canExecute()){//true
DB db = new DB();//error occured. it reached here.
}
render();
}
Execution exception
NoClassDefFoundError occured : Could not initialize class kyotocabinet.DB
Play.getFile found the path "jkyotocabinet.dll" so jkyotocabinet.dll is in current directory so jvm should find it automatically.
anyone can use JNI in playframework?
finally, I could use kyotocabinet as PROD mode but not DEV mode.
Project/conf/application.conf
#application.mode=dev
application.mode=prod
I assume you just need to load the dll into Java via System.load
If you place the ddl on your project, you may load it via Play.getFile inside your #OnApplicationStart controller. This should make it available to your application while the JVM is alive.
EDIT:
#KenichiYamamoto Play.getFile gets files from the application path. You are trying to use a full path in there.
Read this about loading the file in a container. It may be that (due to Play compile-reload system) you are hitting the "already loaded" error. Try to follow the example by adding the System.load inside a static block in your #OnApplicationStart
Do as pere says but use the relative path from your application root. Not the absolute path. I.e. Play.getFile("lib\myfile.dll")
Consider this code (based entirely on flying saucer's "getting started" code, their rights reserved):
package flyingsaucerpdf;
import java.io.File;
import java.io.FileOutputStream;
import java.io.OutputStream;
import org.xhtmlrenderer.pdf.ITextRenderer;
public class PDFMaker {
public static void main(String[] args) throws Exception {
new PDFMaker().go();
}
public void go() throws Exception {
String inputFile = "sample.html";
String url = new File(inputFile).toURI().toURL().toString();
String outputFile = "firstdoc.pdf";
OutputStream os = new FileOutputStream(outputFile);
ITextRenderer renderer = new ITextRenderer();
renderer.setDocument(url);
renderer.layout();
renderer.createPDF(os);
os.close();
}
}
Few facts:
Running it standalone (calling main) with JDK 1.6 or 1.5 works perfectly (PDF is generated)
But when loaded via a URLClassLoader from an existing web application it fails with this error:
Caused by: org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or change an object in a way which is incorrect with regard to namespaces.
at org.apache.xerces.dom.AttrNSImpl.setName(Unknown Source)
at org.apache.xerces.dom.AttrNSImpl.(Unknown Source)
at org.apache.xerces.dom.CoreDocumentImpl.createAttributeNS(Unknown Source)
at org.apache.xerces.dom.ElementImpl.setAttributeNS(Unknown Source)
at org.apache.xml.utils.DOMBuilder.startElement(DOMBuilder.java:307)
... 19 more
After looking for a while in the wrong place (for example, I created a child-first / parent-last class loader suspecting xalan / xerces jars, but it still fails), I finally narrowed down the root cause:
It seems that the web application that loads my code, has an old xalan.jar, specification version 1.2
I did a little test, I ran the code above as standalone (which worked fine before) but this time I added the xalan.jar from the web app to it's classpath, and bingo, the same error as in the web app scenario
So I inspected that old xalan.jar and wondered, what can cause the JVM to load it's old xalan implementation instead of the JDK's? after all my child-first class loader is also parent-last e.g. system in the middle, to say: searching the system classloader before the parent (to avoid loading parent overriden JDK jars, just like this case of the parent's xalan.jar overriding the JDK's xalan implementation)
Then something cought my eyes - a file in: xalan.jar/META-INF/services/ named javax.xml.transform.TransformerFactory with this content:
org.apache.xalan.processor.TransformerFactoryImpl
So I imediately pressed Ctrl+T in eclipse and looked for the full qualified name... only in xalan.jar!
Then I searched only for "TransformerFactoryImpl", and this is what the JDK has:
com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl
Easy to see the difference
So, if you read till here, my bottom line question is: How do I make my TransformerFactory use the JDK's implementation and not the old Xalan's one? (I can't remove that jar from the web app my code will be loaded from)
It seems the answer is simpler than I thought.
In your classloader, add to the classpath (jar not required) this folder: /META-INF/services/
In it, create a file named javax.xml.transform.TransformerFactory
Edit it and set this as it's content to: com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl
Thats it!
Why does it work? see this class used by Java for loading the Xalan implementation.
Notice that it seems de-facto a parent-last (or child-first) loader for that specific "META-INF" entry (the oposite of how regular Java class loaders work, e.g. parent-first / child-last) but feel free to correct me if I'm wrong
Snippet from javax.xml.datatype.FactoryFinder
/*
* Try to find provider using Jar Service Provider Mechanism
*
* #return instance of provider class if found or null
*/
private static Object findJarServiceProvider(String factoryId)
throws ConfigurationError
{
String serviceId = "META-INF/services/" + factoryId;
InputStream is = null;
// First try the Context ClassLoader
ClassLoader cl = ss.getContextClassLoader();
if (cl != null) {
is = ss.getResourceAsStream(cl, serviceId);
// If no provider found then try the current ClassLoader
if (is == null) {
cl = FactoryFinder.class.getClassLoader();
is = ss.getResourceAsStream(cl, serviceId);
}
} else {
// No Context ClassLoader, try the current
// ClassLoader
cl = FactoryFinder.class.getClassLoader();
is = ss.getResourceAsStream(cl, serviceId);
}
if (is == null) {
// No provider found
return null;
}
...
You should also note that you aren't required to use the SPI mechanism at all.
You can use http://download.oracle.com/javase/6/docs/api/javax/xml/xpath/XPathFactory.html#newInstance(java.lang.String, java.lang.String, java.lang.ClassLoader) to use a copy of xalan in your own isolated class loader once you know the name of the relevant class.
If you are developing a web application and thus can't set a system property, then the most direct way is to explicitly request the JDK transformer.
Here is an example for the internal XSLTC transformer (has StAXSupport).
TransformerFactory tf = TransformerFactory.newInstance(
"com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl", null);
I'm using wildfly application server and 3rd jar which uses TransformerFactory.
Overriding TransformerFactory using file resources\META-INF\services\javax.xml.transform.TransformerFactory. Didn't work for me.
When I looked to FactoryFinder implementation (JDK8u201). I found the following code fragment
String systemProp = ss.getSystemProperty(factoryId);
if (systemProp != null) {
dPrint("found system property, value=" + systemProp);
return newInstance(type, systemProp, null, true);
}
Thus, solution was to set system property javax.xml.transform.TransformerFactory to com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.
Cheers
I need to add plugin functionality to an existing application for certain parts of the application. I want to be able to add a jar at runtime and the application should be able to load a class from the jar without restarting the app. So far so good. I found some samples online using URLClassLoader and it works fine.
I also wanted the ability to reload the same class when an updated version of the jar is available. I again found some samples and the key to achieving this as I understand is that I need to use a new classloader instance for each new load.
I wrote some sample code but hit a NullPointerException. First let me show you guys the code:
package test.misc;
import java.io.File;
import java.net.URL;
import java.net.URLClassLoader;
import plugin.misc.IPlugin;
public class TestJarLoading {
public static void main(String[] args) {
IPlugin plugin = null;
while(true) {
try {
File file = new File("C:\\plugins\\test.jar");
String classToLoad = "jartest.TestPlugin";
URL jarUrl = new URL("jar", "","file:" + file.getAbsolutePath()+"!/");
URLClassLoader cl = new URLClassLoader(new URL[] {jarUrl}, TestJarLoading.class.getClassLoader());
Class loadedClass = cl.loadClass(classToLoad);
plugin = (IPlugin) loadedClass.newInstance();
plugin.doProc();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
Thread.sleep(30000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
IPlugin is a simple interface with just one method doProc:
public interface IPlugin {
void doProc();
}
and jartest.TestPlugin is an implementation of this interface where doProc just prints out some statements.
Now, I package the jartest.TestPlugin class into a jar called test.jar and place it under C:\plugins and run this code. The first iteration runs smoothly and the class loads without issues.
When the program is executing the sleep statement, I replace C:\plugins\test.jar with a new jar containing an updated version of the same class and wait for the next iteration of while. Now here's what I don't understand. Sometimes the updated class gets reloaded without issues i.e. the next iteration runs fine. But sometimes, I see an exception thrown:
java.lang.NullPointerException
at java.io.FilterInputStream.close(FilterInputStream.java:155)
at sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream.close(JarURLConnection.java:90)
at sun.misc.Resource.getBytes(Resource.java:137)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:256)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at test.misc.TestJarLoading.main(TestJarLoading.java:22)
I have searched on the net and scratched my head but can't really arrive at any conclusion as to why this exception is thrown and that too - only sometimes, not always.
I need your experience and expertise to understand this. What's wrong with this code? Please help!!
Let me know if you need any more info. Thanks for looking!
For everyone's benefit, let me summarize the real problem and the solution that worked for me.
As Ryan pointed out, there is a bug in JVM, which affects Windows Platform. URLClassLoader does not close the open jar files after it opens them for loading classes, effectively locking the jar files. The jar files can't be deleted or replaced.
The solution is simple: close the open jar files after they've been read. However, to get a handle to the open jar files, we need to use reflection since the properties we need to traverse down are not public. So we traverse down this path
URLClassLoader -> URLClassPath ucp -> ArrayList<Loader> loaders
JarLoader -> JarFile jar -> jar.close()
The code to close the open jar files can be added to a close() method in a class extending URLClassLoader:
public class MyURLClassLoader extends URLClassLoader {
public PluginClassLoader(URL[] urls, ClassLoader parent) {
super(urls, parent);
}
/**
* Closes all open jar files
*/
public void close() {
try {
Class clazz = java.net.URLClassLoader.class;
Field ucp = clazz.getDeclaredField("ucp");
ucp.setAccessible(true);
Object sunMiscURLClassPath = ucp.get(this);
Field loaders = sunMiscURLClassPath.getClass().getDeclaredField("loaders");
loaders.setAccessible(true);
Object collection = loaders.get(sunMiscURLClassPath);
for (Object sunMiscURLClassPathJarLoader : ((Collection) collection).toArray()) {
try {
Field loader = sunMiscURLClassPathJarLoader.getClass().getDeclaredField("jar");
loader.setAccessible(true);
Object jarFile = loader.get(sunMiscURLClassPathJarLoader);
((JarFile) jarFile).close();
} catch (Throwable t) {
// if we got this far, this is probably not a JAR loader so skip it
}
}
} catch (Throwable t) {
// probably not a SUN VM
}
return;
}
}
(This code was taken from the second link that Ryan posted. This code is also posted on the bug report page.)
However, there's a catch: For this code to work and be able to get a handle to the open jar files to close them, the loader used to load the classes from the file by URLClassLoader implementation has to be a JarLoader. Looking at the source code of URLClassPath (method getLoader(URL url)), I noticed that it uses a JARLoader only if the file string used to create the URL does not end in "/". So, the URL must be defined like this:
URL jarUrl = new URL("file:" + file.getAbsolutePath());
The overall class loading code should look something like this:
void loadAndInstantiate() {
MyURLClassLoader cl = null;
try {
File file = new File("C:\\jars\\sample.jar");
String classToLoad = "com.abc.ClassToLoad";
URL jarUrl = new URL("file:" + file.getAbsolutePath());
cl = new MyURLClassLoader(new URL[] {jarUrl}, getClass().getClassLoader());
Class loadedClass = cl.loadClass(classToLoad);
Object o = loadedClass.getConstructor().newInstance();
} finally {
if(cl != null)
cl.close();
}
}
Update: JRE 7 has introduced a close() method in the class URLClassLoader which may have solved this issue. I haven't verified it.
This behaviour is related to a bug in the jvm
2 workarounds are documented here
Starting from Java 7, you indeed have a close() method in URLClassLoader but it is not enough to release completely the jar files if you call directly or indirectly methods of type ClassLoader#getResource(String), ClassLoader#getResourceAsStream(String) or ClassLoader#getResources(String). Indeed by default, the JarFile instances are automatically stored into the cache of JarFileFactory in case we call directly or indirectly one of the previous methods and those instances are not released even if we call java.net.URLClassLoader#close().
So a hack is still needed in this particular case even with Java 1.8.0_74, here is my hack https://github.com/essobedo/application-manager/blob/master/src/main/java/com/github/essobedo/appma/core/util/Classpath.java#L83 that I use here https://github.com/essobedo/application-manager/blob/master/src/main/java/com/github/essobedo/appma/core/DefaultApplicationManager.java#L388. Even with this hack, I still had to call the GC explicitly to fully release the jar files as you can see here https://github.com/essobedo/application-manager/blob/master/src/main/java/com/github/essobedo/appma/core/DefaultApplicationManager.java#L419
This is an update tested on java 7 with success. Now the URLClassLoader works fine for me
MyReloader
class MyReloaderMain {
...
//assuming ___BASE_DIRECTORY__/lib for jar and ___BASE_DIRECTORY__/conf for configuration
String dirBase = ___BASE_DIRECTORY__;
File file = new File(dirBase, "lib");
String[] jars = file.list();
URL[] jarUrls = new URL[jars.length + 1];
int i = 0;
for (String jar : jars) {
File fileJar = new File(file, jar);
jarUrls[i++] = fileJar.toURI().toURL();
System.out.println(fileJar);
}
jarUrls[i] = new File(dirBase, "conf").toURI().toURL();
URLClassLoader classLoader = new URLClassLoader(jarUrls, MyReloaderMain.class.getClassLoader());
// this is required to load file (such as spring/context.xml) into the jar
Thread.currentThread().setContextClassLoader(classLoader);
Class classToLoad = Class.forName("my.app.Main", true, classLoader);
instance = classToLoad.newInstance();
Method method = classToLoad.getDeclaredMethod("start", args.getClass());
Object result = method.invoke(instance, args);
...
}
Close and Restart the ClassReloader
then update your jar and call
classLoader.close();
then you can restart the app with the new version.
Do not include your jar into your base class loader
Do not include your jar into your base class loader "MyReloaderMain.class.getClassLoader()" of the "MyReloaderMain", in other words develop 2 project with 2 jars one for "MyReloaderMain" and the other one for your real application without dependency between the two, or you will not able to understand who i loading what.
The error is still present in jdk1.8.0_25 on Windows. Although #Nicolas' answer helps, I hit a ClassNotFound for sun.net.www.protocol.jar.JarFileFactory when running it on WildFly, and several vm crashes while debugging some box tests...
Therefore I ended up extracting the part of the code which deals with loading and unloading, to an external jar. From the main code I just call this with java -jar.... all looks fine for now.
NOTE: Windows does release the locks on the loaded jar files when the jvm exits, that is why this works.
In principle, a class that has already been loaded cannot be reloaded with the same classloader.
For a new load, it is necessary to create a new classloader and thus load the class.
Using URLClassLoader has one problem and that is that the jar file remains open.
If you have multiple classes loaded from one jar file by different instances of URLClassLoader and you change the jar file at runtime, you will usually get this error: java.util.zip.ZipException: ZipFile invalid LOC header (bad signature). The error may be different.
In order for the above errors not to occur, it is necessary to use the close method on all URLClassLoaders using the given jar file. But this is a solution that actually leads to a restart of the entire application.
A better solution is to modify the URLClassLoader so that the contents of the jar file are loaded into the RAM cache. This no longer affects other URLClassloaders that read data from the same jar file. The jar file can then be freely changed while the application is running. For example, you can use this modification of URLClassLoader for this purpose: in-memory URLClassLoader
I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process. The following is not enough, because it doesn't include any dynamically loaded classes:
String classpath = System.getProperty("java.class.path");
Currently I'm searching for each needed class with the code below. However, on some machines this fails for some classes/libs, the source variable is null. Is there a more reliable and simpler way to get the location of libs that are used by the current jvm process?
String stax = ClassFinder.classPath("javax.xml.stream.Location");
public static String classPath(String qualifiedClassName) throws NotFoundException {
try {
Class qc = Class.forName( qualifiedClassName );
CodeSource source = qc.getProtectionDomain().getCodeSource();
if ( source != null ) {
URL location = source.getLocation();
String f = location.getPath();
f = URLDecoder.decode(f, "UTF-8"); // decode URL to avoid spaces being replaced by %20
return f.substring(1);
} else {
throw new ClassFinder().new NotFoundException(qualifiedClassName+" (unknown source, likely rt.jar)");
}
} catch ( Exception e ) {
throw new ClassFinder().new NotFoundException(qualifiedClassName);
}
}
See my previous question which covers getting the classpath as well as how to launch a sub-process.
I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process.
You mean invoke a new JVM?
Given that...
it is possible to plug in all sorts of agents and instrumentation into a JVM that can transform classes at load time
it is possible to take a byte array and turn it into a class
it is possible to have complex class loader hierarchies with varying visibility between classes and have the same classes loaded multiple times
...there is no general, magic, catch-all and foolproof way to do this. You should design your application and its class loading mechanisms to achieve this goal. If you allow 3rd party plug-ins, you'll have to document how this works and how they have to register their libraries.
If you look at the javadoc for Class.getClassLoader, you'll see that the "bootstrap" classloader is typically represented as the null. "String.class.getClassLoader()" will return null on the normal sun jvm implementations. i think this implementation detail carries over into the CodeSource stuff. As such, I wouldn't imagine you would need to worry about any class which comes from the bootstrap classloader as long as your sub-process uses the same jvm impl as the current process.