I have the following code:
// Test TODO remove
try {
System.out.println(System.getProperties().getProperty("java.class.path"));
this.getClass().getClassLoader().loadClass("mypackage.MyClass");
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Now the output shows me, that the class is on the classpath, i.e.:
/...some/path.../workspace/project/target/test-classes:/rest/of/the/classpath
java.lang.ClassNotFoundException: mypackage.MyClass
...here be stacktrace...
I also made sure, that the class-file acutaly IS in the given location, i.e. this file exists:
/...some/path.../workspace/project/target/test-classes/mypackage/MyClass.class
Maybe the following is important: the shown code is executed in a javaagent attached to a jUnit test, that I start programtically (via Runtime.execute(...)) - so there is probably something beyond the obvious that can go wrong in the background... but still: if the classpath contains the folder with the class, how come it cannot be loaded?
The Java agent is loaded early in starting the JVM (for obvious reasons) and has its own "classpath", so it isn't actually loaded by the (historically named) system class loader. That is why you have a 'jarpath' as part of the command line argument.
SO you will need something like System.getSystemClassLoader, URLClassLoader.newInstance (with java.class.path) or Thread.getContextClassLoader depending upon your circumstances.
For all who are interested:
I have no idea what the problem was.
I fiddled a bit with it, and it turned out that the command string that was executed by Runtime.exec(...) worked well if executed in the shell.
I fiddled a bit more, but finally gave up searching for the "real" reason. Instead of
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(command);
I now use apache exec:
CommandLine commandLine = CommandLine.parse(command);
DefaultExecutor executor = new DefaultExecutor();
int exitValue = executor.execute(commandLine);
with the exact same command String, and all of a sudden it woks!
You assume that if the target bytecode remains at classpath corresponding class can be loaded by the current class's class loader. However, that is not the case if current class is loaded by tricky/buggy class loader.
I'd suggest to do the following:
Check the used class loader:
System.out.println(this.getClass().getClassLoader());
System.out.println(ClassLoader.getSystemClassLoader() == this.getClass().getClassLoader());
this.getClass().getClassLoader().loadClass("mypackage.MyClass");
Provide minimal but complete reproducible test-case that illustrates the problem
Related
I'm having problems using ProcessBuilder to run a class in my project.
My code:
public class Main {
public static void main(String[] args) {
try {
String pathToJar = Main.class.getProtectionDomain().getCodeSource()
.getLocation().toURI().getPath();
ArrayList<String> params = new ArrayList<String>();
params.add("javaw");
params.add("-classpath");
params.add(pathToJar);
params.add("Program");
ProcessBuilder pb = new ProcessBuilder(params);
Process process = pb.start();
} catch (Exception e) {
e.printStackTrace();
}
}
}
class Program is in same project (same bin folder) and works fine if ran directly but this way I get the error "Could not find the main class: Program". Where is the error in my code?
Thanks in advance.
[EDIT]
I came to the conclution that is some code on my Program class giving error. Basicly only runs with "clean" main. At eclipse, Program class is importing some libraries that are inside a jar file. Don't I need to reference it in ProcessBuilder? If so, how?
In response to your edit:
You can add the current path by switching params.add(pathToJar); with params.add(System.getProperty("java.class.path").concat(";").concat(pathToJar));.
Where is the error in my code?
(You are launching the javaw executable, so that is not the problem. It is also not that your entry point method's signature is incorrect, because that would have given a different diagnostic.)
The problem is either that the class name is incorrect (e.g. if should be "come.pkg.Program"), or the pathname for the JAR file is incorrect.
Assuming that you have eliminated the possibility that the class name is incorrect, my guess is that you are trying to use a relative pathname for the JAR file, but there is some confusion over what the current directory is; i.e. the directory in which the pathname needs to be resolved. Try using an absolute pathname in the classpath parameter.
I need to add plugin functionality to an existing application for certain parts of the application. I want to be able to add a jar at runtime and the application should be able to load a class from the jar without restarting the app. So far so good. I found some samples online using URLClassLoader and it works fine.
I also wanted the ability to reload the same class when an updated version of the jar is available. I again found some samples and the key to achieving this as I understand is that I need to use a new classloader instance for each new load.
I wrote some sample code but hit a NullPointerException. First let me show you guys the code:
package test.misc;
import java.io.File;
import java.net.URL;
import java.net.URLClassLoader;
import plugin.misc.IPlugin;
public class TestJarLoading {
public static void main(String[] args) {
IPlugin plugin = null;
while(true) {
try {
File file = new File("C:\\plugins\\test.jar");
String classToLoad = "jartest.TestPlugin";
URL jarUrl = new URL("jar", "","file:" + file.getAbsolutePath()+"!/");
URLClassLoader cl = new URLClassLoader(new URL[] {jarUrl}, TestJarLoading.class.getClassLoader());
Class loadedClass = cl.loadClass(classToLoad);
plugin = (IPlugin) loadedClass.newInstance();
plugin.doProc();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
Thread.sleep(30000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
IPlugin is a simple interface with just one method doProc:
public interface IPlugin {
void doProc();
}
and jartest.TestPlugin is an implementation of this interface where doProc just prints out some statements.
Now, I package the jartest.TestPlugin class into a jar called test.jar and place it under C:\plugins and run this code. The first iteration runs smoothly and the class loads without issues.
When the program is executing the sleep statement, I replace C:\plugins\test.jar with a new jar containing an updated version of the same class and wait for the next iteration of while. Now here's what I don't understand. Sometimes the updated class gets reloaded without issues i.e. the next iteration runs fine. But sometimes, I see an exception thrown:
java.lang.NullPointerException
at java.io.FilterInputStream.close(FilterInputStream.java:155)
at sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream.close(JarURLConnection.java:90)
at sun.misc.Resource.getBytes(Resource.java:137)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:256)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at test.misc.TestJarLoading.main(TestJarLoading.java:22)
I have searched on the net and scratched my head but can't really arrive at any conclusion as to why this exception is thrown and that too - only sometimes, not always.
I need your experience and expertise to understand this. What's wrong with this code? Please help!!
Let me know if you need any more info. Thanks for looking!
For everyone's benefit, let me summarize the real problem and the solution that worked for me.
As Ryan pointed out, there is a bug in JVM, which affects Windows Platform. URLClassLoader does not close the open jar files after it opens them for loading classes, effectively locking the jar files. The jar files can't be deleted or replaced.
The solution is simple: close the open jar files after they've been read. However, to get a handle to the open jar files, we need to use reflection since the properties we need to traverse down are not public. So we traverse down this path
URLClassLoader -> URLClassPath ucp -> ArrayList<Loader> loaders
JarLoader -> JarFile jar -> jar.close()
The code to close the open jar files can be added to a close() method in a class extending URLClassLoader:
public class MyURLClassLoader extends URLClassLoader {
public PluginClassLoader(URL[] urls, ClassLoader parent) {
super(urls, parent);
}
/**
* Closes all open jar files
*/
public void close() {
try {
Class clazz = java.net.URLClassLoader.class;
Field ucp = clazz.getDeclaredField("ucp");
ucp.setAccessible(true);
Object sunMiscURLClassPath = ucp.get(this);
Field loaders = sunMiscURLClassPath.getClass().getDeclaredField("loaders");
loaders.setAccessible(true);
Object collection = loaders.get(sunMiscURLClassPath);
for (Object sunMiscURLClassPathJarLoader : ((Collection) collection).toArray()) {
try {
Field loader = sunMiscURLClassPathJarLoader.getClass().getDeclaredField("jar");
loader.setAccessible(true);
Object jarFile = loader.get(sunMiscURLClassPathJarLoader);
((JarFile) jarFile).close();
} catch (Throwable t) {
// if we got this far, this is probably not a JAR loader so skip it
}
}
} catch (Throwable t) {
// probably not a SUN VM
}
return;
}
}
(This code was taken from the second link that Ryan posted. This code is also posted on the bug report page.)
However, there's a catch: For this code to work and be able to get a handle to the open jar files to close them, the loader used to load the classes from the file by URLClassLoader implementation has to be a JarLoader. Looking at the source code of URLClassPath (method getLoader(URL url)), I noticed that it uses a JARLoader only if the file string used to create the URL does not end in "/". So, the URL must be defined like this:
URL jarUrl = new URL("file:" + file.getAbsolutePath());
The overall class loading code should look something like this:
void loadAndInstantiate() {
MyURLClassLoader cl = null;
try {
File file = new File("C:\\jars\\sample.jar");
String classToLoad = "com.abc.ClassToLoad";
URL jarUrl = new URL("file:" + file.getAbsolutePath());
cl = new MyURLClassLoader(new URL[] {jarUrl}, getClass().getClassLoader());
Class loadedClass = cl.loadClass(classToLoad);
Object o = loadedClass.getConstructor().newInstance();
} finally {
if(cl != null)
cl.close();
}
}
Update: JRE 7 has introduced a close() method in the class URLClassLoader which may have solved this issue. I haven't verified it.
This behaviour is related to a bug in the jvm
2 workarounds are documented here
Starting from Java 7, you indeed have a close() method in URLClassLoader but it is not enough to release completely the jar files if you call directly or indirectly methods of type ClassLoader#getResource(String), ClassLoader#getResourceAsStream(String) or ClassLoader#getResources(String). Indeed by default, the JarFile instances are automatically stored into the cache of JarFileFactory in case we call directly or indirectly one of the previous methods and those instances are not released even if we call java.net.URLClassLoader#close().
So a hack is still needed in this particular case even with Java 1.8.0_74, here is my hack https://github.com/essobedo/application-manager/blob/master/src/main/java/com/github/essobedo/appma/core/util/Classpath.java#L83 that I use here https://github.com/essobedo/application-manager/blob/master/src/main/java/com/github/essobedo/appma/core/DefaultApplicationManager.java#L388. Even with this hack, I still had to call the GC explicitly to fully release the jar files as you can see here https://github.com/essobedo/application-manager/blob/master/src/main/java/com/github/essobedo/appma/core/DefaultApplicationManager.java#L419
This is an update tested on java 7 with success. Now the URLClassLoader works fine for me
MyReloader
class MyReloaderMain {
...
//assuming ___BASE_DIRECTORY__/lib for jar and ___BASE_DIRECTORY__/conf for configuration
String dirBase = ___BASE_DIRECTORY__;
File file = new File(dirBase, "lib");
String[] jars = file.list();
URL[] jarUrls = new URL[jars.length + 1];
int i = 0;
for (String jar : jars) {
File fileJar = new File(file, jar);
jarUrls[i++] = fileJar.toURI().toURL();
System.out.println(fileJar);
}
jarUrls[i] = new File(dirBase, "conf").toURI().toURL();
URLClassLoader classLoader = new URLClassLoader(jarUrls, MyReloaderMain.class.getClassLoader());
// this is required to load file (such as spring/context.xml) into the jar
Thread.currentThread().setContextClassLoader(classLoader);
Class classToLoad = Class.forName("my.app.Main", true, classLoader);
instance = classToLoad.newInstance();
Method method = classToLoad.getDeclaredMethod("start", args.getClass());
Object result = method.invoke(instance, args);
...
}
Close and Restart the ClassReloader
then update your jar and call
classLoader.close();
then you can restart the app with the new version.
Do not include your jar into your base class loader
Do not include your jar into your base class loader "MyReloaderMain.class.getClassLoader()" of the "MyReloaderMain", in other words develop 2 project with 2 jars one for "MyReloaderMain" and the other one for your real application without dependency between the two, or you will not able to understand who i loading what.
The error is still present in jdk1.8.0_25 on Windows. Although #Nicolas' answer helps, I hit a ClassNotFound for sun.net.www.protocol.jar.JarFileFactory when running it on WildFly, and several vm crashes while debugging some box tests...
Therefore I ended up extracting the part of the code which deals with loading and unloading, to an external jar. From the main code I just call this with java -jar.... all looks fine for now.
NOTE: Windows does release the locks on the loaded jar files when the jvm exits, that is why this works.
In principle, a class that has already been loaded cannot be reloaded with the same classloader.
For a new load, it is necessary to create a new classloader and thus load the class.
Using URLClassLoader has one problem and that is that the jar file remains open.
If you have multiple classes loaded from one jar file by different instances of URLClassLoader and you change the jar file at runtime, you will usually get this error: java.util.zip.ZipException: ZipFile invalid LOC header (bad signature). The error may be different.
In order for the above errors not to occur, it is necessary to use the close method on all URLClassLoaders using the given jar file. But this is a solution that actually leads to a restart of the entire application.
A better solution is to modify the URLClassLoader so that the contents of the jar file are loaded into the RAM cache. This no longer affects other URLClassloaders that read data from the same jar file. The jar file can then be freely changed while the application is running. For example, you can use this modification of URLClassLoader for this purpose: in-memory URLClassLoader
Is there a way in using externally stored sourcecode and loading it into a Java programm, so that it can be used by it?
I would like to have a program that can be altered without editing the complete source code and that this is even possible without compiling this every time. Another advantage is, that I can change parts of the code like I want.
Of course I have to have interfaces so that it is possible to send data into this and getting it back into the fixed source program again.
And of course it should be faster than a pure interpreting system.
So is there a way in doing this like an additional compiling of these external source code parts and a start of the programm after this is done?
Thank you in advance, Andreas :)
You need the javax.tools API for this. Thus, you need to have at least the JDK installed to get it to work (and let your IDE point to it instead of the JRE). Here's a basic kickoff example (without proper exception and encoding handling just to make the basic example less opaque, cough):
public static void main(String... args) throws Exception {
String source = "public class Test { static { System.out.println(\"test\"); } }";
File root = new File("/test");
File sourceFile = new File(root, "Test.java");
Writer writer = new FileWriter(sourceFile);
writer.write(source);
writer.close();
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
compiler.run(null, null, null, sourceFile.getPath());
URLClassLoader classLoader = URLClassLoader.newInstance(new URL[] { root.toURI().toURL() });
Class<?> cls = Class.forName("Test", true, classLoader);
}
This should print test in stdout, as done by the static initializer in the test source code. Further use would be more easy if those classes implements a certain interface which is already in the classpath. Otherwise you need to involve the Reflection API to access and invoke the methods/fields.
In Java 6 or later, you can get access to the compiler through the javax.tools package. ToolProvider.getSystemJavaCompiler() will get you a javax.tools.JavaCompiler, which you can configure to compile your source. If you are using earlier versions of Java, you can still get at it through the internal com.sun.tools.javac.Main interface, although it's a lot less flexible.
Java6 has a scripting API. I've used it with Javascript, but I believe you can have it compile external Java code as well.
http://java.sun.com/developer/technicalArticles/J2SE/Desktop/scripting/
Edit: Here is a more relevant link:
"Dynamic source" code in Java applications
I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process. The following is not enough, because it doesn't include any dynamically loaded classes:
String classpath = System.getProperty("java.class.path");
Currently I'm searching for each needed class with the code below. However, on some machines this fails for some classes/libs, the source variable is null. Is there a more reliable and simpler way to get the location of libs that are used by the current jvm process?
String stax = ClassFinder.classPath("javax.xml.stream.Location");
public static String classPath(String qualifiedClassName) throws NotFoundException {
try {
Class qc = Class.forName( qualifiedClassName );
CodeSource source = qc.getProtectionDomain().getCodeSource();
if ( source != null ) {
URL location = source.getLocation();
String f = location.getPath();
f = URLDecoder.decode(f, "UTF-8"); // decode URL to avoid spaces being replaced by %20
return f.substring(1);
} else {
throw new ClassFinder().new NotFoundException(qualifiedClassName+" (unknown source, likely rt.jar)");
}
} catch ( Exception e ) {
throw new ClassFinder().new NotFoundException(qualifiedClassName);
}
}
See my previous question which covers getting the classpath as well as how to launch a sub-process.
I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process.
You mean invoke a new JVM?
Given that...
it is possible to plug in all sorts of agents and instrumentation into a JVM that can transform classes at load time
it is possible to take a byte array and turn it into a class
it is possible to have complex class loader hierarchies with varying visibility between classes and have the same classes loaded multiple times
...there is no general, magic, catch-all and foolproof way to do this. You should design your application and its class loading mechanisms to achieve this goal. If you allow 3rd party plug-ins, you'll have to document how this works and how they have to register their libraries.
If you look at the javadoc for Class.getClassLoader, you'll see that the "bootstrap" classloader is typically represented as the null. "String.class.getClassLoader()" will return null on the normal sun jvm implementations. i think this implementation detail carries over into the CodeSource stuff. As such, I wouldn't imagine you would need to worry about any class which comes from the bootstrap classloader as long as your sub-process uses the same jvm impl as the current process.
I have a server written in Java that runs as a Windows service (thanks to Install4J). I want this service to be able to download the latest version of the JAR file it runs from, and start running the new code. The stitch is that I don't want the Windows service to fully exit.
Ideally, I would accomplish this by a unix-style exec() call to stop the current version and run the new one. How can I best accomplish this?
Here is a complicated, but portable, way.
Split your code into two jars. One very small jar is there just to manage process startup. It creates a ClassLoader that holds the other jar on its classpath.
When you want to load a new version, you terminate all threads running code from the old jar. Null out all references to instances of classes from the old jar. Null out all references to the ClassLoader that loaded the old jar. At this point, if you didn't miss anything, the old classes and ClassLoader should be eligible for garbage collection.
Now you start over with a new ClassLoader instance pointing at the new jar, and restart your application code.
As far as I know, there is no way to do this in Java.
I suppose you could work around it by using the Java Runtime.exec or ProcessBuilder's start() command (which start new processes) then letting the current one end... the docs state
The subprocess is not killed when
there are no more references to the
Process object, but rather the
subprocess continues executing
asynchronously.
I'm assuming the same is true if the parent finishes and is garbage collected.
The catch is Runtime.exec's process will no longer have valid in, out, and err streams.
Java Service Wrapper does this and more.
You could use the built in reloading functionality of for example Tomcat, Jetty or JBoss, they can be run as a service, and you don't have to use them as a web container or Java EE container.
Other options are OSGi, and your own class loading functionality.
But be aware of reloading in a production environment. It is not always the best solution.
General approach:
import java.io.BufferedInputStream;
import java.util.Arrays;
public class ProcessSpawner {
public static void main(String[] args) {
//You can change env variables and working directory, and
//have better control over arguments.
//See [ProcessBuilder javadocs][1]
ProcessBuilder builder = new ProcessBuilder("ls", "-l");
try {
Process p = builder.start();
//here we just echo stdout from the process to java's stdout.
//of course, this might not be what you're after.
BufferedInputStream stream =
new BufferedInputStream(p.getInputStream());
byte[] b = new byte[80];
while(stream.available() > 0) {
stream.read(b);
String s = new String(b);
System.out.print(s);
Arrays.fill(b, (byte)0);
}
//exit with the exit code of the spawned process.
System.exit(p.waitFor());
} catch(Exception e) {
System.err.println("Exception: "+e.getMessage());
System.exit(1);
}
}
}