I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process. The following is not enough, because it doesn't include any dynamically loaded classes:
String classpath = System.getProperty("java.class.path");
Currently I'm searching for each needed class with the code below. However, on some machines this fails for some classes/libs, the source variable is null. Is there a more reliable and simpler way to get the location of libs that are used by the current jvm process?
String stax = ClassFinder.classPath("javax.xml.stream.Location");
public static String classPath(String qualifiedClassName) throws NotFoundException {
try {
Class qc = Class.forName( qualifiedClassName );
CodeSource source = qc.getProtectionDomain().getCodeSource();
if ( source != null ) {
URL location = source.getLocation();
String f = location.getPath();
f = URLDecoder.decode(f, "UTF-8"); // decode URL to avoid spaces being replaced by %20
return f.substring(1);
} else {
throw new ClassFinder().new NotFoundException(qualifiedClassName+" (unknown source, likely rt.jar)");
}
} catch ( Exception e ) {
throw new ClassFinder().new NotFoundException(qualifiedClassName);
}
}
See my previous question which covers getting the classpath as well as how to launch a sub-process.
I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process.
You mean invoke a new JVM?
Given that...
it is possible to plug in all sorts of agents and instrumentation into a JVM that can transform classes at load time
it is possible to take a byte array and turn it into a class
it is possible to have complex class loader hierarchies with varying visibility between classes and have the same classes loaded multiple times
...there is no general, magic, catch-all and foolproof way to do this. You should design your application and its class loading mechanisms to achieve this goal. If you allow 3rd party plug-ins, you'll have to document how this works and how they have to register their libraries.
If you look at the javadoc for Class.getClassLoader, you'll see that the "bootstrap" classloader is typically represented as the null. "String.class.getClassLoader()" will return null on the normal sun jvm implementations. i think this implementation detail carries over into the CodeSource stuff. As such, I wouldn't imagine you would need to worry about any class which comes from the bootstrap classloader as long as your sub-process uses the same jvm impl as the current process.
Related
I'm trying to redefine a method at runtime using javassist, but I'm running into some issues on the last step, because of the weird requirements I have for this:
I can't require the user to add startup flags
My code will necessarily run after the class has already been defined/loaded
My code looks like this:
val cp = ClassPool.getDefault()
val clazz = cp.get("net.minecraft.world.item.ItemStack")
val method = clazz.getDeclaredMethod(
"a",
arrayOf(cp.get("net.minecraft.world.level.block.state.IBlockData"))
)
method.setBody(
"""
{
double destroySpeed = this.c().a(this, $1);
if (this.s()) {
return destroySpeed * this.t().k("DestroySpeedMultiplier");
} else {
return destroySpeed;
}
}
""".trimIndent()
)
clazz.toClass(Items::class.java)
(I'm dealing with obfuscated method references, hence the weird names)
However, calling .toClass() causes an error as there are then two duplicate classes on the class loader - and to my knowledge there's no way to unload a single class.
My next port of call to update the class was to use the attach API and an agent, but that requires a startup flag to be added (on Java 9+, I'm running J17), which I can't do given my requirements. I have the same problem trying to load an agent on startup.
I have tried patching the server's jar file itself by using .toBytecode(), but I didn't manage to write the new class file to the jar - this method sounds promising, so it's absolutely on the table to restart the server after patching the jar.
Is there any way I can get this to work with my requirements? Or is there any alternative I can use to change a method's behavior?
We learned from the release notes of Java 9 that
The application class loader is no longer an instance of java.net.URLClassLoader (an implementation detail that was never specified in previous releases). Code that assumes that ClassLoader::getSytemClassLoader returns a URLClassLoader object will need to be updated.
This breaks old code, which scans the classpath as follows:
Java <= 8
URL[] ressources = ((URLClassLoader) classLoader).getURLs();
which runs into a
java.lang.ClassCastException:
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to
java.base/java.net.URLClassLoader
So for Java 9+ the following workaround was proposed as a PR at the Apache Ignite Project, which works as intended given adjustments in the JVM runtime options: --add-opens java.base/jdk.internal.loader=ALL-UNNAMED. However, as mentioned in the comments below, this PR was never merged into their Master branch.
/*
* Java 9 + Bridge to obtain URLs from classpath...
*/
private static URL[] getURLs(ClassLoader classLoader) {
URL[] urls = new URL[0];
try {
//see https://github.com/apache/ignite/pull/2970
Class builtinClazzLoader = Class.forName("jdk.internal.loader.BuiltinClassLoader");
if (builtinClazzLoader != null) {
Field ucpField = builtinClazzLoader.getDeclaredField("ucp");
ucpField.setAccessible(true);
Object ucpObject = ucpField.get(classLoader);
Class clazz = Class.forName("jdk.internal.loader.URLClassPath");
if (clazz != null && ucpObject != null) {
Method getURLs = clazz.getMethod("getURLs");
if (getURLs != null) {
urls = (URL[]) getURLs.invoke(ucpObject);
}
}
}
} catch (NoSuchMethodException | InvocationTargetException | NoSuchFieldException | IllegalAccessException | ClassNotFoundException e) {
logger.error("Could not obtain classpath URLs in Java 9+ - Exception was:");
logger.error(e.getLocalizedMessage(), e);
}
return urls;
}
However, this causes some severe headache due to the use of Reflection here. This is kind of an anti-pattern and is strictly criticized by the forbidden-apis maven plugin:
Forbidden method invocation: java.lang.reflect.AccessibleObject#setAccessible(boolean) [Reflection usage to work around access flags fails with SecurityManagers and likely will not work anymore on runtime classes in Java 9]
Question
Is there a safe way to access the list of all resource URLs in the class- / module path, which can be accessed by the given classloader, in OpenJDK 9/10 without using sun.misc.* imports (e.g. by using Unsafe)?
UPDATE (related to the comments)
I know, that I can do
String[] pathElements = System.getProperty("java.class.path").split(System.getProperty("path.separator"));
to obtain the elements in the classpath and then parse them to URLs. However - as far as I know - this property only returns the classpath given at the time of the application launch. However, in a container environment this will be the one of the application server and might not be sufficient, e.g. then using EAR bundles.
UPDATE 2
Thank your for all your comments. I will test, if System.getProperty("java.class.path") will work for our purposes and update the question, if this fullfills our needs.
However, it seems that other projects (maybe for other reasons, e.g Apache TomEE 8) suffer the same pain related to the URLClassLoader- for this reason, I think it is a valueable question.
UPDATE 3
Finally, we did switch to classgraph and migrated our code to this library to resolve our use-case to load ML resources bundled as JARs from the classpath.
I think this is an XY problem. Accessing the URLs of all resources on the classpath is not a supported operation in Java and is not a good thing to try to do. As you have already seen in this question, you will be fighting against the framework all the way if you try to do this. There will be a million edge cases that will break your solution (custom classloaders, EE containers, etc. etc.).
Please could you expand on why you want to do this?
If you have some kind of plugin system and are looking for modules that interface with your code which may have been provided at runtime, then you should use the ServiceLoader API, i.e.:
A service provider that is packaged as a JAR file for the class path is identified by placing a provider-configuration file in the resource directory META-INF/services. The name of the provider-configuration file is the fully qualified binary name of the service. The provider-configuration file contains a list of fully qualified binary names of service providers, one per line.
For example, suppose the service provider com.example.impl.StandardCodecs is packaged in a JAR file for the class path. The JAR file will contain a provider-configuration file named:
META-INF/services/com.example.CodecFactory
that contains the line:
com.example.impl.StandardCodecs # Standard codecs
AFAIK you can parse the java.class.path system property to get the urls:
String classpath = System.getProperty("java.class.path");
String[] entries = classpath.split(File.pathSeparator);
URL[] result = new URL[entries.length];
for(int i = 0; i < entries.length; i++) {
result[i] = Paths.get(entries[i]).toAbsolutePath().toUri().toURL();
}
System.out.println(Arrays.toString(result)); // e.g. [file:/J:/WS/Oxygen-Stable/jdk10/bin/]
We learned from the release notes of Java 9 that
The application class loader is no longer an instance of java.net.URLClassLoader (an implementation detail that was never specified in previous releases). Code that assumes that ClassLoader::getSytemClassLoader returns a URLClassLoader object will need to be updated.
This breaks old code, which scans the classpath as follows:
Java <= 8
URL[] ressources = ((URLClassLoader) classLoader).getURLs();
which runs into a
java.lang.ClassCastException:
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to
java.base/java.net.URLClassLoader
So for Java 9+ the following workaround was proposed as a PR at the Apache Ignite Project, which works as intended given adjustments in the JVM runtime options: --add-opens java.base/jdk.internal.loader=ALL-UNNAMED. However, as mentioned in the comments below, this PR was never merged into their Master branch.
/*
* Java 9 + Bridge to obtain URLs from classpath...
*/
private static URL[] getURLs(ClassLoader classLoader) {
URL[] urls = new URL[0];
try {
//see https://github.com/apache/ignite/pull/2970
Class builtinClazzLoader = Class.forName("jdk.internal.loader.BuiltinClassLoader");
if (builtinClazzLoader != null) {
Field ucpField = builtinClazzLoader.getDeclaredField("ucp");
ucpField.setAccessible(true);
Object ucpObject = ucpField.get(classLoader);
Class clazz = Class.forName("jdk.internal.loader.URLClassPath");
if (clazz != null && ucpObject != null) {
Method getURLs = clazz.getMethod("getURLs");
if (getURLs != null) {
urls = (URL[]) getURLs.invoke(ucpObject);
}
}
}
} catch (NoSuchMethodException | InvocationTargetException | NoSuchFieldException | IllegalAccessException | ClassNotFoundException e) {
logger.error("Could not obtain classpath URLs in Java 9+ - Exception was:");
logger.error(e.getLocalizedMessage(), e);
}
return urls;
}
However, this causes some severe headache due to the use of Reflection here. This is kind of an anti-pattern and is strictly criticized by the forbidden-apis maven plugin:
Forbidden method invocation: java.lang.reflect.AccessibleObject#setAccessible(boolean) [Reflection usage to work around access flags fails with SecurityManagers and likely will not work anymore on runtime classes in Java 9]
Question
Is there a safe way to access the list of all resource URLs in the class- / module path, which can be accessed by the given classloader, in OpenJDK 9/10 without using sun.misc.* imports (e.g. by using Unsafe)?
UPDATE (related to the comments)
I know, that I can do
String[] pathElements = System.getProperty("java.class.path").split(System.getProperty("path.separator"));
to obtain the elements in the classpath and then parse them to URLs. However - as far as I know - this property only returns the classpath given at the time of the application launch. However, in a container environment this will be the one of the application server and might not be sufficient, e.g. then using EAR bundles.
UPDATE 2
Thank your for all your comments. I will test, if System.getProperty("java.class.path") will work for our purposes and update the question, if this fullfills our needs.
However, it seems that other projects (maybe for other reasons, e.g Apache TomEE 8) suffer the same pain related to the URLClassLoader- for this reason, I think it is a valueable question.
UPDATE 3
Finally, we did switch to classgraph and migrated our code to this library to resolve our use-case to load ML resources bundled as JARs from the classpath.
I think this is an XY problem. Accessing the URLs of all resources on the classpath is not a supported operation in Java and is not a good thing to try to do. As you have already seen in this question, you will be fighting against the framework all the way if you try to do this. There will be a million edge cases that will break your solution (custom classloaders, EE containers, etc. etc.).
Please could you expand on why you want to do this?
If you have some kind of plugin system and are looking for modules that interface with your code which may have been provided at runtime, then you should use the ServiceLoader API, i.e.:
A service provider that is packaged as a JAR file for the class path is identified by placing a provider-configuration file in the resource directory META-INF/services. The name of the provider-configuration file is the fully qualified binary name of the service. The provider-configuration file contains a list of fully qualified binary names of service providers, one per line.
For example, suppose the service provider com.example.impl.StandardCodecs is packaged in a JAR file for the class path. The JAR file will contain a provider-configuration file named:
META-INF/services/com.example.CodecFactory
that contains the line:
com.example.impl.StandardCodecs # Standard codecs
AFAIK you can parse the java.class.path system property to get the urls:
String classpath = System.getProperty("java.class.path");
String[] entries = classpath.split(File.pathSeparator);
URL[] result = new URL[entries.length];
for(int i = 0; i < entries.length; i++) {
result[i] = Paths.get(entries[i]).toAbsolutePath().toUri().toURL();
}
System.out.println(Arrays.toString(result)); // e.g. [file:/J:/WS/Oxygen-Stable/jdk10/bin/]
I got a little project where I have to compute a list. The computation depends on serveal factors.
The point is that these factors change from time to time and the user should be allowed to change this by it's self.
Up to now, the factors are hard-coded and no changes can be done without recompiling the code.
At the moment the code looks like this:
if (someStatement.equals("someString")) {
computedList.remove("something");
}
My idea is to make an editable and human readable textfile, configfile, etc. which is loaded at runtime/ at startup? This file should hold the java code from above.
Any ideas how to do that? Please note: The targeted PCs do not have the JDK installed, only an JRE.
An effective way of going about this is using a static initializer. Static Block in Java A good and concise explanation can be found under this link.
One option here that would allow this would be to use User Input Dialogs from the swing API - then you could store the users answer's in variables and export them to a text file/config file, or just use them right in the program without saving them. You would just have the input dialogs pop up at the very beginning of the program before anything else happens, and then the program would run based off those responses.
You could use Javascript for the configuration file language, instead of java. Java 7 SE and later includes a javascript interpreter that you can call from Java. it's not difficult to use, and you can inject java objects into the javascript environment.
Basically, you'd create a Javascript environment, insert the java objects into it which the config file is expected to configure, and then run the config file as javascript.
Okay, here we go... I found an quite simple solution for my problem.
I am using Janino by Codehaus (Link). This library has an integrated Java compiler and seems to work like the JavaCompiler class in Java 7.
BUT without having the JDK to be installed.
Through Janino you can load and compile *.java files(which are human readable) at runtime, which was exactly what I needed.
I think the examples and code-snippets on their homepage are just painful, so here's my own implementation:
Step one is to implement an interface with the same methods your Java file has which is loaded at runtime:
public interface ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList);}
Then you call the Janino classloader when you need the class:
File janinoSourceDir = new File(PATH_TO_JAVAFILE);
File[] srcDir = new File[] { janinoSourceDir };
String encoding = null;
ClassLoader parentClassLoader = this.getClass().getClassLoader();
ClassLoader cl = new JavaSourceClassLoader(parentClassLoader, srcDir,
encoding);
And create an new instance
ZuordnungsInterface myZuordnung = (ZuordnungInterface) cl.loadClass("zuordnung")
.newInstance();
Note: The class which is loaded is named zuordnung.java, so there is no extension needed in the call cl.loadClass("zuordnung").
And finaly the class I want to load and compile at runtime of my program, which can be located wherever you want it to be (PATH_TO_JAVAFILE):
public class zuordnung implements ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList){
ArrayList<String> computedList = (ArrayList<String>) rawList.clone();
if (Model.getSomeString().equals("Some other string")) {
computedList.add("Yeah, I loaded an external Java class");
}
return computedList;
}}
That's it. Hope it helps others with similar problems!
Is there a way in using externally stored sourcecode and loading it into a Java programm, so that it can be used by it?
I would like to have a program that can be altered without editing the complete source code and that this is even possible without compiling this every time. Another advantage is, that I can change parts of the code like I want.
Of course I have to have interfaces so that it is possible to send data into this and getting it back into the fixed source program again.
And of course it should be faster than a pure interpreting system.
So is there a way in doing this like an additional compiling of these external source code parts and a start of the programm after this is done?
Thank you in advance, Andreas :)
You need the javax.tools API for this. Thus, you need to have at least the JDK installed to get it to work (and let your IDE point to it instead of the JRE). Here's a basic kickoff example (without proper exception and encoding handling just to make the basic example less opaque, cough):
public static void main(String... args) throws Exception {
String source = "public class Test { static { System.out.println(\"test\"); } }";
File root = new File("/test");
File sourceFile = new File(root, "Test.java");
Writer writer = new FileWriter(sourceFile);
writer.write(source);
writer.close();
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
compiler.run(null, null, null, sourceFile.getPath());
URLClassLoader classLoader = URLClassLoader.newInstance(new URL[] { root.toURI().toURL() });
Class<?> cls = Class.forName("Test", true, classLoader);
}
This should print test in stdout, as done by the static initializer in the test source code. Further use would be more easy if those classes implements a certain interface which is already in the classpath. Otherwise you need to involve the Reflection API to access and invoke the methods/fields.
In Java 6 or later, you can get access to the compiler through the javax.tools package. ToolProvider.getSystemJavaCompiler() will get you a javax.tools.JavaCompiler, which you can configure to compile your source. If you are using earlier versions of Java, you can still get at it through the internal com.sun.tools.javac.Main interface, although it's a lot less flexible.
Java6 has a scripting API. I've used it with Javascript, but I believe you can have it compile external Java code as well.
http://java.sun.com/developer/technicalArticles/J2SE/Desktop/scripting/
Edit: Here is a more relevant link:
"Dynamic source" code in Java applications