Develop Reference

Java Urlclass loader cast exception [duplicate]

We learned from the release notes of Java 9 that
The application class loader is no longer an instance of java.net.URLClassLoader (an implementation detail that was never specified in previous releases). Code that assumes that ClassLoader::getSytemClassLoader returns a URLClassLoader object will need to be updated.
This breaks old code, which scans the classpath as follows:
Java <= 8
URL[] ressources = ((URLClassLoader) classLoader).getURLs();
which runs into a
java.lang.ClassCastException:
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to
java.base/java.net.URLClassLoader
So for Java 9+ the following workaround was proposed as a PR at the Apache Ignite Project, which works as intended given adjustments in the JVM runtime options: --add-opens java.base/jdk.internal.loader=ALL-UNNAMED. However, as mentioned in the comments below, this PR was never merged into their Master branch.
/*
* Java 9 + Bridge to obtain URLs from classpath...
*/
private static URL[] getURLs(ClassLoader classLoader) {
URL[] urls = new URL[0];
try {
//see https://github.com/apache/ignite/pull/2970
Class builtinClazzLoader = Class.forName("jdk.internal.loader.BuiltinClassLoader");
if (builtinClazzLoader != null) {
Field ucpField = builtinClazzLoader.getDeclaredField("ucp");
ucpField.setAccessible(true);
Object ucpObject = ucpField.get(classLoader);
Class clazz = Class.forName("jdk.internal.loader.URLClassPath");
if (clazz != null && ucpObject != null) {
Method getURLs = clazz.getMethod("getURLs");
if (getURLs != null) {
urls = (URL[]) getURLs.invoke(ucpObject);
}
}
}
} catch (NoSuchMethodException | InvocationTargetException | NoSuchFieldException | IllegalAccessException | ClassNotFoundException e) {
logger.error("Could not obtain classpath URLs in Java 9+ - Exception was:");
logger.error(e.getLocalizedMessage(), e);
}
return urls;
}
However, this causes some severe headache due to the use of Reflection here. This is kind of an anti-pattern and is strictly criticized by the forbidden-apis maven plugin:
Forbidden method invocation: java.lang.reflect.AccessibleObject#setAccessible(boolean) [Reflection usage to work around access flags fails with SecurityManagers and likely will not work anymore on runtime classes in Java 9]
Question
Is there a safe way to access the list of all resource URLs in the class- / module path, which can be accessed by the given classloader, in OpenJDK 9/10 without using sun.misc.* imports (e.g. by using Unsafe)?
UPDATE (related to the comments)
I know, that I can do
String[] pathElements = System.getProperty("java.class.path").split(System.getProperty("path.separator"));
to obtain the elements in the classpath and then parse them to URLs. However - as far as I know - this property only returns the classpath given at the time of the application launch. However, in a container environment this will be the one of the application server and might not be sufficient, e.g. then using EAR bundles.
UPDATE 2
Thank your for all your comments. I will test, if System.getProperty("java.class.path") will work for our purposes and update the question, if this fullfills our needs.
However, it seems that other projects (maybe for other reasons, e.g Apache TomEE 8) suffer the same pain related to the URLClassLoader- for this reason, I think it is a valueable question.
UPDATE 3
Finally, we did switch to classgraph and migrated our code to this library to resolve our use-case to load ML resources bundled as JARs from the classpath.

I think this is an XY problem. Accessing the URLs of all resources on the classpath is not a supported operation in Java and is not a good thing to try to do. As you have already seen in this question, you will be fighting against the framework all the way if you try to do this. There will be a million edge cases that will break your solution (custom classloaders, EE containers, etc. etc.).
Please could you expand on why you want to do this?
If you have some kind of plugin system and are looking for modules that interface with your code which may have been provided at runtime, then you should use the ServiceLoader API, i.e.:
A service provider that is packaged as a JAR file for the class path is identified by placing a provider-configuration file in the resource directory META-INF/services. The name of the provider-configuration file is the fully qualified binary name of the service. The provider-configuration file contains a list of fully qualified binary names of service providers, one per line.
For example, suppose the service provider com.example.impl.StandardCodecs is packaged in a JAR file for the class path. The JAR file will contain a provider-configuration file named:
META-INF/services/com.example.CodecFactory
that contains the line:
com.example.impl.StandardCodecs # Standard codecs

AFAIK you can parse the java.class.path system property to get the urls:
String classpath = System.getProperty("java.class.path");
String[] entries = classpath.split(File.pathSeparator);
URL[] result = new URL[entries.length];
for(int i = 0; i < entries.length; i++) {
result[i] = Paths.get(entries[i]).toAbsolutePath().toUri().toURL();
}
System.out.println(Arrays.toString(result)); // e.g. [file:/J:/WS/Oxygen-Stable/jdk10/bin/]

How to safely access the URLs of all resource files in the classpath in Java 9+?

We learned from the release notes of Java 9 that
The application class loader is no longer an instance of java.net.URLClassLoader (an implementation detail that was never specified in previous releases). Code that assumes that ClassLoader::getSytemClassLoader returns a URLClassLoader object will need to be updated.
This breaks old code, which scans the classpath as follows:
Java <= 8
URL[] ressources = ((URLClassLoader) classLoader).getURLs();
which runs into a
java.lang.ClassCastException:
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to
java.base/java.net.URLClassLoader
So for Java 9+ the following workaround was proposed as a PR at the Apache Ignite Project, which works as intended given adjustments in the JVM runtime options: --add-opens java.base/jdk.internal.loader=ALL-UNNAMED. However, as mentioned in the comments below, this PR was never merged into their Master branch.
/*
* Java 9 + Bridge to obtain URLs from classpath...
*/
private static URL[] getURLs(ClassLoader classLoader) {
URL[] urls = new URL[0];
try {
//see https://github.com/apache/ignite/pull/2970
Class builtinClazzLoader = Class.forName("jdk.internal.loader.BuiltinClassLoader");
if (builtinClazzLoader != null) {
Field ucpField = builtinClazzLoader.getDeclaredField("ucp");
ucpField.setAccessible(true);
Object ucpObject = ucpField.get(classLoader);
Class clazz = Class.forName("jdk.internal.loader.URLClassPath");
if (clazz != null && ucpObject != null) {
Method getURLs = clazz.getMethod("getURLs");
if (getURLs != null) {
urls = (URL[]) getURLs.invoke(ucpObject);
}
}
}
} catch (NoSuchMethodException | InvocationTargetException | NoSuchFieldException | IllegalAccessException | ClassNotFoundException e) {
logger.error("Could not obtain classpath URLs in Java 9+ - Exception was:");
logger.error(e.getLocalizedMessage(), e);
}
return urls;
}
However, this causes some severe headache due to the use of Reflection here. This is kind of an anti-pattern and is strictly criticized by the forbidden-apis maven plugin:
Forbidden method invocation: java.lang.reflect.AccessibleObject#setAccessible(boolean) [Reflection usage to work around access flags fails with SecurityManagers and likely will not work anymore on runtime classes in Java 9]
Question
Is there a safe way to access the list of all resource URLs in the class- / module path, which can be accessed by the given classloader, in OpenJDK 9/10 without using sun.misc.* imports (e.g. by using Unsafe)?
UPDATE (related to the comments)
I know, that I can do
String[] pathElements = System.getProperty("java.class.path").split(System.getProperty("path.separator"));
to obtain the elements in the classpath and then parse them to URLs. However - as far as I know - this property only returns the classpath given at the time of the application launch. However, in a container environment this will be the one of the application server and might not be sufficient, e.g. then using EAR bundles.
UPDATE 2
Thank your for all your comments. I will test, if System.getProperty("java.class.path") will work for our purposes and update the question, if this fullfills our needs.
However, it seems that other projects (maybe for other reasons, e.g Apache TomEE 8) suffer the same pain related to the URLClassLoader- for this reason, I think it is a valueable question.
UPDATE 3
Finally, we did switch to classgraph and migrated our code to this library to resolve our use-case to load ML resources bundled as JARs from the classpath.

I think this is an XY problem. Accessing the URLs of all resources on the classpath is not a supported operation in Java and is not a good thing to try to do. As you have already seen in this question, you will be fighting against the framework all the way if you try to do this. There will be a million edge cases that will break your solution (custom classloaders, EE containers, etc. etc.).
Please could you expand on why you want to do this?
If you have some kind of plugin system and are looking for modules that interface with your code which may have been provided at runtime, then you should use the ServiceLoader API, i.e.:
A service provider that is packaged as a JAR file for the class path is identified by placing a provider-configuration file in the resource directory META-INF/services. The name of the provider-configuration file is the fully qualified binary name of the service. The provider-configuration file contains a list of fully qualified binary names of service providers, one per line.
For example, suppose the service provider com.example.impl.StandardCodecs is packaged in a JAR file for the class path. The JAR file will contain a provider-configuration file named:
META-INF/services/com.example.CodecFactory
that contains the line:
com.example.impl.StandardCodecs # Standard codecs

AFAIK you can parse the java.class.path system property to get the urls:
String classpath = System.getProperty("java.class.path");
String[] entries = classpath.split(File.pathSeparator);
URL[] result = new URL[entries.length];
for(int i = 0; i < entries.length; i++) {
result[i] = Paths.get(entries[i]).toAbsolutePath().toUri().toURL();
}
System.out.println(Arrays.toString(result)); // e.g. [file:/J:/WS/Oxygen-Stable/jdk10/bin/]

How to get File Permission Mode programmatically in Java

I know it is possible to change the permission mode of a file using:
Runtime.getRuntime().exec( "chmod 777 myfile" );.
This example sets the permission bits to 777. Is it possible to set the permission bits to 777 programmatically using Java? Can this be done to every file?

Using chmod in Android
Java doesn't have native support for platform dependent operations like chmod. However, Android provides utilities for some of these operations via android.os.FileUtils. The FileUtils class is not part of the public SDK and is therefore not supported. So, use this at your own risk:
public int chmod(File path, int mode) throws Exception {
Class fileUtils = Class.forName("android.os.FileUtils");
Method setPermissions =
fileUtils.getMethod("setPermissions", String.class, int.class, int.class, int.class);
return (Integer) setPermissions.invoke(null, path.getAbsolutePath(), mode, -1, -1);
}
...
chmod("/foo/bar/baz", 0755);
...
Reference : http://www.damonkohler.com/2010/05/using-chmod-in-android.html?showComment=1341900716400#c4186506545056003185

Android make it difficult to interact with other applications and their data except through Intents. Intents won't work for permissions because you're dependent on the application receiving the Intent to do/provide what you want; they probably weren't designed to tell anybody their files' permissions. There are ways around it, but only when the applications are desgned to operate in the same JVM.
So each application can only change it's file. for more detail in file permission see http://docs.oracle.com/javase/1.4.2/docs/guide/security/permissions.html

As commented previously, android.os.FileUtils has changed and the solution posted by Ashraf no longer works. The following method should work on all versions of Android (although it does use reflection and if a manufacturer made major changes this may not work).
public static void chmod(String path, int mode) throws Exception {
Class<?> libcore = Class.forName("libcore.io.Libcore");
Field field = libcore.getDeclaredField("os");
if (!field.isAccessible()) {
field.setAccessible(true);
}
Object os = field.get(field);
Method chmod = os.getClass().getMethod("chmod", String.class, int.class);
chmod.invoke(os, path, mode);
}
Obviously you will need to own the file to make any permission changes.

Here is a solution using Apache Commons.IO FileUtils, and the appropriate methods on File objects.
for (File f : FileUtils.listFilesAndDirs(new File('/some/path'), TrueFileFilter.TRUE, TrueFileFilter.TRUE)) {
if (!f.setReadable(true, false)) {
throw new IOException(String.format("Failed to setReadable for all on %s", f));
}
if (!f.setWritable(true, false)) {
throw new IOException(String.format("Failed to setWritable for all on %s", f));
}
if (!f.setExecutable(true, false)) {
throw new IOException(String.format("Failed to setExecutable for all on %s", f));
}
}
This is the equivalent of chmod -R 0777 /some/path. Adjust the set{Read,Writ,Execut}able calls to implement other modes. (I'll happily update this answer if someone posts appropriate code to do this.)

Launching Java Subprocess using parent process Classpath

I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process. The following is not enough, because it doesn't include any dynamically loaded classes:
String classpath = System.getProperty("java.class.path");
Currently I'm searching for each needed class with the code below. However, on some machines this fails for some classes/libs, the source variable is null. Is there a more reliable and simpler way to get the location of libs that are used by the current jvm process?
String stax = ClassFinder.classPath("javax.xml.stream.Location");
public static String classPath(String qualifiedClassName) throws NotFoundException {
try {
Class qc = Class.forName( qualifiedClassName );
CodeSource source = qc.getProtectionDomain().getCodeSource();
if ( source != null ) {
URL location = source.getLocation();
String f = location.getPath();
f = URLDecoder.decode(f, "UTF-8"); // decode URL to avoid spaces being replaced by %20
return f.substring(1);
} else {
throw new ClassFinder().new NotFoundException(qualifiedClassName+" (unknown source, likely rt.jar)");
}
} catch ( Exception e ) {
throw new ClassFinder().new NotFoundException(qualifiedClassName);
}
}

See my previous question which covers getting the classpath as well as how to launch a sub-process.

I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process.
You mean invoke a new JVM?
Given that...
it is possible to plug in all sorts of agents and instrumentation into a JVM that can transform classes at load time
it is possible to take a byte array and turn it into a class
it is possible to have complex class loader hierarchies with varying visibility between classes and have the same classes loaded multiple times
...there is no general, magic, catch-all and foolproof way to do this. You should design your application and its class loading mechanisms to achieve this goal. If you allow 3rd party plug-ins, you'll have to document how this works and how they have to register their libraries.

If you look at the javadoc for Class.getClassLoader, you'll see that the "bootstrap" classloader is typically represented as the null. "String.class.getClassLoader()" will return null on the normal sun jvm implementations. i think this implementation detail carries over into the CodeSource stuff. As such, I wouldn't imagine you would need to worry about any class which comes from the bootstrap classloader as long as your sub-process uses the same jvm impl as the current process.

NoClassDefFoundError while accessing GraphicsEnvironment.getLocalGraphicsEnvironment on Tomcat

I have an application which is running on tomcat, one of the methods is, creating a simple thumbnail from an jpeg image. The functions works fine offline and a week ago also on tomcat. But now i get the following error:
java.lang.NoClassDefFoundError
java.lang.Class.forName0(Native Method)
java.lang.Class.forName(Class.java:164)
java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:68)
java.awt.image.BufferedImage.createGraphics(BufferedImage.java:1141)
eval.impl.ImageEval.getThumbnail(ImageEval.java:155)
eval.impl.ImageServlet.doGet(ImageServlet.java:79)
javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
I don't think that i have change anything what should influence this (actually i didn't change the function at all according to the svn repository), so it must be a library problem. But i can't figure out what is missing.
Here are the actual lines from the getThumbnail function, where the error occures:
BufferedImage thumbImage = new BufferedImage(thumbWidth,
thumbHeight, BufferedImage.TYPE_INT_RGB);
Graphics2D graphics2D = thumbImage.createGraphics();
graphics2D.setRenderingHint(RenderingHints.KEY_INTERPOLATION,
RenderingHints.VALUE_INTERPOLATION_BILINEAR);
graphics2D.drawImage(simage, 0, 0, thumbWidth, thumbHeight, null);
[edit] I decided to update the problem description a little.
Yes it seems that he can not find some class from java.awt or one related to that. But they do exist on the server in the jvm. Java headless mode doesn't solve the problem.
In another project the exact same code, but inside an axis2 webservice on this server is working fine.
[/edit]

It seems like you've change the configuration of Tomcat.
Either you've changed to a l{0,1}[iu]n[iu]x box or installed on a virtual machine with different security control than the one where you test it.
Apparently the
GraphicsEnvironment.getLocalGraphicsEnvironment()
Is trying to access the property: java.awt.graphicsenv
Which may return null or some non existing class name which is then loaded and throws the ClassNotFoundException. 1
The solution seems to be specifying the "java.awt.headless" property.
This is a similar question: java.awt.Color error
Try this search , it shows similar situations as your.
I remember there was something in the sun bugs database too.
Post the solution when you find it!
1.GraphicsEnvironment.java
EDIT
It is not eclipse!!
In my original post there is a link to the source code of the class which is throwing the exception.
Since I looks like you miss it, I'll post it here for you:
public static synchronized GraphicsEnvironment getLocalGraphicsEnvironment() {
if (localEnv == null) {
// Y O U R E R R O R O R I G I N A T E S H E R E !!!
String nm = (String) java.security.AccessController.doPrivileged
(new sun.security.action.GetPropertyAction
("java.awt.graphicsenv", null));
try {
// long t0 = System.currentTimeMillis();
localEnv =
(GraphicsEnvironment) Class.forName(nm).newInstance();
// long t1 = System.currentTimeMillis();
// System.out.println("GE creation took " + (t1-t0)+ "ms.");
if (isHeadless()) {
localEnv = new HeadlessGraphicsEnvironment(localEnv);
}
} catch (ClassNotFoundException e) {
throw new Error("Could not find class: "+nm);
} catch (InstantiationException e) {
throw new Error("Could not instantiate Graphics Environment: "
+ nm);
} catch (IllegalAccessException e) {
throw new Error ("Could not access Graphics Environment: "
+ nm);
}
}
return localEnv;
}
That's what gets executed.
And in the original post which you don't seem to have read, I said the code is accessing the property "java.awt.graphicsenv"
If that other project using axis doesn't have the same problem it may be because it may be running in a different tomcat configuration or the axis library allowed the access to that property. But we cannot be sure. That's pure speculation. So why don't you test the following and see what gets printed:
String nm = (String) java.security.AccessController.doPrivileged
(new sun.security.action.GetPropertyAction
("java.awt.graphicsenv", null));
System.out.println("java.awt.graphicsenv = " + nm );
It it prints null then you now what the problem is. You don't have that property in your system, or the security forbids you do use it.
It is very hard to tell you from here: "Go and edit file xyz and add : fail = false" So you have to do your work and try to figure out what's the real reason.
Start by researching what's the code being executed is ( which I have just posted ) and follow by understand what it does and how does all that "AccessController.doPrivileged" works. (You may use Google + StackOverflow for that).

We had a similar issue and after much trouble shooting it was identified to be related to the java.awt.headless property. The issue was resolved by explicitly setting the JVM option to
-Djava.awt.headless=true

It was running a week ago, and now it is not.
THEREFORE, YOU CHANGED SOMETHING BETWEEN "working" and "not working".
Go back to the working config (if you can), and rigorously track what you changed. If you don't have a backup of the working config, then meticulously go back through what you've done between working and non-working until you find what you changed.
It may not be code - it could be a config file, etc.
Best of luck,
-R

Is this server running java in server mode - I hear that doesn't load in the AWT classes.

If you are deploying this on *nix, and you don't have an X window system running anymore, that could explain it. Even if you do, if you aren't exporting the DISPLAY system variable to the process that starts the JVM, or if you are but it is not actually valid, it could cause such an issue.
That would at least explain why you didn't change any configuration in tomcat, but still have a problem.

If your NoClassDefFoundError has no message at all, then this means two things:
The JVM has already tried and failed to load a class. Usually, this means the JVM was unable to complete static initialization for that class, i.e. assign values to any static fields and run any static { } blocks. Often, this is because the classes necessary to do this static initialization are missing.
You're using Java 5, not Java 6. (In Java 6, you get a 'Could not initialize class xyz' message instead.)
The problem class appears to be the one whose name is the value of the system property java.awt.graphicsenv. I would start by finding out the value of this property. What happens when you try to instantiate this class?

Since you're getting NoClassDefFoundError from inside the AWT code, it looks like Java is failing to load the X Windows libraries. Note that even if you're running in headless mode ($DISPLAY not pointing to an X Windows server), AWT still needs some subset of the X11 libraries in order to render images. See, for example, this reference:
http://javatechniques.com/blog/linux-x11-libraries-for-headless-mode
If something stopped working and your Java code didn't change, it's possible that the X11 libraries got moved or uninstalled on your machine, or that for some other reason your LD_LIBRARY_PATH environment variable doesn't point to them anymore.