Develop Reference

Java ClassLoader: findClass is not called again after ClassNotFoundException occurs

Context:
I have a microservice that at application BootStrap goes and gets all the classes it needs from another microservice as a Zip, then it loads all the classes this Zip contains and executes some code.
Problem:
What we are experiencing is that in some cases the service that gives the Zip with the classes does not answer (this is not the problem I want to address here).
The problem is that when this happens we throw a ClassNotFoundException and execute again and in this execution the programm again detects it needs to load a class it does not have... So it goes to the findClass() method and tries to get it by calling the Microservices asking for this class, but the service that gives the Zip with the class again does not respond so we throw another ClassNotFoundException... And again execute but this time it does not even try to call findClass() method, it is like Java is saying "okay this is definetely not here so im not even going to bother calling findClass()", I am trying to find where in the documentation is this specified (because I wanna see where is this specified, is it normal?).
Oracle Documentation ClassLoader
The closest I could find to what I experience is this
But in that case they do have the class, but the loader does not bother in loading it again because it detects it already has it, we are experiencing the same but in reverse, the loader does not have it, and it does not bother in trying again. Where is this in the docs?

Actually, the absence of a guaranty that multiple resolution attempts will be made is already enough to allow the behavior of remembering and reusing the result of the first symbol resolution attempt, whether it is a failure or success.
But it’s even mandated behavior, as specified in The Java Virtual Machine Specification, §5.4.3. Resolution:
Subsequent attempts to resolve the symbolic reference always fail with the same error that was thrown as a result of the initial resolution attempt.
This is not be confused with manual invocations of loadClass on a ClassLoader instance.

Can IntelliJ be configured to hit a breakpoint in a Java class that is loaded at runtime through reflection?

I have a Java class that is being loaded at runtime through reflection for a Kafka Streams application, configured with passing the name of the class as a String, like this:
streamsConfig.put(StreamsConfig.DEFAULT_TIMESTAMP_EXTRACTOR_CLASS_CONFIG,
"com.company.data.kstreams.Processor.MediaTimestampExtractor");
Then, at runtime, I have validated that it is properly loading and running the specified class. So, that part is working fine.
However, when I put a breakpoint in IntelliJ inside that custom MediaTimestampExtractor class, the debugger never hits it. I can see in the output that it's executing the code in the class, but for some reason, IntelliJ doesn't "know" that the class is being executed. In fact, the class itself shows a warning of:
Class 'MediaTimestampExtractor' is never used
Is there an option in IntelliJ to enable hitting breakpoints in code dynamically loaded by reflection, or a way to configure the breakpoint so that it hits, even if IntelliJ doesn't believe the class is being used?

Custom classloader trouble with getResources for names ENDING in slash

I am desperate for help but was unable to find anything on the web about this particular subject (many related ones that leave my particular problem unanswered).
Specifically, I need to be able to download code (jars) from a central and external code repository. This is done by the bootstrap code that needs to add this to the classpath of a class loader to be used thereafter. This is when we enter the subject that has been discussed so many times. I don't like hacks, so I tried the following:
Attempt #1: Create an instance of URLClassLoader configured for this purpose, then invoke the "rest" of the code through it.
Failure: There are 1.5 problems here (one may be the cause of another). One is that URLClassLoader, normally, prefers to load stuff from its parent. Some code has to exist in both, possibly different versions. If the code from the parent is used, it continues using the "outer" class loader for the rest of loading, which is not what we want, even when the initial loading is OK. Secondly, some third party libraries seem to access the system class loader directly, either by design or accidentally (may get it from one of the classes loaded by it).
Attempt #2: Create my subclass of the URLClassLoader that prefers self over the parent. Overrode loadClass, getResource, getResources, getPackage, getPackages... later other methods too to make sure of this.
Failure: Didn't help (enough). That third party code still couldn't load some resources.
Attempt #3 Create another custom subclass of the URLClassLoader and set it as the system class loader using -Djava.system.class.loader=...
Failure: This worked better - went further, but still failed trying to get resources. This time it was different resources, though. I added logging to all the overridden methods to log their calls and resource names. Regular resources were MOSTLY found. Some still weren't, even though they are there (confirmed). But something I don't know about even though I tried hard to learn is about many calls with resource names that end with a slash. Some also have slashes where a dollar sign would normally appear (nested/inner class resources). Some examples that were requested but NOT found:
com/acme/foo/bar/ClassName/
com/acme/foo/bar/ClassName/InnerClassName/
When I run the downloaded code with all content on the initial/boot classpath (and do not use my classloader), everything works fine - thus my class loader breaks things, but I need it to work.
My closest guesses are:
Third party code gets hold of the true system class loader somehow, perhaps via some class that was loaded by it, then uses that. I don't see requests to it and they are bound to fail because it does not have the entire class path.
This business with resource names ending in slashes is the cause by being supported by the true system class loader but not by the URLClassLoader I am subclassing. I can only guess that the expected return URL somehow locates the collection of resources with that name as prefix. That would be tough to match, although possible. Furthermore, it appears that some slashes are in positions where a dollar sign separating the inner class name should be, i.e. in the above example (spaces added for clarity):
com/acme/foo/bar/ClassName / InnerClassName/
com/acme/foo/bar/ClassName $ InnerClassName/
Please note that I cannot rely on hacking the actual system classloader by assuming that it is a subclass of the URLClassLoader and using reflection to call its addURL(URL) method.
Is there a way to make this work? Please help!
UPDATE
I just made an additional attempt. I created a dummy wrapper classloader (extending ClassLoader, not URLClassLoader) that only logs requests, then passes them on to the parent (public methods) or superclass (protected methods). I set this to be the system class loader and manually added the entire "inner" class path to the actual outer one, then tried to run the code. That works correctly, just as it does without the custom system class loader. What was logged also identified that even the system class loader return null for these resources ending in slashes for MOST of them, but not all. I did not check whether these also work in the my real code but guessing they may - as they were not the stumbling block. Somehow the custom system classloader is still being bypassed. How?
UPDATE 2
In my custom system class loaders I have let some classes come from the outer/true system class loader, e.g. those in java.lang. I am now suspecting that I should not have and that the inner "world" must be completely isolated. That would make it problematic, though, to communicate with it and all I would have left is reflection... but not sure whether that would even work - i.e. can there be more than one java.lang.Class and/or java.lang.Object?

Given all constraints this does not appear entirely possible in a rock solid fashion as I wanted it:
a) Third party libraries may always "misbehave" and get hold of lassloaders they are not supposed to use one way or another. I looked at OneJar as suggested by fge but they have the same issue - they only detect a possibility of it.
b) There is no way to completely replace/hide the system class loader.
c) Casting the system class loader to a URLClassLoader may stop working at any moment.

It seems, you didn’t understand the class loader structure. Within an ordinary JVM of the current version, there are at least three class loaders:
The bootstrap loader which is responsible for loading the core classes. Since this involves classes like Class and ClassLoader itself, it can’t be represented by a ClassLoader instance. All classes whose getClassLoader() returns null were loaded by the bootstrap loader
The extension loader. It is responsible for loading classes within the ext/ directory of the JRE. Afaik, it may vanish in future versions. Its parent loader is the bootstrap loader
The application loader. This is the one which will be returned by ClassLoader.getSystemClassLoader() and which will be used if no other parent was specified. In the current configurations, it’s parent is the extension loader, but maybe it will have the bootstrap loader as its direct parent in future versions
The conclusion is, if you want to reload your application’s classes without the delegation to the parent loader destroying your effort, you don’t need to manipulate the class loader’s implementation. You just have to specify the right parent. It’s as simple as
URLClassLoader cl=new URLClassLoader(urls, ClassLoader.getSystemClassLoader().getParent());
That way, the new class loader’s parent will be the original application class loader’s parent, thus the already loaded application classes are not in the scope of the new loader while everything else works as usual.
Regarding the resources ending with a slash, they are rather uncommon. They may get resolved when they actually refer to a directory but that depends on the protocol of the URL and the actual handler for that protocol. E.g. it might work for file: URLs but usually doesn’t for jar: URLs unless the jar file contains pseudo-entries for directories. I’ve also seen it working for ftp: URLs.
Another thing to understand is that if one class directly refers to another class, its original defining class loader will be queried, not necessarily the application class loader. E.g. when the class java.lang.String contains a reference to java.lang.Object (it does), this reference will be directly resolved using the bootstrap loader as this is the defining loader of java.lang.String.
This implies that if you manipulate the parent lookup of a loader to not follow the standard parent delegation you are risking to resolve names to different runtime classes as the resolving of the same names when being referenced by classes loaded by the parent loader. You avoid such problems by following the standard procedure as in the solution above. The JRE classes will never contain references to your application classes and the new loader not having the original application loader as its parent will never interfere with the classes loaded by the original application loader.

How do I control which ClassLoader loads a class?

The situation at hand is not as simple as the title seems to indicate.
Java 1.6_17 running via JWS.
I have a class, lets say MyClass and one of its instance member variables is a Type from an errant 3rd party library where during class initialization it dynamically tries loading some of its own classes with Class.forName(String). In one of these cases it happens to dynamically call: Class.forName("foo/Bar").This class name doesn't follow the JLS for binary names and ultimately leads to a java.lang.NoClassDefFoundError: foo/Bar.
We have a custom ClassLoader which I've added a sanitize method to ClassLoader.findClass(String) and ClassLoader.loadClass(String) which fixes this problem.
I can call stuff like:
myCustomClassLoader.findClass("foo/Bar")
Which then loads the class without any problems. But even if I load the class ahead of time, I still get the exception later. This is because during initialization of MyClass which refers to Bar - their code ends up calling Class.forName("foo/Bar") in a static block somewhere. This actually would be OK if the ClassLoader it was trying to use was my custom class loader. But it isn't. It is the com.sun.jnlp.JNLPClassLoader which doesn't do such sanitation, thus my problem.
I've made sure that Thread.currentThread().getContextClassLoader() is set to my custom class loader. But this (as you know) has no effect. I even set it as the first thing i do in main() due to some stuff I read and still, MyClass.class.getClassLoader() - is the JNLPClassLoader. If I could force it to NOT be the JNLPClassLoader and to use mine instead, problem solved.
How can I control which ClassLoader is used to load the class via their static Class.forName("foo/Bar") call made during class initialization? I believe if I can force MyClass.class.getClassLoader() to return my custom class loader, my problem will be resolved.
I'm open to other options if anyone has ideas.
TL;DR: Help me force all Class.forName(String) calls in a third party library which are referenced by MyClass - to use the classloader of my choosing.

This reminds me of an article I read 10 years ago about the classloading arrangements in Java. It's still there on JavaWorld.
The article won't answer your question directly, but it may help understand your problem. You need to cause MyClass to be loaded through your custom class loader and trump the default class loading behavior, which is to first delegate class loading to the parent classloader and only attempt to load a class if that fails.
Allowing MyClass to get loaded by a classloader other than yours will store a relationship from the instantiated class to that classloader (via getClassLoader) and cause Java to use that other classloader to try to discover any referenced classes found at compile time, effectively bypassing your custom class loader by virtue of the class loader hierarchy and the delegation model. If MyClass is instead defined by your class loader, you get a second chance.
It sounds like a job for something like URLClassLoader, overriding loadClass and trumping the delegation model for classes residing in your JARs. You'll probably want to use a bootstrap approach (as suggested by Thomas in a comment above) to force a single entrypoint class to be loaded through your custom class loader, dragging all the others with it.
Also informative is this other JavaWorld article by the same guy, which warns you about the caveats of Class.forName. That too may trip your classloading arrangements.
I hope this helps and proves informative. In any case, it sounds like a difficult solution that is easy to break as your code evolves.

I think everyone gave good solid attempts at answering the problem. However, it turns out that I misdiagnosed the problem.
I had a coworker take over the problem and asked him to get a JDK with debug flags on so we could debug the JNLPClassLoader to see what was going on as I had tried all of the suggestions here + some.
We ended up getting OpenJDK because recompiling the JDK from scratch is a total nightmare (we tried). After getting OpenJDK working with our product and debugging through the JNLPClassLoader - it turns out that it was still using a REALLY old .jnlp from months earlier that had the resource path wrong and thus why it couldn't find the class.
We were confused why it was still using the ancient .jnlp even though we had redeployed the server correctly many times with the correct .jnlp and lots of code changes between which were reflected in our client application when run.
Well, it turns out that on client machines, Java caches the .jnlp file. Even if your application changes and it redownloads your application, it still won't re-download the new .jnlp for whatever reason. So it will use all of the new code, but look up resources/class paths using the cached .jnlp.
If you run:
javaws -uninstall
On the client machine then that will clear the .jnlp cache and next time it will use the correct .jnlp file.
Really sad that this was the problem. Hopefully, this saves someone else endless hours of frustration like it caused us.

If you run out of ideas with patching the ClassLoaders themselves, you might consider rewriting the library bytecode itself -- just replace the "foo/bar" constant with the correct value, and then you don't need to customize further class loading at all!
You could do this either at runtime or beforehand.

Java: Finding out *why* a class is loaded

I am currently having the problem that I have a (partial) program that is trying to load a class but fails because it cannot find this class. Looking at the stack trace, I cannot see any particular reason for why the VM tries to load this particular class at the first place. Are there any tools that would let me figure out why a particular class is being loaded?
Hint:
I am already getting a stack trace at the exact point where the JVM tries to load the class (through an agent). However, the stack trace contains no line numbers. Therefore I only know which method triggers the class being loaded, not which statement. Then, even knowing the statement may not be enough. A single statement can cause a class to be loaded in many ways, because sometimes the VM needs to load part of the transitive closure of classes.

Run your program with the -XX:+TraceClassLoading and -XX:+TraceClassResolution flags. This will create a LOT of output that looks like the following:
[Loaded com.kdgregory.example.memory.PermgenExhaustion$MyClassLoader from file:/home/kgregory/Workspace/Website/programming/examples/bin/]
RESOLVE com.kdgregory.example.memory.PermgenExhaustion$MyClassLoader java.net.URLClassLoader
RESOLVE java.net.URLClassLoader java.lang.Class URLClassLoader.java:188
You'll need to trace the chain of RESOLVE messages for a particular class. Or more likely, you'll see an error when your program attempts to load the class, preceeded by resolve messages for the class that loads it).

You might try a static analysis tool like JDepend to see what classes have references to that class.

Classloaders
If it's an envorinment where multiple class-loaders are in the game (like a web application) you should be careful. Please tell us what's the app.
resource
Tell us where are the jars (you file/dir structure) and what class is loading. Are you loading it dinamically using Class.forName? or using spring or another framework of IOC? Is it the main class?
Some previous testing (to help us)
Maybe you can test some things using Class.getResource() from within the main method. If you class is foo.bar.Clazz try Class.getResource("/foo/bar/Clazz.class") to see if it returns something valid or not. Try to do the same with the class that loads your failing class to see if it's where you expect.