Develop Reference

How to deal with dependency conflicts using custom classloader in java properly?

I have a multimodule project called k-sdk. It has modules like
---> agent
---> core
---> api
---> integration
---> sdk
where (sdk module contains core, api, integration modules and acting as a shaded jar).
This jar can be imported as a dependency in the user application.
Here k-sdk is acting as a middleware to capture request and response of the api calls (both external and internal) and do some custom processing. And to intercept the internal calls i am using servlet-filter & for external calls, i am using ByteBuddy and wrote the required code in the premain method. The corresponding interceptors are written in integration module. User can use this k-sdk by running agent jar(present in agent module) of the k-sdk.
Now comes to the actual problem. Suppose I have written an Interceptor for OkHttp client version 4.10.0 in the integration module and the user is using the same client of version 3.14.9, now these two versions share completely same package name. And there are some uncommon apis, methods etc, which is being used in the OkHttpInterceptor which results in java.lang.NoSuchMethodError because maven only resolves that version of dependency which has the shortest path. And only 1 version of a dependency will be loaded in the classloader.
Now, To resolve this version conflict what i was thinking that i will use the below custom classloader. (ref) as i know that the solution lies in this approach only.
public class ChildFirstClassLoader extends URLClassLoader {
private final ClassLoader sysClzLoader;
public ChildFirstClassLoader(URL[] urls, ClassLoader parent) {
super(urls, parent);
sysClzLoader = getSystemClassLoader();
}
#Override
protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
// has the class loaded already?
Class<?> loadedClass = findLoadedClass(name);
if (loadedClass == null) {
try {
if (sysClzLoader != null) {
loadedClass = sysClzLoader.loadClass(name);
}
} catch (ClassNotFoundException ex) {
// class not found in system class loader... silently skipping
}
try {
// find the class from given jar urls as in first constructor parameter.
if (loadedClass == null) {
loadedClass = findClass(name);
}
} catch (ClassNotFoundException e) {
// class is not found in the given urls.
// Let's try it in parent classloader.
// If class is still not found, then this method will throw class not found ex.
loadedClass = super.loadClass(name, resolve);
}
}
if (resolve) { // marked to resolve
resolveClass(loadedClass);
}
return loadedClass;
}
#Override
public Enumeration<URL> getResources(String name) throws IOException {
List<URL> allRes = new LinkedList<>();
// load resources from sys class loader
Enumeration<URL> sysResources = sysClzLoader.getResources(name);
if (sysResources != null) {
while (sysResources.hasMoreElements()) {
allRes.add(sysResources.nextElement());
}
}
// load resource from this classloader
Enumeration<URL> thisRes = findResources(name);
if (thisRes != null) {
while (thisRes.hasMoreElements()) {
allRes.add(thisRes.nextElement());
}
}
// then try finding resources from parent classloaders
Enumeration<URL> parentRes = super.findResources(name);
if (parentRes != null) {
while (parentRes.hasMoreElements()) {
allRes.add(parentRes.nextElement());
}
}
return new Enumeration<URL>() {
Iterator<URL> it = allRes.iterator();
#Override
public boolean hasMoreElements() {
return it.hasNext();
}
#Override
public URL nextElement() {
return it.next();
}
};
}
#Override
public URL getResource(String name) {
URL res = null;
if (sysClzLoader != null) {
res = sysClzLoader.getResource(name);
}
if (res == null) {
res = findResource(name);
}
if (res == null) {
res = super.getResource(name);
}
return res;
}
}
And to improve the user experience, i have to make changes only in the premain method. Now i don't know how to use this classloader to resolve this conflicting issue.
The above classloader is extending URLClassLoader, and the application class loader is no longer an instance of java.net.URLClassLoader in java 9+.
I am very new to this complex solution, Please! help me out.
Before this solution, i tried to repackage classes using maven-shade-plugin but that solution was not working because in premain method using ByteBuddy i was targeting a class of actual package and was using different package in the interceptor so the flow was not going to the interceptor as the class was not matching.

Load resource bundle at runtime

Here is what I would like to achieve. We have an application that is running as a servlet on an IBM Domino server.
The application uses resource bundle to get translated messages and labels according to the browser language.
We want to enable customers to override some of the values.
We cannot modify the bundle_lang.properties files in the .jar at runtime.
So the idea was to provide additional bundleCustom_lang.properties files along with the .jar
This bundle could be loaded at runtime using
private static void addToClassPath(String s) throws Exception {
File file = new File(s);
URLClassLoader cl = (URLClassLoader) ClassLoader.getSystemClassLoader();
java.lang.reflect.Method m = URLClassLoader.class.getDeclaredMethod("addURL", new Class[] { URL.class });
m.setAccessible(true);
m.invoke(cl, new Object[] { file.toURI().toURL() });
}
So far, so good, this works in Eclipse. Here I had the bundleCustom files in a directory outside the workspace ( /volumes/DATA/Temp/ )
Once the addition ResourceBundle is available, We check this bundle for the key first. If it returns a value than this value is being used for the translation. If no value is returned, or the file does not exist, the value from the bundle inside the .jar is used.
My full code is here
public class BundleTest2 {
static final String CUSTOM_BUNDLE_PATH = "/volumes/DATA/Temp/";
static final String CUSTOM_BUNDLE_MODIFIER = "Custom";
public static void main(String[] args) {
try {
addToClassPath(CUSTOM_BUNDLE_PATH);
System.out.println(_getTranslation("LabelBundle", "OutlineUsersAllVIP"));
} catch (Exception e) {
}
}
private static String _getTranslation(String bundle, String translation) {
return _getTranslation0(bundle, new Locale("de"), translation);
}
private static String _getTranslation0(String bundle, Locale locale, String key) {
String s = null;
try {
try {
ResourceBundle custom = ResourceBundle.getBundle(bundle + CUSTOM_BUNDLE_MODIFIER, locale);
if (custom.containsKey(key)) {
s = custom.getString(key);
}
} catch (MissingResourceException re) {
System.out.println("CANNOT FIND CUSTOM RESOURCE BUNDLE: " + bundle + CUSTOM_BUNDLE_MODIFIER);
}
if (null == s || "".equals(s)) {
s = ResourceBundle.getBundle(bundle, locale).getString(key);
}
} catch (Exception e) {
}
return s;
}
private static void addToClassPath(String s) throws Exception {
File file = new File(s);
URLClassLoader cl = (URLClassLoader) ClassLoader.getSystemClassLoader();
java.lang.reflect.Method m = URLClassLoader.class.getDeclaredMethod("addURL", new Class[] { URL.class });
m.setAccessible(true);
m.invoke(cl, new Object[] { file.toURI().toURL() });
}
}
When I try the same from inside the servlet, I get a MissingResourceException.
I also tried to put the .properties files into a customization.jar and provide the full path ( incl. the .jar ) when invoking addToClassPath().
Apparently, the customization.jar is loaded ( it is locked in the file system ), but I still get the MissingResourceException.
We already use the same code in addToClassPath to load a Db2 driver and this is working as expected.
What am I missing?

Why don't you use Database to store the overriden translations? Persisting something crated by client in the local deployment of application is generally not a good idea, what will happen if you redeploy the app, will these resources be deleted? What if you have to run another node of your app, how will you replicate the custom properties file?

Java dynamic loading a class issue

I am trying to write a code that compiles and runs another java class, after it creates it from a String.
My problem is when I run
Class classToLoad = null;
ClassLoader classLoader = Server.class.getClassLoader();
try {
classToLoad = classLoader.loadClass(className);
} catch (Exception e) {
e.printStackTrace();
}
It throws a ClassNotFoundException. My problem isn't about the package, because if I debug the code and place a breakpoint before the "getClassLoader" and I reload the classes, then my code works fine and it sees the class that was recently created earlier in the app.
How can I reload the classes during runtime so the loadClass will work?

Take a look at this tutorial:
ClassLoader Load / Reload Example
... Let's look at a simple
example. Below is an example of a simple ClassLoader subclass. Notice
how it delegates class loading to its parent except for the one class
it is intended to be able to reload. If the loading of this class is
delegated to the parent class loader, it cannot be reloaded later.
Remember, a class can only be loaded once by the same ClassLoader
instance.
As said earlier, this is just an example that serves to show you the
basics of a ClassLoader's behaviour. It is not a production ready
template for your own class loaders. Your own class loaders should
probably not be limited to a single class, but a collection of classes
that you know you will need to reload. In addition, you should
probably not hardcode the class paths either.
public class MyClassLoader extends ClassLoader{
public MyClassLoader(ClassLoader parent) {
super(parent);
}
public Class loadClass(String name) throws ClassNotFoundException {
if(!"reflection.MyObject".equals(name))
return super.loadClass(name);
try {
String url = "file:C:/data/projects/tutorials/web/WEB-INF/" +
"classes/reflection/MyObject.class";
URL myUrl = new URL(url);
URLConnection connection = myUrl.openConnection();
InputStream input = connection.getInputStream();
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int data = input.read();
while(data != -1){
buffer.write(data);
data = input.read();
}
input.close();
byte[] classData = buffer.toByteArray();
return defineClass("reflection.MyObject",
classData, 0, classData.length);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
}
Below is an example use of the MyClassLoader.
public static void main(String[] args) throws
ClassNotFoundException,
IllegalAccessException,
InstantiationException {
ClassLoader parentClassLoader = MyClassLoader.class.getClassLoader();
MyClassLoader classLoader = new MyClassLoader(parentClassLoader);
Class myObjectClass = classLoader.loadClass("reflection.MyObject");
AnInterface2 object1 =
(AnInterface2) myObjectClass.newInstance();
MyObjectSuperClass object2 =
(MyObjectSuperClass) myObjectClass.newInstance();
//create new class loader so classes can be reloaded.
classLoader = new MyClassLoader(parentClassLoader);
myObjectClass = classLoader.loadClass("reflection.MyObject");
object1 = (AnInterface2) myObjectClass.newInstance();
object2 = (MyObjectSuperClass) myObjectClass.newInstance();
}

Probably asking: "What is the context in which you are loading the class?" will help answer your question better.
Most standard frameworks like Spring handle loading classes internally and exposing only the methods that those classes provide.
Try Class.forName(String name) to attempt to load the class and return the handle to the class object.
If you want to specifically use your own classloader to load the class, use the overloaded: Class.forName(String name, boolean initialize, ClassLoader loader)
But you will need to ensure that your classloader is able to locate the class to load correctly.
For the classloader you are using, try:
Thread.currentThread().getContextClassLoader()

Which classloader will be used in this case?

I have the following problem.
A HashMap is used to set properties and the key is a ClassLoader.
The code that sets a property is the following (AxisProperties):
public static void setProperty(String propertyName, String value, boolean isDefault){
if(propertyName != null)
synchronized(propertiesCache)
{
ClassLoader classLoader = getThreadContextClassLoader();
HashMap properties = (HashMap)propertiesCache.get(classLoader);
if(value == null)
{
if(properties != null)
properties.remove(propertyName);
} else
{
if(properties == null)
{
properties = new HashMap();
propertiesCache.put(classLoader, properties);
}
properties.put(propertyName, new Value(value, isDefault));
}
}
}
One of these values is cached somewhere and I need to reset this hashmap but the problem is I don't know how to do this.
I thought to load the class (delegating to axis using a URLClassLoader) but I see that the code does getThreadContextClassLoader(); which is:
public ClassLoader getThreadContextClassLoader()
{
ClassLoader classLoader;
try
{
classLoader = Thread.currentThread().getContextClassLoader();
}
catch(SecurityException e)
{
classLoader = null;
}
return classLoader;
}
So I think it will use the classloader of my current thread not the one that I used to load the class to use (i.e. axis).
So is there a way around this?
Note: I already have loaded axis as part of my application. So the idea would be to reload it via a different classloader

If you know the class loader in question, you can set the context class loader before making the call into axis:
ClassLoader key = ...;
ClassLoader oldCtx = Thread.currentThread().getContextClassLoader();
try {
Thread.currentThread().setContextClassLoader(key);
// your code here.
}
finally {
Thread.currentThread().setContextClassLoader(oldCtx);
}
You often have to do this in cases when you are outside a servlet container, but the library assumes that you are in one. For instance, you have to do this with CXF in an OSGi container, where the semantics of context class loader are not defined. You can use a template like this to keep things clean:
public abstract class CCLTemplate<R>
{
public R execute( ClassLoader context )
throws Exception
{
ClassLoader oldCtx = Thread.currentThread().getContextClassLoader();
try {
Thread.currentThread().setContextClassLoader(context);
return inContext();
}
finally {
Thread.currentThread().setContextClassLoader(oldCtx);
}
}
public abstract R inContext() throws Exception;
}
And then do this when interacting with Axis:
ClassLoader context = ...;
new CCLTemplate<Void>() {
public Void inContext() {
// your code here.
return null;
}
}.execute(context);

Chained classloader conundrum

I'm having some hard time with Java classloaders, maybe somebody could shed some light on this. I have extracted the essence of the problem to the follwing:
There are three classes - ClassLoaderTest, LoadedClass and LoadedClassDep. They are all on different paths.
ClassLoaderTest instantiates a new URLClassLoader - myClassLoader, priming it with the paths to the remaining two classes and it's own classloader (i.e. the application classloader) as parent. It then uses Class.forName("com.example.LoadedClass", true, myClassLoader) to load the LoadedClass through reflection. The LoadedClass imports the LoadedClassDep. If I run the above, using:
java -cp /path/to/the/ClassLoaderTest ClassLoaderTest "/path/to/LoadedClass" "/path/to/LoadedClassDep"
and using the command line arguments to prime the URLClassLoader everything works fine. Using static initialisers I confirm that the two classes are loaded with an instance of a URLClassLoader.
HOWEVER, and this is the problem, if I do:
java -cp /path/to/the/ClassLoaderTest:/path/to/the/LoadedClass ClassLoaderTest "/path/to/LoadedClassDep"
this fails to load the LoadedClassDep (ClassNotFoundException). The LoadedClass is loaded correctly, but with sun.misc.Launcher$AppClassLoader, not the URLClassLoader!
It would appear that since the application classloader is capable of loading the LoadedClass it also attempts to load the LoadedClassDep, disregarding the URLClassLoader.
Here's the full source code:
package example.bc;
public class ClassloaderTest {
public static void main(String[] args) {
new ClassloaderTest().run(args);
}
private void run(String[] args) {
URLClassLoader myClasLoader = initClassLoader(args);
try {
Class<?> cls = Class.forName("com.example.bc.LoadedClass", true, myClasLoader);
Object obj = cls.newInstance();
cls.getMethod("call").invoke(obj);
} catch (Exception e) {
e.printStackTrace();
}
}
private URLClassLoader initClassLoader(String[] args) {
URL[] urls = new URL[args.length];
try {
for (int i = 0; i < args.length; i++) {
urls[i] = new File(args[i]).toURI().toURL();
}
} catch (MalformedURLException e) {
e.printStackTrace();
}
return new URLClassLoader(urls, getClass().getClassLoader());
}
}
package com.example.bc;
import com.bc.LoadedClassDep;
public class LoadedClass {
static {
System.out.println("LoadedClass " + LoadedClass.class.getClassLoader().getClass());
}
public void call() {
new LoadedClassDep();
}
}
package com.bc;
public class LoadedClassDep {
static {
System.out.println("LoadedClassDep " + LoadedClassDep.class.getClassLoader().getClass());
}
}
I hope I made this clear enough. My issue is, I only know the path to ClassLoadeTest at compile time, I have to use strings at runtime for the other paths. So, any ideas how to make the second scenario work?

I'd expect the application classloader to load LoadedClass in the second case, since classloaders delegate to their parent initially - this is the standard behaviour. In the second case, LoadedClass is on the parent's classpath, so it loads the class instead of giving up and letting the URLClassLoader try.
The application classloader then attempts to load the LoadedClassDep because it is imported and referenced directly in LoadedClass:
public void call() {
new LoadedClassDep();
}
If you need to load these classes dynamically and independently at runtime, you can't have direct references between them in this way.
It is also possible to change the order in which classloaders are tried - see Java classloaders: why search the parent classloader first? for some discussion of this.