Develop Reference

How to exclude classes generated with WSDL from Spring-Devtools restart cycle?

When I start Spring Boot application with Spring-Devtools enabled and classes generated from the WSDL schema I get:
Caused by: java.lang.IllegalArgumentException: org.wsdl.WsdlServiceWs referenced from a method is not visible from class loader
I have a project based on Spring Boot with some of the classes generated from the WSDL file using the org.apache.cxf:cxf-codegen-plugin plugin. Generated classes are stored in target/generated/wsdl/** directory. The name of the package of generated classes differs from the project package name.
I tried several exclusions following the documentation:
https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#using-boot-devtools-restart-exclude
But all my attempts failed.
restart.exclude.wsdl=target/generated/wsdl
restart.exclude.wsdl=org.wsdl.*
restart.exclude.wsdl=**WsdlServiceWs.class
I want to have Spring-Devtools enabled, having org.wsdl.** generated classes excluded from the restart cycle.

The problem was, that I tried to use the WsdlServiceWs which was in fact an interface returned by WsdlServiceWsService. I had the WsdlServiceWs interface returned as a bean in the configuration:
...
#Bean
public WsdlServiceWs wsdlService() {
return new WsdlServiceWsService().getService();
}
...
I have not thought that this will be the problem. Simply changing the bean to the following:
...
#Bean
public WsdlServiceWsService wsdlService() {
return new WsdlServiceWsService();
}
...
Did the work.
Edit:
This solution only moved the invocation of exception from the Bean creation phase to the execution phase. The issue is still not resolved.

You Can't
because the devtools only check the class parent path, not every folder, you can add an breakpoint on ChangeableUrls.java:59
private ChangeableUrls(URL... urls) {
DevToolsSettings settings = DevToolsSettings.get();
List<URL> reloadableUrls = new ArrayList<>(urls.length);
for (URL url : urls) {
if ((settings.isRestartInclude(url) || isDirectoryUrl(url.toString())) && !settings.isRestartExclude(url)) {
reloadableUrls.add(url);
}
}
if (logger.isDebugEnabled()) {
logger.debug("Matching URLs for reloading : " + reloadableUrls);
}
this.urls = Collections.unmodifiableList(reloadableUrls);
}
you can see the url is file:/xxx/target/classes/
so, you can't exclude one class by this way

Using SystemLoader / SpringFactoryLoader to load external Jar in Spring-Project

first: I'm really new to spring-boot and maven. So I still don't get how everything plugs together.
What I'm trying to achieve is some kind of plugin-feature for my application. From my research it seems the best way to do this is using ServiceLoader or the spring-boot implmentation of the SpringFactoriesLoader.
According to several instructions from the web I put two projects together
James (the main application) GitHub
TemperatureSensor (the plugin) GitHub
The JamesApplication provides an interfaces which is supposed to be implemented (de.maxrakete.james.device.domain.DeviceInterface).
The TemperatureSensor implements said class and exposes this in several ways.
For the ServiceLoader in in the file META-INF\services\de.maxrakete.james.device.domain.DeviceInterface with this content
de.maxrakete.james.plugin.TemperatureSensor.TemperatureSensor
For the SpringFactoriesLoader in the file META-INF\spring.factories with this content
de.maxrakete.james.device.domain.DeviceInterface=de.maxrakete.james.plugin.TemperatureSensor.TemperatureSensor
According to this page I tried two different implementations (see in the onApplicationEvent-function) in the MainApplication:
#SpringBootApplication
public class JamesApplication implements ApplicationListener<ApplicationReadyEvent> {
public static void main(String[] args) {
SpringApplication.run(JamesApplication.class, args);
ClassLoader cl = ClassLoader.getSystemClassLoader();
URL[] urls = ((URLClassLoader)cl).getURLs();
for(URL url: urls){
System.out.println("Classpath file: " + url.getFile());
}
}
#Override
public void onApplicationEvent(ApplicationReadyEvent event) {
ServiceLoader<DeviceInterface> loader = ServiceLoader.load(DeviceInterface.class);
loader.iterator();
List<DeviceInterface> foos = SpringFactoriesLoader.loadFactories(DeviceInterface.class, null);
}
}
I'm trying both ways to load the jar, but nothing is happening (I'm supposed to get some log-messages from the plugin) but this is not happening.
The way I'm running the application is like this:
java -cp "./plugins/TemperatureSensor-0.0.1-SNAPSHOT.jar" -jar james.war
As you see I'm trying to add the jar in the subfolder to the classpath, but in the ouput of the main-function (where I try to print all the files in the classpath) I only get Classpath file: /home/max/folder/james.war
Conclusion
So, there are three possible error-sources
Wrong cli command to add classpath files
Wrong declaration of interfaces in the META-INF folder
Wrong implementation of the Loader
Maybe I'm compiling the sources the wrong way?
Wrong configuration of the pom.xml
I really have no idea what the problem might be. I tried to provide you with as much information as possible and all the steps of my research. I hope someone finds some helpful clues, which I might have overlooked.
Thanks veryone!

Instantiating classes from a jar that implement a common interface, and then assigning the instance to the interface causes ClassCastException

This is a classloader issue that I am struggling with. I understand the root cause of the issue (different classloaders), but I'm not sure about the best way to fix it.
I have project with some common interfaces; let's call it api. I have two other projects called runner and module that both use api as a dependency.
The job of runner is to dynamically load a module artifact (from a jar; it's a fat one that includes its dependencies) and then execute it. runner expects module to provide certain concrete implementations from api. To make sure that classes from different versions of module.jar don't clobber each other, I create a new classloader with a URL to module.jar, and set the parent classloader to the classloader of the class that loads and processes module.jar. This works without any issues.
The problem arose when I used runner as a dependency inside a webapp (a spring boot app to be specific), and quickly found that I couldn't load some classes from module.jar because they conflict with classes that already exist in the current classpath (from other dependencies in the webapp).
Since module.jar really only needs the classes from api, I thought that I could create a new URLClassLoader (without a parent) that only has classes from api.jar, and then use that as the parent classloader when I load up the module. This is where I started running into trouble:
CommonInterface commonInterface = null;
Class<CommonInterface> commonInterfaceClass = null;
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL, apiClassesClassLoader);
//...
//...
//clazz is a concrete implementation from module.jar
if(myClassLoader.loadClass(CommonInterface.class.getName()).isAssignableFrom(clazz)) {
commonInterfaceClass = clazz;
}
commonInterface = commonInterfaceClass.newInstance(); //ClassCastException
I understand that my original problem is due to the fact that the classloader first checks to see if the class has already been loaded before attempting to load it, which meant that when it was resolved using the name from module.jar, it was linking against an incompatible version of the class.
What's a good way to deal with this issue? Instead of creating a URL classloader that only has classes from api, does it make sense to create my own implementation that delegates to the parent only if the requested class is one from api?

You have loaded CommonInterface from two different class loaders. Classes with the same name but different class loaders are different classes to the JVM. (Even if the classes are 100% identical in the .class file - the problem is not incompatibility but the fact that they're from different class loaders)
If you do a
System.out.println(CommonInterface.class == myClassLoader.loadClass(CommonInterface.class.getName()));
You'll find that this prints false.
The way your create your classloader:
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL, apiClassesClassLoader);
.. would only work if apiClassesClassLoader is also a parent class loader of the class that contains this code.
You could try:
ClassLoader myClassLoader = URLClassLoader.newInstance(moduleJarURL,
getClass().getClassLoader());
But from your description (it's a "fat" jar that contains its own dependencies) and the intricacies of the web classloader (child first) this may not solve your problem.
In that case, the only solution is to make your module jar "lean" to ensure that you only load each class once with one class loader only.

I forgot to update this question with my solution. I was able to solve this issue by creating a custom class-loader that extends URLClassLoader. This classloader does not have a parent.
I then overrode loadClass to control how classes were being loaded. I first check to see if the class exists in module.jar. If so, I load it from there. Otherwise, I load it using the current classloader. Since my custom classloader doesn't have a parent, it can load classes from module.jar even if they were already loaded by the main classloader, because they do not exist in my custom classloader's hierarchy.
The basic approach was like this:
public class MyClassLoader extends URLClassLoader {
private final ClassLoader mainClassLoader = MyClassLoader.class.getClassLoader();
private final Set<String> moduleClasses;
private MyClassLoader(URL url) {
super(new URL[]{ url });
try {
JarURLConnection connection = (JarURLConnection) url.openConnection();
this.moduleClasses = connection.getJarFile().stream()
.map(JarEntry::getName)
.filter(name -> name.endsWith(".class"))
.map(name -> name.replace(".class", "").replaceAll("/", "."))
.collect(Collectors.toSet());
} catch(IOException e) {
throw new IllegalArgumentException(String.format("Unexpected error while reading module jar: %s", e.getMessage()));
}
}
public static MyClassLoader newInstance(JarFile libraryJar) {
try {
return new MyClassLoader(new URL(String.format("jar:file:%s!/", libraryJar.getName())));
} catch(MalformedURLException e) {
throw new IllegalArgumentException(String.format("Path to module jar could not be converted into proper URL: %s", e.getMessage()));
}
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
if(moduleClasses.contains(name)) {
Class<?> clazz = findLoadedClass(name);
if(clazz != null) {
return clazz;
} else {
return findClass(name);
}
} else {
return mainClassLoader.loadClass(name);
}
}
}

Dynamic ClassLoader

I have a large desktop Java application and I want to allow other developers to develop plugins for. Plugins will be jars placed in a specified dir. They will not be on the classpath at startup. I will load and deploy them at runtime.
The complication is that some plugins will have dependencies on each other, as well as the core application. So I cannot load each plugin/jar in its own URLClassLoader. Therefore I want to load all plugins into 1 URLClassLoader. Furthermore, some plugins may fail to initialise for various reasons. And I only want a ClassLoader at the end of day that knows about the successfully loaded plugins. The reasons are quite bizarre and relate to some legacy stuff that is using reflection to instantiate classes. This needs to fail if the plugin doesn't initialise for classes defined inside the plugin jar that failed.
Without this requirement, the solution would be:
Collect the jar URLs and build a ClassLoader based on them
Try to initialise a plugin class from each jar (defined in config in the manifest)
Now the ClassLoader here would be passed to the legacy system for it to use for its reflection stuff. However, it's my understanding that it will still be able to instantiate classes from plugin jars whose plugin failed to initialise (since the jar will still in the URL[] of the ClassLoader). Hence this breaks my requirement above.
The only solution I have come up with so far is to create a custom URLClassLoader as follows (simply to allow access to findClass()):
public class CustomURLClassLoader extends URLClassLoader {
public CustomURLClassLoader(final URL[] urls, final ClassLoader parent) {
super(urls, parent);
}
#Override
protected Class<?> findClass(final String name) throws ClassNotFoundException {
return super.findClass(name);
}
}
And then I made another custom ClassLoader that essentially knows about multiple child ClassLoaders:
public class MultiURLClassLoader extends ClassLoader {
private Set<CustomURLClassLoader> loaders = new HashSet<CustomURLClassLoader>();
public MultiURLClassLoader(final ClassLoader parent) {
super(parent);
}
#Override
protected Class<?> findClass(final String name) throws ClassNotFoundException {
Iterator<CustomURLClassLoader> loadersIter = loaders.iterator();
boolean first = true;
while (first || loadersIter.hasNext()) {
try {
if (first) {
return super.findClass(name);
} else {
return loadersIter.next().findClass(name);
}
} catch (ClassNotFoundException e) {
first = false;
}
}
throw new ClassNotFoundException(name);
}
public void addClassLoader(final CustomURLClassLoader classLoader) {
loaders.add(classLoader);
}
public void removeClassLoader(final CustomURLClassLoader classLoader) {
loaders.remove(classLoader);
}
}
Then my loading plugin alogorithm will be something like
MultiURLClassLoader multiURLClassLoader = new MultiURLClassLoader(ClassLoader.getSystemClassLoader());
for (File pluginJar : new File("plugindir").listFiles()) {
CustomURLClassLoader classLoader = null;
try {
URL pluginURL = pluginJar.toURI().toURL();
final URL[] pluginJarUrl = new URL[] { pluginURL };
classLoader = new CustomURLClassLoader(pluginJarUrl, multiURLClassLoader);
multiURLClassLoader.addClassLoader(classLoader);
Class<?> clazz = Class.forName("some.PluginClass", false, multiURLClassLoader);
Constructor<?> ctor = clazz.getConstructor();
SomePluginInterface plugin = (SomePluginInterface)ctor1.newInstance();
plugin.initialise();
} catch (SomePluginInitialiseException e) {
multiURLClassLoader.removeClassLoader(classLoader);
}
}
Then I can pass the multiURLClassLoader instance onto the legacy system and it will only be able to find classes (via reflection) whose plugin successfully loaded.
I've done some basic testing and it seems to work as I'd like so far. But I would very much like someones opinion on whether this seems like a good idea or not? I have never played this much with ClassLoaders before and I am wanting to avoid getting myself in too deep before its too late.
Thanks!

The problem I see is that if you don't know in advance which plugin depends on which, it's very hard to do anything reasonable, to debug problems, to isolate non-functional or bad-behaving plugins, etc.
Therefore I'd suggest another option: Add another field into each plugin's manifest, which will say on what other plugins it depends. Perhaps just a list of other plugin JARs it needs to function. (The core application classes would be always available.) I believe this would make the design much more robust and simplify many things.
Then, you could choose from different designs, for example:
For each plugin you could create a separate ClassLoader that would load just the JARs it needs. Probably the most robust solution. But I see a drawback: plugins that act as dependencies for many other ones will be loaded repeatedly in different class-loaders. It depends on circumstances (plugin count, JARs size, ...) if this could be a problem or not, it could even be an advantage.
You could have one big ClassLoader for all plugins, as you suggest, but you could ask it for plugin classes in the order of their dependencies. The ones that don't depend on anything first, then the ones that depend on those first ones etc. If some plugin class fails to load/initialize, you could immediately discard all plugins that depend on it.

Are you looking for something like the OSGi approach?
You could do something like Petr Pudlák has said, however you should take in account the fact that one of the solutions you have can create cyclic dependencies...

Java: Which of multiple resources on classpath JVM takes?

If I have multiple files of the same name on classpath (e.g. I have multiple .jar with log4j.properties), what are the rules JVM follows to chose one?

It is specified by the order in which the resources (i.e. usually jar files) are specified using -classpath option. Resources 'earlier' on the classpath take precedence over resources that are specified after them. This can be also set in the manifest file of your application and then you don't need to provide -classpath option. You may want to check these articles on how to work with manifest files.
The exhaustive description of "how classes are found" can be found here, where the section on JAR-class-path Classes describes the logic of JAR-files searching.

The ClassLoader determines where a resource will be located (taken from ClassLoader JavaDoc):
The ClassLoader class uses a delegation model to search for classes and resources. Each instance of ClassLoader has an associated parent class loader. When requested to find a class or resource, a ClassLoader instance will delegate the search for the class or resource to its parent class loader before attempting to find the class or resource itself. The virtual machine's built-in class loader, called the "bootstrap class loader", does not itself have a parent but may serve as the parent of a ClassLoader instance.
So wherever in your code Class#getResource or Class#getResourceAsStream is called, this happens (taken from Class.java)
public java.net.URL getResource(String name) {
name = resolveName(name);
ClassLoader cl = getClassLoader0();
if (cl==null) {
// A system class.
return ClassLoader.getSystemResource(name);
}
return cl.getResource(name);
}
ClassLoader.java:
public URL getResource(String name) {
URL url;
if (parent != null) {
url = parent.getResource(name);
} else {
url = getBootstrapResource(name);
}
if (url == null) {
url = findResource(name);
}
return url;
}
where ClassLoader#findResource is actually to be overwritten by the ClassLoader implementation. This implies that the behavior is different on an application server, a TomCat or if you are running from a jar file, it depends on the ClassLoader implementations of the environment you are currently in.
Here is an example that you may use to trace what's going under the hood in your particular case.

I am contributing a proven case that if classpath is, say, all jars in a folder, and you want to prioritize one (or some) of them, this does not work:
Windows:
bin/prioritized.jar;bin/*
Linux:
bin/prioritized.jar:bin/*
It appears that the first path bin/prioritized.jar is ignored just because the second one with a wildcard includes it in its own scope. This is what effectivelly breaks the specified order of classpaths.
Therefore, in order to have multiple resources prioritized (tested on Java 10.0.1), you need to put them in non-overlapping scopes and then they will work.