Develop Reference

How to load a JDBC driver dynamically during runtime since Java 9?

I'm currently migrating my Java 8 code to Java 11 and stumbled across a problem. I'm looking for jar files in a directory and add them to the classpath in order to use them as JDBC drivers.
After doing so I can easily use DriverManager.getConnection(jdbcString); to get a connection to any database I loaded a driver beforehand.
I used to load drivers using this bit of code which no longer works since the SystemClassLoader is no longer a URLClassLoader.
Method method = URLClassLoader.class.getDeclaredMethod("addURL", new Class[] { URL.class });
method.setAccessible(true);
method.invoke(ClassLoader.getSystemClassLoader(), new Object[] { jdbcDriver.toURI().toURL() });
So after looking around for alternatives I found this answer on SO:
https://stackoverflow.com/a/14479658/10511969
Unfortunately for this approach I'd need the drivers class name, i.e. "org.postgresql.Driver" which I don't know.
Is there just no way to do this anymore, or am I missing something?

Using a Shim is a good way to load the JDBC driver when the driver is, for some reason, not accessibile via the system class loader context. I have ran into this a few times with multi-threaded scripts that have their own separated classpath context.
http://www.kfu.com/~nsayer/Java/dyn-jdbc.html

Not knowing the driver's class seems like an odd constraint.
I would go for a custom class loader that after ever class initialisation (I think you can do that), calls DriverManager.getDrivers and registers any new drivers it finds. (I have no time at the moment to write the code.)
The hacky alternative would be to load all your code (except a bootstrap) in a URLClassLoader and addURL to that.
Edit: So I wrote some code.
It creates a class loader for the drivers that also contains a "scout" class that forwards DriverManager.drivers (which is a naughty caller sensitive method (a newish one!)). A fake driver within the application class loader forwards connect attempts onto any dynamically loaded drivers at the time of request.
I don't have any JDBC 4.0 or later drivers conveniently around to test this on. You'll probably want to change the URL - you'll need the Scout class and the driver jar.
import java.lang.reflect.*;
import java.net.*;
import java.sql.*;
import java.util.*;
import java.util.logging.*;
import java.util.stream.*;
class FakeJDBCDriver {
public static void main(String[] args) throws Exception {
URLClassLoader loader = URLClassLoader.newInstance(
new URL[] { new java.io.File("dynamic").toURI().toURL() },
FakeJDBCDriver.class.getClassLoader()
);
Class<?> scout = loader.loadClass("Scout");
Method driversMethod = scout.getMethod("drivers");
DriverManager.registerDriver(new Driver() {
public int getMajorVersion() {
return 0;
}
public int getMinorVersion() {
return 0;
}
public Logger getParentLogger() throws SQLFeatureNotSupportedException {
throw new SQLFeatureNotSupportedException();
}
public DriverPropertyInfo[] getPropertyInfo(String url, Properties info) {
return new DriverPropertyInfo[] { };
}
public boolean jdbcCompliant() {
return false;
}
public boolean acceptsURL(String url) throws SQLException {
if (url == null) {
throw new SQLException();
}
for (Iterator<Driver> iter=drivers(); iter.hasNext(); ) {
Driver driver = iter.next();
if (
driver.getClass().getClassLoader() == loader &&
driver.acceptsURL(url)
) {
return true;
}
}
return false;
}
public Connection connect(String url, Properties info) throws SQLException {
if (url == null) {
throw new SQLException();
}
for (Iterator<Driver> iter=drivers(); iter.hasNext(); ) {
Driver driver = iter.next();
if (
driver.getClass().getClassLoader() == loader &&
driver.acceptsURL(url)
) {
Connection connection = driver.connect(url, info);
if (connection != null) {
return connection;
}
}
}
return null;
}
private Iterator<Driver> drivers() {
try {
return ((Stream<Driver>)driversMethod.invoke(null)).iterator();
} catch (IllegalAccessException exc) {
throw new Error(exc);
} catch (InvocationTargetException exc) {
Throwable cause = exc.getTargetException();
if (cause instanceof Error) {
throw (Error)cause;
} else if (cause instanceof RuntimeException) {
throw (RuntimeException)cause;
} else {
throw new Error(exc);
}
}
}
});
// This the driver I'm trying to access, but isn't even in a jar.
Class.forName("MyDriver", true, loader);
// Just some nonsense to smoke test.
System.err.println(DriverManager.drivers().collect(Collectors.toList()));
System.err.println(DriverManager.getConnection("jdbc:mydriver"));
}
}
Within a directory dynamic (relative to current working directory):
import java.sql.*;
public interface Scout {
public static java.util.stream.Stream<Driver> drivers() {
return DriverManager.drivers();
}
}
I would always suggest avoiding setting the thread context class loader to anything other than a loader that denies everything, or perhaps null.
Modules may well allow you to load drivers cleanly, but I've not looked.

if you don`t know the driver name, you cannot use reflect to use urlLoader to load jar, which you exactly want.
I have same problem with dynamically load driver, because of jars are conflict.
Even though, I have to know the driver name to jar, which i want to load use my url class loader.
DriverManager use class loader to load jar, so it could find jdbc driver by name. As usual we use: class.forName。
We use self defined class loader to load our driver, so that it can solve the conflict of jars.

Remove folder from Java classpath at runtime

Is there a way to remove a folder from the classpath similar to adding a folder at runtime (Can a directory be added to the class path at runtime?)

Please find below a snippet as technical example to demonstrate adding / removing a path.
create following source files in any directory
import java.io.File;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.Stack;
import sun.misc.URLClassPath;
public class EvilPathDemo {
public static void addPath(String path) throws Exception {
URL u = new File(path).toURI().toURL();
URLClassLoader urlClassLoader = (URLClassLoader)
ClassLoader.getSystemClassLoader();
Class<?> urlClass = URLClassLoader.class;
Method method = urlClass.getDeclaredMethod("addURL",
new Class[]{URL.class}
);
method.setAccessible(true);
method.invoke(urlClassLoader, new Object[]{u});
}
public static void removePath(String path) throws Exception {
URL url = new File(path).toURI().toURL();
URLClassLoader urlClassLoader = (URLClassLoader)
ClassLoader.getSystemClassLoader();
Class<?> urlClass = URLClassLoader.class;
Field ucpField = urlClass.getDeclaredField("ucp");
ucpField.setAccessible(true);
URLClassPath ucp = (URLClassPath) ucpField.get(urlClassLoader);
Class<?> ucpClass = URLClassPath.class;
Field urlsField = ucpClass.getDeclaredField("urls");
urlsField.setAccessible(true);
Stack urls = (Stack) urlsField.get(ucp);
urls.remove(url);
}
public static void main(String[] args) throws Exception {
String parm = args.length == 1 ? args[0] : "";
String evilPath = "/tmp";
String classpath = System.getProperty("java.class.path");
boolean isEvilPathSet = false;
for (String path : classpath.split(File.pathSeparator)) {
if (path.equalsIgnoreCase(evilPath)) {
System.out.printf("evil path '%s' in classpath%n", evilPath);
isEvilPathSet = true;
break;
}
}
if (isEvilPathSet && parm.equalsIgnoreCase("REMOVE")) {
System.out.printf("evil path '%s' will be removed%n", evilPath);
removePath(evilPath);
}
tryToLoad("Foo");
if (parm.equalsIgnoreCase("ADD")) {
System.out.printf("evil path '%s' will be added%n", evilPath);
addPath(evilPath);
}
tryToLoad("Bar");
}
private static void tryToLoad(String className) {
try {
Class<?> foo = Class.forName(className);
System.out.printf("class loaded: %s%n", foo.getName());
} catch (ClassNotFoundException ex) {
System.out.println(ex);
}
}
}
.
public class Foo {
static {
System.out.println("I'm foo...");
}
}
.
public class Bar {
static {
System.out.println("I'm bar...");
}
}
compile them as follow
javac EvilPathDemo.java
javac -d /tmp Foo.java Bar.java
During the test we will try to load the classes Foo and Bar.
without /tmp in the classpath
java -cp . EvilPathDemo
java.lang.ClassNotFoundException: Foo
java.lang.ClassNotFoundException: Bar
adding /tmp to the classpath
java -cp . EvilPathDemo add
java.lang.ClassNotFoundException: Foo
evil path '/tmp' will be added
I'm bar...
class loaded: Bar
with /tmp in the classpath
java -cp .:/tmp EvilPathDemo
evil path '/tmp' in the classpath
I'm foo...
class loaded: Foo
I'm bar...
class loaded: Bar
remove /tmp from the classpath
java -cp .:/tmp EvilPathDemo remove
evil path '/tmp' in the classpath
evil path '/tmp' will be removed
java.lang.ClassNotFoundException: Foo
java.lang.ClassNotFoundException: Bar
During the testing I found out that following cases are not working.
addPath(evilPath);
tryToLoad("Foo");
removePath(evilPath); // had not effect
tryToLoad("Bar");
removePath(evilPath);
tryToLoad("Foo");
addPath(evilPath); // had no effect
tryToLoad("Bar");
tryToLoad("Foo");
removePath(evilPath); // had no effect
tryToLoad("Bar");
I did not spent time to find out why. Because I don't see any practical use in it. If you really need/wish to play with the classpaths have a look how classloaders are working.

The removePath method from above did not work for me and my Weld Container, the url stack was always emtpy.
The following ugly smugly method worked:
public static void removeLastClasspathEntry() throws Exception {
URLClassLoader urlClassLoader = (URLClassLoader) ClassLoader.getSystemClassLoader();
Class<?> urlClass = URLClassLoader.class;
Field ucpField = urlClass.getDeclaredField("ucp");
ucpField.setAccessible(true);
URLClassPath ucp = (URLClassPath) ucpField.get(urlClassLoader);
Field loadersField = URLClassPath.class.getDeclaredField("loaders");
loadersField.setAccessible(true);
List jarEntries = (List) loadersField.get(ucp);
jarEntries.remove(jarEntries.size() - 1);
Field pathField = URLClassPath.class.getDeclaredField("path");
pathField.setAccessible(true);
List pathList = (List) pathField.get(ucp);
URL jarUrl = (URL) pathList.get(pathList.size() - 1);
String jarName = jarUrl.toString();
pathList.remove(pathList.size() - 1);
Field lmapField = URLClassPath.class.getDeclaredField("lmap");
lmapField.setAccessible(true);
Map lmapMap = (Map) lmapField.get(ucp);
lmapMap.remove(jarName.replaceFirst("file:/", "file:///"));
}

Class loaders can be nested so instead of modifying the system class loader which is the root of the tree of class loaders, it is better to simply create a nested classloader and use that to load classes.
The system classloader itself is immutable (for good reasons) but you can do whatever you want in nested class loaders, including destroying them to unload classes and resources. This is commonly used in e.g. osgi and application servers to load/unload e.g. plugins, applications, etc.
For nested class loaders you can completely customize how to load classes. The URLClassloader is probably a good starting point for what you want.

I dont think there is a straight forward way to do it. You can follow :
Get class path variables using : System.getenv("CLASSPATH"). It will return semi colon separated values.
String classPath = System.getenv("CLASSPATH")
Take the folder path as input and replace it with "" like :
String remainigPath = classPath.replace(inputpath,"");
Put the remaining paths in an array using split method.
String[] paths = remainigPath .split(";");
For adding classPath, You already have the code.

I had the same issue, so I tackled it by creating a library that works on every ClassLoader that uses a URLClassPath (so, currently, URLClassLoader).
The library has methods for:
Adding new entries in front
Appending new entries
Remove existing entries
Please note that, since this library accesses internal and proprietary APIs, it is no guaranteed to work in future versions of the JDK. It currently does for Java 7 and Java 8 (Oracle and OpenJDK).
Here is the GitHub page (contribution is appreciated), and here is the Maven Central artifact

Container-level Versioned Libraries Shared by WARs

In a Java servlet container (preferably Tomcat, but if this can be done in a different container then say so) I desire something which is theoretically possible. My question here is whether tools exist to support it, and if so what tools (or what names I should research further).
Here is my problem: in one servlet container I want to run a large number of different WAR files. They share some large common libraries (such as Spring). At first blush, I have two unacceptable alternatives:
Include the large library (Spring, for example) in each WAR file. This is unacceptable because it will load a large number of copies of Spring, exhausting the memory on the server.
Place the large library in the container classpath. Now all of the WAR files share one instance of the library (good). But this is unacceptable because I cannot upgrade the Spring version without upgrading ALL of the WAR files at once, and such a large change is difficult verging on impossible.
In theory, though, there is an alternative which could work:
Put each version of the large library into the container-level classpath. Do some container level magic so that each WAR file declares which version it wishes to use and it will find that on its classpath.
The "magic" must be done at the container level (I think) because this can only be achieved by loading each version of the library with a different classloader, then adjusting what classloaders are visible to each WAR file.
So, have you ever heard of doing this? If so, how? Or tell me what it is called so I can research further.

Regarding Tomcat, for the 7th version you can use VirtualWebappLocader like so
<Context>
<Loader className="org.apache.catalina.loader.VirtualWebappLoader"
virtualClasspath="/usr/shared/lib/spring-3/*.jar,/usr/shared/classes" />
</Context>
For the 8th version Pre- & Post- Resources should be used instead
<Context>
<Resources>
<PostResources className="org.apache.catalina.webresources.DirResourceSet"
base="/usr/shared/lib/spring-3" webAppMount="/WEB-INF/lib" />
<PostResources className="org.apache.catalina.webresources.DirResourceSet"
base="/usr/shared/classes" webAppMount="/WEB-INF/classes" />
</Resources>
</Context>
Don't forget to put the corresponding context.xml into the META-INF of your webapp.
For the jetty as well as other containers the same technique may be used.
The only difference is in how to specify extra classpath elements for the webapp.
UPDATE
The samples above does not share the loaded classes, but the idea is the same - use custom classloader. Here is just the pretty ugly sample that also tries to prevent classloader leaks during undeployment.
SharedWebappLoader
package com.foo.bar;
import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.WebappLoader;
public class SharedWebappLoader extends WebappLoader {
private String pathID;
private String pathConfig;
static final ThreadLocal<ClassLoaderFactory> classLoaderFactory = new ThreadLocal<>();
public SharedWebappLoader() {
this(null);
}
public SharedWebappLoader(ClassLoader parent) {
super(parent);
setLoaderClass(SharedWebappClassLoader.class.getName());
}
public String getPathID() {
return pathID;
}
public void setPathID(String pathID) {
this.pathID = pathID;
}
public String getPathConfig() {
return pathConfig;
}
public void setPathConfig(String pathConfig) {
this.pathConfig = pathConfig;
}
#Override
protected void startInternal() throws LifecycleException {
classLoaderFactory.set(new ClassLoaderFactory(pathConfig, pathID));
try {
super.startInternal();
} finally {
classLoaderFactory.remove();
}
}
}
SharedWebappClassLoader
package com.foo.bar;
import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.ResourceEntry;
import org.apache.catalina.loader.WebappClassLoader;
import java.net.URL;
public class SharedWebappClassLoader extends WebappClassLoader {
public SharedWebappClassLoader(ClassLoader parent) {
super(SharedWebappLoader.classLoaderFactory.get().create(parent));
}
#Override
protected ResourceEntry findResourceInternal(String name, String path) {
ResourceEntry entry = super.findResourceInternal(name, path);
if(entry == null) {
URL url = parent.getResource(name);
if (url == null) {
return null;
}
entry = new ResourceEntry();
entry.source = url;
entry.codeBase = entry.source;
}
return entry;
}
#Override
public void stop() throws LifecycleException {
ClassLoaderFactory.removeLoader(parent);
}
}
ClassLoaderFactory
package com.foo.bar;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
public class ClassLoaderFactory {
private static final class ConfigKey {
private final String pathConfig;
private final String pathID;
private ConfigKey(String pathConfig, String pathID) {
this.pathConfig = pathConfig;
this.pathID = pathID;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ConfigKey configKey = (ConfigKey) o;
if (pathConfig != null ? !pathConfig.equals(configKey.pathConfig) : configKey.pathConfig != null)
return false;
if (pathID != null ? !pathID.equals(configKey.pathID) : configKey.pathID != null) return false;
return true;
}
#Override
public int hashCode() {
int result = pathConfig != null ? pathConfig.hashCode() : 0;
result = 31 * result + (pathID != null ? pathID.hashCode() : 0);
return result;
}
}
private static final Map<ConfigKey, ClassLoader> loaders = new HashMap<>();
private static final Map<ClassLoader, ConfigKey> revLoaders = new HashMap<>();
private static final Map<ClassLoader, Integer> usages = new HashMap<>();
private final ConfigKey key;
public ClassLoaderFactory(String pathConfig, String pathID) {
this.key = new ConfigKey(pathConfig, pathID);
}
public ClassLoader create(ClassLoader parent) {
synchronized (loaders) {
ClassLoader loader = loaders.get(key);
if(loader != null) {
Integer usageCount = usages.get(loader);
usages.put(loader, ++usageCount);
return loader;
}
Properties props = new Properties();
try (InputStream is = new BufferedInputStream(new FileInputStream(key.pathConfig))) {
props.load(is);
} catch (IOException e) {
throw new RuntimeException(e);
}
String libsStr = props.getProperty(key.pathID);
String[] libs = libsStr.split(File.pathSeparator);
URL[] urls = new URL[libs.length];
try {
for(int i = 0, len = libs.length; i < len; i++) {
urls[i] = new URL(libs[i]);
}
} catch (MalformedURLException e) {
throw new RuntimeException(e);
}
loader = new URLClassLoader(urls, parent);
loaders.put(key, loader);
revLoaders.put(loader, key);
usages.put(loader, 1);
return loader;
}
}
public static void removeLoader(ClassLoader parent) {
synchronized (loaders) {
Integer val = usages.get(parent);
if(val > 1) {
usages.put(parent, --val);
} else {
usages.remove(parent);
ConfigKey key = revLoaders.remove(parent);
loaders.remove(key);
}
}
}
}
context.xml of the first app
<Context>
<Loader className="com.foo.bar.SharedWebappLoader"
pathConfig="${catalina.base}/conf/shared.properties"
pathID="commons_2_1"/>
</Context>
context.xml of the second app
<Context>
<Loader className="com.foo.bar.SharedWebappLoader"
pathConfig="${catalina.base}/conf/shared.properties"
pathID="commons_2_6"/>
</Context>
$TOMCAT_HOME/conf/shared.properties
commons_2_1=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.1/commons-lang-2.1.jar
commons_2_6=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar

I was able to implement this for Tomcat (Tested on Tomcat 7.0.52). My solution involves implementing custom version of WebAppLoader which extends standard Tomcat's WebAppLoader. Thanks to this solution you can pass custom classloader to load classes for each of web application.
To utilize this new loader you need to declare it for each application (either in Context.xml file placed in each war or in Tomcat's server.xml file). This loader takes an extra custom parameter webappName which is later passed to LibrariesStorage class to determine which libraries should be used by which application.
<Context path="/pl-app" >
<Loader className="web.DynamicWebappLoader" webappName="pl-app"/>
</Context>
<Context path="/my-webapp" >
<Loader className="web.DynamicWebappLoader" webappName="myApplication2"/>
</Context>
Once this is defined you need to install this DynamicWebappLoader to Tomcat. To do this copy all copiled classes to lib directory of Tomcat (so you should have following files [tomcat dir]/lib/web/DynamicWebappLoader.class, [tomcat dir]/lib/web/LibrariesStorage.class, [tomcat dir]/lib/web/LibraryAndVersion.class, [tomcat dir]/lib/web/WebAppAwareClassLoader.class).
You need also to download xbean-classloader-4.0.jar and place it in Tomcat's lib dir (so you should have [tomcat dir]/lib/xbean-classloader-4.0.jar. NOTE:xbean-classloader provides special implementation of classloader (org.apache.xbean.classloader.JarFileClassLoader) which allowes to load needed jars at runtime.
Main trick is made in LibraryStorgeClass (full implementation is at the end). It stores a mapping between each application (defined by webappName) and libraries which this application is allowed to load. In current implementation this is hardcoded, but this can be rewritten to dynamically generate list of libs needed by each application. Each library has its own instance of JarFileClassLoader which ensures that each library is only loaded one time (the mapping between library and its classloader is stored in static field "libraryToClassLoader", so this mapping is the same for every web application because of static nature of the field)
class LibrariesStorage {
private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";
private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();
private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();
static {
try {
addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");
mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars
mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");
(...)
}
In above example, suppose that in directory with all the jars (defined here by JARS_DIR) we have only a commons-lang3-3.3.2.jar file. This would mean that application identified by "pl-app" name (the name comes from webappName attribute in tag in Context.xml as mentioned above) will be able to load classes from commons-lang jar. Application identified by "myApplication2" will get ClassNotFoundException at this point because it has access only to commons-lang3-3.3.0.jar, but this file is not present in JARS_DIR directory.
Full implementation here:
package web;
import org.apache.catalina.loader.WebappLoader;
import org.apache.xbean.classloader.JarFileClassLoader;
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class DynamicWebappLoader extends WebappLoader {
private String webappName;
private WebAppAwareClassLoader webAppAwareClassLoader;
public static final ThreadLocal lastCreatedClassLoader = new ThreadLocal();
public DynamicWebappLoader() {
super(new WebAppAwareClassLoader(Thread.currentThread().getContextClassLoader()));
webAppAwareClassLoader = (WebAppAwareClassLoader) lastCreatedClassLoader.get(); // unfortunately I did not find better solution to access new instance of WebAppAwareClassLoader created in previous line so I passed it via thread local
lastCreatedClassLoader.remove();
}
// (this method is called by Tomcat because of Loader attribute in Context.xml - <Context> <Loader className="..." webappName="myApplication2"/> )
public void setWebappName(String name) {
System.out.println("Setting webapp name: " + name);
this.webappName = name;
webAppAwareClassLoader.setWebAppName(name); // pass web app name to ClassLoader
}
}
class WebAppAwareClassLoader extends ClassLoader {
private String webAppName;
public WebAppAwareClassLoader(ClassLoader parent) {
super(parent);
DynamicWebappLoader.lastCreatedClassLoader.set(this); // store newly created instance in ThreadLocal .. did not find better way to access the reference later in code
}
#Override
public Class<?> loadClass(String className) throws ClassNotFoundException {
System.out.println("Load class: " + className + " for webapp: " + webAppName);
try {
return LibrariesStorage.loadClassForWebapp(webAppName, className);
} catch (ClassNotFoundException e) {
System.out.println("JarFileClassLoader did not find class: " + className + " " + e.getMessage());
return super.loadClass(className);
}
}
public void setWebAppName(String webAppName) {
this.webAppName = webAppName;
}
}
class LibrariesStorage {
private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";
private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();
private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();
static {
try {
addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");
mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars
mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");
} catch (MalformedURLException e) {
throw new RuntimeException(e.getMessage(), e);
}
}
private static void mapApplicationToLibrary(String applicationName, String libraryName, String libraryVersion) {
LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
if (!webappLibraries.containsKey(applicationName)) {
webappLibraries.put(applicationName, new ArrayList<LibraryAndVersion>());
}
webappLibraries.get(applicationName).add(libraryAndVersion);
}
private static void addLibrary(String libraryName, String libraryVersion, String filename)
throws MalformedURLException {
LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
URL libraryLocation = new File(JARS_DIR + File.separator + filename).toURI().toURL();
libraryToClassLoader.put(libraryAndVersion,
new JarFileClassLoader("JarFileClassLoader for lib: " + libraryAndVersion,
new URL[] { libraryLocation }));
}
private LibrariesStorage() {
}
public static Class<?> loadClassForWebapp(String webappName, String className) throws ClassNotFoundException {
System.out.println("Loading class: " + className + " for web application: " + webappName);
List<LibraryAndVersion> webappLibraries = LibrariesStorage.webappLibraries.get(webappName);
for (LibraryAndVersion libraryAndVersion : webappLibraries) {
JarFileClassLoader libraryClassLoader = libraryToClassLoader.get(libraryAndVersion);
try {
return libraryClassLoader.loadClass(className); // ok current lib contained class to load
} catch (ClassNotFoundException e) {
// ok.. continue in loop... try to load the class from classloader connected to next library
}
}
throw new ClassNotFoundException("Class " + className + " was not found in any jar connected to webapp: " +
webappLibraries);
}
}
class LibraryAndVersion {
private final String name;
private final String version;
LibraryAndVersion(String name, String version) {
this.name = name;
this.version = version;
}
#Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if ((o == null) || (getClass() != o.getClass())) {
return false;
}
LibraryAndVersion that = (LibraryAndVersion) o;
if ((name != null) ? (!name.equals(that.name)) : (that.name != null)) {
return false;
}
if ((version != null) ? (!version.equals(that.version)) : (that.version != null)) {
return false;
}
return true;
}
#Override
public int hashCode() {
int result = (name != null) ? name.hashCode() : 0;
result = (31 * result) + ((version != null) ? version.hashCode() : 0);
return result;
}
#Override
public String toString() {
return "LibraryAndVersion{" +
"name='" + name + '\'' +
", version='" + version + '\'' +
'}';
}
}

JBoss has a framework called Modules that solves this problem. You can save the shared library with its version and reference it from your war-file.
I have no idea if it works on Tomcat, but it works as a charm on Wildfly.

custom classLoader issue

the problem is next: i took the base classLoader code from here. but my classLoader is specific from a point, that it must be able to load classes from a filesystem(let's take WinOS), so in classLoader must be some setAdditionalPath() method, which sets a path(a directory on a filesystem), from which we'll load class(only *.class, no jars). here is code, which modifies the loader from a link(you can see, that only loadClass is modified), but it doesn't work properly:
public void setAdditionalPath(String dir) {
if(dir == null) {
throw new NullPointerException("");
}
this.Path = dir;
}
public Loader(){
super(Loader.class.getClassLoader());
}
public Class loadClass(String className) throws ClassNotFoundException {
if(Path.length() != 0) {
File file = new File(Path);
try {
// Convert File to an URL
URL url = file.toURL();
URL[] urls = new URL[]{url};
// Create a new class loader with the directory
ClassLoader cl = new URLClassLoader(urls);
ClassLoader c = cl.getSystemClassLoader();
Class cls = c.loadClass(className);
return cls;
} catch (MalformedURLException e) {
} catch (ClassNotFoundException e) {
}
}
return findClass(Path);
}
I'd grateful if anyone helps :)

You can just use framework provided java.net.URLClassLoader. No need to write your own. It supports loading of classes from directories and JAR files.
Any URL that ends with a '/' is assumed to refer to a directory.
Otherwise, the URL is assumed to refer to a JAR file which will be
opened as needed.
It also supports a parent class loader. If this class loader does not suite your requirements, perhaps you can specify in more detail what you need. And in any case, you can look at the source and derive your own class loader class based on that.
Here is a short working snippet of code that should demostrate how to load a class by name from a URLClassLoader:
ClassLoader systemClassLoader = ClassLoader.getSystemClassLoader();
// This URL for a directory will be searched *recursively*
URL classes =
new URL( "file:///D:/code/myCustomClassesAreUnderThisFolder/" );
ClassLoader custom =
new URLClassLoader( new URL[] { classes }, systemClassLoader );
// this class should be loaded from your directory
Class< ? > clazz = custom.loadClass( "my.custom.class.Name" );
// this class will be loaded as well, because you specified the system
// class loader as the parent
Class< ? > clazzString = custom.loadClass( "java.lang.String" );

Can I dynamically discover xml files in the classpath inside an EJB 3 container?

Background:
One of the components of our project operates using spring. Some SQL code is dynamically generated, based on a given XML spring configuration.
At first it was fine to store all the XML configurations in the same package on the classpath, (and then load it as a resource when the service is called) but over time we ended up with a large number of configurations. It came time to separate the configurations into different namespaces.
The Goal
What I want is, given a starting package on the classpath, to recursively walk the directory structure and discover any spring XML files dynamically. (So that as new configurations / packages are added, the files will still be found by the service).
The Problem
I was able to accomplish my goal fine when running outside an EJB container by using Thread.getContextClassloader().getResource(myBasePackage), then getting a File object and using it to walk the tree on the filesystem. Clunky, I know, but it was still classpath relative and it worked.
However, you cannot do this inside an EJB container (you can't interact with the filesystem at all), so I had to use the rather annoying workaround in which I maintain a list of hardcoded packages to search.
The Question
Is there a way (running inside an EJB container) to dynamically walk the classpath (from a given starting location) searching for arbitrary resources?

Short answer: Not while staying in compliance with the EJB spec. Because the spec envisions containers running in all kinds of non-standard situations, it does not make this possible.
Longer answer: Since you are not creating these resources dynamically, I would write a routine that gives you a list of all of the resources at build time and puts them in a dynamically generated file that your EJB knows how to reference. So you basically create a directory listing of packages and files that you can load in the EJB that are referenced in one master file.
Spring answer: Spring supports finding resources on the classpath, although I have no idea how well this works in the EJB context (and I doubt its EJB compliant, but I haven't checked). Some details here.

DISCLAIMER: As already pointed out, creating resources in the classpath is not recommended and depending on the EJB container explicitly forbidden. This may cause you a lot of problems because containers may explode your resources into another folder or even replicate the resources throughout the cluster (if thats the case). In order to create resources dynamically you have to create a custom classloader. So, I would never do it. It is better to access the filesystem directly than the classpath. It is less ugly and eventually cluster-safe if you use a remote filesystem + file locks.
If even after all I explained you still want to play with the classpath, you can try to do something like: get the classloader via
ClassLoader cld = Thread.currentThread().getContextClassLoader();
Starting from a base package enumerate all occurrences
Enumeration<URL> basePackageUrls = cld.getResources(basePackagePath);
Each URL is generally either a file link (file:///home/scott/.../MyResource.properties) or a jar link (file:///lib.jar!/com/domain/MyResource.properties). You have to check the pattern in the URL. Using that, enumerate the contents of the folder using the normal java API and find the subpackages. Proceed until you have scanned all packages.
See the class below (will be released with an open-source project of mine soon). It implemens a classpath scanner that you can pass in a selector. It works like a visitor. It my work for you, if not, get ideas from it. See the sample annotation selector at the end.
public class ClasspathScanner
{
private static final Log log = LogFactory.getLog(ClasspathScanner.class);
private static final String JAR_FILE_PATTERN = ".jar!";
private ClassSelector selector;
private Set<Class<?>> classes;
// PUBLIC METHODS ------------------------------------------------------------------------------
public synchronized Set<Class<?>> scanPackage(String basePackage, ClassSelector selector)
throws Exception
{
if (selector == null)
{
throw new NullPointerException("Selector cannot be NULL");
}
this.selector = selector;
this.classes = new HashSet<Class<?>>();
Set<Class<?>> aux;
try
{
scanClasses0(basePackage);
aux = this.classes;
}
finally
{
this.selector = null;
this.classes = null;
}
return aux;
}
// HELPER CLASSES ------------------------------------------------------------------------------
private void scanClasses0(String basePackage)
throws IOException, ClassNotFoundException, FileNotFoundException
{
File packageDirectory = null;
ClassLoader cld = getLoader();
String basePackagePath = basePackage.replace('.', '/');
Enumeration<URL> basePackageUrls = cld.getResources(basePackagePath);
if (basePackageUrls == null || !basePackageUrls.hasMoreElements())
{
throw new ClassNotFoundException("Base package path not found: [" + basePackagePath
+ "]");
}
while (basePackageUrls.hasMoreElements())
{
String packagePath = basePackageUrls.nextElement().getFile();
if (packagePath.contains(JAR_FILE_PATTERN))
{
scanJarFile(basePackagePath, packagePath);
}
else
{
packageDirectory = new File(packagePath);
scanDirectory(basePackage, packageDirectory);
}
}
}
private void scanDirectory(String packageName, File packagePath)
throws ClassNotFoundException, FileNotFoundException
{
if (packagePath.exists())
{
File[] packageFiles = packagePath.listFiles();
for (File file : packageFiles)
{
if (file.isFile() && file.getName().endsWith(".class"))
{
String fullFileName = packageName + '.' + file.getName();
checkClass(fullFileName);
}
else if (file.isDirectory())
{
scanDirectory(packageName + "." + file.getName(), file);
}
}
}
else
{
throw new FileNotFoundException(packagePath.getPath());
}
}
private void scanJarFile(String basePackagePath, String jarFileUrl)
throws IOException, ClassNotFoundException
{
String jarFilePath = jarFileUrl.substring("file:".length(), jarFileUrl
.indexOf(JAR_FILE_PATTERN)
+ JAR_FILE_PATTERN.length() - 1);
log.debug("URL JAR file path: [" + jarFilePath + "]");
jarFilePath = URLDecoder.decode(jarFilePath, "UTF-8");
log.debug("Decoded JAR file path: [" + jarFilePath + "]");
JarFile jar = new JarFile(new File(jarFilePath));
for (Enumeration<JarEntry> jarFiles = jar.entries(); jarFiles.hasMoreElements();)
{
JarEntry file = jarFiles.nextElement();
String fileName = file.getName();
if (!file.isDirectory() && fileName.endsWith(".class")
&& fileName.startsWith(basePackagePath))
{
String className = fileName.replace('/', '.');
checkClass(className);
}
}
}
private void checkClass(String fullFilePath) throws ClassNotFoundException
{
String className = fullFilePath.substring(0, fullFilePath.length() - 6);
Class<?> c = getLoader().loadClass(className);
if (selector.select(c))
{
classes.add(c);
}
}
private ClassLoader getLoader()
{
ClassLoader loader = Thread.currentThread().getContextClassLoader();
if (loader == null)
{
loader = getClass().getClassLoader();
}
return loader;
}
// INNER CLASSES -------------------------------------------------------------------------------
public interface ClassSelector
{
boolean select(Class<?> clazz);
}
public static class AnnotatedClassSelector implements ClassSelector
{
private final Class<? extends Annotation>[] annotations;
public AnnotatedClassSelector(Class<? extends Annotation>... annotations)
{
this.annotations = annotations;
}
public boolean select(Class<?> clazz)
{
for (Class<? extends Annotation> ac : annotations)
{
if (clazz.isAnnotationPresent(ac))
{
return true;
}
}
return false;
}
}
}