Container-level Versioned Libraries Shared by WARs - java

In a Java servlet container (preferably Tomcat, but if this can be done in a different container then say so) I desire something which is theoretically possible. My question here is whether tools exist to support it, and if so what tools (or what names I should research further).
Here is my problem: in one servlet container I want to run a large number of different WAR files. They share some large common libraries (such as Spring). At first blush, I have two unacceptable alternatives:
Include the large library (Spring, for example) in each WAR file. This is unacceptable because it will load a large number of copies of Spring, exhausting the memory on the server.
Place the large library in the container classpath. Now all of the WAR files share one instance of the library (good). But this is unacceptable because I cannot upgrade the Spring version without upgrading ALL of the WAR files at once, and such a large change is difficult verging on impossible.
In theory, though, there is an alternative which could work:
Put each version of the large library into the container-level classpath. Do some container level magic so that each WAR file declares which version it wishes to use and it will find that on its classpath.
The "magic" must be done at the container level (I think) because this can only be achieved by loading each version of the library with a different classloader, then adjusting what classloaders are visible to each WAR file.
So, have you ever heard of doing this? If so, how? Or tell me what it is called so I can research further.

Regarding Tomcat, for the 7th version you can use VirtualWebappLocader like so
<Context>
<Loader className="org.apache.catalina.loader.VirtualWebappLoader"
virtualClasspath="/usr/shared/lib/spring-3/*.jar,/usr/shared/classes" />
</Context>
For the 8th version Pre- & Post- Resources should be used instead
<Context>
<Resources>
<PostResources className="org.apache.catalina.webresources.DirResourceSet"
base="/usr/shared/lib/spring-3" webAppMount="/WEB-INF/lib" />
<PostResources className="org.apache.catalina.webresources.DirResourceSet"
base="/usr/shared/classes" webAppMount="/WEB-INF/classes" />
</Resources>
</Context>
Don't forget to put the corresponding context.xml into the META-INF of your webapp.
For the jetty as well as other containers the same technique may be used.
The only difference is in how to specify extra classpath elements for the webapp.
UPDATE
The samples above does not share the loaded classes, but the idea is the same - use custom classloader. Here is just the pretty ugly sample that also tries to prevent classloader leaks during undeployment.
SharedWebappLoader
package com.foo.bar;
import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.WebappLoader;
public class SharedWebappLoader extends WebappLoader {
private String pathID;
private String pathConfig;
static final ThreadLocal<ClassLoaderFactory> classLoaderFactory = new ThreadLocal<>();
public SharedWebappLoader() {
this(null);
}
public SharedWebappLoader(ClassLoader parent) {
super(parent);
setLoaderClass(SharedWebappClassLoader.class.getName());
}
public String getPathID() {
return pathID;
}
public void setPathID(String pathID) {
this.pathID = pathID;
}
public String getPathConfig() {
return pathConfig;
}
public void setPathConfig(String pathConfig) {
this.pathConfig = pathConfig;
}
#Override
protected void startInternal() throws LifecycleException {
classLoaderFactory.set(new ClassLoaderFactory(pathConfig, pathID));
try {
super.startInternal();
} finally {
classLoaderFactory.remove();
}
}
}
SharedWebappClassLoader
package com.foo.bar;
import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.ResourceEntry;
import org.apache.catalina.loader.WebappClassLoader;
import java.net.URL;
public class SharedWebappClassLoader extends WebappClassLoader {
public SharedWebappClassLoader(ClassLoader parent) {
super(SharedWebappLoader.classLoaderFactory.get().create(parent));
}
#Override
protected ResourceEntry findResourceInternal(String name, String path) {
ResourceEntry entry = super.findResourceInternal(name, path);
if(entry == null) {
URL url = parent.getResource(name);
if (url == null) {
return null;
}
entry = new ResourceEntry();
entry.source = url;
entry.codeBase = entry.source;
}
return entry;
}
#Override
public void stop() throws LifecycleException {
ClassLoaderFactory.removeLoader(parent);
}
}
ClassLoaderFactory
package com.foo.bar;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
public class ClassLoaderFactory {
private static final class ConfigKey {
private final String pathConfig;
private final String pathID;
private ConfigKey(String pathConfig, String pathID) {
this.pathConfig = pathConfig;
this.pathID = pathID;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ConfigKey configKey = (ConfigKey) o;
if (pathConfig != null ? !pathConfig.equals(configKey.pathConfig) : configKey.pathConfig != null)
return false;
if (pathID != null ? !pathID.equals(configKey.pathID) : configKey.pathID != null) return false;
return true;
}
#Override
public int hashCode() {
int result = pathConfig != null ? pathConfig.hashCode() : 0;
result = 31 * result + (pathID != null ? pathID.hashCode() : 0);
return result;
}
}
private static final Map<ConfigKey, ClassLoader> loaders = new HashMap<>();
private static final Map<ClassLoader, ConfigKey> revLoaders = new HashMap<>();
private static final Map<ClassLoader, Integer> usages = new HashMap<>();
private final ConfigKey key;
public ClassLoaderFactory(String pathConfig, String pathID) {
this.key = new ConfigKey(pathConfig, pathID);
}
public ClassLoader create(ClassLoader parent) {
synchronized (loaders) {
ClassLoader loader = loaders.get(key);
if(loader != null) {
Integer usageCount = usages.get(loader);
usages.put(loader, ++usageCount);
return loader;
}
Properties props = new Properties();
try (InputStream is = new BufferedInputStream(new FileInputStream(key.pathConfig))) {
props.load(is);
} catch (IOException e) {
throw new RuntimeException(e);
}
String libsStr = props.getProperty(key.pathID);
String[] libs = libsStr.split(File.pathSeparator);
URL[] urls = new URL[libs.length];
try {
for(int i = 0, len = libs.length; i < len; i++) {
urls[i] = new URL(libs[i]);
}
} catch (MalformedURLException e) {
throw new RuntimeException(e);
}
loader = new URLClassLoader(urls, parent);
loaders.put(key, loader);
revLoaders.put(loader, key);
usages.put(loader, 1);
return loader;
}
}
public static void removeLoader(ClassLoader parent) {
synchronized (loaders) {
Integer val = usages.get(parent);
if(val > 1) {
usages.put(parent, --val);
} else {
usages.remove(parent);
ConfigKey key = revLoaders.remove(parent);
loaders.remove(key);
}
}
}
}
context.xml of the first app
<Context>
<Loader className="com.foo.bar.SharedWebappLoader"
pathConfig="${catalina.base}/conf/shared.properties"
pathID="commons_2_1"/>
</Context>
context.xml of the second app
<Context>
<Loader className="com.foo.bar.SharedWebappLoader"
pathConfig="${catalina.base}/conf/shared.properties"
pathID="commons_2_6"/>
</Context>
$TOMCAT_HOME/conf/shared.properties
commons_2_1=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.1/commons-lang-2.1.jar
commons_2_6=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar

I was able to implement this for Tomcat (Tested on Tomcat 7.0.52). My solution involves implementing custom version of WebAppLoader which extends standard Tomcat's WebAppLoader. Thanks to this solution you can pass custom classloader to load classes for each of web application.
To utilize this new loader you need to declare it for each application (either in Context.xml file placed in each war or in Tomcat's server.xml file). This loader takes an extra custom parameter webappName which is later passed to LibrariesStorage class to determine which libraries should be used by which application.
<Context path="/pl-app" >
<Loader className="web.DynamicWebappLoader" webappName="pl-app"/>
</Context>
<Context path="/my-webapp" >
<Loader className="web.DynamicWebappLoader" webappName="myApplication2"/>
</Context>
Once this is defined you need to install this DynamicWebappLoader to Tomcat. To do this copy all copiled classes to lib directory of Tomcat (so you should have following files [tomcat dir]/lib/web/DynamicWebappLoader.class, [tomcat dir]/lib/web/LibrariesStorage.class, [tomcat dir]/lib/web/LibraryAndVersion.class, [tomcat dir]/lib/web/WebAppAwareClassLoader.class).
You need also to download xbean-classloader-4.0.jar and place it in Tomcat's lib dir (so you should have [tomcat dir]/lib/xbean-classloader-4.0.jar. NOTE:xbean-classloader provides special implementation of classloader (org.apache.xbean.classloader.JarFileClassLoader) which allowes to load needed jars at runtime.
Main trick is made in LibraryStorgeClass (full implementation is at the end). It stores a mapping between each application (defined by webappName) and libraries which this application is allowed to load. In current implementation this is hardcoded, but this can be rewritten to dynamically generate list of libs needed by each application. Each library has its own instance of JarFileClassLoader which ensures that each library is only loaded one time (the mapping between library and its classloader is stored in static field "libraryToClassLoader", so this mapping is the same for every web application because of static nature of the field)
class LibrariesStorage {
private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";
private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();
private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();
static {
try {
addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");
mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars
mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");
(...)
}
In above example, suppose that in directory with all the jars (defined here by JARS_DIR) we have only a commons-lang3-3.3.2.jar file. This would mean that application identified by "pl-app" name (the name comes from webappName attribute in tag in Context.xml as mentioned above) will be able to load classes from commons-lang jar. Application identified by "myApplication2" will get ClassNotFoundException at this point because it has access only to commons-lang3-3.3.0.jar, but this file is not present in JARS_DIR directory.
Full implementation here:
package web;
import org.apache.catalina.loader.WebappLoader;
import org.apache.xbean.classloader.JarFileClassLoader;
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class DynamicWebappLoader extends WebappLoader {
private String webappName;
private WebAppAwareClassLoader webAppAwareClassLoader;
public static final ThreadLocal lastCreatedClassLoader = new ThreadLocal();
public DynamicWebappLoader() {
super(new WebAppAwareClassLoader(Thread.currentThread().getContextClassLoader()));
webAppAwareClassLoader = (WebAppAwareClassLoader) lastCreatedClassLoader.get(); // unfortunately I did not find better solution to access new instance of WebAppAwareClassLoader created in previous line so I passed it via thread local
lastCreatedClassLoader.remove();
}
// (this method is called by Tomcat because of Loader attribute in Context.xml - <Context> <Loader className="..." webappName="myApplication2"/> )
public void setWebappName(String name) {
System.out.println("Setting webapp name: " + name);
this.webappName = name;
webAppAwareClassLoader.setWebAppName(name); // pass web app name to ClassLoader
}
}
class WebAppAwareClassLoader extends ClassLoader {
private String webAppName;
public WebAppAwareClassLoader(ClassLoader parent) {
super(parent);
DynamicWebappLoader.lastCreatedClassLoader.set(this); // store newly created instance in ThreadLocal .. did not find better way to access the reference later in code
}
#Override
public Class<?> loadClass(String className) throws ClassNotFoundException {
System.out.println("Load class: " + className + " for webapp: " + webAppName);
try {
return LibrariesStorage.loadClassForWebapp(webAppName, className);
} catch (ClassNotFoundException e) {
System.out.println("JarFileClassLoader did not find class: " + className + " " + e.getMessage());
return super.loadClass(className);
}
}
public void setWebAppName(String webAppName) {
this.webAppName = webAppName;
}
}
class LibrariesStorage {
private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";
private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();
private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();
static {
try {
addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");
mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars
mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");
} catch (MalformedURLException e) {
throw new RuntimeException(e.getMessage(), e);
}
}
private static void mapApplicationToLibrary(String applicationName, String libraryName, String libraryVersion) {
LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
if (!webappLibraries.containsKey(applicationName)) {
webappLibraries.put(applicationName, new ArrayList<LibraryAndVersion>());
}
webappLibraries.get(applicationName).add(libraryAndVersion);
}
private static void addLibrary(String libraryName, String libraryVersion, String filename)
throws MalformedURLException {
LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
URL libraryLocation = new File(JARS_DIR + File.separator + filename).toURI().toURL();
libraryToClassLoader.put(libraryAndVersion,
new JarFileClassLoader("JarFileClassLoader for lib: " + libraryAndVersion,
new URL[] { libraryLocation }));
}
private LibrariesStorage() {
}
public static Class<?> loadClassForWebapp(String webappName, String className) throws ClassNotFoundException {
System.out.println("Loading class: " + className + " for web application: " + webappName);
List<LibraryAndVersion> webappLibraries = LibrariesStorage.webappLibraries.get(webappName);
for (LibraryAndVersion libraryAndVersion : webappLibraries) {
JarFileClassLoader libraryClassLoader = libraryToClassLoader.get(libraryAndVersion);
try {
return libraryClassLoader.loadClass(className); // ok current lib contained class to load
} catch (ClassNotFoundException e) {
// ok.. continue in loop... try to load the class from classloader connected to next library
}
}
throw new ClassNotFoundException("Class " + className + " was not found in any jar connected to webapp: " +
webappLibraries);
}
}
class LibraryAndVersion {
private final String name;
private final String version;
LibraryAndVersion(String name, String version) {
this.name = name;
this.version = version;
}
#Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if ((o == null) || (getClass() != o.getClass())) {
return false;
}
LibraryAndVersion that = (LibraryAndVersion) o;
if ((name != null) ? (!name.equals(that.name)) : (that.name != null)) {
return false;
}
if ((version != null) ? (!version.equals(that.version)) : (that.version != null)) {
return false;
}
return true;
}
#Override
public int hashCode() {
int result = (name != null) ? name.hashCode() : 0;
result = (31 * result) + ((version != null) ? version.hashCode() : 0);
return result;
}
#Override
public String toString() {
return "LibraryAndVersion{" +
"name='" + name + '\'' +
", version='" + version + '\'' +
'}';
}
}

JBoss has a framework called Modules that solves this problem. You can save the shared library with its version and reference it from your war-file.
I have no idea if it works on Tomcat, but it works as a charm on Wildfly.

Related

Java - How to Load Classes your Main does'nt know the name from? [duplicate]

Is it possible to find all classes or interfaces in a given package? (Quickly looking at e.g. Package, it would seem like no.)
Due to the dynamic nature of class loaders, this is not possible. Class loaders are not required to tell the VM which classes it can provide, instead they are just handed requests for classes, and have to return a class or throw an exception.
However, if you write your own class loaders, or examine the classpaths and it's jars, it's possible to find this information. This will be via filesystem operations though, and not reflection. There might even be libraries that can help you do this.
If there are classes that get generated, or delivered remotely, you will not be able to discover those classes.
The normal method is instead to somewhere register the classes you need access to in a file, or reference them in a different class. Or just use convention when it comes to naming.
Addendum: The Reflections Library will allow you to look up classes in the current classpath. It can be used to get all classes in a package:
Reflections reflections = new Reflections("my.project.prefix");
Set<Class<? extends Object>> allClasses =
reflections.getSubTypesOf(Object.class);
You should probably take a look at the open source Reflections library. With it you can easily achieve what you want.
First, setup the reflections index (it's a bit messy since searching for all classes is disabled by default):
List<ClassLoader> classLoadersList = new LinkedList<ClassLoader>();
classLoadersList.add(ClasspathHelper.contextClassLoader());
classLoadersList.add(ClasspathHelper.staticClassLoader());
Reflections reflections = new Reflections(new ConfigurationBuilder()
.setScanners(new SubTypesScanner(false /* don't exclude Object.class */), new ResourcesScanner())
.setUrls(ClasspathHelper.forClassLoader(classLoadersList.toArray(new ClassLoader[0])))
.filterInputsBy(new FilterBuilder().include(FilterBuilder.prefix("org.your.package"))));
Then you can query for all objects in a given package:
Set<Class<?>> classes = reflections.getSubTypesOf(Object.class);
Google Guava 14 includes a new class ClassPath with three methods to scan for top level classes:
getTopLevelClasses()
getTopLevelClasses(String packageName)
getTopLevelClassesRecursive(String packageName)
See the ClassPath javadocs for more info.
You could use this method1 that uses the ClassLoader.
/**
* Scans all classes accessible from the context class loader which belong to the given package and subpackages.
*
* #param packageName The base package
* #return The classes
* #throws ClassNotFoundException
* #throws IOException
*/
private static Class[] getClasses(String packageName)
throws ClassNotFoundException, IOException {
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
assert classLoader != null;
String path = packageName.replace('.', '/');
Enumeration<URL> resources = classLoader.getResources(path);
List<File> dirs = new ArrayList<File>();
while (resources.hasMoreElements()) {
URL resource = resources.nextElement();
dirs.add(new File(resource.getFile()));
}
ArrayList<Class> classes = new ArrayList<Class>();
for (File directory : dirs) {
classes.addAll(findClasses(directory, packageName));
}
return classes.toArray(new Class[classes.size()]);
}
/**
* Recursive method used to find all classes in a given directory and subdirs.
*
* #param directory The base directory
* #param packageName The package name for classes found inside the base directory
* #return The classes
* #throws ClassNotFoundException
*/
private static List<Class> findClasses(File directory, String packageName) throws ClassNotFoundException {
List<Class> classes = new ArrayList<Class>();
if (!directory.exists()) {
return classes;
}
File[] files = directory.listFiles();
for (File file : files) {
if (file.isDirectory()) {
assert !file.getName().contains(".");
classes.addAll(findClasses(file, packageName + "." + file.getName()));
} else if (file.getName().endsWith(".class")) {
classes.add(Class.forName(packageName + '.' + file.getName().substring(0, file.getName().length() - 6)));
}
}
return classes;
}
__________
1 This method was taken originally from http://snippets.dzone.com/posts/show/4831, which was archived by the Internet Archive, as linked to now. The snippet is also available at https://dzone.com/articles/get-all-classes-within-package.
Spring
This example is for Spring 4, but you can find the classpath scanner in earlier versions as well.
// create scanner and disable default filters (that is the 'false' argument)
final ClassPathScanningCandidateComponentProvider provider = new ClassPathScanningCandidateComponentProvider(false);
// add include filters which matches all the classes (or use your own)
provider.addIncludeFilter(new RegexPatternTypeFilter(Pattern.compile(".*")));
// get matching classes defined in the package
final Set<BeanDefinition> classes = provider.findCandidateComponents("my.package.name");
// this is how you can load the class type from BeanDefinition instance
for (BeanDefinition bean: classes) {
Class<?> clazz = Class.forName(bean.getBeanClassName());
// ... do your magic with the class ...
}
Google Guava
Note: In version 14, the API is still marked as #Beta, so beware in production code.
final ClassLoader loader = Thread.currentThread().getContextClassLoader();
for (final ClassPath.ClassInfo info : ClassPath.from(loader).getTopLevelClasses()) {
if (info.getName().startsWith("my.package.")) {
final Class<?> clazz = info.load();
// do something with your clazz
}
}
Hello. I always have had some issues with the solutions above (and on other sites).
I, as a developer, am programming a addon for a API. The API prevents the use of any external libraries or 3rd party tools. The setup also consists of a mixture of code in jar or zip files and class files located directly in some directories. So my code had to be able to work arround every setup. After a lot of research I have come up with a method that will work in at least 95% of all possible setups.
The following code is basically the overkill method that will always work.
The code:
This code scans a given package for all classes that are included in it. It will only work for all classes in the current ClassLoader.
/**
* Private helper method
*
* #param directory
* The directory to start with
* #param pckgname
* The package name to search for. Will be needed for getting the
* Class object.
* #param classes
* if a file isn't loaded but still is in the directory
* #throws ClassNotFoundException
*/
private static void checkDirectory(File directory, String pckgname,
ArrayList<Class<?>> classes) throws ClassNotFoundException {
File tmpDirectory;
if (directory.exists() && directory.isDirectory()) {
final String[] files = directory.list();
for (final String file : files) {
if (file.endsWith(".class")) {
try {
classes.add(Class.forName(pckgname + '.'
+ file.substring(0, file.length() - 6)));
} catch (final NoClassDefFoundError e) {
// do nothing. this class hasn't been found by the
// loader, and we don't care.
}
} else if ((tmpDirectory = new File(directory, file))
.isDirectory()) {
checkDirectory(tmpDirectory, pckgname + "." + file, classes);
}
}
}
}
/**
* Private helper method.
*
* #param connection
* the connection to the jar
* #param pckgname
* the package name to search for
* #param classes
* the current ArrayList of all classes. This method will simply
* add new classes.
* #throws ClassNotFoundException
* if a file isn't loaded but still is in the jar file
* #throws IOException
* if it can't correctly read from the jar file.
*/
private static void checkJarFile(JarURLConnection connection,
String pckgname, ArrayList<Class<?>> classes)
throws ClassNotFoundException, IOException {
final JarFile jarFile = connection.getJarFile();
final Enumeration<JarEntry> entries = jarFile.entries();
String name;
for (JarEntry jarEntry = null; entries.hasMoreElements()
&& ((jarEntry = entries.nextElement()) != null);) {
name = jarEntry.getName();
if (name.contains(".class")) {
name = name.substring(0, name.length() - 6).replace('/', '.');
if (name.contains(pckgname)) {
classes.add(Class.forName(name));
}
}
}
}
/**
* Attempts to list all the classes in the specified package as determined
* by the context class loader
*
* #param pckgname
* the package name to search
* #return a list of classes that exist within that package
* #throws ClassNotFoundException
* if something went wrong
*/
public static ArrayList<Class<?>> getClassesForPackage(String pckgname)
throws ClassNotFoundException {
final ArrayList<Class<?>> classes = new ArrayList<Class<?>>();
try {
final ClassLoader cld = Thread.currentThread()
.getContextClassLoader();
if (cld == null)
throw new ClassNotFoundException("Can't get class loader.");
final Enumeration<URL> resources = cld.getResources(pckgname
.replace('.', '/'));
URLConnection connection;
for (URL url = null; resources.hasMoreElements()
&& ((url = resources.nextElement()) != null);) {
try {
connection = url.openConnection();
if (connection instanceof JarURLConnection) {
checkJarFile((JarURLConnection) connection, pckgname,
classes);
} else if (connection instanceof FileURLConnection) {
try {
checkDirectory(
new File(URLDecoder.decode(url.getPath(),
"UTF-8")), pckgname, classes);
} catch (final UnsupportedEncodingException ex) {
throw new ClassNotFoundException(
pckgname
+ " does not appear to be a valid package (Unsupported encoding)",
ex);
}
} else
throw new ClassNotFoundException(pckgname + " ("
+ url.getPath()
+ ") does not appear to be a valid package");
} catch (final IOException ioex) {
throw new ClassNotFoundException(
"IOException was thrown when trying to get all resources for "
+ pckgname, ioex);
}
}
} catch (final NullPointerException ex) {
throw new ClassNotFoundException(
pckgname
+ " does not appear to be a valid package (Null pointer exception)",
ex);
} catch (final IOException ioex) {
throw new ClassNotFoundException(
"IOException was thrown when trying to get all resources for "
+ pckgname, ioex);
}
return classes;
}
These three methods provide you with the ability to find all classes in a given package.
You use it like this:
getClassesForPackage("package.your.classes.are.in");
The explanation:
The method first gets the current ClassLoader. It then fetches all resources that contain said package and iterates of these URLs. It then creates a URLConnection and determines what type of URl we have. It can either be a directory (FileURLConnection) or a directory inside a jar or zip file (JarURLConnection). Depending on what type of connection we have two different methods will be called.
First lets see what happens if it is a FileURLConnection.
It first checks if the passed File exists and is a directory. If that's the case it checks if it is a class file. If so a Class object will be created and put in the ArrayList. If it is not a class file but is a directory, we simply iterate into it and do the same thing. All other cases/files will be ignored.
If the URLConnection is a JarURLConnection the other private helper method will be called. This method iterates over all Entries in the zip/jar archive. If one entry is a class file and is inside of the package a Class object will be created and stored in the ArrayList.
After all resources have been parsed it (the main method) returns the ArrayList containig all classes in the given package, that the current ClassLoader knows about.
If the process fails at any point a ClassNotFoundException will be thrown containg detailed information about the exact cause.
The most robust mechanism for listing all classes in a given package is currently ClassGraph, because it handles the widest possible array of classpath specification mechanisms, including the new JPMS module system. (I am the author.)
List<String> classNames = new ArrayList<>();
try (ScanResult scanResult = new ClassGraph().acceptPackages("my.package")
.enableClassInfo().scan()) {
classNames.addAll(scanResult.getAllClasses().getNames());
}
Without using any extra libraries:
package test;
import java.io.DataInputStream;
import java.io.InputStream;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
public class Test {
public static void main(String[] args) throws Exception{
List<Class> classes = getClasses(Test.class.getClassLoader(),"test");
for(Class c:classes){
System.out.println("Class: "+c);
}
}
public static List<Class> getClasses(ClassLoader cl,String pack) throws Exception{
String dottedPackage = pack.replaceAll("[/]", ".");
List<Class> classes = new ArrayList<Class>();
URL upackage = cl.getResource(pack);
DataInputStream dis = new DataInputStream((InputStream) upackage.getContent());
String line = null;
while ((line = dis.readLine()) != null) {
if(line.endsWith(".class")) {
classes.add(Class.forName(dottedPackage+"."+line.substring(0,line.lastIndexOf('.'))));
}
}
return classes;
}
}
In general class loaders do not allow for scanning through all the classes on the classpath. But usually the only used class loader is UrlClassLoader from which we can retrieve the list of directories and jar files (see getURLs) and open them one by one to list available classes. This approach, called class path scanning, is implemented in Scannotation and Reflections.
Reflections reflections = new Reflections("my.package");
Set<Class<? extends Object>> classes = reflections.getSubTypesOf(Object.class);
Another approach is to use Java Pluggable Annotation Processing API to write annotation processor which will collect all annotated classes at compile time and build the index file for runtime use. This mechanism is implemented in ClassIndex library:
// package-info.java
#IndexSubclasses
package my.package;
// your code
Iterable<Class> classes = ClassIndex.getPackageClasses("my.package");
Notice that no additional setup is needed as the scanning is fully automated thanks to Java compiler automatically discovering any processors found on the classpath.
What about this:
public static List<Class<?>> getClassesForPackage(final String pkgName) throws IOException, URISyntaxException {
final String pkgPath = pkgName.replace('.', '/');
final URI pkg = Objects.requireNonNull(ClassLoader.getSystemClassLoader().getResource(pkgPath)).toURI();
final ArrayList<Class<?>> allClasses = new ArrayList<Class<?>>();
Path root;
if (pkg.toString().startsWith("jar:")) {
try {
root = FileSystems.getFileSystem(pkg).getPath(pkgPath);
} catch (final FileSystemNotFoundException e) {
root = FileSystems.newFileSystem(pkg, Collections.emptyMap()).getPath(pkgPath);
}
} else {
root = Paths.get(pkg);
}
final String extension = ".class";
try (final Stream<Path> allPaths = Files.walk(root)) {
allPaths.filter(Files::isRegularFile).forEach(file -> {
try {
final String path = file.toString().replace('/', '.');
final String name = path.substring(path.indexOf(pkgName), path.length() - extension.length());
allClasses.add(Class.forName(name));
} catch (final ClassNotFoundException | StringIndexOutOfBoundsException ignored) {
}
});
}
return allClasses;
}
You can then overload the function:
public static List<Class<?>> getClassesForPackage(final Package pkg) throws IOException, URISyntaxException {
return getClassesForPackage(pkg.getName());
}
If you need to test it:
public static void main(final String[] argv) throws IOException, URISyntaxException {
for (final Class<?> cls : getClassesForPackage("my.package")) {
System.out.println(cls);
}
for (final Class<?> cls : getClassesForPackage(MyClass.class.getPackage())) {
System.out.println(cls);
}
}
If your IDE does not have import helper:
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.nio.file.FileSystemNotFoundException;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Objects;
import java.util.stream.Stream;
It works:
from your IDE
for a JAR file
without external dependencies
Here's how I do it. I scan all the subfolders (sub-packages) and I don't try to load anonymous classes:
/**
* Attempts to list all the classes in the specified package as determined
* by the context class loader, recursively, avoiding anonymous classes
*
* #param pckgname
* the package name to search
* #return a list of classes that exist within that package
* #throws ClassNotFoundException
* if something went wrong
*/
private static List<Class> getClassesForPackage(String pckgname) throws ClassNotFoundException {
// This will hold a list of directories matching the pckgname. There may be more than one if a package is split over multiple jars/paths
ArrayList<File> directories = new ArrayList<File>();
String packageToPath = pckgname.replace('.', '/');
try {
ClassLoader cld = Thread.currentThread().getContextClassLoader();
if (cld == null) {
throw new ClassNotFoundException("Can't get class loader.");
}
// Ask for all resources for the packageToPath
Enumeration<URL> resources = cld.getResources(packageToPath);
while (resources.hasMoreElements()) {
directories.add(new File(URLDecoder.decode(resources.nextElement().getPath(), "UTF-8")));
}
} catch (NullPointerException x) {
throw new ClassNotFoundException(pckgname + " does not appear to be a valid package (Null pointer exception)");
} catch (UnsupportedEncodingException encex) {
throw new ClassNotFoundException(pckgname + " does not appear to be a valid package (Unsupported encoding)");
} catch (IOException ioex) {
throw new ClassNotFoundException("IOException was thrown when trying to get all resources for " + pckgname);
}
ArrayList<Class> classes = new ArrayList<Class>();
// For every directoryFile identified capture all the .class files
while (!directories.isEmpty()){
File directoryFile = directories.remove(0);
if (directoryFile.exists()) {
// Get the list of the files contained in the package
File[] files = directoryFile.listFiles();
for (File file : files) {
// we are only interested in .class files
if ((file.getName().endsWith(".class")) && (!file.getName().contains("$"))) {
// removes the .class extension
int index = directoryFile.getPath().indexOf(packageToPath);
String packagePrefix = directoryFile.getPath().substring(index).replace('/', '.');;
try {
String className = packagePrefix + '.' + file.getName().substring(0, file.getName().length() - 6);
classes.add(Class.forName(className));
} catch (NoClassDefFoundError e)
{
// do nothing. this class hasn't been found by the loader, and we don't care.
}
} else if (file.isDirectory()){ // If we got to a subdirectory
directories.add(new File(file.getPath()));
}
}
} else {
throw new ClassNotFoundException(pckgname + " (" + directoryFile.getPath() + ") does not appear to be a valid package");
}
}
return classes;
}
I put together a simple github project that solves this problem:
https://github.com/ddopson/java-class-enumerator
It should work for BOTH file-based classpaths AND for jar files.
If you run 'make' after checking out the project it will print this out:
Cleaning...
rm -rf build/
Building...
javac -d build/classes src/pro/ddopson/ClassEnumerator.java src/test/ClassIShouldFindOne.java src/test/ClassIShouldFindTwo.java src/test/subpkg/ClassIShouldFindThree.java src/test/TestClassEnumeration.java
Making JAR Files...
jar cf build/ClassEnumerator_test.jar -C build/classes/ .
jar cf build/ClassEnumerator.jar -C build/classes/ pro
Running Filesystem Classpath Test...
java -classpath build/classes test.TestClassEnumeration
ClassDiscovery: Package: 'test' becomes Resource: 'file:/Users/Dopson/work/other/java-class-enumeration/build/classes/test'
ClassDiscovery: Reading Directory '/Users/Dopson/work/other/java-class-enumeration/build/classes/test'
ClassDiscovery: FileName 'ClassIShouldFindOne.class' => class 'test.ClassIShouldFindOne'
ClassDiscovery: FileName 'ClassIShouldFindTwo.class' => class 'test.ClassIShouldFindTwo'
ClassDiscovery: FileName 'subpkg' => class 'null'
ClassDiscovery: Reading Directory '/Users/Dopson/work/other/java-class-enumeration/build/classes/test/subpkg'
ClassDiscovery: FileName 'ClassIShouldFindThree.class' => class 'test.subpkg.ClassIShouldFindThree'
ClassDiscovery: FileName 'TestClassEnumeration.class' => class 'test.TestClassEnumeration'
Running JAR Classpath Test...
java -classpath build/ClassEnumerator_test.jar test.TestClassEnumeration
ClassDiscovery: Package: 'test' becomes Resource: 'jar:file:/Users/Dopson/work/other/java-class-enumeration/build/ClassEnumerator_test.jar!/test'
ClassDiscovery: Reading JAR file: '/Users/Dopson/work/other/java-class-enumeration/build/ClassEnumerator_test.jar'
ClassDiscovery: JarEntry 'META-INF/' => class 'null'
ClassDiscovery: JarEntry 'META-INF/MANIFEST.MF' => class 'null'
ClassDiscovery: JarEntry 'pro/' => class 'null'
ClassDiscovery: JarEntry 'pro/ddopson/' => class 'null'
ClassDiscovery: JarEntry 'pro/ddopson/ClassEnumerator.class' => class 'null'
ClassDiscovery: JarEntry 'test/' => class 'null'
ClassDiscovery: JarEntry 'test/ClassIShouldFindOne.class' => class 'test.ClassIShouldFindOne'
ClassDiscovery: JarEntry 'test/ClassIShouldFindTwo.class' => class 'test.ClassIShouldFindTwo'
ClassDiscovery: JarEntry 'test/subpkg/' => class 'null'
ClassDiscovery: JarEntry 'test/subpkg/ClassIShouldFindThree.class' => class 'test.subpkg.ClassIShouldFindThree'
ClassDiscovery: JarEntry 'test/TestClassEnumeration.class' => class 'test.TestClassEnumeration'
Tests Passed.
See also my other answer
Yeah using few API's you can, here is how I like doing it, faced this problem which I was using hibernate core & had to find classes which where annotated with a certain annotation.
Make these an custom annotation using which you will mark which classes you want to be picked up.
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.TYPE)
public #interface EntityToBeScanned {
}
Then mark your class with it like
#EntityToBeScanned
public MyClass{
}
Make this utility class which has the following method
public class ClassScanner {
public static Set<Class<?>> allFoundClassesAnnotatedWithEntityToBeScanned(){
Reflections reflections = new Reflections(".*");
Set<Class<?>> annotated = reflections.getTypesAnnotatedWith(EntityToBeScanned.class);
return annotated;
}
}
Call the allFoundClassesAnnotatedWithEntityToBeScanned() method to get a Set of Classes found.
You will need libs given below
<!-- https://mvnrepository.com/artifact/com.google.guava/guava -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>21.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.javassist/javassist -->
<dependency>
<groupId>org.javassist</groupId>
<artifactId>javassist</artifactId>
<version>3.22.0-CR1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.reflections/reflections -->
<dependency>
<groupId>org.reflections</groupId>
<artifactId>reflections</artifactId>
<version>0.9.10</version>
</dependency>
If you're in Spring-land you can use PathMatchingResourcePatternResolver;
PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
Resource[] resources = resolver.getResources("classpath*:some/package/name/*.class");
Arrays.asList(resources).forEach(r->{
...
});
You need to look up every class loader entry in the class path:
String pkg = "org/apache/commons/lang";
ClassLoader cl = ClassLoader.getSystemClassLoader();
URL[] urls = ((URLClassLoader) cl).getURLs();
for (URL url : urls) {
System.out.println(url.getFile());
File jar = new File(url.getFile());
// ....
}
If entry is directory, just look up in the right subdirectory:
if (jar.isDirectory()) {
File subdir = new File(jar, pkg);
if (!subdir.exists())
continue;
File[] files = subdir.listFiles();
for (File file : files) {
if (!file.isFile())
continue;
if (file.getName().endsWith(".class"))
System.out.println("Found class: "
+ file.getName().substring(0,
file.getName().length() - 6));
}
}
If the entry is the file, and it's jar, inspect the ZIP entries of it:
else {
// try to open as ZIP
try {
ZipFile zip = new ZipFile(jar);
for (Enumeration<? extends ZipEntry> entries = zip
.entries(); entries.hasMoreElements();) {
ZipEntry entry = entries.nextElement();
String name = entry.getName();
if (!name.startsWith(pkg))
continue;
name = name.substring(pkg.length() + 1);
if (name.indexOf('/') < 0 && name.endsWith(".class"))
System.out.println("Found class: "
+ name.substring(0, name.length() - 6));
}
} catch (ZipException e) {
System.out.println("Not a ZIP: " + e.getMessage());
} catch (IOException e) {
System.err.println(e.getMessage());
}
}
Now once you have all class names withing package, you can try loading them with reflection and analyze if they are classes or interfaces, etc.
I've been trying to use the Reflections library, but had some problems using it, and there were too many jars I should include just to simply obtain the classes on a package.
I'll post a solution I've found in this duplicate question: How to get all classes names in a package?
The answer was written by sp00m; I've added some corrections to make it work:
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.util.Enumeration;
import java.util.LinkedList;
import java.util.List;
public final class ClassFinder {
private final static char DOT = '.';
private final static char SLASH = '/';
private final static String CLASS_SUFFIX = ".class";
private final static String BAD_PACKAGE_ERROR = "Unable to get resources from path '%s'. Are you sure the given '%s' package exists?";
public final static List<Class<?>> find(final String scannedPackage) {
final ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
final String scannedPath = scannedPackage.replace(DOT, SLASH);
final Enumeration<URL> resources;
try {
resources = classLoader.getResources(scannedPath);
} catch (IOException e) {
throw new IllegalArgumentException(String.format(BAD_PACKAGE_ERROR, scannedPath, scannedPackage), e);
}
final List<Class<?>> classes = new LinkedList<Class<?>>();
while (resources.hasMoreElements()) {
final File file = new File(resources.nextElement().getFile());
classes.addAll(find(file, scannedPackage));
}
return classes;
}
private final static List<Class<?>> find(final File file, final String scannedPackage) {
final List<Class<?>> classes = new LinkedList<Class<?>>();
if (file.isDirectory()) {
for (File nestedFile : file.listFiles()) {
classes.addAll(find(nestedFile, scannedPackage));
}
//File names with the $1, $2 holds the anonymous inner classes, we are not interested on them.
} else if (file.getName().endsWith(CLASS_SUFFIX) && !file.getName().contains("$")) {
final int beginIndex = 0;
final int endIndex = file.getName().length() - CLASS_SUFFIX.length();
final String className = file.getName().substring(beginIndex, endIndex);
try {
final String resource = scannedPackage + DOT + className;
classes.add(Class.forName(resource));
} catch (ClassNotFoundException ignore) {
}
}
return classes;
}
}
To use it just call the find method as sp00n mentioned in this example:
I've added the creation of instances of the classes if needed.
List<Class<?>> classes = ClassFinder.find("com.package");
ExcelReporting excelReporting;
for (Class<?> aClass : classes) {
Constructor constructor = aClass.getConstructor();
//Create an object of the class type
constructor.newInstance();
//...
}
I just wrote a util class, it include test methods, you can have a check ~
IteratePackageUtil.java:
package eric.j2se.reflect;
import java.util.Set;
import org.reflections.Reflections;
import org.reflections.scanners.ResourcesScanner;
import org.reflections.scanners.SubTypesScanner;
import org.reflections.util.ClasspathHelper;
import org.reflections.util.ConfigurationBuilder;
import org.reflections.util.FilterBuilder;
/**
* an util to iterate class in a package,
*
* #author eric
* #date Dec 10, 2013 12:36:46 AM
*/
public class IteratePackageUtil {
/**
* <p>
* Get set of all class in a specified package recursively. this only support lib
* </p>
* <p>
* class of sub package will be included, inner class will be included,
* </p>
* <p>
* could load class that use the same classloader of current class, can't load system packages,
* </p>
*
* #param pkg
* path of a package
* #return
*/
public static Set<Class<? extends Object>> getClazzSet(String pkg) {
// prepare reflection, include direct subclass of Object.class
Reflections reflections = new Reflections(new ConfigurationBuilder().setScanners(new SubTypesScanner(false), new ResourcesScanner())
.setUrls(ClasspathHelper.forClassLoader(ClasspathHelper.classLoaders(new ClassLoader[0])))
.filterInputsBy(new FilterBuilder().includePackage(pkg)));
return reflections.getSubTypesOf(Object.class);
}
public static void test() {
String pkg = "org.apache.tomcat.util";
Set<Class<? extends Object>> clazzSet = getClazzSet(pkg);
for (Class<? extends Object> clazz : clazzSet) {
System.out.println(clazz.getName());
}
}
public static void main(String[] args) {
test();
}
}
Almost all the answers either uses Reflections or reads class files from file system. If you try to read classes from file system, you may get errors when you package your application as JAR or other. Also you may not want to use a separate library for that purpose.
Here is another approach which is pure java and not depends on file system.
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.StandardLocation;
import javax.tools.ToolProvider;
import java.io.File;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import java.util.stream.StreamSupport;
public class PackageUtil {
public static Collection<Class> getClasses(final String pack) throws Exception {
final StandardJavaFileManager fileManager = ToolProvider.getSystemJavaCompiler().getStandardFileManager(null, null, null);
return StreamSupport.stream(fileManager.list(StandardLocation.CLASS_PATH, pack, Collections.singleton(JavaFileObject.Kind.CLASS), false).spliterator(), false)
.map(javaFileObject -> {
try {
final String[] split = javaFileObject.getName()
.replace(".class", "")
.replace(")", "")
.split(Pattern.quote(File.separator));
final String fullClassName = pack + "." + split[split.length - 1];
return Class.forName(fullClassName);
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);
}
})
.collect(Collectors.toCollection(ArrayList::new));
}
}
Java 8 is not a must. You can use for loops instead of streams.
And you can test it like this
public static void main(String[] args) throws Exception {
final String pack = "java.nio.file"; // Or any other package
PackageUtil.getClasses(pack).stream().forEach(System.out::println);
}
Aleksander Blomskøld's solution did not work for me for parameterized tests #RunWith(Parameterized.class) when using Maven. The tests were named correctly and also where found but not executed:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running some.properly.named.test.run.with.maven.SomeTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.123 sec
A similar issue has been reported here.
In my case #Parameters is creating instances of each class in a package. The tests worked well when run locally in the IDE. However, when running Maven no classes where found with Aleksander Blomskøld's solution.
I did make it work with the following snipped which was inspired by David Pärsson's comment on Aleksander Blomskøld's answer:
Reflections reflections = new Reflections(new ConfigurationBuilder()
.setScanners(new SubTypesScanner(false /* don't exclude Object.class */), new ResourcesScanner())
.addUrls(ClasspathHelper.forJavaClassPath())
.filterInputsBy(new FilterBuilder()
.include(FilterBuilder.prefix(basePackage))));
Set<Class<?>> subTypesOf = reflections.getSubTypesOf(Object.class);
I couldn't find a short working snippet for something so simple. So here it is, I made it myself after screwing around for a while:
Reflections reflections =
new Reflections(new ConfigurationBuilder()
.filterInputsBy(new FilterBuilder().includePackage(packagePath))
.setUrls(ClasspathHelper.forPackage(packagePath))
.setScanners(new SubTypesScanner(false)));
Set<String> typeList = reflections.getAllTypes();
It uses org.reflections.
Define classes to be scanning in the package test
package test;
public class A {
private class B {}
enum C {}
record D() {}
}
For org.reflections:reflections:0.10.2, it works for me as follows:
Use reflection lib to scan classes in package test
#Test
void t() {
final String packagePath = "test";
final Reflections reflections =
new Reflections(packagePath, Scanners.SubTypes.filterResultsBy(v -> true));
reflections.getAll(Scanners.SubTypes).forEach(System.out::println);
}
Output
java.lang.constant.Constable
java.lang.Enum
java.lang.Comparable
java.lang.Record
java.lang.Object
java.io.Serializable
test.A$C
test.A$D
test.A$B
test.A
For io.github.classgraph:classgraph:4.8.146, it works for me as follows:
#Test
void t() {
final String packagePath = "test";
try (ScanResult scanResult = new ClassGraph()
.enableClassInfo()
.ignoreClassVisibility()
.acceptPackages(packagePath)
.scan()) {
scanResult.getAllClasses()
.forEach(v -> {
System.out.println(v.getName());
});
}
}
Output
test.A
test.A$B
test.A$C
test.A$D
Provided you are not using any dynamic class loaders you can search the classpath and for each entry search the directory or JAR file.
Worth mentioning
If you want to have a list of all classes under some package, you can use Reflection the following way:
List<Class> myTypes = new ArrayList<>();
Reflections reflections = new Reflections("com.package");
for (String s : reflections.getStore().get(SubTypesScanner.class).values()) {
myTypes.add(Class.forName(s));
}
This will create a list of classes that later you can use them as you wish.
It is very possible, but without additional libraries like Reflections it is hard...
It is hard because you haven't full instrument for get class name.
And, I take the code of my ClassFinder class:
package play.util;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.List;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
/**
* Created by LINKOR on 26.05.2017 in 15:12.
* Date: 2017.05.26
*/
public class FileClassFinder {
private JarFile file;
private boolean trouble;
public FileClassFinder(String filePath) {
try {
file = new JarFile(filePath);
} catch (IOException e) {
trouble = true;
}
}
public List<String> findClasses(String pkg) {
ArrayList<String> classes = new ArrayList<>();
Enumeration<JarEntry> entries = file.entries();
while (entries.hasMoreElements()) {
JarEntry cls = entries.nextElement();
if (!cls.isDirectory()) {
String fileName = cls.getName();
String className = fileName.replaceAll("/", ".").replaceAll(File.pathSeparator, ".").substring(0, fileName.lastIndexOf('.'));
if (className.startsWith(pkg)) classes.add(className.substring(pkg.length() + 1));
}
}
return classes;
}
}
this scans the class loaders and all parent loaders for jar files and directories.
the jar files and directories referred by the Class-Path of the jars are also loaded.
this code is testet with Java 8,11,18.
on 8 everything works perfectly using the URLClassLoader and the getURLs() method.
on 11 it works fine using reflections, but the JVM prints a warning on the stderr stream (not redirectible with System.setErr() with my JVM)
on 18 the reflections are useless (throws NoSuchMethod/Field), and the only thing (where I know that it works) is to use the getResource() method. When the class loader loades the resources of the given package from the file system a simple path url is returned. When the class loader loades the resources from a jar a url like 'jar:file:[jar-path]!/[in-jar-path]' is returned.
I have used the answer https://stackoverflow.com/a/1157352/18252455 (from a duplicate question) and added the functionality to read the Class-Path and also search for directory URLs.
/**
* orig description:<br>
* Scans all classloaders for the current thread for loaded jars, and then scans
* each jar for the package name in question, listing all classes directly under
* the package name in question. Assumes directory structure in jar file and class
* package naming follow java conventions (i.e. com.example.test.MyTest would be in
* /com/example/test/MyTest.class)
* <p>
* in addition this method also scans for directories, where also is assumed, that the classes are
* placed followed by the java conventions. (i.e. <code>com.example.test.MyTest</code> would be in
* <code>directory/com/example/test/MyTest.class</code>)
* <p>
* this method also reads the jars Class-Path for other jars and directories. for the jars and
* directories referred in the jars are scanned with the same rules as defined here.<br>
* it is ensured that no jar/directory is scanned exactly one time.
* <p>
* if {#code bailError} is <code>true</code> all errors will be wrapped in a
* {#link RuntimeException}
* and then thrown.<br>
* a {#link RuntimeException} will also be thrown if something unexpected happens.<br>
*
* #param packageName
* the name of the package for which the classes should be searched
* #param allowSubPackages
* <code>true</code> is also classes in sub packages should be found
* #param loader
* the {#link ClassLoader} which should be used to find the URLs and to load classes
* #param bailError
* if all {#link Exception} should be re-thrown wrapped in {#link RuntimeException} and
* if a {#link RuntimeException} should be thrown, when something is not as expected.
* #see https://stackoverflow.com/questions/1156552/java-package-introspection
* #see https://stackoverflow.com/a/1157352/18252455
* #see https://creativecommons.org/licenses/by-sa/2.5/
* #see https://creativecommons.org/licenses/by-sa/2.5/legalcode
*/
public static Set <Class <?>> tryGetClassesForPackage(String packageName, boolean allowSubPackages, ClassLoader loader, boolean bailError) {
Set <URL> jarUrls = new HashSet <URL>();
Set <Path> directorys = new HashSet <Path>();
findClassPools(loader, jarUrls, directorys, bailError, packageName);
Set <Class <?>> jarClasses = findJarClasses(allowSubPackages, packageName, jarUrls, directorys, loader, bailError);
Set <Class <?>> dirClasses = findDirClasses(allowSubPackages, packageName, directorys, loader, bailError);
jarClasses.addAll(dirClasses);
return jarClasses;
}
private static Set <Class <?>> findDirClasses(boolean subPackages, String packageName, Set <Path> directorys, ClassLoader loader, boolean bailError) {
Filter <Path> filter;
Set <Class <?>> result = new HashSet <>();
for (Path startPath : directorys) {
String packagePath = packageName.replace(".", startPath.getFileSystem().getSeparator());
final Path searchPath = startPath.resolve(packagePath).toAbsolutePath();
if (subPackages) {
filter = p -> {
p = p.toAbsolutePath();
Path other;
if (p.getNameCount() >= searchPath.getNameCount()) {
other = searchPath;
} else {
other = searchPath.subpath(0, p.getNameCount());
}
if (p.startsWith(other)) {
return true;
} else {
return false;
}
};
} else {
filter = p -> {
p = p.toAbsolutePath();
if (p.getNameCount() > searchPath.getNameCount() + 1) {
return false;
} else if (p.toAbsolutePath().startsWith(searchPath)) {
return true;
} else {
return false;
}
};
}
if (Files.exists(searchPath)) {
findDirClassFilesRecursive(filter, searchPath, startPath, result, loader, bailError);
} // the package does not have to exist in every directory
}
return result;
}
private static void findDirClassFilesRecursive(Filter <Path> filter, Path path, Path start, Set <Class <?>> classes, ClassLoader loader, boolean bailError) {
try (DirectoryStream <Path> dirStream = Files.newDirectoryStream(path, filter)) {
for (Path p : dirStream) {
if (Files.isDirectory(p)) {
findDirClassFilesRecursive(filter, p, start, classes, loader, bailError);
} else {
Path subp = p.subpath(start.getNameCount(), p.getNameCount());
String str = subp.toString();
if (str.endsWith(".class")) {
str = str.substring(0, str.length() - 6);
String sep = p.getFileSystem().getSeparator();
if (str.startsWith(sep)) {
str = str.substring(sep.length());
}
if (str.endsWith(sep)) {
str = str.substring(0, str.length() - sep.length());
}
String fullClassName = str.replace(sep, ".");
try {
Class <?> cls = Class.forName(fullClassName, false, loader);
classes.add(cls);
} catch (ClassNotFoundException e) {
if (bailError) {
throw new RuntimeException(e);
}
}
}
}
}
} catch (IOException e) {
if (bailError) {
throw new RuntimeException(e);
}
}
}
private static Set <Class <?>> findJarClasses(boolean subPackages, String packageName, Set <URL> nextJarUrls, Set <Path> directories, ClassLoader loader, boolean bailError) {
String packagePath = packageName.replace('.', '/');
Set <Class <?>> result = new HashSet <>();
Set <URL> allJarUrls = new HashSet <>();
while (true) {
Set <URL> thisJarUrls = new HashSet <>(nextJarUrls);
thisJarUrls.removeAll(allJarUrls);
if (thisJarUrls.isEmpty()) {
break;
}
allJarUrls.addAll(thisJarUrls);
for (URL url : thisJarUrls) {
try (JarInputStream stream = new JarInputStream(url.openStream())) {
// may want better way to open url connections
readJarClassPath(stream, nextJarUrls, directories, bailError);
JarEntry entry = stream.getNextJarEntry();
while (entry != null) {
String name = entry.getName();
int i = name.lastIndexOf("/");
if (i > 0 && name.endsWith(".class")) {
try {
if (subPackages) {
if (name.substring(0, i).startsWith(packagePath)) {
Class <?> cls = Class.forName(name.substring(0, name.length() - 6).replace("/", "."), false, loader);
result.add(cls);
}
} else {
if (name.substring(0, i).equals(packagePath)) {
Class <?> cls = Class.forName(name.substring(0, name.length() - 6).replace("/", "."), false, loader);
result.add(cls);
}
}
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
entry = stream.getNextJarEntry();
}
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return result;
}
private static void readJarClassPath(JarInputStream stream, Set <URL> jarUrls, Set <Path> directories, boolean bailError) {
Object classPathObj = stream.getManifest().getMainAttributes().get(new Name("Class-Path"));
if (classPathObj == null) {
return;
}
if (classPathObj instanceof String) {
String[] entries = ((String) classPathObj).split("\\s+");// should also work with a single space (" ")
for (String entry : entries) {
try {
URL url = new URL(entry);
addFromUrl(jarUrls, directories, url, bailError);
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
} else if (bailError) {
throw new RuntimeException("the Class-Path attribute is no String: " + classPathObj.getClass().getName() + " tos='" + classPathObj + "'");
}
}
private static void findClassPools(ClassLoader classLoader, Set <URL> jarUrls, Set <Path> directoryPaths, boolean bailError, String packageName) {
packageName = packageName.replace('.', '/');
while (classLoader != null) {
if (classLoader instanceof URLClassLoader) {
for (URL url : ((URLClassLoader) classLoader).getURLs()) {
addFromUrl(jarUrls, directoryPaths, url, bailError);
System.out.println("rurl-class-loade.url[n]r->'" + url + "'");
}
} else {
URL res = classLoader.getResource("");
if (res != null) {
addFromUrl(jarUrls, directoryPaths, res, bailError);
}
res = classLoader.getResource("/");
if (res != null) {
addFromUrl(jarUrls, directoryPaths, res, bailError);
}
res = classLoader.getResource("/" + packageName);
if (res != null) {
res = removePackageFromUrl(res, packageName, bailError);
if (res != null) {
addFromUrl(jarUrls, directoryPaths, res, bailError);
}
}
res = classLoader.getResource(packageName);
if (res != null) {
res = removePackageFromUrl(res, packageName, bailError);
if (res != null) {
addFromUrl(jarUrls, directoryPaths, res, bailError);
}
}
addFromUnknownClass(classLoader, jarUrls, directoryPaths, bailError, 8);
}
classLoader = classLoader.getParent();
}
}
private static URL removePackageFromUrl(URL res, String packagePath, boolean bailError) {
packagePath = "/" + packagePath;
String urlStr = res.toString();
if ( !urlStr.endsWith(packagePath)) {
if (bailError) {
throw new RuntimeException("the url string does not end with the packagepath! packagePath='" + packagePath + "' urlStr='" + urlStr + "'");
} else {
return null;
}
}
urlStr = urlStr.substring(0, urlStr.length() - packagePath.length());
if (urlStr.endsWith("!")) {
urlStr = urlStr.substring(0, urlStr.length() - 1);
}
if (urlStr.startsWith("jar:")) {
urlStr = urlStr.substring(4);
}
try {
return new URL(urlStr);
} catch (MalformedURLException e) {
if (bailError) {
throw new RuntimeException(e);
} else {
return null;
}
}
}
private static void addFromUnknownClass(Object instance, Set <URL> jarUrls, Set <Path> directoryPaths, boolean bailError, int maxDeep) {
Class <?> cls = instance.getClass();
while (cls != null) {
Field[] fields = cls.getDeclaredFields();
for (Field field : fields) {
Class <?> type = field.getType();
Object value;
try {
value = getValue(instance, field);
if (value != null) {
addFromUnknownValue(value, jarUrls, directoryPaths, bailError, type, field.getName(), maxDeep - 1);
}
} catch (IllegalArgumentException | IllegalAccessException | SecurityException e) {
if (bailError) {
final String version = System.getProperty("java.version");
String vers = version;
if (vers.startsWith("1.")) {
vers = vers.substring(2);
}
int dotindex = vers.indexOf('.');
if (dotindex != -1) {
vers = vers.substring(0, dotindex);
}
int versNum;
try {
versNum = Integer.parseInt(vers);
} catch (NumberFormatException e1) {
throw new RuntimeException("illegal version: '" + version + "' lastError: " + e.getMessage(), e);
}
if (versNum <= 11) {
throw new RuntimeException(e);
}
}
}
}
cls = cls.getSuperclass();
}
}
private static Object getValue(Object instance, Field field) throws IllegalArgumentException, IllegalAccessException, SecurityException {
try {
boolean flag = field.isAccessible();
boolean newflag = flag;
try {
field.setAccessible(true);
newflag = true;
} catch (Exception e) {}
try {
return field.get(instance);
} finally {
if (flag != newflag) {
field.setAccessible(flag);
}
}
} catch (IllegalArgumentException | IllegalAccessException | SecurityException e) {
try {
Field override = AccessibleObject.class.getDeclaredField("override");
boolean flag = override.isAccessible();
boolean newFlag = flag;
try {
override.setAccessible(true);
flag = true;
} catch (Exception s) {}
override.setBoolean(field, true);
if (flag != newFlag) {
override.setAccessible(flag);
}
return field.get(instance);
} catch (IllegalArgumentException | IllegalAccessException | NoSuchFieldException | SecurityException e1) {
e.addSuppressed(e1);
throw e;
}
}
}
private static void addFromUnknownValue(Object value, Set <URL> jarUrls, Set <Path> directoryPaths, boolean bailError, Class <?> type, String fieldName, int maxDeep) {
if (Collection.class.isAssignableFrom(type)) {
for (Object obj : (Collection <?>) value) {
URL url = null;
try {
if (obj instanceof URL) {
url = (URL) obj;
} else if (obj instanceof Path) {
url = ((Path) obj).toUri().toURL();
} else if (obj instanceof File) {
url = ((File) obj).toURI().toURL();
}
} catch (MalformedURLException e) {
if (bailError) {
throw new RuntimeException(e);
}
}
if (url != null) {
addFromUrl(jarUrls, directoryPaths, url, bailError);
}
}
} else if (URL[].class.isAssignableFrom(type)) {
for (URL url : (URL[]) value) {
addFromUrl(jarUrls, directoryPaths, url, bailError);
}
} else if (Path[].class.isAssignableFrom(type)) {
for (Path path : (Path[]) value) {
try {
addFromUrl(jarUrls, directoryPaths, path.toUri().toURL(), bailError);
} catch (MalformedURLException e) {
if (bailError) {
throw new RuntimeException(e);
}
}
}
} else if (File[].class.isAssignableFrom(type)) {
for (File file : (File[]) value) {
try {
addFromUrl(jarUrls, directoryPaths, file.toURI().toURL(), bailError);
} catch (MalformedURLException e) {
if (bailError) {
throw new RuntimeException(e);
}
}
}
} else if (maxDeep > 0) {
addFromUnknownClass(value, jarUrls, directoryPaths, bailError, maxDeep - 1);
}
}
private static void addFromUrl(Set <URL> jarUrls, Set <Path> directoryPaths, URL url, boolean bailError) {
if (url.getFile().endsWith(".jar") || url.getFile().endsWith(".zip")) {
// may want better way to detect jar files
jarUrls.add(url);
} else {
try {
Path path = Paths.get(url.toURI());
if (Files.isDirectory(path)) {
directoryPaths.add(path);
} else if (bailError) {
throw new RuntimeException("unknown url for class loading: " + url);
}
} catch (URISyntaxException e) {
if (bailError) {
throw new RuntimeException(e);
}
}
}
}
imports:
import java.io.File;
import java.io.IOException;
import java.lang.reflect.AccessibleObject;
import java.lang.reflect.Field;
import java.net.MalformedURLException;
import java.net.URISyntaxException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.DirectoryStream;
import java.nio.file.DirectoryStream.Filter;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Collection;
import java.util.HashSet;
import java.util.Set;
import java.util.jar.Attributes.Name;
import java.util.jar.JarEntry;
import java.util.jar.JarInputStream;
Based on #Staale's answer, and in an attempt not to rely on third party libraries, I would implement the File System approach by inspecting first package physical location with:
import java.io.File;
import java.io.FileFilter;
import java.util.ArrayList;
...
Class<?>[] foundClasses = new Class<?>[0];
final ArrayList<Class<?>> foundClassesDyn = new ArrayList<Class<?>>();
new java.io.File(
klass.getResource(
"/" + curPackage.replace( "." , "/")
).getFile()
).listFiles(
new java.io.FileFilter() {
public boolean accept(java.io.File file) {
final String classExtension = ".class";
if ( file.isFile()
&& file.getName().endsWith(classExtension)
// avoid inner classes
&& ! file.getName().contains("$") )
{
try {
String className = file.getName();
className = className.substring(0, className.length() - classExtension.length());
foundClassesDyn.add( Class.forName( curPackage + "." + className ) );
} catch (ClassNotFoundException e) {
e.printStackTrace(System.out);
}
}
return false;
}
}
);
foundClasses = foundClassesDyn.toArray(foundClasses);
plain java: FindAllClassesUsingPlainJavaReflectionTest.java
#Slf4j
class FindAllClassesUsingPlainJavaReflectionTest {
private static final Function<Throwable, RuntimeException> asRuntimeException = throwable -> {
log.error(throwable.getLocalizedMessage());
return new RuntimeException(throwable);
};
private static final Function<String, Collection<Class<?>>> findAllPackageClasses = basePackageName -> {
Locale locale = Locale.getDefault();
Charset charset = StandardCharsets.UTF_8;
val fileManager = ToolProvider.getSystemJavaCompiler()
.getStandardFileManager(/* diagnosticListener */ null, locale, charset);
StandardLocation location = StandardLocation.CLASS_PATH;
JavaFileObject.Kind kind = JavaFileObject.Kind.CLASS;
Set<JavaFileObject.Kind> kinds = Collections.singleton(kind);
val javaFileObjects = Try.of(() -> fileManager.list(location, basePackageName, kinds, /* recurse */ true))
.getOrElseThrow(asRuntimeException);
String pathToPackageAndClass = basePackageName.replace(".", File.separator);
Function<String, String> mapToClassName = s -> {
String prefix = Arrays.stream(s.split(pathToPackageAndClass))
.findFirst()
.orElse("");
return s.replaceFirst(prefix, "")
.replaceAll(File.separator, ".");
};
return StreamSupport.stream(javaFileObjects.spliterator(), /* parallel */ true)
.filter(javaFileObject -> javaFileObject.getKind().equals(kind))
.map(FileObject::getName)
.map(fileObjectName -> fileObjectName.replace(".class", ""))
.map(mapToClassName)
.map(className -> Try.of(() -> Class.forName(className))
.getOrElseThrow(asRuntimeException))
.collect(Collectors.toList());
};
#Test
#DisplayName("should get classes recursively in given package")
void test() {
Collection<Class<?>> classes = findAllPackageClasses.apply(getClass().getPackage().getName());
assertThat(classes).hasSizeGreaterThan(4);
classes.stream().map(String::valueOf).forEach(log::info);
}
}
PS: to simplify boilerplates for handling errors, etc, I'm using here vavr and lombok libraries
other implementations could be found in my GitHub daggerok/java-reflection-find-annotated-classes-or-methods repo
As of org.reflections version 0.10 :
org.reflections.scanners.SubTypesScanner
and
org.reflections.Reflections.getAllTypes()
are deprecated. I userd:
public Set<String> getEntityNamesInPackage(String packagePath) {
Reflections reflections = new Reflections(new ConfigurationBuilder()
.filterInputsBy(new FilterBuilder().includePackage(packagePath))
.setUrls(ClasspathHelper.forPackage(packagePath))
.setScanners(SubTypes.filterResultsBy(s -> true)));
return reflections.getAll(SubTypes).stream()
.filter(s -> s.startsWith(packagePath))
.collect(Collectors.toSet());
}
If you are merely looking to load a group of related classes, then Spring can help you.
Spring can instantiate a list or map of all classes that implement a given interface in one line of code. The list or map will contain instances of all the classes that implement that interface.
That being said, as an alternative to loading the list of classes out of the file system, instead just implement the same interface in all the classes you want to load, regardless of package and use Spring to provide you instances of all of them. That way, you can load (and instantiate) all the classes you desire regardless of what package they are in.
On the other hand, if having them all in a package is what you want, then simply have all the classes in that package implement a given interface.
Note that the interface itself doesn't have to declare any methods - it can be completely empty.
To inject a list of classes implementing a given interface, use the following lines of code...
#Autowired
private List<ISomeInterface> implementationList;
It is also possible to inject a Map of classes using Spring. Read the docs if interested to see how.
Finally, I will offer one other solution that is a bit more elegant than searching the entire file system tree.
Create a custom annotation that builds a catalog of the classes to which it is applied - something like #ClassCatalog.
It is not possible, since all classes in the package might not be loaded, while you always knows package of a class.

Programmatic SchemaExport / SchemaUpdate with Hibernate 5 and Spring 4

With Spring 4 and Hibernate 4, I was able to use Reflection to get the Hibernate Configuration object from the current environment, using this code:
#Autowired LocalContainerEntityManagerFactoryBean lcemfb;
EntityManagerFactoryImpl emf = (EntityManagerFactoryImpl) lcemfb.getNativeEntityManagerFactory();
SessionFactoryImpl sf = emf.getSessionFactory();
SessionFactoryServiceRegistryImpl serviceRegistry = (SessionFactoryServiceRegistryImpl) sf.getServiceRegistry();
Configuration cfg = null;
try {
Field field = SessionFactoryServiceRegistryImpl.class.getDeclaredField("configuration");
field.setAccessible(true);
cfg = (Configuration) field.get(serviceRegistry);
} catch (NoSuchFieldException | SecurityException | IllegalArgumentException | IllegalAccessException e) {
e.printStackTrace();
}
SchemaUpdate update = new SchemaUpdate(serviceRegistry, cfg);
With Hibernate 5, I must use some MetadataImplementor, which doesn't seems to be available from any of those objects. I also tried to use MetadataSources with the serviceRegistry. But it did say that it's the wrong kind of ServiceRegistry.
Is there any other way to get this working?
Basic idea for this problem is:
implementation of org.hibernate.integrator.spi.Integrator which stores required data to some holder. Register implementation as a service and use it where you need.
Work example you can find here https://github.com/valery-barysok/spring4-hibernate5-stackoverflow-34612019
create org.hibernate.integrator.api.integrator.Integrator class
import hello.HibernateInfoHolder;
import org.hibernate.boot.Metadata;
import org.hibernate.engine.spi.SessionFactoryImplementor;
import org.hibernate.service.spi.SessionFactoryServiceRegistry;
public class Integrator implements org.hibernate.integrator.spi.Integrator {
#Override
public void integrate(Metadata metadata, SessionFactoryImplementor sessionFactory, SessionFactoryServiceRegistry serviceRegistry) {
HibernateInfoHolder.setMetadata(metadata);
HibernateInfoHolder.setSessionFactory(sessionFactory);
HibernateInfoHolder.setServiceRegistry(serviceRegistry);
}
#Override
public void disintegrate(SessionFactoryImplementor sessionFactory, SessionFactoryServiceRegistry serviceRegistry) {
}
}
create META-INF/services/org.hibernate.integrator.spi.Integrator file
org.hibernate.integrator.api.integrator.Integrator
import org.hibernate.boot.spi.MetadataImplementor;
import org.hibernate.tool.hbm2ddl.SchemaExport;
import org.hibernate.tool.hbm2ddl.SchemaUpdate;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
#SpringBootApplication
public class Application implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Override
public void run(String... args) throws Exception {
new SchemaExport((MetadataImplementor) HibernateInfoHolder.getMetadata()).create(true, true);
new SchemaUpdate(HibernateInfoHolder.getServiceRegistry(), (MetadataImplementor) HibernateInfoHolder.getMetadata()).execute(true, true);
}
}
I would like to add up on Aviad's answer to make it complete as per OP's request.
The internals:
In order to get an instance of MetadataImplementor, the workaround is to register an instance of SessionFactoryBuilderFactory through Java's ServiceLoader facility. This registered service's getSessionFactoryBuilder method is then invoked by MetadataImplementor with an instance of itself, when hibernate is bootstrapped. The code references are below:
Service Loading
Invocation of getSessionFactoryBuilder
So, ultimately to get an instance of MetadataImplementor, you have to implement SessionFactoryBuilderFactory and register so ServiceLoader can recognize this service:
An implementation of SessionFactoryBuilderFactory:
public class MetadataProvider implements SessionFactoryBuilderFactory {
private static MetadataImplementor metadata;
#Override
public SessionFactoryBuilder getSessionFactoryBuilder(MetadataImplementor metadata, SessionFactoryBuilderImplementor defaultBuilder) {
this.metadata = metadata;
return defaultBuilder; //Just return the one provided in the argument itself. All we care about is the metadata :)
}
public static MetadataImplementor getMetadata() {
return metadata;
}
}
In order to register the above, create simple text file in the following path(assuming it's a maven project, ultimately we need the 'META-INF' folder to be available in the classpath):
src/main/resources/META-INF/services/org.hibernate.boot.spi.SessionFactoryBuilderFactory
And the content of the text file should be a single line(can even be multiple lines if you need to register multiple instances) stating the fully qualified class path of your implementation of SessionFactoryBuilderFactory. For example, for the above class, if your package name is 'com.yourcompany.prj', the following should be the content of the file.
com.yourcompany.prj.MetadataProvider
And that's it, if you run your application, spring app or standalone hibernate, you will have an instance of MetadataImplementor available through a static method once hibernate is bootstraped.
Update 1:
There is no way it can be injected via Spring. I digged into Hibernate's source code and the metadata object is not stored anywhere in SessionFactory(which is what we get from Spring). So, it's not possible to inject it. But there are two options if you want it in Spring's way:
Extend existing classes and customize all the way from
LocalSessionFactoryBean -> MetadataSources -> MetadataBuilder
LocalSessionFactoryBean is what you configure in Spring and it has an object of MetadataSources. MetadataSources creates MetadataBuilder which in turn creates MetadataImplementor. All the above operations don't store anything, they just create object on the fly and return. If you want to have an instance of MetaData, you should extend and modify the above classes so that they store a local copy of respective objects before they return. That way you can have a reference to MetadataImplementor. But I wouldn't really recommend this unless it's really needed, because the APIs might change over time.
On the other hand, if you don't mind building a MetaDataImplemetor from SessionFactory, the following code will help you:
EntityManagerFactoryImpl emf=(EntityManagerFactoryImpl)lcemfb.getNativeEntityManagerFactory();
SessionFactoryImpl sf=emf.getSessionFactory();
StandardServiceRegistry serviceRegistry = sf.getSessionFactoryOptions().getServiceRegistry();
MetadataSources metadataSources = new MetadataSources(new BootstrapServiceRegistryBuilder().build());
Metadata metadata = metadataSources.buildMetadata(serviceRegistry);
SchemaUpdate update=new SchemaUpdate(serviceRegistry,metadata); //To create SchemaUpdate
// You can either create SchemaExport from the above details, or you can get the existing one as follows:
try {
Field field = SessionFactoryImpl.class.getDeclaredField("schemaExport");
field.setAccessible(true);
SchemaExport schemaExport = (SchemaExport) field.get(serviceRegistry);
} catch (NoSuchFieldException | SecurityException | IllegalArgumentException | IllegalAccessException e) {
e.printStackTrace();
}
Take a look on this one:
public class EntityMetaData implements SessionFactoryBuilderFactory {
private static final ThreadLocal<MetadataImplementor> meta = new ThreadLocal<>();
#Override
public SessionFactoryBuilder getSessionFactoryBuilder(MetadataImplementor metadata, SessionFactoryBuilderImplementor defaultBuilder) {
meta.set(metadata);
return defaultBuilder;
}
public static MetadataImplementor getMeta() {
return meta.get();
}
}
Take a look on This Thread which seems to answer your needs
Well, my go to on this:
public class SchemaTranslator {
public static void main(String[] args) throws Exception {
new SchemaTranslator().run();
}
private void run() throws Exception {
String packageName[] = { "model"};
generate(packageName);
}
private List<Class<?>> getClasses(String packageName) throws Exception {
File directory = null;
try {
ClassLoader cld = getClassLoader();
URL resource = getResource(packageName, cld);
directory = new File(resource.getFile());
} catch (NullPointerException ex) {
throw new ClassNotFoundException(packageName + " (" + directory + ") does not appear to be a valid package");
}
return collectClasses(packageName, directory);
}
private ClassLoader getClassLoader() throws ClassNotFoundException {
ClassLoader cld = Thread.currentThread().getContextClassLoader();
if (cld == null) {
throw new ClassNotFoundException("Can't get class loader.");
}
return cld;
}
private URL getResource(String packageName, ClassLoader cld) throws ClassNotFoundException {
String path = packageName.replace('.', '/');
URL resource = cld.getResource(path);
if (resource == null) {
throw new ClassNotFoundException("No resource for " + path);
}
return resource;
}
private List<Class<?>> collectClasses(String packageName, File directory) throws ClassNotFoundException {
List<Class<?>> classes = new ArrayList<>();
if (directory.exists()) {
String[] files = directory.list();
for (String file : files) {
if (file.endsWith(".class")) {
// removes the .class extension
classes.add(Class.forName(packageName + '.' + file.substring(0, file.length() - 6)));
}
}
} else {
throw new ClassNotFoundException(packageName + " is not a valid package");
}
return classes;
}
private void generate(String[] packagesName) throws Exception {
Map<String, String> settings = new HashMap<String, String>();
settings.put("hibernate.hbm2ddl.auto", "drop-create");
settings.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQL94Dialect");
MetadataSources metadata = new MetadataSources(
new StandardServiceRegistryBuilder()
.applySettings(settings)
.build());
for (String packageName : packagesName) {
System.out.println("packageName: " + packageName);
for (Class<?> clazz : getClasses(packageName)) {
System.out.println("Class: " + clazz);
metadata.addAnnotatedClass(clazz);
}
}
SchemaExport export = new SchemaExport(
(MetadataImplementor) metadata.buildMetadata()
);
export.setDelimiter(";");
export.setOutputFile("db-schema.sql");
export.setFormat(true);
export.execute(true, false, false, false);
}
}

Calling same Method having same packageName From Different JARs [duplicate]

This question already has answers here:
Java, Classpath, Classloading => Multiple Versions of the same jar/project
(5 answers)
Closed 8 years ago.
I have three Jar files.All jar files contain Same class TestServicesImpl And Same Method displayWeLcomeMessage() But having different messages(output) of displayWeLcomeMessage().
Example :
public void displayWeLcomeMessage() {
System.out.println("wecome msg of JAR version first");
}
public void displayWeLcomeMessage() {
System.out.println("wecome msg of JAR version two");
}
public void displayWeLcomeMessage() {
System.out.println("wecome msg of JAR version third");
}
I have One main application and it contains jars included. My main application calls displayWeLcomeMessage() method.
first JAR is added in classpath and second JAR is loaded with custom classloader and invoke method displayWeLcomeMessage().
File file = new File("C:/Users/amitk/Desktop/Test_1.0.2.jar");
#SuppressWarnings("deprecation")
URL url = file.toURL();
URL[] urls = new URL[]{url};
URLClassLoader loader = new URLClassLoader(urls);
Class classS = loader.loadClass("com.amit.servicesImpl.TestServicesImpl");
Object object = classS.newInstance();
Method getmsg = classS.getMethod("displayWeLcomeMessage");
getmsg.invoke(object);
but it displays the same message as in method of JAR first.
In my third JAR, i have changed the package name.
that is
com.amit.servicesImpl.TestServicesImpl is changed to com.amit.servicesImpl2.TestServicesImpl
and this time it works properly that is message of method of JAR 3 is displayed here.
so let me know the main issue behind this.and solution for this.
Maybe you have your JAR in your initial class loader.
URLClassLoader will check existing class in parent class loader before checking in its own space.
1) You can extend and modify this behavior:
package com.mytool;
import java.net.URL;
import java.net.URLClassLoader;
import java.net.URLStreamHandlerFactory;
import java.util.HashMap;
import java.util.Map;
public class MyURLClassLoader extends URLClassLoader {
private final Map<String, Class<?>> ourClasses = new HashMap<>();
public MyURLClassLoader(URL[] urls, ClassLoader parent) {
super(urls, parent);
}
public MyURLClassLoader(URL[] urls) {
super(urls);
}
public MyURLClassLoader(URL[] urls, ClassLoader parent, URLStreamHandlerFactory factory) {
super(urls, parent, factory);
}
#Override
protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
synchronized (getClassLoadingLock(name)) {
// First, check if the class has already been loaded
Class<?> c = ourClasses.get(name);
if (c == null) {
// search in our paths
try {
c = findClass(name);
ourClasses.put(name, c);
} catch (ClassNotFoundException e) {
// ignore
}
}
if (c == null) {
c = findLoadedClass(name);
}
if (c != null) {
if (resolve) {
resolveClass(c);
}
return c;
}
// default search
return super.loadClass(name, resolve);
}
}
}
2) Or you can try to move our JAR and not load it at JVM start.
Note:
Instead of using a full reflexivity, I'll use an interface
loaded only by the initial classloader. Your object could implements it, and you'll be able to cast to this interface. If you do this with MyURLClassLoader, please don't add this interface in our dynamic loaded JAR!
Classloader will pick that class which was found first. If you are having 10 packages having same class then only that class will be picked which was introduced first.

Can I dynamically discover xml files in the classpath inside an EJB 3 container?

Background:
One of the components of our project operates using spring. Some SQL code is dynamically generated, based on a given XML spring configuration.
At first it was fine to store all the XML configurations in the same package on the classpath, (and then load it as a resource when the service is called) but over time we ended up with a large number of configurations. It came time to separate the configurations into different namespaces.
The Goal
What I want is, given a starting package on the classpath, to recursively walk the directory structure and discover any spring XML files dynamically. (So that as new configurations / packages are added, the files will still be found by the service).
The Problem
I was able to accomplish my goal fine when running outside an EJB container by using Thread.getContextClassloader().getResource(myBasePackage), then getting a File object and using it to walk the tree on the filesystem. Clunky, I know, but it was still classpath relative and it worked.
However, you cannot do this inside an EJB container (you can't interact with the filesystem at all), so I had to use the rather annoying workaround in which I maintain a list of hardcoded packages to search.
The Question
Is there a way (running inside an EJB container) to dynamically walk the classpath (from a given starting location) searching for arbitrary resources?
Short answer: Not while staying in compliance with the EJB spec. Because the spec envisions containers running in all kinds of non-standard situations, it does not make this possible.
Longer answer: Since you are not creating these resources dynamically, I would write a routine that gives you a list of all of the resources at build time and puts them in a dynamically generated file that your EJB knows how to reference. So you basically create a directory listing of packages and files that you can load in the EJB that are referenced in one master file.
Spring answer: Spring supports finding resources on the classpath, although I have no idea how well this works in the EJB context (and I doubt its EJB compliant, but I haven't checked). Some details here.
DISCLAIMER: As already pointed out, creating resources in the classpath is not recommended and depending on the EJB container explicitly forbidden. This may cause you a lot of problems because containers may explode your resources into another folder or even replicate the resources throughout the cluster (if thats the case). In order to create resources dynamically you have to create a custom classloader. So, I would never do it. It is better to access the filesystem directly than the classpath. It is less ugly and eventually cluster-safe if you use a remote filesystem + file locks.
If even after all I explained you still want to play with the classpath, you can try to do something like: get the classloader via
ClassLoader cld = Thread.currentThread().getContextClassLoader();
Starting from a base package enumerate all occurrences
Enumeration<URL> basePackageUrls = cld.getResources(basePackagePath);
Each URL is generally either a file link (file:///home/scott/.../MyResource.properties) or a jar link (file:///lib.jar!/com/domain/MyResource.properties). You have to check the pattern in the URL. Using that, enumerate the contents of the folder using the normal java API and find the subpackages. Proceed until you have scanned all packages.
See the class below (will be released with an open-source project of mine soon). It implemens a classpath scanner that you can pass in a selector. It works like a visitor. It my work for you, if not, get ideas from it. See the sample annotation selector at the end.
public class ClasspathScanner
{
private static final Log log = LogFactory.getLog(ClasspathScanner.class);
private static final String JAR_FILE_PATTERN = ".jar!";
private ClassSelector selector;
private Set<Class<?>> classes;
// PUBLIC METHODS ------------------------------------------------------------------------------
public synchronized Set<Class<?>> scanPackage(String basePackage, ClassSelector selector)
throws Exception
{
if (selector == null)
{
throw new NullPointerException("Selector cannot be NULL");
}
this.selector = selector;
this.classes = new HashSet<Class<?>>();
Set<Class<?>> aux;
try
{
scanClasses0(basePackage);
aux = this.classes;
}
finally
{
this.selector = null;
this.classes = null;
}
return aux;
}
// HELPER CLASSES ------------------------------------------------------------------------------
private void scanClasses0(String basePackage)
throws IOException, ClassNotFoundException, FileNotFoundException
{
File packageDirectory = null;
ClassLoader cld = getLoader();
String basePackagePath = basePackage.replace('.', '/');
Enumeration<URL> basePackageUrls = cld.getResources(basePackagePath);
if (basePackageUrls == null || !basePackageUrls.hasMoreElements())
{
throw new ClassNotFoundException("Base package path not found: [" + basePackagePath
+ "]");
}
while (basePackageUrls.hasMoreElements())
{
String packagePath = basePackageUrls.nextElement().getFile();
if (packagePath.contains(JAR_FILE_PATTERN))
{
scanJarFile(basePackagePath, packagePath);
}
else
{
packageDirectory = new File(packagePath);
scanDirectory(basePackage, packageDirectory);
}
}
}
private void scanDirectory(String packageName, File packagePath)
throws ClassNotFoundException, FileNotFoundException
{
if (packagePath.exists())
{
File[] packageFiles = packagePath.listFiles();
for (File file : packageFiles)
{
if (file.isFile() && file.getName().endsWith(".class"))
{
String fullFileName = packageName + '.' + file.getName();
checkClass(fullFileName);
}
else if (file.isDirectory())
{
scanDirectory(packageName + "." + file.getName(), file);
}
}
}
else
{
throw new FileNotFoundException(packagePath.getPath());
}
}
private void scanJarFile(String basePackagePath, String jarFileUrl)
throws IOException, ClassNotFoundException
{
String jarFilePath = jarFileUrl.substring("file:".length(), jarFileUrl
.indexOf(JAR_FILE_PATTERN)
+ JAR_FILE_PATTERN.length() - 1);
log.debug("URL JAR file path: [" + jarFilePath + "]");
jarFilePath = URLDecoder.decode(jarFilePath, "UTF-8");
log.debug("Decoded JAR file path: [" + jarFilePath + "]");
JarFile jar = new JarFile(new File(jarFilePath));
for (Enumeration<JarEntry> jarFiles = jar.entries(); jarFiles.hasMoreElements();)
{
JarEntry file = jarFiles.nextElement();
String fileName = file.getName();
if (!file.isDirectory() && fileName.endsWith(".class")
&& fileName.startsWith(basePackagePath))
{
String className = fileName.replace('/', '.');
checkClass(className);
}
}
}
private void checkClass(String fullFilePath) throws ClassNotFoundException
{
String className = fullFilePath.substring(0, fullFilePath.length() - 6);
Class<?> c = getLoader().loadClass(className);
if (selector.select(c))
{
classes.add(c);
}
}
private ClassLoader getLoader()
{
ClassLoader loader = Thread.currentThread().getContextClassLoader();
if (loader == null)
{
loader = getClass().getClassLoader();
}
return loader;
}
// INNER CLASSES -------------------------------------------------------------------------------
public interface ClassSelector
{
boolean select(Class<?> clazz);
}
public static class AnnotatedClassSelector implements ClassSelector
{
private final Class<? extends Annotation>[] annotations;
public AnnotatedClassSelector(Class<? extends Annotation>... annotations)
{
this.annotations = annotations;
}
public boolean select(Class<?> clazz)
{
for (Class<? extends Annotation> ac : annotations)
{
if (clazz.isAnnotationPresent(ac))
{
return true;
}
}
return false;
}
}
}

Getting all Classes from a Package

Lets say I have a java package commands which contains classes that all inherit from ICommand can I get all of those classes somehow? I'm locking for something among the lines of:
Package p = Package.getPackage("commands");
Class<ICommand>[] c = p.getAllPackagedClasses(); //not real
Is something like that possible?
Here's a basic example, assuming that classes are not JAR-packaged:
// Prepare.
String packageName = "com.example.commands";
List<Class<ICommand>> commands = new ArrayList<Class<ICommand>>();
URL root = Thread.currentThread().getContextClassLoader().getResource(packageName.replace(".", "/"));
// Filter .class files.
File[] files = new File(root.getFile()).listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.endsWith(".class");
}
});
// Find classes implementing ICommand.
for (File file : files) {
String className = file.getName().replaceAll(".class$", "");
Class<?> cls = Class.forName(packageName + "." + className);
if (ICommand.class.isAssignableFrom(cls)) {
commands.add((Class<ICommand>) cls);
}
}
Below is an implementation using the JSR-199 API, i.e. classes from javax.tools.*:
List<Class> commands = new ArrayList<>();
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
StandardJavaFileManager fileManager = compiler.getStandardFileManager(
null, null, null);
StandardLocation location = StandardLocation.CLASS_PATH;
String packageName = "commands";
Set<JavaFileObject.Kind> kinds = new HashSet<>();
kinds.add(JavaFileObject.Kind.CLASS);
boolean recurse = false;
Iterable<JavaFileObject> list = fileManager.list(location, packageName,
kinds, recurse);
for (JavaFileObject classFile : list) {
String name = classFile.getName().replaceAll(".*/|[.]class.*","");
commands.add(Class.forName(packageName + "." + name));
}
Works for all packages and classes on the class path, packaged in jar files or without. For classes not explicitly added to the class path, i.e. those loaded by the bootstrap class loader, try setting location to PLATFORM_CLASS_PATH instead.
Here is an utility method, using Spring.
Details about the pattern can be found here
public static List<Class> listMatchingClasses(String matchPattern) throws IOException {
List<Class> classes = new LinkedList<Class>();
PathMatchingResourcePatternResolver scanner = new PathMatchingResourcePatternResolver();
Resource[] resources = scanner.getResources(matchPattern);
for (Resource resource : resources) {
Class<?> clazz = getClassFromResource(resource);
classes.add(clazz);
}
return classes;
}
public static Class getClassFromResource(Resource resource) {
try {
String resourceUri = resource.getURI().toString();
resourceUri = resourceUri.replace(esourceUri.indexOf(".class"), "").replace("/", ".");
// try printing the resourceUri before calling forName, to see if it is OK.
return Class.forName(resourceUri);
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}
If you do not want to use external depencies and you want to work on your IDE / on a JAR file, you can try this:
public static List<Class<?>> getClassesForPackage(final String pkgName) throws IOException, URISyntaxException {
final String pkgPath = pkgName.replace('.', '/');
final URI pkg = Objects.requireNonNull(ClassLoader.getSystemClassLoader().getResource(pkgPath)).toURI();
final ArrayList<Class<?>> allClasses = new ArrayList<Class<?>>();
Path root;
if (pkg.toString().startsWith("jar:")) {
try {
root = FileSystems.getFileSystem(pkg).getPath(pkgPath);
} catch (final FileSystemNotFoundException e) {
root = FileSystems.newFileSystem(pkg, Collections.emptyMap()).getPath(pkgPath);
}
} else {
root = Paths.get(pkg);
}
final String extension = ".class";
try (final Stream<Path> allPaths = Files.walk(root)) {
allPaths.filter(Files::isRegularFile).forEach(file -> {
try {
final String path = file.toString().replace('/', '.');
final String name = path.substring(path.indexOf(pkgName), path.length() - extension.length());
allClasses.add(Class.forName(name));
} catch (final ClassNotFoundException | StringIndexOutOfBoundsException ignored) {
}
});
}
return allClasses;
}
From: Can you find all classes in a package using reflection?
Start with public Classloader.getResources(String name). Ask the classloader for a class corresponding to each name in the package you are interested. Repeat for all classloaders of relevance.
Yes but its not the easiest thing to do. There are lots of issues with this. Not all of the classes are easy to find. Some classes could be in a: Jar, as a class file, over the network etc.
Take a look at this thread.
To make sure they were the ICommand type then you would have to use reflection to check for the inheriting class.
This would be a very useful tool we need, and JDK should provide some support.
But it's probably better done during build. You know where all your class files are and you can inspect them statically and build a graph. At runtime you can query this graph to get all subtypes. This requires more work, but I believe it really belongs to the build process.
Using Johannes Link's ClasspathSuite, I was able to do it like this:
import org.junit.extensions.cpsuite.ClassTester;
import org.junit.extensions.cpsuite.ClasspathClassesFinder;
public static List<Class<?>> getClasses(final Package pkg, final boolean includeChildPackages) {
return new ClasspathClassesFinder(new ClassTester() {
#Override public boolean searchInJars() { return true; }
#Override public boolean acceptInnerClass() { return false; }
#Override public boolean acceptClassName(String name) {
return name.startsWith(pkg.getName()) && (includeChildPackages || name.indexOf(".", pkg.getName().length()) != -1);
}
#Override public boolean acceptClass(Class<?> c) { return true; }
}, System.getProperty("java.class.path")).find();
}
The ClasspathClassesFinder looks for class files and jars in the system classpath.
In your specific case, you could modify acceptClass like this:
#Override public boolean acceptClass(Class<?> c) {
return ICommand.class.isAssignableFrom(c);
}
One thing to note: be careful what you return in acceptClassName, as the next thing ClasspathClassesFinder does is to load the class and call acceptClass. If acceptClassName always return true, you'll end up loading every class in the classpath and that may cause an OutOfMemoryError.
You could use OpenPojo and do this:
final List<PojoClass> pojoClasses = PojoClassFactory.getPojoClassesRecursively("my.package.path", null);
Then you can go over the list and perform any functionality you desire.

Categories

Resources