Develop Reference

How to get SpringBoot to find my file in the src/main/resources directory

I have the following class (below). The file corresponding to vocabLookupFile is found when in the root directory of my SpringBoot project. However, I really want it in the src/main/resources directory of the project. With the below setup, it is not found there. By the way, the LookupMapper component is autowired in a #Service class, and other than not finding the file in src/main/resources, it works fine.
I am hoping someone can tell me how to modify the below so it can be found there. Thanks for any ideas.
#Component
public class LookupMapper {
public HashMap<String, LookUp> entry = new HashMap<>();
#Autowired
public LookupMapper(#Value("${vocab.lookup.mapper}") String vocabLookupFile) throws IOException {
try (CSVReader csvReader = new CSVReader(new FileReader(vocabLookupFile))) {
String[] values = null;
while ((values = csvReader.readNext()) != null) {
LookUp lookUp = new LookUp(values[1], Boolean.parseBoolean(values[2]));
this.entry.put(values[0].toUpperCase(), lookUp);
}
}
}
}

as per Mark Rotteveel, suggestion, with my file in the resource directory, in general, I need a solution that could retrieve the file from the context of the jar (and those things in the jar are considered "resources"). I used Classloader to get the resource as a stream. So the below works for me. Thanks to Mark.
#Component
public class LookupMapper {
public HashMap<String, LookUp> entry = new HashMap<>();
#Autowired
public LookupMapper(#Value("${vocab.lookup.mapper}") String vocabLookupFile) throws IOException {
ClassLoader classLoader = getClass().getClassLoader();
try (CSVReader csvReader = new CSVReader(new InputStreamReader(classLoader.getResourceAsStream(vocabLookupFile)))) {
String[] values = null;
while ((values = csvReader.readNext()) != null) {
LookUp lookUp = new LookUp(values[1], Boolean.parseBoolean(values[2]));
this.entry.put(values[0].toUpperCase(), lookUp);
}
}
}
}

Jetty ServletContextHandler setClassLoader not working on every request thread

I tried to develop multiple webservices using RESTEasy and Jetty. Im planning to make each of the webservice to have its own set of JAR files which will be loaded from certain directory.
What I have done is I create custom ClassLoader like this
public class AppClassLoader extends ClassLoader{
static Logger log = Logger.getLogger(AppClassLoader.class.getName());
String libPath = "";
public AppClassLoader(String libPath) {
this.libPath = libPath;
}
#Override
public Class loadClass(String name) throws ClassNotFoundException {
Class clazz = findLoadedClass(name);
if(clazz == null) {
try {
clazz = ClassLoader.getSystemClassLoader().loadClass(name);
if(clazz == null) {
clazz = getClass(name);
if(clazz == null) {
throw new ClassNotFoundException();
}
}
return clazz;
}catch (ClassNotFoundException e) {
// TODO: handle exception
throw new ClassNotFoundException();
}
}else {
return getSystemClassLoader().loadClass(name);
}
}
private Class<?> getClass(String name) throws ClassNotFoundException {
try {
File dir = new File(this.libPath);
if(dir.isDirectory()) {
for(File jar : dir.listFiles()) {
JarFile jarFile = new JarFile(jar.getPath());
Enumeration<JarEntry> e = jarFile.entries();
URL[] urls = { new URL("jar:file:" + jar.getPath()+"!/") };
URLClassLoader cl = URLClassLoader.newInstance(urls);
while (e.hasMoreElements()) {
JarEntry je = e.nextElement();
if(je.isDirectory() || !je.getName().endsWith(".class")){
continue;
}
String className = je.getName().substring(0,je.getName().length()-6);
className = className.replace('/', '.');
if(className.equals(name)) {
return cl.loadClass(className);
}
}
}
}
} catch (IOException e) {
e.printStackTrace();
return null;
}
return null;
}
And then what I did is assign this custom class loader into ServletContextHandler when I initialized Jetty and RestEasy to start the server, like below
final int port = 8080;
final Server server = new Server(port);
// Setup the basic Application "context" at "/".
// This is also known as the handler tree (in Jetty speak).
final ServletContextHandler context = new ServletContextHandler(server, CONTEXT_ROOT);
AppClassLoader classLoader = new AppClassLoader("../apps/dummy/lib");
context.setClassLoader(classLoader);
// Setup RESTEasy's HttpServletDispatcher at "/api/*".
final ServletHolder restEasyServlet = new ServletHolder(new HttpServletDispatcher());
restEasyServlet.setInitParameter("resteasy.servlet.mapping.prefix",APPLICATION_PATH);
restEasyServlet.setInitParameter("javax.ws.rs.Application",App.class.getCanonicalName());
final ServletHolder servlet = new ServletHolder(new HttpServletDispatcher());
context.addServlet(restEasyServlet, APPLICATION_PATH + "/*");
server.start();
server.join();
And then in the jax-rs endpoint I have this code
#Path("/")
public class Dummy {
Logger log = Logger.getLogger(Dummy.class.getName());
#GET
#Path("dummy")
#Produces(MediaType.TEXT_PLAIN)
public String test() {
HikariConfig src = new HikariConfig();
JwtMap jw = new JwtMap();
return "This is DUMMY service : "+src.getClass().getName().toString()+" ### "+jw.getClass().getName();
}}
I managed to start the server just fine, but when I tried to call the webservice, it return
java.lang.NoClassDefFoundError: com/zaxxer/hikari/HikariConfig
And then I see the classLoader used in the thread is not my custom class loader but the java standard class loader.
Which part that I did wrong here? Im so new to this classs loading stuff and I am not sure I literally understand how to used it.
Regards

By Default, ClassLoaders works on Parent first strategy. That means that Classes are searched and loaded through the sequence
Bootstrap Class Loader -> Ext Class Loader -> System Class Loader ->
Custom Class Loader
So, with that approach Dummy Class is loaded using System Class Loader. Now, classes loaded through a ClassLoader only has the visibility to the classes from Parent ClassLoader and not vice versa. So, HikariConfig class is not visible to the Dummy Class. Hence, the exception.
But, you should be able to load a class this way using the ServletContext Classloader which in your case is the Custom ClassLoader.
Inject the Servlet Context in your Dummy class and then
servletContext.getClassLoader().loadClass("com.zaxxer.hikari.HikariConfig");

Tomcat 8 new Resource implementation to map Jar files in a separate directory

With tomcat 8 I have extend the WebAppClassLoader and add some jar filed from a shared location to the classloader path using addRepository() method. With tomcat 8 addRepository have been removed and new resource implementation have been introduced. I'm still able to use the addUrl method to add jar files. But I would like to implement the new resource based implementation.
I've tried with
DirResourceSet dirResourceSet = new DirResourceSet(getContext().getResources(), "/WEB-INF/lib", "/home/thusitha/lib/runtimes/cxf", "/");
WebResourceRoot webResourceRoot = getContext().getResources();
webResourceRoot.getContext().getResources().addPreResources(dirResourceSet);
But this is not working and still it throws classnotfoundexception
Can someone tell me how to map a directory which contains jars to a particular webapp using Tomcat new resource implementation?

A solution to this problem is to register your resources by overriding the ContextConfig class (org.apache.catalina.startup.ContextConfig). Catalina enters a starting state immediately after it scans your document path for resources. Most of the processing of those resources, such as annotations, is handled by the ContextConfig LifecycleListener. To ensure the resources are added before the context configuration takes place, override the ContextConfig.
final Context currentContext = ctx;
ContextConfig ctxCfg = new ContextConfig() {
#Override
public void lifecycleEvent(LifecycleEvent event) {
if (event.getType().equals(Lifecycle.CONFIGURE_START_EVENT)) {
WebResourceRoot webResourcesRoot = currentContext.getResources();
String baseDir = Platform.getBaseDir(); // Server Base Directory
File libDir = new File(baseDir + File.separator + "lib");
DirResourceSet dirResourceSet = null;
try {
dirResourceSet = new DirResourceSet(webResourcesRoot, "/WEB-INF/lib", libDir.getCanonicalPath(), "/");
} catch (IOException e) {
throw new RuntimeException(e);
}
webResourcesRoot.addPostResources(dirResourceSet);
String[] possibleJars = dirResourceSet.list("/WEB-INF/lib");
for(String libfile : possibleJars) {
WebResource possibleJar = dirResourceSet.getResource("/WEB-INF/lib/"+libfile);
System.err.println(String.format("Loading possible jar %s",possibleJar.getCanonicalPath())); // Just checking...
if (possibleJar.isFile() && possibleJar.getName().endsWith(".jar")) {
WebResourceSet resourceSet = new JarResourceSet(webResourcesRoot, "/WEB-INF/classes", possibleJar.getCanonicalPath(),"/");
webResourcesRoot.addPostResources(resourceSet);
}
}
}
super.lifecycleEvent(event);
}
};
ctx.addLifecycleListener(ctxCfg);
This is an undocumented solution that works on Tomcat 8.0.23. Considering the complexity and difficulty of this I can't say it is a better solution than adding jars directly to ClassLoaders.

Can I dynamically discover xml files in the classpath inside an EJB 3 container?

Background:
One of the components of our project operates using spring. Some SQL code is dynamically generated, based on a given XML spring configuration.
At first it was fine to store all the XML configurations in the same package on the classpath, (and then load it as a resource when the service is called) but over time we ended up with a large number of configurations. It came time to separate the configurations into different namespaces.
The Goal
What I want is, given a starting package on the classpath, to recursively walk the directory structure and discover any spring XML files dynamically. (So that as new configurations / packages are added, the files will still be found by the service).
The Problem
I was able to accomplish my goal fine when running outside an EJB container by using Thread.getContextClassloader().getResource(myBasePackage), then getting a File object and using it to walk the tree on the filesystem. Clunky, I know, but it was still classpath relative and it worked.
However, you cannot do this inside an EJB container (you can't interact with the filesystem at all), so I had to use the rather annoying workaround in which I maintain a list of hardcoded packages to search.
The Question
Is there a way (running inside an EJB container) to dynamically walk the classpath (from a given starting location) searching for arbitrary resources?

Short answer: Not while staying in compliance with the EJB spec. Because the spec envisions containers running in all kinds of non-standard situations, it does not make this possible.
Longer answer: Since you are not creating these resources dynamically, I would write a routine that gives you a list of all of the resources at build time and puts them in a dynamically generated file that your EJB knows how to reference. So you basically create a directory listing of packages and files that you can load in the EJB that are referenced in one master file.
Spring answer: Spring supports finding resources on the classpath, although I have no idea how well this works in the EJB context (and I doubt its EJB compliant, but I haven't checked). Some details here.

DISCLAIMER: As already pointed out, creating resources in the classpath is not recommended and depending on the EJB container explicitly forbidden. This may cause you a lot of problems because containers may explode your resources into another folder or even replicate the resources throughout the cluster (if thats the case). In order to create resources dynamically you have to create a custom classloader. So, I would never do it. It is better to access the filesystem directly than the classpath. It is less ugly and eventually cluster-safe if you use a remote filesystem + file locks.
If even after all I explained you still want to play with the classpath, you can try to do something like: get the classloader via
ClassLoader cld = Thread.currentThread().getContextClassLoader();
Starting from a base package enumerate all occurrences
Enumeration<URL> basePackageUrls = cld.getResources(basePackagePath);
Each URL is generally either a file link (file:///home/scott/.../MyResource.properties) or a jar link (file:///lib.jar!/com/domain/MyResource.properties). You have to check the pattern in the URL. Using that, enumerate the contents of the folder using the normal java API and find the subpackages. Proceed until you have scanned all packages.
See the class below (will be released with an open-source project of mine soon). It implemens a classpath scanner that you can pass in a selector. It works like a visitor. It my work for you, if not, get ideas from it. See the sample annotation selector at the end.
public class ClasspathScanner
{
private static final Log log = LogFactory.getLog(ClasspathScanner.class);
private static final String JAR_FILE_PATTERN = ".jar!";
private ClassSelector selector;
private Set<Class<?>> classes;
// PUBLIC METHODS ------------------------------------------------------------------------------
public synchronized Set<Class<?>> scanPackage(String basePackage, ClassSelector selector)
throws Exception
{
if (selector == null)
{
throw new NullPointerException("Selector cannot be NULL");
}
this.selector = selector;
this.classes = new HashSet<Class<?>>();
Set<Class<?>> aux;
try
{
scanClasses0(basePackage);
aux = this.classes;
}
finally
{
this.selector = null;
this.classes = null;
}
return aux;
}
// HELPER CLASSES ------------------------------------------------------------------------------
private void scanClasses0(String basePackage)
throws IOException, ClassNotFoundException, FileNotFoundException
{
File packageDirectory = null;
ClassLoader cld = getLoader();
String basePackagePath = basePackage.replace('.', '/');
Enumeration<URL> basePackageUrls = cld.getResources(basePackagePath);
if (basePackageUrls == null || !basePackageUrls.hasMoreElements())
{
throw new ClassNotFoundException("Base package path not found: [" + basePackagePath
+ "]");
}
while (basePackageUrls.hasMoreElements())
{
String packagePath = basePackageUrls.nextElement().getFile();
if (packagePath.contains(JAR_FILE_PATTERN))
{
scanJarFile(basePackagePath, packagePath);
}
else
{
packageDirectory = new File(packagePath);
scanDirectory(basePackage, packageDirectory);
}
}
}
private void scanDirectory(String packageName, File packagePath)
throws ClassNotFoundException, FileNotFoundException
{
if (packagePath.exists())
{
File[] packageFiles = packagePath.listFiles();
for (File file : packageFiles)
{
if (file.isFile() && file.getName().endsWith(".class"))
{
String fullFileName = packageName + '.' + file.getName();
checkClass(fullFileName);
}
else if (file.isDirectory())
{
scanDirectory(packageName + "." + file.getName(), file);
}
}
}
else
{
throw new FileNotFoundException(packagePath.getPath());
}
}
private void scanJarFile(String basePackagePath, String jarFileUrl)
throws IOException, ClassNotFoundException
{
String jarFilePath = jarFileUrl.substring("file:".length(), jarFileUrl
.indexOf(JAR_FILE_PATTERN)
+ JAR_FILE_PATTERN.length() - 1);
log.debug("URL JAR file path: [" + jarFilePath + "]");
jarFilePath = URLDecoder.decode(jarFilePath, "UTF-8");
log.debug("Decoded JAR file path: [" + jarFilePath + "]");
JarFile jar = new JarFile(new File(jarFilePath));
for (Enumeration<JarEntry> jarFiles = jar.entries(); jarFiles.hasMoreElements();)
{
JarEntry file = jarFiles.nextElement();
String fileName = file.getName();
if (!file.isDirectory() && fileName.endsWith(".class")
&& fileName.startsWith(basePackagePath))
{
String className = fileName.replace('/', '.');
checkClass(className);
}
}
}
private void checkClass(String fullFilePath) throws ClassNotFoundException
{
String className = fullFilePath.substring(0, fullFilePath.length() - 6);
Class<?> c = getLoader().loadClass(className);
if (selector.select(c))
{
classes.add(c);
}
}
private ClassLoader getLoader()
{
ClassLoader loader = Thread.currentThread().getContextClassLoader();
if (loader == null)
{
loader = getClass().getClassLoader();
}
return loader;
}
// INNER CLASSES -------------------------------------------------------------------------------
public interface ClassSelector
{
boolean select(Class<?> clazz);
}
public static class AnnotatedClassSelector implements ClassSelector
{
private final Class<? extends Annotation>[] annotations;
public AnnotatedClassSelector(Class<? extends Annotation>... annotations)
{
this.annotations = annotations;
}
public boolean select(Class<?> clazz)
{
for (Class<? extends Annotation> ac : annotations)
{
if (clazz.isAnnotationPresent(ac))
{
return true;
}
}
return false;
}
}
}

Getting all Classes from a Package

Lets say I have a java package commands which contains classes that all inherit from ICommand can I get all of those classes somehow? I'm locking for something among the lines of:
Package p = Package.getPackage("commands");
Class<ICommand>[] c = p.getAllPackagedClasses(); //not real
Is something like that possible?

Here's a basic example, assuming that classes are not JAR-packaged:
// Prepare.
String packageName = "com.example.commands";
List<Class<ICommand>> commands = new ArrayList<Class<ICommand>>();
URL root = Thread.currentThread().getContextClassLoader().getResource(packageName.replace(".", "/"));
// Filter .class files.
File[] files = new File(root.getFile()).listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.endsWith(".class");
}
});
// Find classes implementing ICommand.
for (File file : files) {
String className = file.getName().replaceAll(".class$", "");
Class<?> cls = Class.forName(packageName + "." + className);
if (ICommand.class.isAssignableFrom(cls)) {
commands.add((Class<ICommand>) cls);
}
}

Below is an implementation using the JSR-199 API, i.e. classes from javax.tools.*:
List<Class> commands = new ArrayList<>();
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
StandardJavaFileManager fileManager = compiler.getStandardFileManager(
null, null, null);
StandardLocation location = StandardLocation.CLASS_PATH;
String packageName = "commands";
Set<JavaFileObject.Kind> kinds = new HashSet<>();
kinds.add(JavaFileObject.Kind.CLASS);
boolean recurse = false;
Iterable<JavaFileObject> list = fileManager.list(location, packageName,
kinds, recurse);
for (JavaFileObject classFile : list) {
String name = classFile.getName().replaceAll(".*/|[.]class.*","");
commands.add(Class.forName(packageName + "." + name));
}
Works for all packages and classes on the class path, packaged in jar files or without. For classes not explicitly added to the class path, i.e. those loaded by the bootstrap class loader, try setting location to PLATFORM_CLASS_PATH instead.

Here is an utility method, using Spring.
Details about the pattern can be found here
public static List<Class> listMatchingClasses(String matchPattern) throws IOException {
List<Class> classes = new LinkedList<Class>();
PathMatchingResourcePatternResolver scanner = new PathMatchingResourcePatternResolver();
Resource[] resources = scanner.getResources(matchPattern);
for (Resource resource : resources) {
Class<?> clazz = getClassFromResource(resource);
classes.add(clazz);
}
return classes;
}
public static Class getClassFromResource(Resource resource) {
try {
String resourceUri = resource.getURI().toString();
resourceUri = resourceUri.replace(esourceUri.indexOf(".class"), "").replace("/", ".");
// try printing the resourceUri before calling forName, to see if it is OK.
return Class.forName(resourceUri);
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}

If you do not want to use external depencies and you want to work on your IDE / on a JAR file, you can try this:
public static List<Class<?>> getClassesForPackage(final String pkgName) throws IOException, URISyntaxException {
final String pkgPath = pkgName.replace('.', '/');
final URI pkg = Objects.requireNonNull(ClassLoader.getSystemClassLoader().getResource(pkgPath)).toURI();
final ArrayList<Class<?>> allClasses = new ArrayList<Class<?>>();
Path root;
if (pkg.toString().startsWith("jar:")) {
try {
root = FileSystems.getFileSystem(pkg).getPath(pkgPath);
} catch (final FileSystemNotFoundException e) {
root = FileSystems.newFileSystem(pkg, Collections.emptyMap()).getPath(pkgPath);
}
} else {
root = Paths.get(pkg);
}
final String extension = ".class";
try (final Stream<Path> allPaths = Files.walk(root)) {
allPaths.filter(Files::isRegularFile).forEach(file -> {
try {
final String path = file.toString().replace('/', '.');
final String name = path.substring(path.indexOf(pkgName), path.length() - extension.length());
allClasses.add(Class.forName(name));
} catch (final ClassNotFoundException | StringIndexOutOfBoundsException ignored) {
}
});
}
return allClasses;
}
From: Can you find all classes in a package using reflection?

Start with public Classloader.getResources(String name). Ask the classloader for a class corresponding to each name in the package you are interested. Repeat for all classloaders of relevance.

Yes but its not the easiest thing to do. There are lots of issues with this. Not all of the classes are easy to find. Some classes could be in a: Jar, as a class file, over the network etc.
Take a look at this thread.
To make sure they were the ICommand type then you would have to use reflection to check for the inheriting class.

This would be a very useful tool we need, and JDK should provide some support.
But it's probably better done during build. You know where all your class files are and you can inspect them statically and build a graph. At runtime you can query this graph to get all subtypes. This requires more work, but I believe it really belongs to the build process.

Using Johannes Link's ClasspathSuite, I was able to do it like this:
import org.junit.extensions.cpsuite.ClassTester;
import org.junit.extensions.cpsuite.ClasspathClassesFinder;
public static List<Class<?>> getClasses(final Package pkg, final boolean includeChildPackages) {
return new ClasspathClassesFinder(new ClassTester() {
#Override public boolean searchInJars() { return true; }
#Override public boolean acceptInnerClass() { return false; }
#Override public boolean acceptClassName(String name) {
return name.startsWith(pkg.getName()) && (includeChildPackages || name.indexOf(".", pkg.getName().length()) != -1);
}
#Override public boolean acceptClass(Class<?> c) { return true; }
}, System.getProperty("java.class.path")).find();
}
The ClasspathClassesFinder looks for class files and jars in the system classpath.
In your specific case, you could modify acceptClass like this:
#Override public boolean acceptClass(Class<?> c) {
return ICommand.class.isAssignableFrom(c);
}
One thing to note: be careful what you return in acceptClassName, as the next thing ClasspathClassesFinder does is to load the class and call acceptClass. If acceptClassName always return true, you'll end up loading every class in the classpath and that may cause an OutOfMemoryError.

You could use OpenPojo and do this:
final List<PojoClass> pojoClasses = PojoClassFactory.getPojoClassesRecursively("my.package.path", null);
Then you can go over the list and perform any functionality you desire.