I am looking for a way to get a list of all resource names from a given classpath directory, something like a method List<String> getResourceNames (String directoryName).
For example, given a classpath directory x/y/z containing files a.html, b.html, c.html and a subdirectory d, getResourceNames("x/y/z") should return a List<String> containing the following strings:['a.html', 'b.html', 'c.html', 'd'].
It should work both for resources in filesystem and jars.
I know that I can write a quick snippet with Files, JarFiles and URLs, but I do not want to reinvent the wheel. My question is, given existing publicly available libraries, what is the quickest way to implement getResourceNames? Spring and Apache Commons stacks are both feasible.
Custom Scanner
Implement your own scanner. For example:
(limitations of this solution are mentioned in the comments)
private List<String> getResourceFiles(String path) throws IOException {
List<String> filenames = new ArrayList<>();
try (
InputStream in = getResourceAsStream(path);
BufferedReader br = new BufferedReader(new InputStreamReader(in))) {
String resource;
while ((resource = br.readLine()) != null) {
filenames.add(resource);
}
}
return filenames;
}
private InputStream getResourceAsStream(String resource) {
final InputStream in
= getContextClassLoader().getResourceAsStream(resource);
return in == null ? getClass().getResourceAsStream(resource) : in;
}
private ClassLoader getContextClassLoader() {
return Thread.currentThread().getContextClassLoader();
}
Spring Framework
Use PathMatchingResourcePatternResolver from Spring Framework.
Ronmamo Reflections
The other techniques might be slow at runtime for huge CLASSPATH values. A faster solution is to use ronmamo's Reflections API, which precompiles the search at compile time.
Here is the code
Source: forums.devx.com/showthread.php?t=153784
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Enumeration;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
import java.util.zip.ZipException;
import java.util.zip.ZipFile;
/**
* list resources available from the classpath # *
*/
public class ResourceList{
/**
* for all elements of java.class.path get a Collection of resources Pattern
* pattern = Pattern.compile(".*"); gets all resources
*
* #param pattern
* the pattern to match
* #return the resources in the order they are found
*/
public static Collection<String> getResources(
final Pattern pattern){
final ArrayList<String> retval = new ArrayList<String>();
final String classPath = System.getProperty("java.class.path", ".");
final String[] classPathElements = classPath.split(System.getProperty("path.separator"));
for(final String element : classPathElements){
retval.addAll(getResources(element, pattern));
}
return retval;
}
private static Collection<String> getResources(
final String element,
final Pattern pattern){
final ArrayList<String> retval = new ArrayList<String>();
final File file = new File(element);
if(file.isDirectory()){
retval.addAll(getResourcesFromDirectory(file, pattern));
} else{
retval.addAll(getResourcesFromJarFile(file, pattern));
}
return retval;
}
private static Collection<String> getResourcesFromJarFile(
final File file,
final Pattern pattern){
final ArrayList<String> retval = new ArrayList<String>();
ZipFile zf;
try{
zf = new ZipFile(file);
} catch(final ZipException e){
throw new Error(e);
} catch(final IOException e){
throw new Error(e);
}
final Enumeration e = zf.entries();
while(e.hasMoreElements()){
final ZipEntry ze = (ZipEntry) e.nextElement();
final String fileName = ze.getName();
final boolean accept = pattern.matcher(fileName).matches();
if(accept){
retval.add(fileName);
}
}
try{
zf.close();
} catch(final IOException e1){
throw new Error(e1);
}
return retval;
}
private static Collection<String> getResourcesFromDirectory(
final File directory,
final Pattern pattern){
final ArrayList<String> retval = new ArrayList<String>();
final File[] fileList = directory.listFiles();
for(final File file : fileList){
if(file.isDirectory()){
retval.addAll(getResourcesFromDirectory(file, pattern));
} else{
try{
final String fileName = file.getCanonicalPath();
final boolean accept = pattern.matcher(fileName).matches();
if(accept){
retval.add(fileName);
}
} catch(final IOException e){
throw new Error(e);
}
}
}
return retval;
}
/**
* list the resources that match args[0]
*
* #param args
* args[0] is the pattern to match, or list all resources if
* there are no args
*/
public static void main(final String[] args){
Pattern pattern;
if(args.length < 1){
pattern = Pattern.compile(".*");
} else{
pattern = Pattern.compile(args[0]);
}
final Collection<String> list = ResourceList.getResources(pattern);
for(final String name : list){
System.out.println(name);
}
}
}
If you are using Spring Have a look at PathMatchingResourcePatternResolver
Using Reflections
Get everything on the classpath:
Reflections reflections = new Reflections(null, new ResourcesScanner());
Set<String> resourceList = reflections.getResources(x -> true);
Another example - get all files with extension .csv from some.package:
Reflections reflections = new Reflections("some.package", new ResourcesScanner());
Set<String> resourceList = reflections.getResources(Pattern.compile(".*\\.csv"));
So in terms of the PathMatchingResourcePatternResolver this is what is needed in the code:
#Autowired
ResourcePatternResolver resourceResolver;
public void getResources() {
resourceResolver.getResources("classpath:config/*.xml");
}
If you use apache commonsIO you can use for the filesystem (optionally with extension filter):
Collection<File> files = FileUtils.listFiles(new File("directory/"), null, false);
and for resources/classpath:
List<String> files = IOUtils.readLines(MyClass.class.getClassLoader().getResourceAsStream("directory/"), Charsets.UTF_8);
If you don't know if "directoy/" is in the filesystem or in resources you may add a
if (new File("directory/").isDirectory())
or
if (MyClass.class.getClassLoader().getResource("directory/") != null)
before the calls and use both in combination...
The most robust mechanism for listing all resources in the classpath is currently to use this pattern with ClassGraph, because it handles the widest possible array of classpath specification mechanisms, including the new JPMS module system. (I am the author of ClassGraph.)
List<String> resourceNames;
try (ScanResult scanResult = new ClassGraph().acceptPaths("x/y/z").scan()) {
resourceNames = scanResult.getAllResources().getNames();
}
The Spring framework's PathMatchingResourcePatternResolver is really awesome for these things:
private Resource[] getXMLResources() throws IOException
{
ClassLoader classLoader = MethodHandles.lookup().getClass().getClassLoader();
PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(classLoader);
return resolver.getResources("classpath:x/y/z/*.xml");
}
Maven dependency:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>LATEST</version>
</dependency>
This should work (if spring is not an option):
public static List<String> getFilenamesForDirnameFromCP(String directoryName) throws URISyntaxException, UnsupportedEncodingException, IOException {
List<String> filenames = new ArrayList<>();
URL url = Thread.currentThread().getContextClassLoader().getResource(directoryName);
if (url != null) {
if (url.getProtocol().equals("file")) {
File file = Paths.get(url.toURI()).toFile();
if (file != null) {
File[] files = file.listFiles();
if (files != null) {
for (File filename : files) {
filenames.add(filename.toString());
}
}
}
} else if (url.getProtocol().equals("jar")) {
String dirname = directoryName + "/";
String path = url.getPath();
String jarPath = path.substring(5, path.indexOf("!"));
try (JarFile jar = new JarFile(URLDecoder.decode(jarPath, StandardCharsets.UTF_8.name()))) {
Enumeration<JarEntry> entries = jar.entries();
while (entries.hasMoreElements()) {
JarEntry entry = entries.nextElement();
String name = entry.getName();
if (name.startsWith(dirname) && !dirname.equals(name)) {
URL resource = Thread.currentThread().getContextClassLoader().getResource(name);
filenames.add(resource.toString());
}
}
}
}
}
return filenames;
}
My way, no Spring, used during a unit test:
URI uri = TestClass.class.getResource("/resources").toURI();
Path myPath = Paths.get(uri);
Stream<Path> walk = Files.walk(myPath, 1);
for (Iterator<Path> it = walk.iterator(); it.hasNext(); ) {
Path filename = it.next();
System.out.println(filename);
}
With Spring it's easy. Be it a file, or folder, or even multiple files, there are chances, you can do it via injection.
This example demonstrates the injection of multiple files located in x/y/z folder.
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;
#Service
public class StackoverflowService {
#Value("classpath:x/y/z/*")
private Resource[] resources;
public List<String> getResourceNames() {
return Arrays.stream(resources)
.map(Resource::getFilename)
.collect(Collectors.toList());
}
}
It does work for resources in the filesystem as well as in JARs.
Used a combination of Rob's response.
final String resourceDir = "resourceDirectory/";
List<String> files = IOUtils.readLines(Thread.currentThread().getClass().getClassLoader().getResourceAsStream(resourceDir), Charsets.UTF_8);
for (String f : files) {
String data = IOUtils.toString(Thread.currentThread().getClass().getClassLoader().getResourceAsStream(resourceDir + f));
// ... process data
}
I think you can leverage the [Zip File System Provider][1] to achieve this. When using FileSystems.newFileSystem it looks like you can treat the objects in that ZIP as a "regular" file.
In the linked documentation above:
Specify the configuration options for the zip file system in the java.util.Map object passed to the FileSystems.newFileSystem method. See the [Zip File System Properties][2] topic for information about the provider-specific configuration properties for the zip file system.
Once you have an instance of a zip file system, you can invoke the methods of the [java.nio.file.FileSystem][3] and [java.nio.file.Path][4] classes to perform operations such as copying, moving, and renaming files, as well as modifying file attributes.
The documentation for the jdk.zipfs module in [Java 11 states][5]:
The zip file system provider treats a zip or JAR file as a file system and provides the ability to manipulate the contents of the file. The zip file system provider can be created by [FileSystems.newFileSystem][6] if installed.
Here is a contrived example I did using your example resources. Note that a .zip is a .jar, but you could adapt your code to instead use classpath resources:
Setup
cd /tmp
mkdir -p x/y/z
touch x/y/z/{a,b,c}.html
echo 'hello world' > x/y/z/d
zip -r example.zip x
Java
import java.io.IOException;
import java.net.URI;
import java.nio.file.FileSystem;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.util.Collections;
import java.util.stream.Collectors;
public class MkobitZipRead {
public static void main(String[] args) throws IOException {
final URI uri = URI.create("jar:file:/tmp/example.zip");
try (
final FileSystem zipfs = FileSystems.newFileSystem(uri, Collections.emptyMap());
) {
Files.walk(zipfs.getPath("/")).forEach(path -> System.out.println("Files in zip:" + path));
System.out.println("-----");
final String manifest = Files.readAllLines(
zipfs.getPath("x", "y", "z").resolve("d")
).stream().collect(Collectors.joining(System.lineSeparator()));
System.out.println(manifest);
}
}
}
Output
Files in zip:/
Files in zip:/x/
Files in zip:/x/y/
Files in zip:/x/y/z/
Files in zip:/x/y/z/c.html
Files in zip:/x/y/z/b.html
Files in zip:/x/y/z/a.html
Files in zip:/x/y/z/d
-----
hello world
Neither of answers worked for me even though I had my resources put in resources folders and followed the above answers. What did make a trick was:
#Value("file:*/**/resources/**/schema/*.json")
private Resource[] resources;
Expanding on Luke Hutchinsons answer above, using his ClassGraph library, I was able to easily get a list of all files in a Resource folder with almost no effort at all.
Let's say that in your resource folder, you have a folder called MyImages. This is how easy it is to get a URL list of all the files in that folder:
import io.github.classgraph.ClassGraph;
import io.github.classgraph.ResourceList;
import io.github.classgraph.ScanResult;
public static LinkedList<URL> getURLList (String folder) {
LinkedList<URL> urlList = new LinkedList<>();
ScanResult scanResult = new ClassGraph().enableAllInfo().scan();
ResourceList resources = scanResult.getAllResources();
for (URL url : resources.getURLs()) {
if (url.toString().contains(folder)) {
urlList.addLast(url);
}
}
return urlList;
}
Then you simply do this:
LinkedList<URL> myURLFileList = getURLList("MyImages");
The URLs can then be loaded into streams or use Apache's FileUtils to copy the files somewhere else like this:
String outPath = "/My/Output/Path";
for(URL url : myURLFileList) {
FileUtils.copyURLToFile(url, new File(outPath, url.getFile()));
}
I think ClassGraph is a pretty slick library for making tasks like this very simple and easy to comprehend.
Based on #rob 's information above, I created the implementation which I am releasing to the public domain:
private static List<String> getClasspathEntriesByPath(String path) throws IOException {
InputStream is = Main.class.getClassLoader().getResourceAsStream(path);
StringBuilder sb = new StringBuilder();
while (is.available()>0) {
byte[] buffer = new byte[1024];
sb.append(new String(buffer, Charset.defaultCharset()));
}
return Arrays
.asList(sb.toString().split("\n")) // Convert StringBuilder to individual lines
.stream() // Stream the list
.filter(line -> line.trim().length()>0) // Filter out empty lines
.collect(Collectors.toList()); // Collect remaining lines into a List again
}
While I would not have expected getResourcesAsStream to work like that on a directory, it really does and it works well.
I am very lost with the concept of getResources.
I have put a simple text file in a bin folder which I would like to access as a resource so I can then build and deploy. However when I try to run the jar file I get a file not found error which I think is down to how I am accessing the resource. How can I use it?
public class Iterator {
static ArrayList<String> myFiles = new ArrayList<String>();
static URL filename= Iterator.class.getResource("/Files/FilesLogged.txt");
static String folderName;
static Path p;
public Iterator() { }
public static void main(String[] args) throws IOException, SAXException, TikaException, SQLException, ParseException, URISyntaxException, BackingStoreException {
Preferences userPrefs = Preferences.userNodeForPackage(TBB_SQLBuilder.class);
p = Paths.get(filename.toURI());
//This iterates through each of the files in the specified folder and copies them to a log.
//It also checks to see if that file has been read already so that it isn't re-inputted into the database if run again
//Loop through the ArrayList with the full path names of each folder in the outer loop
for (String line : Files.readAllLines(p)){
myFiles.add(line);
}
}
}
The error I get
Exception in thread "main" java.nio.file.FileSystemNotFoundException
at com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
at com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
at java.nio.file.Paths.get(Paths.java:143)
at Overview.Iterator.main(Iterator.java:46)
**Edit with #BorisTheSpiders' answer:
public class Iterator {
static ArrayList<String> myFiles = new ArrayList<String>();
static URL filename= Iterator.class.getResource("/Files/FilesLogged.txt");
static String folderName;
static Path p;
public Iterator() {
}
public static void main(String[] args) throws IOException, SAXException, TikaException, SQLException, ParseException, URISyntaxException, BackingStoreException {
Preferences userPrefs = Preferences.userNodeForPackage(TBB_SQLBuilder.class);
InputStream in = filename.openStream( );
BufferedReader reader = new BufferedReader( new InputStreamReader( in ) );
p = Paths.get(filename.toURI());
//This iterates through each of the files in the specified folder and copies them to a log.
//It also checks to see if that file has been read already so that it isn't re-inputted into the database if run again
//Loop through the ArrayList with the full path names of each folder in the outer loop
for (String line : Files.readAllLines(p)){
myFiles.add(line);
}
but I'm not really sure how I then use the reader to provide a Paths.get with a uri. I think I'm probably not understanding something fundamental here...
As pointed out in the comments, the file in question cannot be found in the file system.
As a suggestion, try replacing
static URL filename= Iterator.class.getResource("/Files/FilesLogged.txt");
with
static InputStream is = Iterator.class.getResourceAsStream("/Files/FilesLogged.txt");
and the block where the file is read with the following:
try (Scanner scanner = new Scanner(is)) {
while(scanner.hasNextLine()){
String line = scanner.nextLine();
myFiles.add(line);
}
}
I'm using this code to load a .java file and do the search:
public class FindClassName {
public static void main(String[] args) {
Logger logger = Logger.getLogger("MOJ.Logger");
try(Scanner scanner = new Scanner(new FileReader("./src/string/FindClassName.java"))){
String pattern = "class\\s*(\\w+)\\s*";
List<String> list = new LinkedList<>();
while(scanner.hasNext()){
String line = scanner.nextLine();
Matcher matcher = Pattern.compile(pattern).matcher(line);
while(matcher.find()){
list.add(matcher.group(1));
}
}
System.out.println(list);
}catch(IOException exception){
logger.info("Couldn't read file");
}
}
static class CHUJ{
}
}
it works but when I export this to executable .jar file then It can't load file to reader. I've read that I need to use:
FindClassName.class.getResource("FindClassName.java");
but this gives me NullPointerException. I tried many different approaches but still couldn't load that .java file to reader with getResource() or getResourcesAsStream().
I know there are tons of questions like that, but I couldn't find the solution.
EDIT
this is strange, this code now runs in executable jar(with resources included) but not within eclipse... why? is there a better way?
public class FindClassName {
public static void main(String[] args) throws FileNotFoundException {
Logger logger = Logger.getLogger("MOJ.Logger");
//String directory = "./src/string/FindClassName.java"; //to będzie działać w eclipsie, ale jak zrobisz z tego jar to wszystkie pliki .java zostaną skompilowane na .class
InputStream directory = FindClassName.class.getResourceAsStream("FindClassName.java");
try(Scanner scanner = new Scanner(new InputStreamReader(directory))){
String pattern = "class\\s*(\\w+)\\s*";
List<String> list = new LinkedList<>();
while(scanner.hasNext()){
String line = scanner.nextLine();
Matcher matcher = Pattern.compile(pattern).matcher(line);
while(matcher.find()){
list.add(matcher.group(1));
}
}
System.out.println(list);
}
}
static class CHUJ{
}
}
Problem:
When you create an excecutable JAR (Actually any jar) all the .java files are compiled and transformed into .class files that can be used then by the JVM.
Solution:
One solution is to include the source code directory as additinal resource directory (Not a normal usecase):
<build>
<resources>
<resource>
<directory>${project.build.sourceDirectory}</directory>
</resource>
<resource>
<directory>src/main/resources</directory>
</resource>
</resources>
</build>
See this link for more information or this video using Eclipse
Something like this should do it:
C:\>java -jar JarFileExample.jar
package com.mycompany.myproject;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
public class JarFileExample {
public static void main(String[] args) throws Exception {
InputStream is = JarFileExample.class.getResourceAsStream("/com/mycompany/myproject/JarFileExample.java");
writeFileToConsole(is);
}
private static void writeFileToConsole(InputStream is) throws IOException {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
for(String str = reader.readLine(); str != null; str = reader.readLine()) {
System.out.println(str);
}
reader.close();
}
}
C:\>
The source code can easily be included in the jar file but depends on the tool you are using to create the jar. I used eclipse and basically the method described here: Eclipse: include source code while exporting as runnable jar
I'm trying to use the following approach to identify specific file types using Java. I need to implement such things in my web application.
package filetype;
import java.io.File;
import java.net.URLConnection;
final public class FileType
{
public static void main(String[] args)
{
File temp=new File("G:/mountain.jpg");
System.out.println(URLConnection.guessContentTypeFromName(temp.getAbsolutePath()));
temp=new File("G:/myFile.txt");
System.out.println(URLConnection.guessContentTypeFromName(temp.getAbsolutePath()));
temp=new File("G:/zipByJava.zip");
System.out.println(URLConnection.guessContentTypeFromName(temp.getAbsolutePath()));
temp=new File("G:/MLM/Login.aspx");
System.out.println(URLConnection.guessContentTypeFromName(temp.getAbsolutePath()));
temp=new File("G:/power_point.pptx");
System.out.println(URLConnection.guessContentTypeFromName(temp.getAbsolutePath()));
temp=new File("G:/excel_sheet.xlsx");
System.out.println(URLConnection.guessContentTypeFromName(temp.getAbsolutePath()));
temp=new File("G:/word_document.docx");
System.out.println(URLConnection.guessContentTypeFromName(temp.getAbsolutePath()));
}
}
It displays the following output on the console.
image/jpeg
text/plain
application/zip
null
null
null
null
In the last four cases, it displays null and fails to recognize the file type of a given file. What is the best approach to identify a specific file type in Java?
Use
Files.probeContentType()
http://openjdk.java.net/projects/nio/javadoc/java/nio/file/Files.html#probeContentType(java.nio.file.Path)
And add custom detectors for any odd ball types you think you will encounter.
Edit:
Here is the oracle JavaDoc for v7
http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#probeContentType(java.nio.file.Path)
Here is the method I use to open a file with an external activity:
private void openFileWithExternalActivity(String path) {
if (path == null) {
return;
}
File file = new File(path);
if (file.exists()) {
Intent target = new Intent(Intent.ACTION_VIEW);
Uri uri = FileProvider.getUriForFile(view.getActivity(), BuildConfig.APPLICATION_ID + ".provider", file);
String mimeType = URLConnection.guessContentTypeFromName(file.getName());
target.setDataAndType(uri, mimeType);
target.setFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION);
try {
view.getActivity().startActivity(target);
} catch (ActivityNotFoundException e) {
// TODO Instruct the user to install a viewer here
Log.e("No viewer found", e);
}
}
}
The instruction
String mimeType = URLConnection.guessContentTypeFromName(file.getName());
determines the mime type
I am trying to create a file using
File newFile = new File("myFile");
However no file called "myFile" is created. This is within a Web application Project i.e. proper form to be pakaged as a WAR but I am calling it as part of a main method (just to see how this works).
How can I make it so that a new file is created at a location relative to the current one i.e not have to put in an absolute path.
EDIT:
newFile.createFile();
Doesn't seem to work:
Here is the entire code:
import java.io.File;
import java.io.IOException;
public class Tester {
public static void main(String[] args) throws IOException{
Tester test = new Tester();
test.makeFile();
}
public void makeFile() throws IOException{
File newFile = new File("myFile");
newFile.createNewFile();
}
}
In answer to your comment. The file will be created in the current directory of the process, unless you specifiy otherwise.
// new file in current directory
File f = new File("yourFile");
f.createNewFile();
System.out.println("Path:" + f.getAbsolutePath());
To create it in a directory of your choosing:
File f = new File("c:\\yourDirectory","yourFile");
f.createNewFile();
System.out.println("Path:" + f.getAbsolutePath());
newFile.createNewFile();
you could use newFile.createNewFile();