Scala way to do apache commons lang3 Validate? - java

What is the Scala way to accomplish the same as apache commons lang3 Validate? i.e. the validation aimed for user input validation as opposed to coding errors via assertions where failing the condition would lead to an IllegalArgumentException e.g.
/**
* Returns the newly created file only if the user entered a valid path.
* #param path input path where to store the new file
* #param fileName name of the file to be created in directory path
* #throws IllegalArgumentException when the input path doesn't exist.
*/
public File createFile(File path, String fileName) {
// make sure that the path exists before creating the file
// TODO: is there a way to do this in Scala without the need for 3rd party libraries
org.apache.commons.lang3.Validate.isTrue(path.exists(), "Illegal input path '" + path.getAbsolutePath() + "', it doesn't exist")
// now it is safe to create the file ...
File result = new File(path, fileName)
// ...
return result;
}

By coincidence I just found out that require would be the method of choice in Scala e.g.
/**
* Returns the newly created file only if the user entered a valid path.
* #param path input path where to store the new file
* #param fileName name of the file to be created in directory path
* #throws IllegalArgumentException when the input path doesn't exist.
*/
def createFile(path: File, fileName: String) : File = {
require(path.exists, s"""Illegal input path "${path.getAbsolutePath()}", it doesn't exist""")
// now it is safe to create the file ...
val result = new File(path, fileName)
// ...
result
}

Related

Load resources from a jar

How can I get the file object from the resource folder from a built jar? I can easily do it from the IDE by doing the following however this does not work with jar files. The code bellow just creates a copy of a file in my resource folder and saves it on the users machine.
final ClassLoader classloader = Thread.currentThread().getContextClassLoader();
final InputStream configIs = classloader.getResourceAsStream("config.yml");
if(configIs != null) {
final File configFile = new File(chosenPath + "/config.yml");
Files.copy(configIs, configFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
configIs.close();
}
I have been trying to figure out how to do the same from within a jar without much success even after reading many other articles. Based on my research the following was suggested. The code bellow loops through all of the paths that reference the resource path. This produces a 500+ KB text file so I am not exactly sure which one is correct but even if I find the correct one what do I do with it? Since the last if statement checks name starts with resourcePath I assume this is the correct entry
Entry:config.yml path:resources/problems.json
But how do I go from that to an input stream? If there is a better way of doing this let me know but so far I have not found any other resources on this topic.
final String resourcePath = "resources/problems.json";
final File jarFile = new File(getClass().getProtectionDomain().getCodeSource().getLocation().getPath());
if(jarFile.exists() && jarFile.isFile()) { // Run with JAR file
System.out.println("Run as jar");
try {
final JarFile jar = new JarFile(jarFile);
final Enumeration<JarEntry> entries = jar.entries(); //gives ALL entries in jar
while(entries.hasMoreElements()) {
final String name = entries.nextElement().getName();
System.out.println("Entry:" + name + " path:" + resourcePath);
if (name.startsWith(resourcePath)) { //filter according to the path
System.out.println(name);
}
}
jar.close();
} catch (IOException exception) {
exception.printStackTrace();
}
}
Perhaps this will help a bit:
/**
* Text files loaded with this method should work either within the IDE or your
* distributive JAR file.<br><br>
*
* <b>Example Usage:</b><pre>
* {#code
* try {
* List<String> list = loadFileFromResources("/resources/textfiles/data_2.txt");
* for (String str : list) {
* System.out.println(str);
* }
* }
* catch (IOException ex) {
* System.err.println(ex);
* } }</pre><br>
*
* As shown in the example usage above the file path requires the Resource
* folder to be named "resources" which also contains a sub-folder named
* "textfiles". This resource folder is to be located directly within the
* Project "src" folder, for example:<pre>
*
* ProjectDirectoryName
* bin
* build
* lib
* dist
* src
* resources
* images
* myImage_1.png
* myImage_2.jpg
* myImage_3.gif
* textfiles
* data_1.txt
* data_2.txt
* data_3.txt
* test</pre><br>
*
* Upon creating the resources/images and resources/textfiles folders within
* the src folder, your IDE should have created two packages within the Project
* named resources.images and resources.textfiles. Images would of course be
* related to the resources.images package and Text Files would be related to
* the resources.textfiles package.<br>
*
* #param filePath (String) Based on the example folder structure above this
* would be (as an example):<pre>
*
* <b> "/resources/textfiles/data_2.txt" </b></pre>
*
* #return ({#code List<String>}) A List of String containing the file's content.
*/
public List<String> loadFileFromResources(String filePath) throws java.io.IOException {
List<String> lines = new ArrayList<>();
try (java.io.InputStream in = getClass().getResourceAsStream(filePath);
java.io.BufferedReader reader = new java.io.BufferedReader(new java.io.InputStreamReader(in))) {
String line;
while ((line = reader.readLine()) != null) {
lines.add(line);
}
reader.close();
}
return lines;
}

Can't copy from HDFS to S3A

I have a class to copy directory content from one location to another using Apache FileUtil:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.FileUtil;
import org.apache.hadoop.fs.LocatedFileStatus;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.RemoteIterator;
class Folder {
private final FileSystem fs;
private final Path pth;
// ... constructors and other methods
/**
* Copy contents (files and files in subfolders) to another folder.
* Merges overlapping folders
* Overwrites already existing files
* #param destination Folder where content will be moved to
* #throws IOException If fails
*/
public void copyFilesTo(final Folder destination) throws IOException {
final RemoteIterator<LocatedFileStatus> iter = this.fs.listFiles(
this.pth,
true
);
final URI root = this.pth.toUri();
while (iter.hasNext()) {
final Path source = iter.next().getPath();
FileUtil.copy(
this.fs,
source,
destination.fs,
new Path(
destination.pth,
root.relativize(source.toUri()).toString()
),
false,
true,
this.fs.getConf()
);
}
}
}
This class is working fine with local (file:///) directories in a unit test,
but when I'm trying to use it in Hadoop cluster to copy files from HDFS (hdfs:///tmp/result) to Amazon S3 (s3a://mybucket/out) it doesn't copy anything and doesn't throw error, just silently skip copying.
When I'm using same class (with both HDFS or S3a filesystems) for another purpose it's working fine, so the configuration and fs reference should be OK here.
What I'm doing wrong? How to copy files from HDFS to S3A correctly?
I'm using Hadoop 2.7.3.
UPDATE
I've added more logs to copyFilesTo method to log root, source and target variables (and extracted rebase() method without changing the code):
/**
* Copy contents (files and files in subfolders) to another folder.
* Merges overlapping folders
* Overwrites already existing files
* #param dst Folder where content will be moved to
* #throws IOException If fails
*/
public void copyFilesTo(final Folder dst) throws IOException {
Logger.info(
this, "copyFilesTo(%s): from %s fs=%s",
dst, this, this.hdfs
);
final RemoteIterator<LocatedFileStatus> iter = this.hdfs.listFiles(
this.pth,
true
);
final URI root = this.pth.toUri();
Logger.info(this, "copyFilesTo(%s): root=%s", dst, root);
while (iter.hasNext()) {
final Path source = iter.next().getPath();
final Path target = Folder.rebase(dst.path(), this.path(), source);
Logger.info(
this, "copyFilesTo(%s): src=%s target=%s",
dst, source, target
);
FileUtil.copy(
this.hdfs,
source,
dst.hdfs,
target,
false,
true,
this.hdfs.getConf()
);
}
}
/**
* Change the base of target URI to new base, using root
* as common path.
* #param base New base
* #param root Common root
* #param target Target to rebase
* #return Path with new base
*/
static Path rebase(final Path base, final Path root, final Path target) {
return new Path(
base, root.toUri().relativize(target.toUri()).toString()
);
}
After running in the cluster I've got these logs:
io.Folder: copyFilesTo(hdfs:///tmp/_dst): from hdfs:///tmp/_src fs=DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_182008924_1, ugi=hadoop (auth:SIMPLE)]]
io.Folder: copyFilesTo(hdfs:///tmp/_dst): root=hdfs:///tmp/_src
INFO io.Folder: copyFilesTo(hdfs:///tmp/_dst): src=hdfs://ip-172-31-2-12.us-east-2.compute.internal:8020/tmp/_src/one.file target=hdfs://ip-172-31-2-12.us-east-2.compute.internal:8020/tmp/_src/one.file
I localized the wrong code in rebase() method, it's not working correctly when running in EMR cluster because RemoteIterator is returning URIs in remote format: hdfs://ip-172-31-2-12.us-east-2.compute.internal:8020/tmp/_src/one.file but this method is expecting format hdfs:///tmp/_src/one.file, this is why it's working locally with file:/// FS.
I don't see anything obviously wrong.
Does it do hdfs-hdfs or s3a-s3a?
Upgrade your hadoop version; 2.7.x is woefully out of date, especially with the S3A code. It's unlikely to make whatever this problem go away, but it will avoid other issues. Once you've upgraded, switch to the fast upload and it will do incremental updates of large files; currently your code will be saving each file to /tmp somewhere and then uploading it in the close() call.
turn on the logging for the org.apache.hadoop.fs.s3a module and see what it says
I'm not sure that it's the best and fully correct solution, but it's working for me. The idea is to fix host and port of local paths before rebasing, the working rebase method will be:
/**
* Change the base of target URI to new base, using root
* as common path.
* #param base New base
* #param root Common root
* #param target Target to rebase
* #return Path with new base
* #throws IOException If fails
*/
#SuppressWarnings("PMD.DefaultPackage")
static Path rebase(final Path base, final Path root, final Path target)
throws IOException {
final URI uri = target.toUri();
try {
return new Path(
new Path(
new URIBuilder(base.toUri())
.setHost(uri.getHost())
.setPort(uri.getPort())
.build()
),
new Path(
new URIBuilder(root.toUri())
.setHost(uri.getHost())
.setPort(uri.getPort())
.build()
.relativize(uri)
)
);
} catch (final URISyntaxException err) {
throw new IOException("Failed to rebase", err);
}
}

How to read an excel file inside the jar? [duplicate]

I'm trying to access an XML file within a jar file, from a separate jar that's running as a desktop application. I can get the URL to the file I need, but when I pass that to a FileReader (as a String) I get a FileNotFoundException saying "The file name, directory name, or volume label syntax is incorrect."
As a point of reference, I have no trouble reading image resources from the same jar, passing the URL to an ImageIcon constructor. This seems to indicate that the method I'm using to get the URL is correct.
URL url = getClass().getResource("/xxx/xxx/xxx/services.xml");
ServicesLoader jsl = new ServicesLoader( url.toString() );
Inside the ServicesLoader class I have
XMLReader xr = XMLReaderFactory.createXMLReader();
xr.setContentHandler( this );
xr.setErrorHandler( this );
xr.parse( new InputSource( new FileReader( filename )));
What's wrong with using this technique to read the XML file?
Looks like you want to use java.lang.Class.getResourceAsStream(String), see
https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getResourceAsStream-java.lang.String-
You don't say if this is a desktop or web app. I would use the getResourceAsStream() method from an appropriate ClassLoader if it's a desktop or the Context if it's a web app.
It looks as if you are using the URL.toString result as the argument to the FileReader constructor. URL.toString is a bit broken, and instead you should generally use url.toURI().toString(). In any case, the string is not a file path.
Instead, you should either:
Pass the URL to ServicesLoader and let it call openStream or similar.
Use Class.getResourceAsStream and just pass the stream over, possibly inside an InputSource. (Remember to check for nulls as the API is a bit messy.)
The problem was that I was going a step too far in calling the parse method of XMLReader. The parse method accepts an InputSource, so there was no reason to even use a FileReader. Changing the last line of the code above to
xr.parse( new InputSource( filename ));
works just fine.
I'd like to point out that one issues is what if the same resources are in multiple jar files.
Let's say you want to read /org/node/foo.txt, but not from one file, but from each and every jar file.
I have run into this same issue several times before.
I was hoping in JDK 7 that someone would write a classpath filesystem, but alas not yet.
Spring has the Resource class which allows you to load classpath resources quite nicely.
I wrote a little prototype to solve this very problem of reading resources form multiple jar files. The prototype does not handle every edge case, but it does handle looking for resources in directories that are in the jar files.
I have used Stack Overflow for quite sometime. This is the second answer that I remember answering a question so forgive me if I go too long (it is my nature).
This is a prototype resource reader. The prototype is devoid of robust error checking.
I have two prototype jar files that I have setup.
<pre>
<dependency>
<groupId>invoke</groupId>
<artifactId>invoke</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>node</groupId>
<artifactId>node</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
The jar files each have a file under /org/node/ called resource.txt.
This is just a prototype of what a handler would look like with classpath://
I also have a resource.foo.txt in my local resources for this project.
It picks them all up and prints them out.
package com.foo;
import java.io.File;
import java.io.FileReader;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URI;
import java.net.URL;
import java.util.Enumeration;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
/**
* Prototype resource reader.
* This prototype is devoid of error checking.
*
*
* I have two prototype jar files that I have setup.
* <pre>
* <dependency>
* <groupId>invoke</groupId>
* <artifactId>invoke</artifactId>
* <version>1.0-SNAPSHOT</version>
* </dependency>
*
* <dependency>
* <groupId>node</groupId>
* <artifactId>node</artifactId>
* <version>1.0-SNAPSHOT</version>
* </dependency>
* </pre>
* The jar files each have a file under /org/node/ called resource.txt.
* <br />
* This is just a prototype of what a handler would look like with classpath://
* I also have a resource.foo.txt in my local resources for this project.
* <br />
*/
public class ClasspathReader {
public static void main(String[] args) throws Exception {
/* This project includes two jar files that each have a resource located
in /org/node/ called resource.txt.
*/
/*
Name space is just a device I am using to see if a file in a dir
starts with a name space. Think of namespace like a file extension
but it is the start of the file not the end.
*/
String namespace = "resource";
//someResource is classpath.
String someResource = args.length > 0 ? args[0] :
//"classpath:///org/node/resource.txt"; It works with files
"classpath:///org/node/"; //It also works with directories
URI someResourceURI = URI.create(someResource);
System.out.println("URI of resource = " + someResourceURI);
someResource = someResourceURI.getPath();
System.out.println("PATH of resource =" + someResource);
boolean isDir = !someResource.endsWith(".txt");
/** Classpath resource can never really start with a starting slash.
* Logically they do, but in reality you have to strip it.
* This is a known behavior of classpath resources.
* It works with a slash unless the resource is in a jar file.
* Bottom line, by stripping it, it always works.
*/
if (someResource.startsWith("/")) {
someResource = someResource.substring(1);
}
/* Use the ClassLoader to lookup all resources that have this name.
Look for all resources that match the location we are looking for. */
Enumeration resources = null;
/* Check the context classloader first. Always use this if available. */
try {
resources =
Thread.currentThread().getContextClassLoader().getResources(someResource);
} catch (Exception ex) {
ex.printStackTrace();
}
if (resources == null || !resources.hasMoreElements()) {
resources = ClasspathReader.class.getClassLoader().getResources(someResource);
}
//Now iterate over the URLs of the resources from the classpath
while (resources.hasMoreElements()) {
URL resource = resources.nextElement();
/* if the resource is a file, it just means that we can use normal mechanism
to scan the directory.
*/
if (resource.getProtocol().equals("file")) {
//if it is a file then we can handle it the normal way.
handleFile(resource, namespace);
continue;
}
System.out.println("Resource " + resource);
/*
Split up the string that looks like this:
jar:file:/Users/rick/.m2/repository/invoke/invoke/1.0-SNAPSHOT/invoke-1.0-SNAPSHOT.jar!/org/node/
into
this /Users/rick/.m2/repository/invoke/invoke/1.0-SNAPSHOT/invoke-1.0-SNAPSHOT.jar
and this
/org/node/
*/
String[] split = resource.toString().split(":");
String[] split2 = split[2].split("!");
String zipFileName = split2[0];
String sresource = split2[1];
System.out.printf("After split zip file name = %s," +
" \nresource in zip %s \n", zipFileName, sresource);
/* Open up the zip file. */
ZipFile zipFile = new ZipFile(zipFileName);
/* Iterate through the entries. */
Enumeration entries = zipFile.entries();
while (entries.hasMoreElements()) {
ZipEntry entry = entries.nextElement();
/* If it is a directory, then skip it. */
if (entry.isDirectory()) {
continue;
}
String entryName = entry.getName();
System.out.printf("zip entry name %s \n", entryName);
/* If it does not start with our someResource String
then it is not our resource so continue.
*/
if (!entryName.startsWith(someResource)) {
continue;
}
/* the fileName part from the entry name.
* where /foo/bar/foo/bee/bar.txt, bar.txt is the file
*/
String fileName = entryName.substring(entryName.lastIndexOf("/") + 1);
System.out.printf("fileName %s \n", fileName);
/* See if the file starts with our namespace and ends with our extension.
*/
if (fileName.startsWith(namespace) && fileName.endsWith(".txt")) {
/* If you found the file, print out
the contents fo the file to System.out.*/
try (Reader reader = new InputStreamReader(zipFile.getInputStream(entry))) {
StringBuilder builder = new StringBuilder();
int ch = 0;
while ((ch = reader.read()) != -1) {
builder.append((char) ch);
}
System.out.printf("zip fileName = %s\n\n####\n contents of file %s\n###\n", entryName, builder);
} catch (Exception ex) {
ex.printStackTrace();
}
}
//use the entry to see if it's the file '1.txt'
//Read from the byte using file.getInputStream(entry)
}
}
}
/**
* The file was on the file system not a zip file,
* this is here for completeness for this example.
* otherwise.
*
* #param resource
* #param namespace
* #throws Exception
*/
private static void handleFile(URL resource, String namespace) throws Exception {
System.out.println("Handle this resource as a file " + resource);
URI uri = resource.toURI();
File file = new File(uri.getPath());
if (file.isDirectory()) {
for (File childFile : file.listFiles()) {
if (childFile.isDirectory()) {
continue;
}
String fileName = childFile.getName();
if (fileName.startsWith(namespace) && fileName.endsWith("txt")) {
try (FileReader reader = new FileReader(childFile)) {
StringBuilder builder = new StringBuilder();
int ch = 0;
while ((ch = reader.read()) != -1) {
builder.append((char) ch);
}
System.out.printf("fileName = %s\n\n####\n contents of file %s\n###\n", childFile, builder);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
} else {
String fileName = file.getName();
if (fileName.startsWith(namespace) && fileName.endsWith("txt")) {
try (FileReader reader = new FileReader(file)) {
StringBuilder builder = new StringBuilder();
int ch = 0;
while ((ch = reader.read()) != -1) {
builder.append((char) ch);
}
System.out.printf("fileName = %s\n\n####\n contents of file %s\n###\n", fileName, builder);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
}
You can see a fuller example here with the sample output.
Here's a sample code on how to read a file properly inside a jar file (in this case, the current executing jar file)
Just change executable with the path of your jar file if it is not the current running one.
Then change the filePath to the path of the file you want to use inside the jar file. I.E. if your file is in
someJar.jar\img\test.gif
. Set the filePath to "img\test.gif"
File executable = new File(BrowserViewControl.class.getProtectionDomain().getCodeSource().getLocation().toURI());
JarFile jar = new JarFile(executable);
InputStream fileInputStreamReader = jar.getInputStream(jar.getJarEntry(filePath));
byte[] bytes = new byte[fileInputStreamReader.available()];
int sizeOrig = fileInputStreamReader.available();
int size = fileInputStreamReader.available();
int offset = 0;
while (size != 0){
fileInputStreamReader.read(bytes, offset, size);
offset = sizeOrig - fileInputStreamReader.available();
size = fileInputStreamReader.available();
}
Outside of your technique, why not use the standard Java JarFile class to get the references you want? From there most of your problems should go away.
If you use resources extensively, you might consider using
Commons VFS.
Also supports:
* Local Files
* FTP, SFTP
* HTTP and HTTPS
* Temporary Files "normal FS backed)
* Zip, Jar and Tar (uncompressed, tgz or tbz2)
* gzip and bzip2
* resources
* ram - "ramdrive"
* mime
There's also JBoss VFS - but it's not much documented.
I have 2 CSV files that I use to read data. The java program is exported as a runnable jar file. When you export it, you will figure out it doesn't export your resources with it.
I added a folder under project called data in eclipse. In that folder i stored my csv files.
When I need to reference those files I do it like this...
private static final String ZIP_FILE_LOCATION_PRIMARY = "free-zipcode-database-Primary.csv";
private static final String ZIP_FILE_LOCATION = "free-zipcode-database.csv";
private static String getFileLocation(){
String loc = new File("").getAbsolutePath() + File.separatorChar +
"data" + File.separatorChar;
if (usePrimaryZipCodesOnly()){
loc = loc.concat(ZIP_FILE_LOCATION_PRIMARY);
} else {
loc = loc.concat(ZIP_FILE_LOCATION);
}
return loc;
}
Then when you put the jar in a location so it can be ran via commandline, make sure that you add the data folder with the resources into the same location as the jar file.

How to get the complete real file path for "../../dir/file.ext" in java?

I there a way in the java.io package to convert a relative path containing "../" to an absolute path?
My goal is to remove the "../" part of the path because
java.awt.Desktop.getDesktop().open()
Does not seem to support ../ in file path under windows
--- Edited when comment was made that the ../ was still in the path ---
import java.io.File;
public class Test {
public static void main(String[] args) throws Exception {
File file = new File("../../home");
System.out.println(file.getCanonicalPath());
System.out.println(file.getAbsolutePath());
}
}
will run with the output
/home/ebuck/home
/home/ebuck/workspace/State/../../home
based on my current working directory of /home/ebuck/workspace/State
Note that you asked for the complete real file path, which technically an absolute path is a complete, real file path, it's just not the shortest complete real file path. So, if you want to do it fast and dirty, one can just add "../../home" to the current working directory and obtain a full, complete file path (albeit a wordy one that contains unnecessary information).
If you want the shortest full, complete file path, that's what getCanonicalPath() is used for. It throws an exception; because, some joker out there will probably ask for "../../home" when in the root directory.
--- Original post follows, with edits ---
new File("../../dir/file.ext").getCanonoicalPath();
Will do so collapsing (following) the relative path links (. and ..).
new File("../../dir/file.ext").getAbsolutePath();
Will do so without collapsing (following) the relative path links.
String path = new File("../../dir/file.ext").getCanonicalPath();
File f = new File("..");
String path = f.getAbsolutePath();
Sometimes File.getCanonicalPath() may not be desired since it may resolve things like symlinks, so if you want to maintain the "logical" path that File.getAbsolutePath() provides, you cannot use getCanonicalPath(). Also, IIRC, gCP() can thrown an exception while gAP() does not, and gAP() can refer to a path does not exist.
I've encounter the '..' problem a long time ago. Here is a utility method I wrote to remove the '..'s in a path:
/**
* Retrieve "clean" absolute path of a file.
* <p>This method retrieves the absolute pathname of file,
* with relative components (e.g. <tt>..</tt>) removed.
* Java's <tt>File.getAbsolutePath()</tt> does not remove
* relative components. For example, if given the pathname:
* </p>
* <pre>
* dir/subdir/subsubdir/../../file.txt
* </pre>
* <p>{#link File#getAbsolutePath()} will return something like:
* </p>
* <pre>
* /home/whomever/dir/subdir/subsubdir/../../file.txt
* </pre>
* <p>This method will return:
* </p>
* <pre>
* /home/whomever/dir/file.txt
* </pre>
*
* #param f File to get clean absolute path of.
* #return Clean absolute pathname of <i>f</i>.
*/
public static String cleanAbsolutePath(
File f
) {
String abs = f.getAbsolutePath();
if (!relDirPattern.matcher(abs).find()) {
// Nothing to do, so just return what Java provided
return abs;
}
String[] parts = abs.split(fileSepRex);
ArrayList<String> newPath = new ArrayList<String>(parts.length);
int capacity = 0;
for (String p : parts) {
if (p.equals(".")) continue;
if (p.equals("..")) {
if (newPath.size() == 0) continue;
String removed = newPath.remove(newPath.size() -1);
capacity -= removed.length();
continue;
}
newPath.add(p);
capacity += p.length();
}
int size = newPath.size();
if (size == 0) {
return File.separator;
}
StringBuilder result = new StringBuilder(capacity);
int i = 0;
for (String p : newPath) {
++i;
result.append(p);
if (i < size) {
result.append(File.separatorChar);
}
}
return result.toString();
}
/** Regex string representing file name separator. */
private static String fileSepRex = "\\"+File.separator;
/** Pattern for checking if pathname has relative components. */
private static Pattern relDirPattern = Pattern.compile(
"(?:\\A|" + fileSepRex + ")\\.{1,2}(?:" + fileSepRex + "|\\z)");

How do I read a resource file from a Java jar file?

I'm trying to access an XML file within a jar file, from a separate jar that's running as a desktop application. I can get the URL to the file I need, but when I pass that to a FileReader (as a String) I get a FileNotFoundException saying "The file name, directory name, or volume label syntax is incorrect."
As a point of reference, I have no trouble reading image resources from the same jar, passing the URL to an ImageIcon constructor. This seems to indicate that the method I'm using to get the URL is correct.
URL url = getClass().getResource("/xxx/xxx/xxx/services.xml");
ServicesLoader jsl = new ServicesLoader( url.toString() );
Inside the ServicesLoader class I have
XMLReader xr = XMLReaderFactory.createXMLReader();
xr.setContentHandler( this );
xr.setErrorHandler( this );
xr.parse( new InputSource( new FileReader( filename )));
What's wrong with using this technique to read the XML file?
Looks like you want to use java.lang.Class.getResourceAsStream(String), see
https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getResourceAsStream-java.lang.String-
You don't say if this is a desktop or web app. I would use the getResourceAsStream() method from an appropriate ClassLoader if it's a desktop or the Context if it's a web app.
It looks as if you are using the URL.toString result as the argument to the FileReader constructor. URL.toString is a bit broken, and instead you should generally use url.toURI().toString(). In any case, the string is not a file path.
Instead, you should either:
Pass the URL to ServicesLoader and let it call openStream or similar.
Use Class.getResourceAsStream and just pass the stream over, possibly inside an InputSource. (Remember to check for nulls as the API is a bit messy.)
The problem was that I was going a step too far in calling the parse method of XMLReader. The parse method accepts an InputSource, so there was no reason to even use a FileReader. Changing the last line of the code above to
xr.parse( new InputSource( filename ));
works just fine.
I'd like to point out that one issues is what if the same resources are in multiple jar files.
Let's say you want to read /org/node/foo.txt, but not from one file, but from each and every jar file.
I have run into this same issue several times before.
I was hoping in JDK 7 that someone would write a classpath filesystem, but alas not yet.
Spring has the Resource class which allows you to load classpath resources quite nicely.
I wrote a little prototype to solve this very problem of reading resources form multiple jar files. The prototype does not handle every edge case, but it does handle looking for resources in directories that are in the jar files.
I have used Stack Overflow for quite sometime. This is the second answer that I remember answering a question so forgive me if I go too long (it is my nature).
This is a prototype resource reader. The prototype is devoid of robust error checking.
I have two prototype jar files that I have setup.
<pre>
<dependency>
<groupId>invoke</groupId>
<artifactId>invoke</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>node</groupId>
<artifactId>node</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
The jar files each have a file under /org/node/ called resource.txt.
This is just a prototype of what a handler would look like with classpath://
I also have a resource.foo.txt in my local resources for this project.
It picks them all up and prints them out.
package com.foo;
import java.io.File;
import java.io.FileReader;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URI;
import java.net.URL;
import java.util.Enumeration;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
/**
* Prototype resource reader.
* This prototype is devoid of error checking.
*
*
* I have two prototype jar files that I have setup.
* <pre>
* <dependency>
* <groupId>invoke</groupId>
* <artifactId>invoke</artifactId>
* <version>1.0-SNAPSHOT</version>
* </dependency>
*
* <dependency>
* <groupId>node</groupId>
* <artifactId>node</artifactId>
* <version>1.0-SNAPSHOT</version>
* </dependency>
* </pre>
* The jar files each have a file under /org/node/ called resource.txt.
* <br />
* This is just a prototype of what a handler would look like with classpath://
* I also have a resource.foo.txt in my local resources for this project.
* <br />
*/
public class ClasspathReader {
public static void main(String[] args) throws Exception {
/* This project includes two jar files that each have a resource located
in /org/node/ called resource.txt.
*/
/*
Name space is just a device I am using to see if a file in a dir
starts with a name space. Think of namespace like a file extension
but it is the start of the file not the end.
*/
String namespace = "resource";
//someResource is classpath.
String someResource = args.length > 0 ? args[0] :
//"classpath:///org/node/resource.txt"; It works with files
"classpath:///org/node/"; //It also works with directories
URI someResourceURI = URI.create(someResource);
System.out.println("URI of resource = " + someResourceURI);
someResource = someResourceURI.getPath();
System.out.println("PATH of resource =" + someResource);
boolean isDir = !someResource.endsWith(".txt");
/** Classpath resource can never really start with a starting slash.
* Logically they do, but in reality you have to strip it.
* This is a known behavior of classpath resources.
* It works with a slash unless the resource is in a jar file.
* Bottom line, by stripping it, it always works.
*/
if (someResource.startsWith("/")) {
someResource = someResource.substring(1);
}
/* Use the ClassLoader to lookup all resources that have this name.
Look for all resources that match the location we are looking for. */
Enumeration resources = null;
/* Check the context classloader first. Always use this if available. */
try {
resources =
Thread.currentThread().getContextClassLoader().getResources(someResource);
} catch (Exception ex) {
ex.printStackTrace();
}
if (resources == null || !resources.hasMoreElements()) {
resources = ClasspathReader.class.getClassLoader().getResources(someResource);
}
//Now iterate over the URLs of the resources from the classpath
while (resources.hasMoreElements()) {
URL resource = resources.nextElement();
/* if the resource is a file, it just means that we can use normal mechanism
to scan the directory.
*/
if (resource.getProtocol().equals("file")) {
//if it is a file then we can handle it the normal way.
handleFile(resource, namespace);
continue;
}
System.out.println("Resource " + resource);
/*
Split up the string that looks like this:
jar:file:/Users/rick/.m2/repository/invoke/invoke/1.0-SNAPSHOT/invoke-1.0-SNAPSHOT.jar!/org/node/
into
this /Users/rick/.m2/repository/invoke/invoke/1.0-SNAPSHOT/invoke-1.0-SNAPSHOT.jar
and this
/org/node/
*/
String[] split = resource.toString().split(":");
String[] split2 = split[2].split("!");
String zipFileName = split2[0];
String sresource = split2[1];
System.out.printf("After split zip file name = %s," +
" \nresource in zip %s \n", zipFileName, sresource);
/* Open up the zip file. */
ZipFile zipFile = new ZipFile(zipFileName);
/* Iterate through the entries. */
Enumeration entries = zipFile.entries();
while (entries.hasMoreElements()) {
ZipEntry entry = entries.nextElement();
/* If it is a directory, then skip it. */
if (entry.isDirectory()) {
continue;
}
String entryName = entry.getName();
System.out.printf("zip entry name %s \n", entryName);
/* If it does not start with our someResource String
then it is not our resource so continue.
*/
if (!entryName.startsWith(someResource)) {
continue;
}
/* the fileName part from the entry name.
* where /foo/bar/foo/bee/bar.txt, bar.txt is the file
*/
String fileName = entryName.substring(entryName.lastIndexOf("/") + 1);
System.out.printf("fileName %s \n", fileName);
/* See if the file starts with our namespace and ends with our extension.
*/
if (fileName.startsWith(namespace) && fileName.endsWith(".txt")) {
/* If you found the file, print out
the contents fo the file to System.out.*/
try (Reader reader = new InputStreamReader(zipFile.getInputStream(entry))) {
StringBuilder builder = new StringBuilder();
int ch = 0;
while ((ch = reader.read()) != -1) {
builder.append((char) ch);
}
System.out.printf("zip fileName = %s\n\n####\n contents of file %s\n###\n", entryName, builder);
} catch (Exception ex) {
ex.printStackTrace();
}
}
//use the entry to see if it's the file '1.txt'
//Read from the byte using file.getInputStream(entry)
}
}
}
/**
* The file was on the file system not a zip file,
* this is here for completeness for this example.
* otherwise.
*
* #param resource
* #param namespace
* #throws Exception
*/
private static void handleFile(URL resource, String namespace) throws Exception {
System.out.println("Handle this resource as a file " + resource);
URI uri = resource.toURI();
File file = new File(uri.getPath());
if (file.isDirectory()) {
for (File childFile : file.listFiles()) {
if (childFile.isDirectory()) {
continue;
}
String fileName = childFile.getName();
if (fileName.startsWith(namespace) && fileName.endsWith("txt")) {
try (FileReader reader = new FileReader(childFile)) {
StringBuilder builder = new StringBuilder();
int ch = 0;
while ((ch = reader.read()) != -1) {
builder.append((char) ch);
}
System.out.printf("fileName = %s\n\n####\n contents of file %s\n###\n", childFile, builder);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
} else {
String fileName = file.getName();
if (fileName.startsWith(namespace) && fileName.endsWith("txt")) {
try (FileReader reader = new FileReader(file)) {
StringBuilder builder = new StringBuilder();
int ch = 0;
while ((ch = reader.read()) != -1) {
builder.append((char) ch);
}
System.out.printf("fileName = %s\n\n####\n contents of file %s\n###\n", fileName, builder);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
}
You can see a fuller example here with the sample output.
Here's a sample code on how to read a file properly inside a jar file (in this case, the current executing jar file)
Just change executable with the path of your jar file if it is not the current running one.
Then change the filePath to the path of the file you want to use inside the jar file. I.E. if your file is in
someJar.jar\img\test.gif
. Set the filePath to "img\test.gif"
File executable = new File(BrowserViewControl.class.getProtectionDomain().getCodeSource().getLocation().toURI());
JarFile jar = new JarFile(executable);
InputStream fileInputStreamReader = jar.getInputStream(jar.getJarEntry(filePath));
byte[] bytes = new byte[fileInputStreamReader.available()];
int sizeOrig = fileInputStreamReader.available();
int size = fileInputStreamReader.available();
int offset = 0;
while (size != 0){
fileInputStreamReader.read(bytes, offset, size);
offset = sizeOrig - fileInputStreamReader.available();
size = fileInputStreamReader.available();
}
Outside of your technique, why not use the standard Java JarFile class to get the references you want? From there most of your problems should go away.
If you use resources extensively, you might consider using
Commons VFS.
Also supports:
* Local Files
* FTP, SFTP
* HTTP and HTTPS
* Temporary Files "normal FS backed)
* Zip, Jar and Tar (uncompressed, tgz or tbz2)
* gzip and bzip2
* resources
* ram - "ramdrive"
* mime
There's also JBoss VFS - but it's not much documented.
I have 2 CSV files that I use to read data. The java program is exported as a runnable jar file. When you export it, you will figure out it doesn't export your resources with it.
I added a folder under project called data in eclipse. In that folder i stored my csv files.
When I need to reference those files I do it like this...
private static final String ZIP_FILE_LOCATION_PRIMARY = "free-zipcode-database-Primary.csv";
private static final String ZIP_FILE_LOCATION = "free-zipcode-database.csv";
private static String getFileLocation(){
String loc = new File("").getAbsolutePath() + File.separatorChar +
"data" + File.separatorChar;
if (usePrimaryZipCodesOnly()){
loc = loc.concat(ZIP_FILE_LOCATION_PRIMARY);
} else {
loc = loc.concat(ZIP_FILE_LOCATION);
}
return loc;
}
Then when you put the jar in a location so it can be ran via commandline, make sure that you add the data folder with the resources into the same location as the jar file.

Categories

Resources