Develop Reference

How do i find a filepath given only the name of the file?

I am currently working on transferring a file from a server to a client via tcp connection. The file exists somewhere within a sharedroot directory of the server. This is the sample code of my upload method for server upload to client.
public void upload(String filename, DataOutputStream out) throws IOException {
File fname = null;
if (filename.contains(sharedroot)) { //this is if the client provides a proper filepath with the filename
fname = new File(filename);
}else { //if client only provides a filename without path
fname = new File(filename);
//"..\\..\\"+ //i was working around with this, but somehow just making the file whether or not it contains the sharedroot seems to give me the "best" output so far...
}
System.out.println(fname.getCanonicalPath());
if (fname.isDirectory()) {
System.out.println("File is a directory");
String quit = "404 not found";
sendOut(quit, out);
return;
}
String path = fname.getAbsolutePath();
System.out.println(path);
if (fname.isFile()) {
String canonpath = fname.getCanonicalPath();
if (canonpath.contains(sharedroot)) {
try {
FileInputStream in = new FileInputStream(fname);
byte[] buffer = new byte[1024];
out.writeInt(fname.getName().length());
out.write(fname.getName().getBytes(), 0, fname.getName().length()); // writes file name only, not
// including the path
long size = fname.length();
out.writeLong(size);
while (size > 0) {
int len = in.read(buffer, 0, buffer.length);
out.write(buffer, 0, len);
size -= len;
}
} catch (Exception e) {
System.out.println("Error occurred in uploading file to client. Please try again");
}
}else {
System.out.println("File not in shared directory");
String quit = "404 not found";
sendOut(quit, out);
}
}else {
System.out.println("File not exists");
String quit = "404 not found";
sendOut(quit, out);
}
}
The output given by getCanonicalPath() and getAbsolutePath() as seen below is wrong because it is checking inside the directory of my eclipse and not the sharedroot directory. How can I get the filepath of the file so that i can compare it to my sharedroot and ensure it exists within the sharedroot? The sharedroot would be for example: D:\seant\2uniFiles\1. FINE2005 Year 3
D:\seant\eclipse-workspace\DCN3005\Lecture 1 Exercise.pdf
D:\seant\eclipse-workspace\DCN3005\Lecture 1 Exercise.pdf
File not exists

Your creation of File does not specify a dedicated directory. There are two constructors requiring a (root) directory and a file name – one as File itself, the other as String. I assume one of your paths is relative but your else-branch creates the file the same way as the full qualified path. You should pass the sharedRoot instead as first parameter and the fileName as second.
File fname = null;
// sharedRoot is more like a constant and startsWith
// avoids reading somewhere else that looks similar
if (filename.startsWith(sharedRoot)) {
fname = new File(filename);
} else {
fname = new File(sharedRoot, filename);
}
In all other cases relative paths are relative to the root directory of the VM process – and I mean process. If for example a user starts this in the user's HOME directory it'll be relative to this. If an operating system task starts the VM it'll be relative to the OS process' root – which might be a Unix cron job or a Windows scheduling thing.
Maybe you introduce a sort of configuration of sharedRoot so you don't need to recompile if this changes in the future.

Vaadin EasyUpload add-on sometimes could not open or find file specified

We are currently using the EasyUpload add-on, and we have specified the criteria for this component:
a) only CSV files are allowed, with a cap size of 1MB per file.
b) only one file can be submitted at a time.
We just did an uploading test on small-sized CSV files that are below 100Kb. Usually, the upload process completes successfully. Occasionally, the error of "Could not open file, The system cannot find the file specified" is displayed although the file is already inside the temp folder, and we found that this happens either when:
a) If the same file is uploaded again after making a small change and within a few seconds after the file has been uploaded successfully.
b) If there are two tabs of the web app, logged under different users were uploading their respective csv files and they also do the same thing of changing values in the csv before uploading them again.
We tried forcing the file upload through (as another testing method) and noticed after a while that the files sometimes get stuck in the queue although we have imposed a one file at a submission time rule. It was displayed in a message "There are too many files over the count limit". We also considered of putting a sleep / wait command of 3-5 seconds after the file submission.
MultiFileUpload multiFileUpload = new MultiFileUpload() {
#Override
protected void handleFile(File tmpFile, String fileName, String mimeType, long length) {
String[] header = {"EOD_NUM","OUTLET_NAME","POSM_NAME","EOD_DATE","TOTAL_SALES","GROSS_SALES",
"TRAN_COUNT","VOID_COUNT","SERVICE_CHARGE","DISCOUNT_AMT","VAT_TAX_AMT","SVC_TAX_AMT","ROUNDING_ADJ"};
uploadLogger.debug("File: " + tmpFile.getAbsolutePath());
uploadLogger.debug("FileName: " + fileName);
uploadLogger.debug("MimeType: " + mimeType);
uploadLogger.debug("File Length: " + length);
DateTimeFormatter dtf = DateTimeFormatter.ofPattern("ddMMyyyyHHmmss");
LocalDateTime now = LocalDateTime.now();
File f2 = null;
f2 = new File(busId+"_"+dtf.format(now)+".csv");
tmpFile.renameTo(f2);
try {
///var/lib/tomcat8/ in linux
///D:\\home\\site\\wwwroot\\ in Windows
uploadLogger.debug("f2 absolutepath: " + f2.getAbsolutePath());
uploadLogger.debug("f2 canonical path: " + f2.getCanonicalPath());
CloudBlockBlob blob = container.getBlockBlobReference(f2.getName());
if(f2.length() > 0){
blob.uploadFromFile(f2.getAbsolutePath());
Notification.show("File upload completed.",Notification.Type.TRAY_NOTIFICATION);
}
CSVReader reader = new CSVReader(new FileReader(f2.getAbsolutePath()), ',' , '"' , 0);
//read header name
//String[] myheader = reader.readNext();
//NOTE :: Store all row and column from csv info List of String Array
myEntries = reader.readAll();
if (myEntries != null && !myEntries.isEmpty()) {
boolean success = uploadDAO.insertUploaderEntry(myEntries,busId, userId,"");
uploadLogger.debug("SUCCESSS??? " + success);
if(success){
Notification successNotify = new Notification("Record has been created successfully.","Upload Successful!");
successNotify.setDelayMsec(3000);
successNotify.setStyleName(ValoTheme.NOTIFICATION_SUCCESS);
successNotify.setPosition(Position.MIDDLE_CENTER);
successNotify.show(Page.getCurrent());
}else {
Notification.show("Error in submitting uploaded record.","Upload failed!"
, Notification.Type.ERROR_MESSAGE).setDelayMsec(3000);
}
Thread.sleep(3000); //added to see if the delay solves the problem or not.
}
} catch (URISyntaxException | StorageException | IOException ex) {
new Notification("Could not open file",ex.getMessage(),Notification.Type.ERROR_MESSAGE).show(Page.getCurrent());
uploadLogger.debug(ex);
} catch (InterruptedException ix) {
uploadLogger.debug("Interrupted Exception found: " + ix.getMessage());
}
}
#Override
protected boolean supportsFileDrops() {
return false;
}
};
multiFileUpload.setMaxFileCount(1);
multiFileUpload.setUploadButtonCaption("Upload CSV file here");
multiFileUpload.setMaxFileSize(fileSizeLimit); // 2MB
multiFileUpload.setAcceptFilter(".csv");
We are unsure whether if this problem is a known limitation of the component or not.
Some of the questions we have discovered along the way are:
a) Is there a better way or to control on the file uploading and to avoid the open file / file not found error?
b) Are the values in the setAcceptedFilter method the mime/type values or something else. We noticed for images, it's "images/*" but for csv, we had to put in as ".csv"

Answering to your second question. The acceptFilter is directly passed to upload inputs "accept" attribute, so both .csv and text/csv should do fine. See https://www.w3schools.com/tags/att_input_accept.asp for more instructions.

Intelligently serving jar files from a web server

I am writing a simple (generic) wrapper Java class that will execute on various computers separate from a deployed web server. I want to download the latest version of a jar file that is the application from that associated Web Server (currently Jetty 8).
I have code like this:
// Get the jar URL which contains the application
URL jarFileURL = new URL("jar:http://localhost:8081/myapplication.jar!/");
JarURLConnection jcl = (JarURLConnection) jarFileURL.openConnection();
Attributes attr = jcl.getMainAttributes();
String mainClass = (attr != null)
? attr.getValue(Attributes.Name.MAIN_CLASS)
: null;
if (mainClass != null) // launch the program
This works well, except that myapplication.jar is a large jar file (a OneJar jarfile, so a lot is in there). I would like this to be as efficient as possible. The jar file isn't going to change very often.
Can the jar file be saved to disk (I see how to get a JarFile object, but not to save it)?
More importantly, but related to #1, can the jar file be cached somehow?
2.1 can I (easily) request the MD5 of the jar file on the web server and only download it when that has changed?
2.2 If not is there another caching mechanism, maybe request only the Manifest? Version/Build info could be stored there.
If anyone done something similar could you sketch out in as much detail what to do?
UPDATES PER INITIAL RESPONSES
The suggestion is to use an If-Modified-Since header in the request and the openStream method on the URL to get the jar file to save.
Based on this feedback, I have added one critical piece of info and some more focused questions.
The java program I am describing above runs the program downloaded from the jar file referenced. This program will run from around 30 seconds to maybe 5 minutes or so. Then it is done and exits. Some user may run this program multiple times per day (say even up to 100 times), others may run it as infrequently as once every other week. It should still be smart enough to know if it has the most current version of the jar file.
More Focused Questions:
Will the If-Modified-Since header still work in this usage? If so, will I need completely different code to add that? That is, can you show me how to modify the code presented to include that? Same question with regard to saving the jar file - ultimately I am really surprised (frustrated!) that I can get a JarFile object, but have no way to persist it - will I even need the JarURLConnection class?
Bounty Question
I didn't initially realize the precise question I was trying to ask. It is this:
How can I save a jar file from a web server locally in a command-line program that exits and ONLY update that jar file when it has been changed on the server?
Any answer that, via code examples, shows how that may be done will be awarded the bounty.

Yes, the file can be saved to the disk, you can get the input stream using the method openStream() in URL class.
As per the comment mentioned by #fge there is a way to detect whether the file is modified.
Sample Code:
private void launch() throws IOException {
// Get the jar URL which contains the application
String jarName = "myapplication.jar";
String strUrl = "jar:http://localhost:8081/" + jarName + "!/";
Path cacheDir = Paths.get("cache");
Files.createDirectories(cacheDir);
Path fetchUrl = fetchUrl(cacheDir, jarName, strUrl);
JarURLConnection jcl = (JarURLConnection) fetchUrl.toUri().toURL().openConnection();
Attributes attr = jcl.getMainAttributes();
String mainClass = (attr != null) ? attr.getValue(Attributes.Name.MAIN_CLASS) : null;
if (mainClass != null) {
// launch the program
}
}
private Path fetchUrl(Path cacheDir, String title, String strUrl) throws IOException {
Path cacheFile = cacheDir.resolve(title);
Path cacheFileDate = cacheDir.resolve(title + "_date");
URL url = new URL(strUrl);
URLConnection connection = url.openConnection();
if (Files.exists(cacheFile) && Files.exists(cacheFileDate)) {
String dateValue = Files.readAllLines(cacheFileDate).get(0);
connection.addRequestProperty("If-Modified-Since", dateValue);
String httpStatus = connection.getHeaderField(0);
if (httpStatus.indexOf(" 304 ") == -1) { // assuming that we get status 200 here instead
writeFiles(connection, cacheFile, cacheFileDate);
} else { // else not modified, so do not do anything, we return the cache file
System.out.println("Using cached file");
}
} else {
writeFiles(connection, cacheFile, cacheFileDate);
}
return cacheFile;
}
private void writeFiles(URLConnection connection, Path cacheFile, Path cacheFileDate) throws IOException {
System.out.println("Creating cache entry");
try (InputStream inputStream = connection.getInputStream()) {
Files.copy(inputStream, cacheFile, StandardCopyOption.REPLACE_EXISTING);
}
String lastModified = connection.getHeaderField("Last-Modified");
Files.write(cacheFileDate, lastModified.getBytes());
System.out.println(connection.getHeaderFields());
}

How can I save a jar file from a web server locally in a command-line program that exits and ONLY update that jar file when it has been changed on the server?
With JWS. It has an API so you can control it from your existing code. It already has versioning and caching, and comes with a JAR-serving servlet.

I have assumed that a .md5 file will be available both locally and at the web server. Same logic will apply if you wanted this to be a version control file.
The urls given in the following code need to updated according to your web server location and app context. Here is how your command line code would go
public class Main {
public static void main(String[] args) {
String jarPath = "/Users/nrj/Downloads/local/";
String jarfile = "apache-storm-0.9.3.tar.gz";
String md5File = jarfile + ".md5";
try {
// Update the URL to your real server location and application
// context
URL url = new URL(
"http://localhost:8090/JarServer/myjar?hash=md5&file="
+ URLEncoder.encode(jarfile, "UTF-8"));
BufferedReader in = new BufferedReader(new InputStreamReader(
url.openStream()));
// get the md5 value from server
String servermd5 = in.readLine();
in.close();
// Read the local md5 file
in = new BufferedReader(new FileReader(jarPath + md5File));
String localmd5 = in.readLine();
in.close();
// compare
if (null != servermd5 && null != localmd5
&& localmd5.trim().equals(servermd5.trim())) {
// TODO - Execute the existing jar
} else {
// Rename the old jar
if (!(new File(jarPath + jarfile).renameTo((new File(jarPath + jarfile
+ String.valueOf(System.currentTimeMillis())))))) {
System.err
.println("Unable to rename old jar file.. please check write access");
}
// Download the new jar
System.out
.println("New jar file found...downloading from server");
url = new URL(
"http://localhost:8090/JarServer/myjar?download=1&file="
+ URLEncoder.encode(jarfile, "UTF-8"));
// Code to download
byte[] buf;
int byteRead = 0;
BufferedOutputStream outStream = new BufferedOutputStream(
new FileOutputStream(jarPath + jarfile));
InputStream is = url.openConnection().getInputStream();
buf = new byte[10240];
while ((byteRead = is.read(buf)) != -1) {
outStream.write(buf, 0, byteRead);
}
outStream.close();
System.out.println("Downloaded Successfully.");
// Now update the md5 file with the new md5
BufferedWriter bw = new BufferedWriter(new FileWriter(md5File));
bw.write(servermd5);
bw.close();
// TODO - Execute the jar, its saved in the same path
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
And just in case you had control over the servlet code as well, this is how the servlet code goes:-
#WebServlet(name = "jarervlet", urlPatterns = { "/myjar" })
public class JarServlet extends HttpServlet {
private static final long serialVersionUID = 1L;
// Remember to have a '/' at the end, otherwise code will fail
private static final String PATH_TO_FILES = "/Users/nrj/Downloads/";
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
String fileName = req.getParameter("file");
if (null != fileName) {
fileName = URLDecoder.decode(fileName, "UTF-8");
}
String hash = req.getParameter("hash");
if (null != hash && hash.equalsIgnoreCase("md5")) {
resp.getWriter().write(readMd5Hash(fileName));
return;
}
String download = req.getParameter("download");
if (null != download) {
InputStream fis = new FileInputStream(PATH_TO_FILES + fileName);
String mimeType = getServletContext().getMimeType(
PATH_TO_FILES + fileName);
resp.setContentType(mimeType != null ? mimeType
: "application/octet-stream");
resp.setContentLength((int) new File(PATH_TO_FILES + fileName)
.length());
resp.setHeader("Content-Disposition", "attachment; filename=\""
+ fileName + "\"");
ServletOutputStream os = resp.getOutputStream();
byte[] bufferData = new byte[10240];
int read = 0;
while ((read = fis.read(bufferData)) != -1) {
os.write(bufferData, 0, read);
}
os.close();
fis.close();
// Download finished
}
}
private String readMd5Hash(String fileName) {
// We are assuming there is a .md5 file present for each file
// so we read the hash file to return hash
try (BufferedReader br = new BufferedReader(new FileReader(
PATH_TO_FILES + fileName + ".md5"))) {
return br.readLine();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
}

I can share experience of solving the same problem in our team. We have several desktop product written in java which are updated regularly.
Couple years ago we had separate update server for every product and following process of update: Client application has an updater wrapper that starts before main logic, and stored in a udpater.jar. Before start, application send request to update server with MD5-hash of application.jar file. Server compares received hash with the one that it has, and send new jar file to updater if hashes are different.
But after many cases, where we confused which build is now in production, and update-server failures we switched to continuous integration practice with TeamCity on top of it.
Every commit done by developer is now tracked by build server. After compilation and test passing build server assigns build number to application and shares app distribution in local network.
Update server now is a simple web server with special structure of static files:
$WEB_SERVER_HOME/
application-builds/
987/
988/
989/
libs/
app.jar
...
changes.txt <- files, that changed from last build
lastversion.txt <- last build number
Updater on client side requests lastversion.txt via HttpClient, retrieves last build number and compares it with client build number stored in manifest.mf.
If update is required, updater harvests all changes made since last update iterating over application-builds/$BUILD_NUM/changes.txt files. After that, updater downloads harvested list of files. There could be jar-files, config files, additional resources etc.
This scheme is seems complex for client updater, but in practice it is very clear and robust.
There is also a bash script that composes structure of files on updater server. Script request TeamCity every minute to get new builds and calculates diff between builds. We also upgrading now this solution to integrate with project management system (Redmine, Youtrack or Jira). The aim is to able product manager to mark build that are approved to be updated.
UPDATE.
I've moved our updater to github, check here: github.com/ancalled/simple-updater
Project contains updater-client on Java, server-side bash scripts (retrieves updates from build-server) and sample application to test updates on it.

Download entire FTP directory in Java (Apache Net Commons)

I am trying to recursively iterate through the entire root directory that I arrive at after login to the FTP server.
I am able to connect, all I really want to do from there is recurse through the entire structure and and download each file and folder and have it in the same structure as it is on the FTP. What I have so far is a working download method, it goes to the server and gets my entire structure of files, which is brilliant, except it fails on the first attempt, then works the second time around. The error I get is as follows:
java.io.FileNotFoundException: output-directory\test\testFile.png
(The system cannot find the path specified)
I managed to do upload functionality of a directory that I have locally, but can't quite get downloading to work, after numerous attempts I really need some help.
public static void download(String filename, String base)
{
File basedir = new File(base);
basedir.mkdirs();
try
{
FTPFile[] ftpFiles = ftpClient.listFiles();
for (FTPFile file : ftpFiles)
{
if (!file.getName().equals(".") && !file.getName().equals("..")) {
// If Dealing with a directory, change to it and call the function again
if (file.isDirectory())
{
// Change working Directory to this directory.
ftpClient.changeWorkingDirectory(file.getName());
// Recursive call to this method.
download(ftpClient.printWorkingDirectory(), base);
// Create the directory locally - in the right place
File newDir = new File (base + "/" + ftpClient.printWorkingDirectory());
newDir.mkdirs();
// Come back out to the parent level.
ftpClient.changeToParentDirectory();
}
else
{
ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
String remoteFile1 = ftpClient.printWorkingDirectory() + "/" + file.getName();
File downloadFile1 = new File(base + "/" + ftpClient.printWorkingDirectory() + "/" + file.getName());
OutputStream outputStream1 = new BufferedOutputStream(new FileOutputStream(downloadFile1));
boolean success = ftpClient.retrieveFile(remoteFile1, outputStream1);
outputStream1.close();
}
}
}
}
catch(IOException ex)
{
System.out.println(ex);
}
}

Your problem (well, your current problem after we got rid of the . and .. and you got past the binary issue) is that you are doing the recursion step before calling newDir.mkdirs().
So suppose you have a tree like
.
..
someDir
.
..
someFile.txt
someOtherDir
.
..
someOtherFile.png
What you do is skip the dot files, see that someDir is a directory, then immediately go inside it, skip its dot files, and see someFile.txt, and process it. You have not created someDir locally as yet, so you get an exception.
Your exception handler does not stop execution, so control goes back to the upper level of the recursion. At this point it creates the directory.
So next time you run your program, the local someDir directory is already created from the previous run, and you see no problem.
Basically, you should change your code to:
if (file.isDirectory())
{
// Change working Directory to this directory.
ftpClient.changeWorkingDirectory(file.getName());
// Create the directory locally - in the right place
File newDir = new File (base + "/" + ftpClient.printWorkingDirectory());
newDir.mkdirs();
// Recursive call to this method.
download(ftpClient.printWorkingDirectory(), base);
// Come back out to the parent level.
ftpClient.changeToParentDirectory();
}

A complete standalone code to download all files recursively from an FTP folder:
private static void downloadFolder(
FTPClient ftpClient, String remotePath, String localPath) throws IOException
{
System.out.println("Downloading folder " + remotePath + " to " + localPath);
FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);
for (FTPFile remoteFile : remoteFiles)
{
if (!remoteFile.getName().equals(".") && !remoteFile.getName().equals(".."))
{
String remoteFilePath = remotePath + "/" + remoteFile.getName();
String localFilePath = localPath + "/" + remoteFile.getName();
if (remoteFile.isDirectory())
{
new File(localFilePath).mkdirs();
downloadFolder(ftpClient, remoteFilePath, localFilePath);
}
else
{
System.out.println("Downloading file " + remoteFilePath + " to " +
localFilePath);
OutputStream outputStream =
new BufferedOutputStream(new FileOutputStream(localFilePath));
if (!ftpClient.retrieveFile(remoteFilePath, outputStream))
{
System.out.println("Failed to download file " + remoteFilePath);
}
outputStream.close();
}
}
}
}

Zip4j missing the last file added

I'm trying to zip a bunch of files using the Zip4j library. I pass a list of the file paths of the files I want to compress and I add them one by one into the zip file. For some reason, the last file does not get added. I checked the indexes of the loop and I'm pretty sure they're correct. I am not getting any exceptions or error messages. Here's my code:
// get the path; paths refers to the list of files to compress
String uuidString = UUID.randomUUID().toString();
String path = "H:/public/ZipFiles/" + uuidString + ".zip";
try {
// create the new zip file
ZipFile zipFile = new ZipFile(path);
File fileToAdd;
String message = "";
ZipParameters parameters = new ZipParameters();
// set compression method to store compression
parameters.setCompressionMethod(Zip4jConstants.COMP_DEFLATE);
// Set the compression level
parameters.setCompressionLevel(Zip4jConstants.DEFLATE_LEVEL_NORMAL);
// add each file to the zipFile
for(int i = 0; i < paths.size(); i++)
{
fileToAdd = new File(paths.get(i));
if(fileToAdd.exists())
{
System.out.println("writing file at " + paths.get(i) + " to the zip file");
zipFile.addFile(fileToAdd, parameters);
}
else
{
message += "File with at path " + paths.get(i) + " was not found.\n";
}
}
} catch (ZipException e) {
e.printStackTrace();
}
All the file paths get printed when they are added. Any ideas?

You're not closing the ZipFile.

I think there is a problem with the jar file from their own website at http://www.lingala.net/zip4j/download.php
But when I downloaded it from the maven repository at https://mvnrepository.com/artifact/net.lingala.zip4j/zip4j/1.3.2 , it is working perfectly.