I'm currently using commons-net library for FTP client in my app. I have to download from remote server some files, by some criteria based on the file name. This is a very simplified and reduced version of my actual code (because I do some checks and catch all possible exceptions), but the essence is there:
//ftp is FTPClient object
//...
files = ftp.listFiles();
for (FTPFile ftpFile : files) {
String name = ftpFile.getName();
if(conformsCriteria(name)) {
String path = outDirectory + File.separatorChar + name;
os = new FileOutputStream(path);
ftp.retrieveFile(name, os);
}
}
Now, what I noticed is that when I run this code, wait a few seconds, and then plug out network cable, output directory contains some "empty" files plus the files actually downloaded, which leads me to believe that this method is working somewhat asynchronously... But then again, some files are downloaded (size > 0KB), and there are these empty files (size = 0KB), which leads me to believe that it is still serialized download... Also, function retrieveFile() returns, I quote documentation:
True if successfully completetd, false if not
What I need is serialized download, because I need to log every unsuccessful download.
What I saw browsing through the commons-net source is that, if I'm not wrong, new Socket is created for each retrieveFile() call.
I'm pretty confused about this, so If someone could explain what is actually happening, and offer solution with this library, or recommend some other FTP java library that supports blocking download per file, that would be nice.
Thanks.
You could just use the java.net.URLConnection class that has been present forever. It should know how to handle FTP URLs just fine. Here is a simple example that should give the blocking behavior that you are looking for.
The caveat is that you have to manage the input/output streams yourself, but this should be pretty simple.
Ok, to briefly answer this in order not to confuse people who might see this question.
Yes, commons-net for FTP is working as I thought it would, that is, retrieveFile() method blocks until it's finished with the download.
It was (of course) my own "mistake" in the code that let me think otherwise.
Related
I know there are no standard Java mechanisms for testing if a file is open or not.
Can anyone please tell me whether there is any libraries through which we can achieve this task, or some other ways,
This is my scenario, say I've open a docx file manually, when I try to upload that file itself its not getting uploaded since the file is opened, so if we tries to upload a file which is open outside it needs to display a message telling "The file is opened".
Using Java
If your client is a Java client (which you don't say in your question), you could try the following code from this answer:
Using the Apache Commons IO library...
boolean isFileUnlocked = false;
try {
org.apache.commons.io.FileUtils.touch(yourFile);
isFileUnlocked = true;
} catch (IOException e) {
isFileUnlocked = false;
}
if(isFileUnlocked){
// Do stuff you need to do with a file that is NOT locked.
} else {
// Do stuff you need to do with a file that IS locked
}
BTW: This is the very first answer when you search for "java file is open" on Google.
Using JavaScript
However, I assume that you don't have a Java client, only a Java server and you want to upload a file via a webpage (so you have a HTML + JS client). I assume this, because you don't tell us.
JavaScript doesn't allow you to write to the disk since this would be a security issue (I think Chrome supports some kind of the JavaScript File API which allows you to write to the disk, but none of the other browsers). So you cannot test this via JavaScript.
In general
The general way to test if a file is open, would be to test if there is some lock on the file. But this would not guarantee that the file is not opened without a lock, but it should be sufficient.
I'm writing a web service using Axis and Apache Tomcat 7.
The service uses a third party library to do some conversions to a file and ends up creating a folder which contains more files (subfolders and regular files). When the conversion is completed the service creates a zip archive and returns it.
When it receives a new request, first of all it removes the files created during the last request and the it starts handling the request.
The service itself works fine, at least the first request is satisfied.
The problem is that when a second request is received, the service cannot delete all the files generated during the last request.
I'm using Windows XP and with Process Explorer i see that Tomcat is keeping some files (bt not all of them) open and that's why i can't delete it.
Is that possibile that the library i'm using keeps the files open even when the service operation ends?
In the code that i use to create the zip archive it seems that i close all the streams that i open. Btw even if i forgive to close them, can they stay still open after the service operation returns his results to the client?
And if so, why the process Tomcat keeps open only some of the files?
It seems that after some time some file are "released", but other file are always kept open...
I hope someone can give me some advice on how to handle this situation :)
Repost of my comment which seems to be useful.
If a file handler is not released, it will never be released until the servlet container is shutdown. Some implementations may also delay the releasing of file handlers to when the object is garbage collected. Nothing you can do except to make sure that you close all handlers. If it's the third party libary, then you have to report a bug or fix it yourself.
My best practice to prevent this sort of problem is to make sure that the file handler is closed in the same method it is opened. If it is not opened in that method, never close it.
public void method() {
//open file handler
//do something
//close file handler. make sure it is closed even if there is an exception.
}
And never make file handler a field.
public class A {
private FileInputStream fin = null; // never do this. you will have hard time keeping track of when to release it.
}
Well, as I was wondering, the library I'm using doesn't close all the file streams..
I found a workaround that can be used in similar situations. I post it, maybe someone can find it useful or advise me a better way to solve the problem :)
Lucky the jar library can be executed from command line with the java command, so i just executed it with Runtime's exec method in this way:
try {
Process p = Runtime.getRuntime().exec("java -jar library.jar parameters");
p.waitFor();
} catch (InterruptedException | IOException e) {
e.printStackTrace();
}
In this way a new process is created by the JVM, and, when it dies, all the pending handlers are relased.
If you can't execute the library with the java command, such library can be simply wrapped in a custom jar made by a simple class who uses it and takes the needed parameters. The wrapper jar can be then executed with exec().
I have written and tested Java code that uses Jsch to transfer files. The transfer from the source computer, HERE, to a destination computer, THERE, works flawlessly. Using the same unmodified code, the transfer from HERE to another computer, PROBLEM, works about half the time. The problem being that the code will, on random file, hang on the write or close, and there are no exceptions thrown, even after a extremely long timeout. (After the next use of the channel after the extremely long timeout causes an exception.) Using the linux command "scp" (openssh-clients) works flawlessly copying the same set of files, both from HERE to THERE and HERE to PROBLEM.
I assume there is an imperfection in the transmission or reception of the files that openssh::scp has been designed to detect and work around. Any suggestions as to how to
proceed.
Detail(s):
methods used for write/close
OutputStream fos = put(String rp3);
fos.close();
Is there a means similar to unix alarm/SIGALRM to interrupt the write/close so that
the attempt can be retried?
Is there a session setConfig parameter that instructs Jsch to be more fault tolerant? Where are these documented?
Should I switch to another Java implementation of scp?
I'm trying to build a simple HTTP Server using Java, using
java.net.ServerSocket = new ServerSocket(this.port, 0, this.ip);
java.net.Socket connection = null;
connection = server.accept();
java.io.OutputStream out = new BufferedOutputStream(connection.getOutputStream());
when connected using web browser, i'm simply write the output (HTTP headers + html code) from a string
String headers = "http headers";
String response = "this is the response";
out.write(headers.getBytes());
out.write(response.getBytes());
out.flush();
connection.close();
and the browser display it correctly.
And now my problem is, i want to construct a full webpage (html, javascript, css, images) and put those files into the Java package (JAR) file, and of course, those files are designed not-to-be modified after the JAR is ready to use. And here's the questions:
how to achieve this? storing the files inside the JAR and then output them when a connection is made.
how output images file (non-text) just like output-ing String by out.write() ?
Thanks, any sample or code is appreciated.
Is implementing an HTTP server your primary problem or just a way to achieve some other goal? If the latter, consider embedding Tomcat or Jetty, much simpler and with standard servlet API.
Back to your question: JAR is just a ZIP file, you can put anything there, including files, images, movies, etc. If you place a file inside a JAR file you can load it easily with:
InputStream is = getClass().getResourceAsStream("/dir/file.png");
See these questions for details how getResourceAsStream() works:
Junit + getResourceAsStream Returning Null
getResourceAsStream() vs FileInputStream
how do you make getResourceAsStream work while debugging Java in Eclipse?
Different ways of loading a file as an InputStream
getResourceAsStream() is always returning null
About your second question: when you have an InputStream instance you can just read it byte-by-byte and copy to target out OutputStream. Of course there are better, safer and faster ways, but that's beyond the scope of this question. Just have a look at IOUtils.copy():
IOUtils.copy(is, out);
And the last hint concerning your code: if you are sending Strings , consider OutputStreamWriter and PrintWriter which have easier API.
To work with JAR files use JarOutputStream or ZipOutputStream. To output binary data just do not wrap your output stream with Writer. OuputStream knows to write bytes using method write(byte) and write(byte[]).
The only question here is "Why are you developing HTTP server yourself?"
As long as it is not a housework I would not try to reinvent the wheel and develop another web server. There a small embedded Java web-servers available which can be used for that purpose.
I have for example use the Tiny Java Web Server and Servlet Container several times.
If you have integrated it into your application you can implement a new javax.servlet.http.HttpServlet that reads the files from the resources of your JAR file. The content can be loaded as Tomasz Nurkiewicz already pointed out getClass().getRourceAsStream(...).
I want to achieve that when downloading, it can be paused, stopped by user. How can I achieve this with DownloadManager?
You can remove a download from the queue. Then you can start it up again, passing the Range HTTP Header when re-adding it to the queue (like TryTryAgain mentioned). I don't believe this currently can be done more easily.
It may be that "pausing" isn't exactly necessary for you? As DownloadManager actually doesn't need to pause to be resumed.
Instead you may be getting tripped up with resuming? See if this helps:
how to resume an interrupted download Where the solution is implementing a Range request header...
// Open connection to URL.
HttpURLConnection connection =
(HttpURLConnection) url.openConnection();
// Specify what portion of file to download.
connection.setRequestProperty("Range", "bytes=" + downloaded + "-");
// here "downloaded" is the data length already previously downloaded.
// Connect to server.
connection.connect();
Unless you need to actually allow "pausing" (because of switching file download priority, maintaining a list or something else more complicated) and not just provide resuming capability, that should help.
Do you mean to do this in Web Application ?
I am also thinking about the same thing. This might have been possible with flash or sliverlight etc., but in pure HTML?
On the server side we can use RandomAccessFileReader(Apache) or similar FileReader (in bytes) to access file and write to output stream. We can remember how much bytes was downloaded in previous session by the user too. But the problem is there is no standard way to save file to disk in the client's local file system. You can use ActiveX Object in IE to write some file provided it is a sequential (plain text) file and not binary file(like mp3, flv etc). But this ActiveX is not a solution. So you may need to design something like 'Downloader' for you website. Just like Youtube Downloader.