In my application I download several critical files from a server, and I want to write some code that handles the case where the a file download didn't complete for a reason or other ,to retry downloading it at next startup. The function that downloads a file at a time however throws only MalformedURLException and IOException , but if these exceptions are thrown that means that the download didn't even begin. How should I arrange things so I can treat the case where a download failed , even if it began ?
void download(String file) throws MalformedURLException ,IOException
{
BufferedInputStream getit = new BufferedInputStream(new URL(file).openStream());
FileOutputStream saveit = new FileOutputStream(DOWNLOAD_PATH+fileName+"."+ZIP_EXTENSION);
BufferedOutputStream bout = new BufferedOutputStream(saveit,1024);
byte data[] = new byte[1024];
int readed = getit.read(data,0,1024);
while(readed != -1)
{
bout.write(data,0,readed);
readed = getit.read(data,0,1024);
}
bout.close();
getit.close();
saveit.close();
}
You might be better using the Jakarta Commons HttpClient API.
However, for your custom function take a look at InterruptedIOException and bytesTransferred on https://docs.oracle.com/javase/1.5.0/docs/api/java/io/InterruptedIOException.html
public class InterruptedIOException
extends IOException
Signals that an I/O operation has been interrupted.
The field bytesTransferred indicates how many bytes were successfully transferred before the interruption occurred.
If your download is synchronous, you should be able to add an appropriate exception (or return an appropriate value) to indicate failure.
If your download is asynchronous, consider using the observer pattern. You can pass in an observer implementation as an extra parameter to your download method.
The observer in your case (for example) might look something like:
public interface FileDownloadObserver
{
public void downloadFailed(String file, Object error);
public void downloadSucceeded(String file);
}
Then the download method would look like:
void download(String file, FileDownloadObserver observer)
throws MalformedURLException, IOException
This is all assuming you can actually detect that the download failed. If not, you might have to provide some more information about how you're doing the download.
Related
I am writing an API service that fetches data from a stream, and outputs it in a file. I can't output it as a stream because I use Swagger (now OpenAPI) 2.0, which doesn't support output streams (Swagger 3.0 does, but i can't use it).
What would be the cleanest way to make a file, output it via the service, and then make sure it gets deleted?
I initially thought I might use a temp file and delete in finally clause. However, there is no guarantee that the file finished downloading on the client side before that clause is reached and file is deleted.
Am I right? Wrong? Is there a better way to do this?
I was talking about using a closeable in the comments. This is it.
Usage:
try (TempFile file = new TempFile("tempfile", ".txt")) {
// do stuff with file
} catch (IOException e) {
// error handling.
// file should be automatically deleted.
}
TempFile:
public class TempFile implements AutoCloseable {
private final File file;
public TempFile(String prefix, String suffix) {
this.file = File.createTempFile(prefix, suffix);
}
public File getFile() {
return this.file;
}
#Override
public void close() throws IOException {
this.file.delete();
}
}
A bit of background, I'm trying to create a URL Stream Handler so I can keep track of how many connections I have active on my webview in my javafx application. Essentially, I'm running an AngularJs app in the WebView, and I'd like to know when it's finished. I can't touch the web site code, so adding a js notifier is not on the table. So, no matter what I put together, the setup always errors with 'protocol doesn't support input.' I've tried to override 'getDoInput' with a method that only returns false, but I still get the error. Any ideas?
Here is something close to what I'm doing:
public class MyUrlStreamHandlerFactory implements URLStreamHandlerFactory {
public URLStreamHandler createURLStreamHandler(String protocol) {
if (protocol.equalsIgnoreCase("http") || protocol.equalsIgnoreCase("https")) {
return new URLStreamHandler() {
#Override
protected URLConnection openConnection(URL url) throws IOException {
return new HttpURLConnection(url) {
#Override
public void connect() throws IOException {
}
#Override
public void disconnect() {
}
#Override
public boolean usingProxy() {
return false;
}
#Override
public boolean getDoInput() {
return false;
}
};
}
};
}
return null;
}
}
I'm installing it with:
URL.setURLStreamHandlerFactory(new MyUrlStreamHandlerFactory());
I understand what you're trying to accomplish however, I think this is the wrong way to go about it.
From: Java Network Programming by Elliotte Rusty Harold
Only abstract URLConnection classes are present in the java.net package. The concrete subclasses are hidden inside the sun.net package hierarchy. It is rare to instantiate URLConnection objects directly in your source code; instead, the runtime environment creates these objects as needed, depending on the protocol in use. The class (which is unknown at compile time) is then instantiated using the forName() and newInstance() methods of the java.lang.Class class.
For example, the connect() method of sun.net.www.protocol.http.HttpURLConnection creates a sun.net.www.http.HttpClient object, which is responsible for connecting to the server.
So unless you want to write your own http protocol handler and an HttpClient, I would suggest exploring other avenues.
Other Things
The only method, that I could find, that throws an UnknownServiceException with the message being "protocol doesn't support input" is:
java.net.URLConnection#getInputStream
/**
* Returns an input stream that reads from this open connection.
*
* #return an input stream that reads from this open connection.
* #exception IOException if an I/O error occurs while
* creating the input stream.
* #exception UnknownServiceException if the protocol does not support
* input.
*/
public InputStream getInputStream() throws IOException {
throw new UnknownServiceException("protocol doesn't support input");
}
Overriding getDoInput
You should not override getDoInput to only return false. Instead you should use setDoInput(false). However, you don't want to set doInput to false. You always want to read something, for instance the response code.
How do I send an email using the Java Mail API that contains large attachments (~25 MB)? The attachments are files stored online with our cloud storage provider (Google Cloud Storage). The API for the service returns an InputStream object or a ReadableByteChannel object for each file.
I can't use ByteArrayDataSource to create a MimeBodyPart because it creates a copy of the entire file that resides in the memory, and we get a OutOfMemoryError.
If its a physical file, we can create a FileDataSource object and attach to the email. But can we do it with an InputStream object?
I can't increase the heap size because increasing it to 25MB seems like a very bad idea. If you have any other ideas too, please let me know. We're working on the Google App Engine platform.
Try the javax.activation.URLDataSource or the javax.activation.FileDataSource instead. Otherwise, you can create your own DataSource adapter class to directly return the given InputStream.
public class InputStreamDataSource implements javax.activation.DataSource {
private final InputStream in;
public InputStreamDataSource(final InputStream in) {
if (in == null) {
throw new NullPointerException();
}
this.in = in;
}
#Override
public InputStream getInputStream() throws IOException {
return in;
}
#Override
public OutputStream getOutputStream() throws IOException {
throw new IOException();
}
#Override
public String getContentType() {
return "application/octet-stream";
}
#Override
public String getName() {
return "some name";
}
}
This might be a time to copy the InputStream to a physical file and use the FileDataSource object to attach it. But better still would be not to send as an attachment, just send the link to the object in cloud storage. If you need to control access to the item in cloud storage, send a signed url.
I'm writing a play 2.0 java application that allows users to upload files. Those files are stored on a third-party service I access using a Java library, the method I use in this API has the following signature:
void store(InputStream stream, String path, String contentType)
I've managed to make uploads working using the following simple controller:
public static Result uploadFile(String path) {
MultipartFormData body = request().body().asMultipartFormData();
FilePart filePart = body.getFile("files[]");
InputStream is = new FileInputStream(filePart.getFile())
myApi.store(is,path,filePart.getContentType());
return ok();
}
My concern is that this solution is not efficient because by default the play framework stores all the data uploaded by the client in a temporary file on the server then calls my uploadFile() method in the controller.
In a traditional servlet application I would have written a servlet behaving this way:
myApi.store(request.getInputStream(), ...)
I have been searching everywhere and didn't find any solution. The closest example I found is Why makes calling error or done in a BodyParser's Iteratee the request hang in Play Framework 2.0? but I didn't found how to modify it to fit my needs.
Is there a way in play2 to achieve this behavior, i.e. having the data uploaded by the client to go "through" the web-application directly to another system ?
Thanks.
I've been able to stream data to my third-party API using the following Scala controller code:
def uploadFile() =
Action( parse.multipartFormData(myPartHandler) )
{
request => Ok("Done")
}
def myPartHandler: BodyParsers.parse.Multipart.PartHandler[MultipartFormData.FilePart[Result]] = {
parse.Multipart.handleFilePart {
case parse.Multipart.FileInfo(partName, filename, contentType) =>
//Still dirty: the path of the file is in the partName...
String path = partName;
//Set up the PipedOutputStream here, give the input stream to a worker thread
val pos:PipedOutputStream = new PipedOutputStream();
val pis:PipedInputStream = new PipedInputStream(pos);
val worker:UploadFileWorker = new UploadFileWorker(path,pis);
worker.contentType = contentType.get;
worker.start();
//Read content to the POS
Iteratee.fold[Array[Byte], PipedOutputStream](pos) { (os, data) =>
os.write(data)
os
}.mapDone { os =>
os.close()
Ok("upload done")
}
}
}
The UploadFileWorker is a really simple Java class that contains the call to the thrid-party API.
public class UploadFileWorker extends Thread {
String path;
PipedInputStream pis;
public String contentType = "";
public UploadFileWorker(String path, PipedInputStream pis) {
super();
this.path = path;
this.pis = pis;
}
public void run() {
try {
myApi.store(pis, path, contentType);
pis.close();
} catch (Exception ex) {
ex.printStackTrace();
try {pis.close();} catch (Exception ex2) {}
}
}
}
It's not completely perfect because I would have preferred to recover the path as a parameter to the Action but I haven't been able to do so. I thus have added a piece of javascript that updates the name of the input field (and thus the partName) and it does the trick.
I'm using JarFile and JarURLConnection to load files out of a jar file. I'm then taking the classes, and loading them via BCEL (ByteCode Engineering Library, apache library). I cant just directly use a class loader because im modifying some classes slightly with the BCEL. I need to load the classes by their bytes into my bcel loader. However, one of the classes I'm loading references a resource. This resource is inside of the jar, so I can get the file (When iterating over the entries in the JarFile, I ignore the regular files, and take the class files for loading later). But just having the file won't do me any good, as the class loads it as a resource. Is there any way I can take that resource from the jar (well I can take it and load it into a byte[], the next part is the issue) and dynamically add it as a resource for my program, so that the classes that I load wont be missing their resources?
Got a lot of stuff here, if anythings confusing, ask in comments, I might've said something wrong, or missed something altogether :) Thanks
I'll show a little of my class loader here (extends ClassLoader):
#Override
public URL getResource(String name) {
System.out.println("LOADING RESOURCE: " + name);
try {
return new URL(null, name, new Handler(files));
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
Now, it is printing out "LOADING RESOURCE: filename", but its then giving me a MalformedURLException (I have no protocol atm, just a file path, that's not a true valid path, but it's just an attempt to give it to my Handler class below).
class Handler extends URLStreamHandler {
#Override
protected URLConnection openConnection(URL u) throws IOException {
return new URLConnection(u) {
#Override
public void connect() throws IOException {
}
#Override
public InputStream getInputStream() throws IOException {
System.out.println("IS: " + url);
return /*method to get input steam*/;
}
};
}
}
The /*method to get input steam*/ is set in my real code, but that's not relevant here. So any further ideas with this?