Develop Reference

Download file hosted on a website

I am trying to download a file that I have hosted on mega. The code is as follows:
AsyncTask.execute(new Runnable() {
#Override
public void run() {
LoggerDebug.d("Prakhar", "Inside run");
try {
URL downloadUrl = new URL(url);
HttpURLConnection httpURLConnection = (HttpURLConnection) downloadUrl.openConnection();
httpURLConnection.setRequestMethod("GET");
httpURLConnection.setDoOutput(true);
httpURLConnection.connect();
File SDCardRoot = Environment.getExternalStorageDirectory();
File downloadFile = new File(SDCardRoot, "downloaded.zip");
FileOutputStream fileOutputStream = new FileOutputStream(downloadFile);
InputStream inputStream = httpURLConnection.getInputStream();
int totalSize = httpURLConnection.getContentLength();
int downloadedSize = 0;
byte[] buffer = new byte[1024];
int bufferLength = 0;
LoggerDebug.d("Prakhar", String.valueOf(totalSize));
while ((bufferLength = inputStream.read(buffer)) > 0 ) {
LoggerDebug.d("Prakhar", String.valueOf(downloadedSize));
fileOutputStream.write(buffer, 0, bufferLength);
downloadedSize += bufferLength;
}
} catch (Exception e) {
e.printStackTrace();
}
}
});
}
Now after I run this code sample I get the following:
Total size is : -1
Downloaded size : 1024
On the mobile I can see the downloaded.zip file, its size is 1.97KB and the original size of the upload is 87.1KB
Where am I going wrong?

So I don't know if you solved your problem yet, but I can see a couple potential issues.
setDoOutput(true) indicates you're going to be uploading according to the documentation here. Specifically, it says:
To upload data to a web server, configure the connection for output using setDoOutput(true).
You're not uploading from what I'm seeing, so this isn't needed, and it's unclear how the server would respond since it appears you're requesting the ability to upload.
A return of -1 indicates an EOF. You're not accessing the file you're intending to, likely due to #3.
You probably can't just download from their site willy-nilly. You need to authenticate somehow. This could be an API key, session id, yadda yadda. They have their own API, and according to this document there is logging in via e-mail in their API. The page doesn't render correctly in GitHub, but you'll need to figure out how to log in. They have an Android app example, but it looks like the API is all native code.

Have you tried changing:
while ((bufferLength = inputStream.read(buffer)) > 0 ) {
to
while ((bufferLength = inputStream.read(buffer)) != null) {

Download files with Android Webview

I am making an Android app that uses a WebView to access to a webpage. To handle downloads I am using AsyncTask in method onDownloadStart of WebView's DownloadListener. However files downloaded are blank (although the filename and extension are correct). My Java code is this:
protected String doInBackground(String... url) {
try {
URL url = new URL(url[0]);
//Creating directory if not exists
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setDoOutput(true);
connection.connect();
//Obtaining filename
File outputFile = new File(directory, filename);
InputStream input = new BufferedInputStream(connection.getInputStream());
OutputStream output = new FileOutputStream(outputFile);
byte data[] = new byte[1024];
int count = 0;
Log.e(null, "input.read(data) = "+input.read(data), null);
while ((count = input.read(data)) != -1) {
output.write(data, 0, count);
}
connection.disconnect();
output.flush();
output.close();
input.close();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
log.e line gives -1 value for input.read(data).
PHP code of download page is this (works in all platforms). Files are stored in non-public directories of my HTML server.
<?php
$guid = $_GET['id'];
$file = get_file($guid);
if (isset($file['path'])) {
$mime = $file['MIMEType'];
if (!$mime) {
$mime = "application/octet-stream";
}
header("Pragma: public");
header("Content-type: $mime");
header("Content-Disposition: attachment; filename=\"{$file['filename']}\"");
header('Content-Transfer-Encoding: binary');
ob_clean();
flush();
readfile($file['path']);
exit();
}
?>
I've noticed that if I write some text after "?>" of PHP file, this text is written in the file downloaded.

In your code, you are using ob_clean(), which will just erase the output buffer. Your subsequent call to flush() therefore doesn't return anything, because the output buffer has been flushed beforehand.
Instead of ob_clean() and flush(), use ob_end_flush(). This will stop output buffering and it will send all the output it withheld.
ob_end_flush — Flush (send) the output buffer and turn off output buffering
If you want to stop output buffering without outputting whatever is saved, you can use ob_end_clean(). Anything after this command will be output again, but anything between ob_start() and ob_end_clean() will be "swallowed."
ob_end_clean — Clean (erase) the output buffer and turn off output buffering
What are the benefits of output buffering in the first place? If you are doing ob_start() and then using flush() on everything you might as well output everything directly.

How to detect that whole image was downloaded in Android

I download images into my android application with this code:
private void download(URL url, File file) throws IOException {
Log.d(TAG, "download(): downloading file: " + url);
URLConnection urlConnection = url.openConnection();
InputStream inputStream = urlConnection.getInputStream();
BufferedInputStream bufferStream;
OutputStream outputStream = null;
try {
bufferStream = new BufferedInputStream(inputStream, 512);
outputStream = new FileOutputStream(file);
byte[] buffer = new byte[512];
int current;
while ((current = bufferStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, current);
}
} finally {
if (outputStream != null) {
outputStream.close();
}
if (inputStream != null) {
inputStream.close();
}
}
}
This code works fine, but some users and testers complained about incomplete photos. I suspect small network lags which interrupt connection.
So I would like to detect if whole image was downloaded and saved file is complete image. Is there any way how to detect file size from BufferedInputStream or is there another way how detect download completion?

I suggest using Google Volley which provides a super simple interface for networking in general, and image loading specifically. It takes care of threading and batching for you.
It's what Google use on the Google Play app.
It will solve your problem by providing listeners that notify you when the job is complete.

If you are downloading an ordinary file over HTTP, the method getContentLength() of URLConnection gives you the length that the file should have in the end.
You can compare the returned value of this method to the file length/length of downloaded data. If it's equal, then the file is complete:
int contentLength = urlConnection.getContentLength();
if (contentLength != -1) {
if (contentLength == file.length()) {
System.out.println("file is complete");
} else {
System.out.println("file is incomplete");
}
} else {
System.out.println("unknown if file is complete");
}

Try something like this. I think it could help you.

Read first bytes of a file

I need a very simple function that allows me to read the first 1k bytes of a file through FTP. I want to use it in MATLAB to read the first lines and, according to some parameters, to download only files I really need eventually. I found some examples online that unfortunately do not work. Here I'm proposing the sample code where I'm trying to download one single file (I'm using the Apache libraries).
FTPClient client = new FTPClient();
FileOutputStream fos = null;
try {
client.connect("data.site.org");
// filename to be downloaded.
String filename = "filename.Z";
fos = new FileOutputStream(filename);
// Download file from FTP server
InputStream stream = client.retrieveFileStream("/pub/obs/2008/021/ab120210.08d.Z");
byte[] b = new byte[1024];
stream.read(b);
fos.write(b);
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fos != null) {
fos.close();
}
client.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}
the error is in stream which is returned empty. I know I'm passing the folder name in a wrong way, but I cannot understand how I have to do. I've tried in many way.
I've also tried with the URL's Java classes as:
URL url;
url = new URL("ftp://data.site.org/pub/obs/2008/021/ab120210.08d.Z");
URLConnection con = url.openConnection();
BufferedInputStream in =
new BufferedInputStream(con.getInputStream());
FileOutputStream out =
new FileOutputStream("C:\\filename.Z");
int i;
byte[] bytesIn = new byte[1024];
if ((i = in.read(bytesIn)) >= 0) {
out.write(bytesIn);
}
out.close();
in.close();
but it is giving an error when I'm closing the InputStream in!
I'm definitely stuck. Some comments about would be very useful!

Try this test
InputStream is = new URL("ftp://test:test#ftp.secureftp-test.com/bookstore.xml").openStream();
byte[] a = new byte[1000];
int n = is.read(a);
is.close();
System.out.println(new String(a, 0, n));
it definitely works

From my experience when you read bytes from a stream acquired from ftpClient.retrieveFileStream, for the first run it is not guarantied that you get your byte buffer filled up. However, either you should read the return value of stream.read(b); surrounded with a cycle based on it or use an advanced library to fill up the 1024 length byte[] buffer:
InputStream stream = null;
try {
// Download file from FTP server
stream = client.retrieveFileStream("/pub/obs/2008/021/ab120210.08d.Z");
byte[] b = new byte[1024];
IOUtils.read(stream, b); // will call periodically stream.read() until it fills up your buffer or reaches end-of-file
fos.write(b);
} catch (IOException e) {
e.printStackTrace();
} finally {
IOUtils.closeQuietly(inputStream);
}

I cannot understand why it doesn't work. I found this link where they used the Apache library to read 4096 bytes each time. I read the first 1024 bytes and it works eventually, the only thing is that if completePendingCommand() is used, the program is held for ever. Thus I've removed it and everything works fine.

How can I send a generic file to a jersey service and receive it correctly?

I'm developing a Jersey service that uses Dropbox's API.
I need to post a generic file to my service (the service would be able to manage every kind of file as well as you can do with the Dropbox API).
Client Side
So, I've implemented a simple client that:
opens the file,
creates a connection to the URL,
sets the correct HTTP method,
creates a FileInputStream and writes the file on the connection's outputstream using byte buffer.
This is the client test code.
public class Client {
public static void main(String args[]) throws IOException, InterruptedException {
String target = "http://localhost:8080/DCService/REST/professor/upload";
URL putUrl = new URL(target);
HttpURLConnection connection = (HttpURLConnection) putUrl.openConnection();
connection.setDoOutput(true);
connection.setInstanceFollowRedirects(false);
connection.setRequestMethod("POST");
connection.setRequestProperty("content-Type", "application/pdf");
OutputStream os = connection.getOutputStream();
InputStream is = new FileInputStream("welcome.pdf");
byte buf[] = new byte[1024];
int len;
int lung = 0;
while ((len = is.read(buf)) > 0) {
System.out.print(len);
lung += len;
is.read(buf);
os.write(buf, 0, len);
}
}
}
Server Side
I've a method that:
gets an InputStream as an argument,
creates a file with the same name and type of the original file.
The following code implements a test method to receive a specific PDF file.
#PUT
#Path("/upload")
#Consumes("application/pdf")
public Response uploadMaterial(InputStream is) throws IOException {
String name = "info";
String type = "exerc";
String description = "not defined";
Integer c = 10;
Integer p = 131;
File f = null;
try {
f = new File("welcome.pdf");
OutputStream out = new FileOutputStream(f);
byte buf[] = new byte[1024];
int len;
while ((len = is.read(buf)) > 0)
out.write(buf, 0, len);
out.close();
is.close();
System.out.println("\nFile is created........");
} catch (IOException e) {
throw new WebApplicationException(Response.Status.BAD_REQUEST);
}
//I need to pass a java.io.file object to this method
professorManager.uploadMaterial(name, type, description, c,p, f);
return Response.ok("<result>File " + name + " was uploaded</result>").build();
}
This implementation works only with text files. If I try to send a simple PDF the received file is not readable (after I've saved it on disk).
How can I satisfy my requirements? Could anyone suggest me solution?

You're client code is faulty.
while ((len = is.read(buf)) > 0) {
...
is.read(buf);
...
}
You're reading from the InputStream twice in every iteration. Remove the read statement from the loop's body and you'll be fine.
You've also said that the code presented in your question works with text files. I think that doesn't work either. Reading twice from the file you're trying to upload means you're uploading only half of its contents. Half a text file is still a text file, but half a PDF is only rubbish, so you can't open the latter. You should have double checked if the contents of your uploaded and saved text file is the same as the original.