I download images into my android application with this code:
private void download(URL url, File file) throws IOException {
Log.d(TAG, "download(): downloading file: " + url);
URLConnection urlConnection = url.openConnection();
InputStream inputStream = urlConnection.getInputStream();
BufferedInputStream bufferStream;
OutputStream outputStream = null;
try {
bufferStream = new BufferedInputStream(inputStream, 512);
outputStream = new FileOutputStream(file);
byte[] buffer = new byte[512];
int current;
while ((current = bufferStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, current);
}
} finally {
if (outputStream != null) {
outputStream.close();
}
if (inputStream != null) {
inputStream.close();
}
}
}
This code works fine, but some users and testers complained about incomplete photos. I suspect small network lags which interrupt connection.
So I would like to detect if whole image was downloaded and saved file is complete image. Is there any way how to detect file size from BufferedInputStream or is there another way how detect download completion?
I suggest using Google Volley which provides a super simple interface for networking in general, and image loading specifically. It takes care of threading and batching for you.
It's what Google use on the Google Play app.
It will solve your problem by providing listeners that notify you when the job is complete.
If you are downloading an ordinary file over HTTP, the method getContentLength() of URLConnection gives you the length that the file should have in the end.
You can compare the returned value of this method to the file length/length of downloaded data. If it's equal, then the file is complete:
int contentLength = urlConnection.getContentLength();
if (contentLength != -1) {
if (contentLength == file.length()) {
System.out.println("file is complete");
} else {
System.out.println("file is incomplete");
}
} else {
System.out.println("unknown if file is complete");
}
Try something like this. I think it could help you.
Related
I am trying to download a file that I have hosted on mega. The code is as follows:
AsyncTask.execute(new Runnable() {
#Override
public void run() {
LoggerDebug.d("Prakhar", "Inside run");
try {
URL downloadUrl = new URL(url);
HttpURLConnection httpURLConnection = (HttpURLConnection) downloadUrl.openConnection();
httpURLConnection.setRequestMethod("GET");
httpURLConnection.setDoOutput(true);
httpURLConnection.connect();
File SDCardRoot = Environment.getExternalStorageDirectory();
File downloadFile = new File(SDCardRoot, "downloaded.zip");
FileOutputStream fileOutputStream = new FileOutputStream(downloadFile);
InputStream inputStream = httpURLConnection.getInputStream();
int totalSize = httpURLConnection.getContentLength();
int downloadedSize = 0;
byte[] buffer = new byte[1024];
int bufferLength = 0;
LoggerDebug.d("Prakhar", String.valueOf(totalSize));
while ((bufferLength = inputStream.read(buffer)) > 0 ) {
LoggerDebug.d("Prakhar", String.valueOf(downloadedSize));
fileOutputStream.write(buffer, 0, bufferLength);
downloadedSize += bufferLength;
}
} catch (Exception e) {
e.printStackTrace();
}
}
});
}
Now after I run this code sample I get the following:
Total size is : -1
Downloaded size : 1024
On the mobile I can see the downloaded.zip file, its size is 1.97KB and the original size of the upload is 87.1KB
Where am I going wrong?
So I don't know if you solved your problem yet, but I can see a couple potential issues.
setDoOutput(true) indicates you're going to be uploading according to the documentation here. Specifically, it says:
To upload data to a web server, configure the connection for output using setDoOutput(true).
You're not uploading from what I'm seeing, so this isn't needed, and it's unclear how the server would respond since it appears you're requesting the ability to upload.
A return of -1 indicates an EOF. You're not accessing the file you're intending to, likely due to #3.
You probably can't just download from their site willy-nilly. You need to authenticate somehow. This could be an API key, session id, yadda yadda. They have their own API, and according to this document there is logging in via e-mail in their API. The page doesn't render correctly in GitHub, but you'll need to figure out how to log in. They have an Android app example, but it looks like the API is all native code.
Have you tried changing:
while ((bufferLength = inputStream.read(buffer)) > 0 ) {
to
while ((bufferLength = inputStream.read(buffer)) != null) {
I am making an Android app that uses a WebView to access to a webpage. To handle downloads I am using AsyncTask in method onDownloadStart of WebView's DownloadListener. However files downloaded are blank (although the filename and extension are correct). My Java code is this:
protected String doInBackground(String... url) {
try {
URL url = new URL(url[0]);
//Creating directory if not exists
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setDoOutput(true);
connection.connect();
//Obtaining filename
File outputFile = new File(directory, filename);
InputStream input = new BufferedInputStream(connection.getInputStream());
OutputStream output = new FileOutputStream(outputFile);
byte data[] = new byte[1024];
int count = 0;
Log.e(null, "input.read(data) = "+input.read(data), null);
while ((count = input.read(data)) != -1) {
output.write(data, 0, count);
}
connection.disconnect();
output.flush();
output.close();
input.close();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
log.e line gives -1 value for input.read(data).
PHP code of download page is this (works in all platforms). Files are stored in non-public directories of my HTML server.
<?php
$guid = $_GET['id'];
$file = get_file($guid);
if (isset($file['path'])) {
$mime = $file['MIMEType'];
if (!$mime) {
$mime = "application/octet-stream";
}
header("Pragma: public");
header("Content-type: $mime");
header("Content-Disposition: attachment; filename=\"{$file['filename']}\"");
header('Content-Transfer-Encoding: binary');
ob_clean();
flush();
readfile($file['path']);
exit();
}
?>
I've noticed that if I write some text after "?>" of PHP file, this text is written in the file downloaded.
In your code, you are using ob_clean(), which will just erase the output buffer. Your subsequent call to flush() therefore doesn't return anything, because the output buffer has been flushed beforehand.
Instead of ob_clean() and flush(), use ob_end_flush(). This will stop output buffering and it will send all the output it withheld.
ob_end_flush — Flush (send) the output buffer and turn off output buffering
If you want to stop output buffering without outputting whatever is saved, you can use ob_end_clean(). Anything after this command will be output again, but anything between ob_start() and ob_end_clean() will be "swallowed."
ob_end_clean — Clean (erase) the output buffer and turn off output buffering
What are the benefits of output buffering in the first place? If you are doing ob_start() and then using flush() on everything you might as well output everything directly.
I need a very simple function that allows me to read the first 1k bytes of a file through FTP. I want to use it in MATLAB to read the first lines and, according to some parameters, to download only files I really need eventually. I found some examples online that unfortunately do not work. Here I'm proposing the sample code where I'm trying to download one single file (I'm using the Apache libraries).
FTPClient client = new FTPClient();
FileOutputStream fos = null;
try {
client.connect("data.site.org");
// filename to be downloaded.
String filename = "filename.Z";
fos = new FileOutputStream(filename);
// Download file from FTP server
InputStream stream = client.retrieveFileStream("/pub/obs/2008/021/ab120210.08d.Z");
byte[] b = new byte[1024];
stream.read(b);
fos.write(b);
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fos != null) {
fos.close();
}
client.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}
the error is in stream which is returned empty. I know I'm passing the folder name in a wrong way, but I cannot understand how I have to do. I've tried in many way.
I've also tried with the URL's Java classes as:
URL url;
url = new URL("ftp://data.site.org/pub/obs/2008/021/ab120210.08d.Z");
URLConnection con = url.openConnection();
BufferedInputStream in =
new BufferedInputStream(con.getInputStream());
FileOutputStream out =
new FileOutputStream("C:\\filename.Z");
int i;
byte[] bytesIn = new byte[1024];
if ((i = in.read(bytesIn)) >= 0) {
out.write(bytesIn);
}
out.close();
in.close();
but it is giving an error when I'm closing the InputStream in!
I'm definitely stuck. Some comments about would be very useful!
Try this test
InputStream is = new URL("ftp://test:test#ftp.secureftp-test.com/bookstore.xml").openStream();
byte[] a = new byte[1000];
int n = is.read(a);
is.close();
System.out.println(new String(a, 0, n));
it definitely works
From my experience when you read bytes from a stream acquired from ftpClient.retrieveFileStream, for the first run it is not guarantied that you get your byte buffer filled up. However, either you should read the return value of stream.read(b); surrounded with a cycle based on it or use an advanced library to fill up the 1024 length byte[] buffer:
InputStream stream = null;
try {
// Download file from FTP server
stream = client.retrieveFileStream("/pub/obs/2008/021/ab120210.08d.Z");
byte[] b = new byte[1024];
IOUtils.read(stream, b); // will call periodically stream.read() until it fills up your buffer or reaches end-of-file
fos.write(b);
} catch (IOException e) {
e.printStackTrace();
} finally {
IOUtils.closeQuietly(inputStream);
}
I cannot understand why it doesn't work. I found this link where they used the Apache library to read 4096 bytes each time. I read the first 1024 bytes and it works eventually, the only thing is that if completePendingCommand() is used, the program is held for ever. Thus I've removed it and everything works fine.
I want to download a HTTP query with java, but the file I download has an undetermined length when downloading.
I thought this would be quite standard, so I searched and found a code snippet for it: http://snipplr.com/view/33805/
But it has a problem with the contentLength variable. As the length is unknown, I get -1 back. This creates an error. When I omit the entire check about contentLength, that means I always have to use the maximum buffer.
But the problem is that the file is not ready yet. So the flush gets only partially filled, and parts of the file get lost.
If you try downloading a link like http://overpass-api.de/api/interpreter?data=area%5Bname%3D%22Hoogstade%22%5D%3B%0A%28%0A++node%28area%29%3B%0A++%3C%3B%0A%29+%3B%0Aout+meta+qt%3B with that snippet, you'll notice the error, and when you always download the maximum buffer to omit the error, you end up with a corrupt XML file.
Is there some way to only download the ready part of the file? I would like if this could download big files (up to a few GB).
This should work, i tested it and it works for me:
void downloadFromUrl(URL url, String localFilename) throws IOException {
InputStream is = null;
FileOutputStream fos = null;
try {
URLConnection urlConn = url.openConnection();//connect
is = urlConn.getInputStream(); //get connection inputstream
fos = new FileOutputStream(localFilename); //open outputstream to local file
byte[] buffer = new byte[4096]; //declare 4KB buffer
int len;
//while we have availble data, continue downloading and storing to local file
while ((len = is.read(buffer)) > 0) {
fos.write(buffer, 0, len);
}
} finally {
try {
if (is != null) {
is.close();
}
} finally {
if (fos != null) {
fos.close();
}
}
}
}
If you want this to run in background, simply call it in a Thread:
Thread download = new Thread(){
public void run(){
URL url= new URL("http://overpass-api.de/api/interpreter?data=area%5Bname%3D%22Hoogstade%22%5D%3B%0A%28%0A++node%28area%29%3B%0A++%3C%3B%0A%29+%3B%0Aout+meta+qt%3B");
String localFilename="mylocalfile"; //needs to be replaced with local file path
downloadFromUrl(url, localFilename);
}
};
download.start();//start the thread
This question already has an answer here:
Closed 11 years ago.
Possible Duplicate:
Read Image File Through Java Socket
void readImage() throws IOException
{
socket = new Socket("upload.wikimedia.org", 80);
DataOutputStream bw = new DataOutputStream(new DataOutputStream(socket.getOutputStream()));
bw.writeBytes("GET /wikipedia/commons/8/80/Knut_IMG_8095.jpg HTTP/1.1\n");
bw.writeBytes("Host: wlab.cs.bilkent.edu.tr:80\n\n");
DataInputStream in = new DataInputStream(socket.getInputStream());
File file = new File("imgg.jpg");
file.createNewFile();
DataOutputStream dos = new DataOutputStream(new FileOutputStream(file));
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0)
{
dos.write(buffer, 0, count);
dos.flush();
}
dos.close();
System.out.println("image transfer done");
socket.close();
}
-Create a socket
-Create output stream
-Request the page that includes image
-Read socket to an input stream
-Write to file
I am trying to read an image from socket.
But it is not working.
It seems to read and the image is opened but can not be seen
Where is the problem?
You need to skip HTTP headers to get correct image.
I've already answered to this question today, look at: Read Image File Through Java Socket
The second problem, that you are trying to receive an image from wikipedia without referer and wikipedia restrict to do that (you receiving access denied every time). Try to use another image URL (google image for example).
You can use URL objects directly to fetch HTTP content. The input stream returned by the URL object will only contain content at the URL. The example method below takes a URL, fetches its content and writes the content to a given file.
public static void createImageFile(URL url, File file) throws IOException{
FileOutputStream fos = null;
InputStream is = null;
byte[] b = new byte[1024]; // 1 kB read blocks.
URLConnection conn;
try{
conn = url.openConnection();
/* Set some connection options here
before opening the stream
(i.e. connect and read timeouts) */
is = conn.getInputStream();
fos = new FileOutputStream(file);
int i = 0;
do{
i = is.read(b);
if(i != -1)
fos.write(b, 0, i);
}while(i != -1)
}finally{
/* Don't forget to clean up. */
if(is != null){
try{
is.close();
}catch(Exception e){
/* Don't care */
}
}
if(fos != null){
try{
fos.close();
}catch(Exception e){
/* Don't care */
}
}
}
}