I am trying to fetch input stream pdf from URL Connection but I am getting an empty input stream. Can anyone please tell me what is I am doing wrong? Following is the code:
<!-- language: java -->
URL fileUrl = new URL("https://www.dropbox.com/s/ao3up7xudju4qm0/Amalgabond%20Adhesive%20Agent.pdf");
HttpURLConnection connection = (HttpURLConnection)fileUrl.openConnection();
connection.connect();
InputStream is = connection.getInputStream();
Log.i("TAG", "is.available(): " + is.available());
is.available() is returning 0 empty stream.
According to the javadoc, available() does not block and wait until all data is available, so you might have not completely received your stuff when its called.
You should use something like this instead of available() :
int bytesRead;
byte[] buffer = new byte[100000];
while((bytesRead = is.read(buffer)) > 0){
// Do something here with buffer
}
read() is a blocking method.
You're misusing the available() method. It doesn't tell you the length of the input stream, so the fact that it returns zero doesn't indicate that it's empty. See the Javadoc, where all this is explicitly stated.
Just read it until end of stream.
If your ultimate goal is to download a file from Dropbox, you should use the Dropbox Java API, or maybe this simpler solution. Otherwise, a URLConnection to a file on Dropbox will download a web page (in HTML) showing you a link to click (with a lot of other stuff !) for downloading your file.
Related
I'm dealing with some Java code in which there's an InputStream that I read one time and then I need to read it once again in the same method.
The problem is that I need to reset it's position to the start in order to read it twice.
I've found a hack-ish solution to the problem:
is.mark(Integer.MAX_VALUE);
//Read the InputStream is fully
// { ... }
try
{
is.reset();
}
catch (IOException e)
{
e.printStackTrace();
}
Does this solution lead to some unespected behaviours? Or it will work in it's dumbness?
As written, you have no guarantees, because mark() is not required to report whether it was successful. To get a guarantee, you must first call markSupported(), and it must return true.
Also as written, the specified read limit is very dangerous. If you happen to be using a stream that buffers in-memory, it will potentially allocate a 2GB buffer. On the other hand, if you happen to be using a FileInputStream, you're fine.
A better approach is to use a BufferedInputStream with an explicit buffer.
It depends on the InputStream implementation. You can also think whether it will be better if you use byte[]. The easiest way is to use Apache commons-io:
byte[] bytes = IOUtils.toByteArray(inputSream);
You can't do this reliably; some InputStreams (such as ones connected to terminals or sockets) don't support mark and reset (see markSupported). If you really have to traverse the data twice, you need to read it into your own buffer.
Instead of trying to reset the InputStream load it into a buffer like a StringBuilder or if it's a binary data stream a ByteArrayOutputStream. You can then process the buffer within the method as many times as you want.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
int read = 0;
byte[] buff = new byte[1024];
while ((read = inStream.read(buff)) != -1) {
bos.write(buff, 0, read);
}
byte[] streamData = bos.toByteArray();
For me, the easiest solution was to pass the object from which the InputStream could be obtained, and just obtain it again. In my case, it was from a ContentResolver.
I am writing a Java code to download large amount of zip files on site using http protocol, and each file is around 1MB(1024KB) size.
I know there are a lot of ways to doing that. I am just wandering which is the fastest, and I would like to know the progress of each downloading like showing a percentage number or something.
I am just giving my version of code , any ideas on how to improve it?
Thanks All.
public static void downloadFile(String downloadUrl , String fileName) throws Exception {
URL url=new URL(downloadUrl);
URLConnection connection = url.openConnection();
int filesize = connection.getContentLength();
float totalDataRead=0;
java.io.BufferedInputStream in = new java.io.BufferedInputStream(connection.getInputStream());
java.io.FileOutputStream fos = new java.io.FileOutputStream(fileName);
java.io.BufferedOutputStream bout = new BufferedOutputStream(fos,1024);
byte[] data = new byte[1024];
int i=0;
while((i=in.read(data,0,1024))>=0) {
totalDataRead=totalDataRead+i;
bout.write(data,0,i);
float Percent=(totalDataRead*100)/filesize;
System.out.println((int)Percent);
}
bout.close();
in.close();
}
You are optimizing prematurely. The network bandwidth bottleneck is likely going to far outweigh any processing you are doing.
You don't need to wrap the InputStream in a BufferedInputStream. You may want to favor larger read buffer sizes, but that may have minimal effect depending on the underlying implementation of the InputStream returned by the connection, kernel level buffering, etc.
For a progress bar, take what you've read so far and divide it by connection.getContentLength(), but note that getContentLength() may return -1 if the length is unknown (it simply gives you the value of the Content-length header). As you're reading the data, pass the progress info along to whatever you choose to do to display it to the user.
I don't know, mine took 8 hours. To reduce it from 24 hours I cancelled all other downloads, didn't use the internet, and killed all other background tasks.
What I want to do is log the output from an inputstream that I go using
org.apache.http.HttpEntity entity = response.getEntity();
org.apache.http.HttpResponse content =entity.getContent();
//Print the result to the screen for debugging
//puroposes
if(Logging.DEBUG) {
InputStream content =entity.getContent();
int i;
StringBuilder b = new StringBuilder();
while( (i=content.read()) != -1 ) {
b.append((char)i);
}
Log.d(TAG, b.toString());
}
Now after I have finished logging, I want to use the exact same stream through an XML parser. The problem is that it tells me that the steam has already been used.
I tried to the use mark() and reset() calls before and after debugging but it didn't work.
It depends whether the inputstream that is returned supports it. The default implementation in the InputStream class does nothing, as described in the API. So you can't be sure whether the returned Stream actually supports it. To be sure of this, you should wrap it in a BufferedInputStream, which does supports these methods.
In general mark() and reset() won't work on an arbitrary InputStream. They only work on subclasses like FileInputStream where the underlying data source supports these operations.
For something like a SocketInputStream or a console InputStream, your only option will be to read and buffer the entire stream contents somewhere; e.g. in memory or by writing it to a temporary file.
Is there an easy (therefore quick) way to accomplish this? Basically just take some input stream, could be something like a socket.getInputStream(), and have the stream's buffer autmoatically redirect to standard out?
There are no easy ways to do it, because InputStream has a pull-style interface, when OutputStream has a push-style one. You need some kind of pumping loop to pull data from InputStream and push them into OutputStream. Something like this (run it in a separate thread if necessary):
int size = 0;
byte[] buffer = new byte[1024];
while ((size = in.read(buffer)) != -1) out.write(buffer, 0, size);
It's already implemented in Apache Commons IO as IOUtils.copy()
You need a simple thread which reads from the input stream and writes to standard output. Make sure it yields to other threads.
Since Java 9, you can use InputStream.transferTo
Example
try (InputStream stream = Application.class.getResourceAsStream("/test.txt")) {
stream.transferTo(System.out);
}
This problem seems to happen inconsistently. We are using a java applet to download a file from our site, which we store temporarily on the client's machine.
Here is the code that we are using to save the file:
URL targetUrl = new URL(urlForFile);
InputStream content = (InputStream)targetUrl.getContent();
BufferedInputStream buffered = new BufferedInputStream(content);
File savedFile = File.createTempFile("temp",".dat");
FileOutputStream fos = new FileOutputStream(savedFile);
int letter;
while((letter = buffered.read()) != -1)
fos.write(letter);
fos.close();
Later, I try to access that file by using:
ObjectInputStream keyInStream = new ObjectInputStream(new FileInputStream(savedFile));
Most of the time it works without a problem, but every once in a while we get the error:
java.io.StreamCorruptedException: invalid stream header: 0D0A0D0A
which makes me believe that it isn't saving the file correctly.
I'm guessing that the operations you've done with getContent and BufferedInputStream have treated the file like an ascii file which has converted newlines or carriage returns into carriage return + newline (0x0d0a), which has confused ObjectInputStream (which expects serialized data objects.
If you are using an FTP URL, the transfer may be occurring in ASCII mode.
Try appending ";type=I" to the end of your URL.
Why are you using ObjectInputStream to read it?
As per the javadoc:
An ObjectInputStream deserializes primitive data and objects previously written using an ObjectOutputStream.
Probably the error comes from the fact you didn't write it with ObjectOutputStream.
Try reading it wit FileInputStream only.
Here's a sample for binary ( although not the most efficient way )
Here's another used for text files.
There are 3 big problems in your sample code:
You're not just treating the input as bytes
You're needlessly pulling the entire object into memory at once
You're doing multiple method calls for every single byte read and written -- use the array based read/write!
Here's a redo:
URL targetUrl = new URL(urlForFile);
InputStream is = targetUrl.getInputStream();
File savedFile = File.createTempFile("temp",".dat");
FileOutputStream fos = new FileOutputStream(savedFile);
int count;
byte[] buff = new byte[16 * 1024];
while((count = is.read(buff)) != -1) {
fos.write(buff, 0, count);
}
fos.close();
content.close();
You could also step back from the code and check to see if the file on your client is the same as the file on the server. If you get both files on an XP machine, you should be able to use the FC utility to do a compare (check FC's help if you need to run this as a binary compare as there is a switch for that). If you're on Unix, I don't know the file compare program, but I'm sure there's something.
If the files are identical, then you're looking at a problem with the code that reads the file.
If the files are not identical, focus on the code that writes your file.
Good luck!