i want to download a file through InputStream from some offSet,
i know that i can do this by read method of InputStream but it requires a parameter byteCount
but i don't know how to calculate byteCount for read Method
My Code is
try {
URL url;
url = new URL(uri);
HttpURLConnection c = (HttpURLConnection) url
.openConnection();
c.setConnectTimeout(1000000);
c.connect();
int lenghtOfFile = c.getContentLength();
Log.i("fileSize", lenghtOfFile + "");
File file = new File(
Environment.getExternalStorageDirectory() + "/"
+ Environment.DIRECTORY_DOWNLOADS);
file.mkdirs();
File outputFile = new File(file, "4k1.jpg");
FileOutputStream fos = new FileOutputStream(outputFile);
Log.v("Place", "trying...");
InputStream is = c.getInputStream();
Log.v("errorPart", c.getURL().toString());
if (c.getURL().toString().contains("404"))
Log.v("Error", "404 Error");
Log.v("Place", "1st attempt success");
byte[] buffer = new byte[1024];
int len1 = 0;
Log.v("Place", "Download started!");
int on = 0;
while ((len1 = is.read(buffer)) != -1) {
fos.write(buffer, 0, len1);
Log.v("is", "running " + len1);
}
Log.v("Place", "Download Finished!");
fos.close();
is.close();
} catch (Exception e) {
e.printStackTrace();
}
Please try to explain your answer, i am newbiee
You need to tell it how many bytes it can read. This is to prevent the read() method copy data over memory that it shouldn't. You should therefore pass the size of the buffer. In the code above this will be 1024. You may want to use private static int BUFFER_SIZE = 1024 and use this constant in both the new and read() so that if you change the size of the buffer you don't forget to change the parameter to the read() function as well.
private static final int BUFFER_SIZE = 1024;
...
byte[] buffer = new byte[BUFFER_SIZE];
int len1 = 0;
Log.v("Place", "Download started!");
int on = 0;
while ((len1 = is.read(buffer,0,BUFFER_SIZE)) != -1) {
fos.write(buffer, 0, len1);
Log.v("is", "running " + len1);
}
...
Why not directly use the class, CountingInputStream, provided by Apache. This class has been perfectly designed to match your requirment.
There is no direct access to the file size from HttpURLConnection You can use the nio cache under your file stream In a one-time generated file.
You don't really need to provide byteCount for read method. Using is.read(buffer)) will simply read out 1024 bytes (since defined by the length of your buffer array) at every round till -1 is returned.
Related
The code I have posted below works for single file transfer over a socket. But it doesn't work for multiple file transfers over a socket. When trying multiple file transfers over the socket the code crashes.
I send multiple files by looping over the server sending code x amount of times, and run the receiving code x amount of times. When trying to send multiple files, the first file will send successfully, the second file name and size will be read successfully but the error in my code happens after this.
In my receiving client I tried to use to suggestion posted here: Java multiple file transfer over socket but had no success.
The error is on the client side.
The question I am asking is: Why isn't this code working for multiple files, and how can I fix it?
Server Sending
try{
byte[] bytes = new byte[(int)file.length()];
FileInputStream fis = new FileInputStream(file);
OutputStream os = socket.getOutputStream();
out.println(file.getName()); // Send Filename
out.println(file.length()); // Send filesize
int count;
while ((count = fis.read(bytes)) > 0) {
os.write(bytes, 0, count);
}
os.flush();
fis.close();
}catch(IOException e){
e.printStackTrace();
}
}
Client Recieving
try{
String file = in.readLine(); // Read filename
int fileSize = Integer.parseInt(in.readLine()); // Read Filesize
//ERROR HAPPENING ON LINE ABOVE IN LOOPS AFTER THE FIRST
byte [] buf = new byte [fileSize];
FileOutputStream fos = new FileOutputStream(file);
InputStream is = socket.getInputStream();
int count = 0;
while (fileSize > 0 && (count = is.read(buf, 0, (int)Math.min(buf.length, fileSize))) != -1){
fos.write(buf, 0, count);
fileSize -= count;
}
fos.close();
}catch(IOException e){
e.printStackTrace();
}
The error is a NumberFormatException, on loops after the first when the client is receiving part of a file for the input to the fileSize.
Make sure you flush the PrintWriter before you then write raw bytes directly to the OutputStream that the PrintWriter is attached to. Otherwise, you could write any buffer data out of order to the underlying socket.
But more importantly, make sure that if you use buffered reading on the receiving end that you read the file bytes using the same buffer that receives the file name and file size. You should also transfer the File using smaller fixed chunks, don't allocate a single byte[] array for the entire file size, that is a waste of memory for large files, and likely to fail.
Server:
try{
byte[] bytes = new byte[1024];
FileInputStream fis = new FileInputStream(file);
OutputStream os = socket.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
PrinterWriter pw = new PrintWriter(bos);
pw.println(file.getName()); // Send Filename
pw.println(file.length()); // Send filesize
pw.flush();
int count;
while ((count = fis.read(bytes)) > 0) {
bos.write(bytes, 0, count);
}
bos.flush();
fis.close();
}catch(IOException e){
e.printStackTrace();
}
}
Client:
try{
byte [] buf = new byte [1024];
FileOutputStream fos = new FileOutputStream(file);
InputStream is = socket.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
InputStreamReader isr = new InputStreamReader(bis);
String file = isr.readLine(); // Read filename
long fileSize = Long.parseLong(isr.readLine()); // Read Filesize
int count = 0;
while ((fileSize > 0) && (count = bis.read(buf, 0, (int)Math.min(buf.length, fileSize))) > 0){
fos.write(buf, 0, count);
fileSize -= count;
}
fos.close();
}catch(IOException e){
e.printStackTrace();
}
That being said, you might also consider using DataOutputStream.writeLong() and DataInputStream.readLong() to send/receive the file size in its original binary format instead of as a textual string:
Server:
try{
byte[] bytes = new byte[1024];
FileInputStream fis = new FileInputStream(file);
OutputStream os = socket.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
PrinterWriter pw = new PrintWriter(bos);
pw.println(file.getName()); // Send Filename
pw.flush();
DataOutputStream dos = new DataOutputStream(bos);
dos.writeLong(file.length()); // Send filesize
dos.flush();
int count;
while ((count = fis.read(bytes)) > 0) {
bos.write(bytes, 0, count);
}
bos.flush();
fis.close();
}catch(IOException e){
e.printStackTrace();
}
}
Client:
try{
byte [] buf = new byte [1024];
FileOutputStream fos = new FileOutputStream(file);
InputStream is = socket.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
InputStreamReader isr = new InputStreamReader(bis);
String file = isr.readLine(); // Read filename
DataInputStream dis = new DataInputStream(bos);
long fileSize = dis.readLong(); // Read Filesize
int count = 0;
while ((fileSize > 0) && (count = bis.read(buf, 0, (int)Math.min(buf.length, fileSize))) > 0){
fos.write(buf, 0, count);
fileSize -= count;
}
fos.close();
}catch(IOException e){
e.printStackTrace();
}
I'm trying to get a BufferedInputStream from an uploaded cvs file.
I'm working with a Multipart derived from the cvs file.
When I first get the Multipart, it's a BufferedInputStream, but the buffer is all null.
But if I look deeper down, there's another buffer in the CoyoteInputStream and that has data.
How can I get at this second buffer? My code is below.
And of course it's throwing a null exception when it gets to
while ((multiPartDataPos = stream.read(buffer)) >= 0)
What am I doing wrong? Am I mistaken that the CoyoteInputStream is the data I want?
public byte[] handleUploadedFile(Multipart multiPart) throws EOFException {
Multipart multiPartData = null;
BufferedInputStream stream = null;
int basicBufferSize = 0x2000;
byte[] buffer = new byte[basicBufferSize];
int bufferPos = 0;
try {
while (multiPart.hasNext()) {
int multiPartDataPos = bufferPos;
multiPartData = (Multipart) multiPart.next();
stream = new BufferedInputStream(multiPartData.getInputStream());
while ((multiPartDataPos = stream.read(buffer)) >= 0) {
int len = stream.read(buffer, multiPartDataPos, buffer.length - multiPartDataPos);
multiPartDataPos += len;
}
bufferPos = bufferPos + multiPartDataPos;
}
} ...
Your code doesn't make any sense.
while ((multiPartDataPos = stream.read(buffer)) >= 0) {
At this point you have read multiPartDataPos bytes into buffer, so that buffer[0..multiPartDataPos-1] contains the data just read.
int len = stream.read(buffer, multiPartDataPos, buffer.length - multiPartDataPos);
At this point you are doing another read, which could return -1, which will otherwise add some data from multiPartPos to multiPartDataPos+len-.
multiPartDataPos += len;
This step is only valid if len > 0.
And you are doing nothing with the buffer; and next time around the loop you will clobber whatever you just read.
The correct way to read any stream in Java is as follows:
while ((count = in.read(buffer)) > 0)
{
// use buffer[9..count-1], for example out.write(buffer, 0, count);
}
I don't understand why you think access to an underlying stream is required or what it's going to give you that you don't already have.
Turns out the better solution was to use move the data from an InputStream to a ByteArrayOutputStream and then return ByteArrayOutputStream.toByteArray()
Multipart multiPartData = null;
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int read;
byte[] input = new byte[4096];
InputStream is;
try {
multiPartData = (Multipart)multipart.next();
is = multiPartData.getInputStream();
while ((read = is.read(input, 0, input.length)) != -1) {
buffer.write(input, 0, read);
}
buffer.flush();
return buffer.toByteArray(); // just a test right now
}
My server is sending several files to my client. However at the client side, it only receives the first file because I don't know how to iterate and get the second file.
The Server sends like this:
ListIterator iter = missingfiles.listIterator();
//missingfiles contain all the filenames to be sent
String filename;
while (iter.hasNext()) {
// System.out.println(iter.next());
filename=(String) iter.next();
File myFile = new File("src/ee4210/files/"+filename);
byte[] mybytearray = new byte[(int) myFile.length()];
FileInputStream fis = new FileInputStream(myFile);
BufferedInputStream bis = new BufferedInputStream(fis);
//bis.read(mybytearray, 0, mybytearray.length);
DataInputStream dis = new DataInputStream(bis);
dis.readFully(mybytearray, 0, mybytearray.length);
OutputStream os = _socket.getOutputStream();
//Sending file name and file size to the server
DataOutputStream dos = new DataOutputStream(os);
dos.writeUTF(myFile.getName());
dos.writeLong(mybytearray.length);
dos.write(mybytearray, 0, mybytearray.length);
dos.flush();
}
The client receives like this: (It will only receive the first file and I don't know how to make it loop to receive the next file)
int bytesRead;
int current = 0;
int filecount = 0;
InputStream in;
try {
in = _socket.getInputStream();
DataInputStream clientData = new DataInputStream(in);
String fileName = clientData.readUTF();
OutputStream output = new FileOutputStream(
"src/ee4210/files/"+ fileName);
long size = clientData.readLong();
byte[] buffer = new byte[1024];
while (size > 0
&& (bytesRead = clientData.read(buffer, 0,
(int) Math.min(buffer.length, size))) != -1) {
output.write(buffer, 0, bytesRead);
size -= bytesRead;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
How to receive multiple files from outputstream?
The obvious answer is 'one at a time, with extra information to tell you where one stops and another starts'. The most usual technique is to send the file size ahead of the file, e.g. as a long via DataOutputStream.writeLong(), and at the receiver change your read loop to stop after exactly that many bytes, close the output file, and continue the outer loop that reads the next long or end-of-stream.
You can try this.
I've used a lazy method to check that the end of all 3 files have been received.
int bytesRead;
int current = 0;
int filecount = 0;
InputStream in;
try
{
in = _socket.getInputStream();
DataInputStream clientData = new DataInputStream(in);
while(true)
{
String fileName = clientData.readUTF();
// will throw an EOFException when the end of file is reached. Exit loop then.
OutputStream output = new FileOutputStream("src/ee4210/files/"+ fileName);
long size = clientData.readLong();
byte[] buffer = new byte[1024];
while (size > 0
&& (bytesRead = clientData.read(buffer, 0,
(int) Math.min(buffer.length, size))) != -1)
{
output.write(buffer, 0, bytesRead);
size -= bytesRead;
}
output.close();
}
}
catch (EOFException e)
{
// means we have read all the files
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
In java, how to read a fixed length from the inputstream and save as a file?
eg. I want to read 5M from inputStream, and save as downloadFile.txt or whatever.(BUFFERSIZE=1024)
FileOutputStream fos = new FileOutputStream(downloadFile);
byte buffer [] = new byte[BUFFERSIZE];
int temp = 0;
while ((temp = inputStream.read(buffer)) != -1)
{
fos.write(buffer, 0, temp);
}
Two options:
Just keep reading and writing until you either reach the end of the input or you've copied enough:
byte[] buffer = new byte[1024];
int bytesLeft = 5 * 1024 * 1024; // Or whatever
FileInputStream fis = new FileInputStream(input);
try {
FileOutputStream fos = new FileOutputStream(output);
try {
while (bytesLeft > 0) {
int read = fis.read(buffer, 0, Math.min(bytesLeft, buffer.length);
if (read == -1) {
throw new EOFException("Unexpected end of data");
}
fos.write(buffer, 0, read);
bytesLeft -= read;
}
} finally {
fos.close(); // Or use Guava's Closeables.closeQuietly,
// or try-with-resources in Java 7
}
} finally {
fis.close();
}
Read all 5M into memory in one call, e.g. using DataInputStream.readFully, and then write it out in one go. Simpler, but obviously uses more memory.
I have a bin file that I need to convert to a byte array. Can anyone tell me how to do this?
Here is what I have so far:
File f = new File("notification.bin");
is = new FileInputStream(f);
long length = f.length();
/*if (length > Integer.MAX_VALUE) {
// File is too large
}*/
// Create the byte array to hold the data
byte[] bytes = new byte[(int)length];
// Read in the bytes
int offset = 0;
int numRead = 0;
while (offset < bytes.length && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) {
offset += numRead;
}
// Ensure all the bytes have been read in
if (offset < bytes.length) {
throw new IOException("Could not completely read file "+f.getName());
}
But it's not working...
Kaddy
try using this
public byte[] readFromStream(InputStream inputStream) throws Exception
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(baos);
byte[] data = new byte[4096];
int count = inputStream.read(data);
while(count != -1)
{
dos.write(data, 0, count);
count = inputStream.read(data);
}
return baos.toByteArray();
}
Btw, do you want a Java code or C++ code. Seeing the code in your question, I assumed it to be a java code and hence gave a java answer to it
You're probably better off using a memory mapped file. See this question
In Java, a simple solution is:
InputStream is = ...
ByteArrayOutputStream os = new ByteArrayOutputStream();
byte[] data = new byte[4096]; // A larger buffer size would probably help
int count;
while ((count = is.read(data)) != -1) {
os.write(data, 0, count);
}
byte[] result = os.toByteArray();
If the input is a file, we can preallocate a byte array of the right size:
File f = ...
long fileSize = f.length();
if (fileSize > Integer.MAX_VALUE) {
// file too big
}
InputStream is = new FileInputStream(f);
byte[] data = new byte[fileSize];
if (is.read(data)) != data.length) {
// file truncated while we were reading it???
}
However, there is probably a more efficient way to do this task using NIO.
Unless you really need to do it just that way, maybe simplify what you're doing.
Doing everything in the for loop may seem like a very slick way of doing it, but it's shooting yourself in the foot when you need to debug and don't immediately see the solution.
In this answer I read from an URL
You could modify it so the InputStream is from a File instead of a URLConnection.
Something like:
FileInputStream inputStream = new FileInputStream("your.binary.file");
ByteArrayOutputStream output = new ByteArrayOutputStream();
byte [] buffer = new byte[ 1024 ];
int n = 0;
while (-1 != (n = inputStream.read(buffer))) {
output.write(buffer, 0, n);
}
inputStream.close();
etc
Try open source library apache commons-io
IOUtils.toByteArray(inputStream)
You are not the first and not the last developer who needs to read a file, no need to reinvent it each time.