The following code doesn't work to download a file (btw clen is file's length):
int pos = 0, total_pos = 0;
byte[] buffer = new byte[BUFFER_SIZE];
while (pos != -1) {
pos = in.read(buffer, 0, BUFFER_SIZE);
total_pos += pos;
out.write(buffer);
setProgress((int) (total_pos * 100 / clen));
}
...but this works fine:
int buf;
while ((buf = in.read()) != -1)
out.write(buf);
I'm wondering why, even though the second code segment works quickly. On that note, is there any particular reason to use a byte[] buffer (since it doesn't seem to be faster, and BufferedInputStream already uses a buffer of its own....?)
Here's how it should be done.
public static void copyStream(InputStream is, OutputStream os)
{
byte[] buff = new byte[4096];
int count;
try {
while((count = is.read(buff)) > 0)
os.write(buff, 0, count);
}catch (Exception e) {
e.printStackTrace();
}finally {
try {
if(is != null)
is.close();
} catch (IOException e) {
e.printStackTrace();
}
try {
if(os != null)
os.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
I've tried to make the minimum changes necessary to your code to get it working. st0le did a good job of providing a neater version of stream copying.
public class Test {
private static final String FORMAT = "UTF-8";
private static final int BUFFER_SIZE = 10; // for demonstration purposes.
public static void main(String[] args) throws Exception {
String string = "This is a test of the public broadcast system";
int clen = string.length();
ByteArrayInputStream in = new ByteArrayInputStream(string.getBytes(FORMAT));
OutputStream out = System.out;
int pos = 0, total_pos = 0;
byte[] buffer = new byte[BUFFER_SIZE];
while (pos != -1) {
pos = in.read(buffer, 0, BUFFER_SIZE);
if (pos > 0) {
total_pos += pos;
out.write(buffer, 0, pos);
setProgress((int) (total_pos * 100 / clen));
}
}
}
private static void setProgress(int i) {
}
}
You were ignoring the value of pos when you were writing out the buffer to the output stream.
You also need to re-check the value of pos because it may have just read the end of the file. You don't increment the total_pos in that case (although you should probably report that you are 100% complete)
Be sure to handle your resources correctly with close()s in the appropriate places.
-edit-
The general reason for using an array as a buffer is so that the output stream can do as much work as it can with a larger set of data.
Writing to a console there might not be much of a delay, but it might be a network socket being written to or some other slow device. As the JavaDoc states
The write method of OutputStream calls the write method of one argument on each of the bytes to be written out. Subclasses are encouraged to override this method and provide a more efficient implementation.
The benefit of using it when using a Buffered Input/Output Stream are probably minimal.
Related
Is there a way to read all InputStream values at once without a need of using some Apache IO lib?
I am reading IR signal and saving it from the InputStream into the byte[] array. While debugging, I have noticed that it works only if I put a delay there, so that I read all bytes at once and then process it.
Is there a smarter way to do it?
CODE:
public void run() {
Log.i(TAG, "BEGIN mConnectedThread");
byte[] buffer = new byte[100];
int numberOfBytes;
removeSharedPrefs("mSharedPrefs");
// Keep listening to the InputStream while connected
while (true) {
try {
// Read from the InputStream
numberOfBytes = mmInStream.read(buffer);
Thread.sleep(700); //If I stop it here for a while, all works fine, because array is fully populated
if (numberOfBytes > 90){
// GET AXIS VALUES FROM THE SHARED PREFS
String[] refValues = loadArray("gestureBuffer", context);
if (refValues!=null && refValues.length>90) {
int incorrectPoints;
if ((incorrectPoints = checkIfGesureIsSameAsPrevious(buffer, refValues, numberOfBytes)) < 5) {
//Correct
} else {
//Incorrect
}
}
saveArray(buffer, numberOfBytes);
}else{
System.out.println("Transmission of the data was corrupted.");
}
buffer = new byte[100];
// Send the obtained bytes to the UI Activity
mHandler.obtainMessage(Constants.MESSAGE_READ, numberOfBytes, -1, buffer)
.sendToTarget();
} catch (IOException e) {
Log.e(TAG, "disconnected", e);
connectionLost();
// Start the service over to restart listening mode
BluetoothChatService.this.start();
break;
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Edit:
My old answer is wrong, see EJPs comment! Please don't use it. The behaviour of ByteChannels depend on wether InputStreams are blocking or not.
So this is why I would suggest, you just copy IOUtils.read from Apache Commons:
public static int read(final InputStream input, final byte[] buffer) throws IOException {
int remaining = buffer.length;
while (remaining > 0) {
final int location = buffer.length - remaining;
final int count = input.read(buffer, location, remaining);
if (count == -1) { // EOF
break;
}
remaining -= count;
}
return buffer.length - remaining;
}
Old answer:
You can use ByteChannels and read into a ByteBuffer:
ReadableByteChannel c = Channels.newChannel(inputstream);
ByteBuffer buf = ByteBuffer.allocate(numBytesExpected);
int numBytesActuallyRead = c.read(buf);
This read method is attempting to read as many bytes as there is remaining space in the buffer. If the stream ends before the buffer is fully filled, the number of bytes actually read is returned. See JavaDoc.
I have to fill a byte[] in my Android application. Sometime, this one is bigger than 4KB.
I initialize my byte[] like this :
int size = ReadTools.getPacketSize(ptr.dataInputStream);
byte[] myByteArray = new byte[size];
Here, my size = 22625. But when I fill up my byte[] like this :
while (i != size) {
myByteArray[i] = ptr.dataInputStream.readByte();
i++;
}
But when I print the content of my byte[], I have a byte[] with size = 4060.
Does Java split my byte[] if this one is bigger than 4060 ? And if yes, how can I have a byte[] superior to 4060 ?
Here is my full code:
public class ReadSocket extends Thread{
DataInputStream inputStream;
BufferedReader reader;
GlobalContent ptr;
public ReadSocket(DataInputStream inputStream, GlobalContent ptr)
{
this.inputStream = inputStream;
this.ptr = ptr;
}
public void run() {
int i = 0;
int j = 0;
try {
ptr.StatusThreadReadSocket = 1;
while(ptr.dataInputStream.available() == 0)
{
if(ptr.StatusThreadReadSocket == 0)
{
ptr.dataInputStream.close();
break;
}
}
if(ptr.StatusThreadReadSocket == 1)
{
int end = ReadTools.getPacketSize(ptr.dataInputStream);
byte[] buffer = new byte[end];
while (i != end) {
buffer[j] = ptr.dataInputStream.readByte();
i++;
j++;
}
ptr.StatusThreadReadSocket = 0;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
...
}
Java doesn't split anything. You should post the minimal code which reproduces your error, and tell where ReadTools comes from.
There are two options here:
ReadTools.getPacketSize() returns 4096
You inadevertedly reassign myByteArray to another array
You should really post your full code and tell what library you use. Likely, it will have a method like
read(byte[] buffer, int offset, int length);
Which will save you some typing and also give better performance if all you need is bulk reading the content of the input in memory
I know that the canonical way of handling the output/error streams of an external Process in Java, is to use two extra-threads in order to pull the data from the output and error streams lest the process might be blocked.
Now what about the following?
public static void main(String[] args) throws IOException, InterruptedException {
ProcessBuilder processBuilder = new ProcessBuilder(args);
Process process = processBuilder.start();
InputStream outputStream = null, errorStream = null;
ByteArrayOutputStream outputBuffer = new ByteArrayOutputStream();
ByteArrayOutputStream errorBuffer = new ByteArrayOutputStream();
try {
outputStream = process.getInputStream();
errorStream = process.getErrorStream();
byte[] tmp = new byte[1024];
while (true) {
int outputBytes = readAvailablOnce(outputStream, outputBuffer, tmp);
int errorBytes = readAvailablOnce(errorStream, errorBuffer, tmp);
if (outputBytes == 0 && errorBytes == 0) {
try {
process.exitValue();
break;
} catch (IllegalThreadStateException e) {
// keep on looping
}
}
}
readAvailableAll(outputStream, outputBuffer, tmp);
readAvailableAll(errorStream, errorBuffer, tmp);
} finally {
closeQuietly(outputStream);
closeQuietly(errorStream);
}
System.out.println(outputBuffer.toString("ASCII"));
System.err.println(errorBuffer.toString("ASCII"));
System.err.println("exit code: " + process.exitValue());
}
private static void closeQuietly(InputStream in) {
if (in != null) {
try {
in.close();
} catch (IOException e) {
// ignored
}
}
}
private static int readAvailablOnce(
InputStream inputStream, OutputStream outputStream, byte[] buffer)
throws IOException {
int bytesRead = 0;
if (inputStream.available() > 0) {
bytesRead = inputStream.read(buffer);
outputStream.write(buffer, 0, bytesRead);
}
return bytesRead;
}
private static void readAvailableAll(
InputStream inputStream, OutputStream outputStream, byte[] buffer)
throws IOException {
if (inputStream.available() > 0) {
int bytesRead = 0;
while ((bytesRead = inputStream.read(buffer)) >= 0) {
outputStream.write(buffer, 0, bytesRead);
}
}
}
This actually runs fine in the few examples I tried (running "dir", "ps aux", etc).
It also has the disadvantage of not allowing you to easily handle the outputs line by line (here you buffer everything before doing anything), unless you start doing some more or less complicated stuff with Buffers and CharsetDecoders.
Still, it looks useful for anything without unacceptably big output (although nothing forces us to buffer the whole output before using it).
I only tried the 1.5 and 1.6 JVM (Windows XP and Linux).
Also, this code makes the assumption that the final output bits of the process will be readily available for reading (InputStream.available() > 0).
Anyone would know what (or if something) is wrong with this code (or have a better idea)?
I would use ProcessBuilder.redirectErrorStream(true) which allows you to read one stream in the current thread. You wouldn't need any background threads.
Server side code
public static boolean sendFile() {
int start = Integer.parseInt(startAndEnd[0]) - 1;
int end = Integer.parseInt(startAndEnd[1]) - 1;
int size = (end - start) + 1;
try {
bos = new BufferedOutputStream(initSocket.getOutputStream());
bos.write(byteArr,start,size);
bos.flush();
bos.close();
initSocket.close();
System.out.println("Send file to : " + initSocket);
} catch (IOException e) {
System.out.println(e.getLocalizedMessage());
disconnected();
return false;
}
return true;
}
Client Side
public boolean receiveFile() {
int current = 0;
try {
int bytesRead = bis.read(byteArr,0,byteArr.length);
System.out.println("Receive file from : " + client);
current = bytesRead;
do {
bytesRead =
bis.read(byteArr, current, (byteArr.length-current));
if(bytesRead >= 0) current += bytesRead;
} while(bytesRead != -1);
bis.close();
bos.write(byteArr, 0 , current);
bos.flush();
bos.close();
} catch (IOException e) {
System.out.println(e.getLocalizedMessage());
disconnected();
return false;
}
return true;
}
Client side is multithreading,server side not use multithreading. I just paste some code that made problem if you want see all code please tell me.
After I debug the code, I found that if I set max thread to any and then the first thread always stuck in this loop. That bis.read(....) always return 0. Although, server had close stream and it not get out of the loop. I don't know why ... But another threads are work correctly.
do {
bytesRead =
bis.read(byteArr, current, (byteArr.length-current));
if(bytesRead >= 0) current += bytesRead;
} while(bytesRead != -1);
How large is your input file (the one you send?) and how large is "byteArr"?
Also, by the time your check how many bytes are read, you already called bis.read(..) twice:
int bytesRead = bis.read(byteArr,0,byteArr.length);
You probably want to read/send files larger than your buffer, so you probably want to do something like this:
byte [] buffer = new byte[4096];
int bytesRead;
int totalLength = 0;
while(-1 != (bytesRead = is.read(buffer))) {
bos.write(buffer, 0, bytesRead);
totalLength += bytesRead;
}
bos.close();
is.close();
"is" would be a plain InputStream, Peter is right, you do not need to buffer it.
read() will return 0 when you give it a buffer with no room left. (Which appears to be the case here)
I would suggest you use a DataInputStream.readFully() which does this for you.
dis.readFully(byteArr); // keeps reading until the byte[] is full.
If you are only writing large byte[] or only writing one piece of data, using a Buffered Stream just adds overhead. You don't need it.
BTW: When you call close() it will call flush() for you.
I am writing code for uploading a file from a client to my server and the performance isn't as fast as I think it should be.
I have the current code snippet that is doing the file transfer and I was wondering how I could speed up the transfer.
Sorry about all of the code:
InputStream fileItemInputStream ;
OutputStream saveFileStream;
int[] buffer;
while (fileItemInputStream.available() > 0) {
buffer = Util.getBytesFromStream(fileItemInputStream);
Util.writeIntArrToStream(saveFileStream, buffer);
}
saveFileStream.close();
fileItemInputStream.close();
The Util methods are as follows:
public static int[] getBytesFromStream(InputStream in, int size) throws IOException {
int[] b = new int[size];
int count = 0;
while (count < size) {
b[count++] = in.read();
}
return b;
}
and:
public static void writeIntArrToStream(OutputStream out, int[] arrToWrite) throws IOException {
for (int i = 0; i < arrToWrite.length; i++) {
out.write(arrToWrite[i]);
}
}
Reading a single byte at a time will be horribly inefficient. You're also relying on available, which is rarely a good idea. (It will return 0 if there are no bytes currently available, but there may be more to come.)
This is the right sort of code to copy a stream:
public void copyStream(InputStream input, OutputStream output) throws IOException
{
byte[] buffer = new byte[32*1024];
int bytesRead;
while ((bytesRead = input.read(buffer, 0, buffer.length)) > 0)
{
output.write(buffer, 0, bytesRead);
}
}
(The caller should close both streams.)