I'm writing a code to read the bytes of a request body and this requires knowing the Content-Length or Transfer-encoding ahead of time to safely transfer the message to the client. According to the RCF2616 Section 14.13:
Any Content-Length greater than or equal to zero is a valid value.
In my code Implementation, I achieved this by getting the Content-Length: header field which returns 0,which I guess is a valid response but not the required amount of bytes. Have tried in the below code to read the InputStream from the socket still the amount is achieved but this seem to be failing.Any pointers achieving this? Can provide more code if necessary.
Here is the calling method to get content-length header and read the bytes in chunk till the exact amount is achieved:
//Gets the Content-Length header value
int contLengthOffset = Integer.parseInt(newRequest.getHeaderField("Content-Length"));
int Offset = contLengthOffset;
if(Offset >= 0) {
//Any Content-Length greater than or equal to zero is a valid value.
count = QueryStreamClass.ReadFullyHelper(socket.getInputStream(), Offset);
}
Below is the method that reads the content-length:
/**
* Read the content-length to determine the transfer-length of the message.
* We need enough bytes to get the required message.
* #param Stream
* #param size
*/
public static String ReadFullyHelper(InputStream Stream, int size) {
int Read;
int totalRead = 0;
int toRead = GMInjectHandler.buffer;;
StringBuilder Request = new StringBuilder();
if(toRead > size) {
toRead = size;
}
while(true) {
try {
final byte[] by = new byte[toRead];
Read = Stream.read(by, 0, toRead);
if(Read == -1){
break;
}
Request.append(new String(by, 0, Read));
totalRead += Read;
if (size - totalRead < toRead) {
toRead = size - totalRead;
}
if (totalRead == size) {
break;
}
} catch (IOException e) {
Log.e(TAG, "Error reading stream", e);
}
}
return Request.toString();
}
'This seems to be failing' is not a problem description, but:
public static String ReadFullyHelper(InputStream Stream, int size) {
You can reduce this entire method to the following:
DataInputStream din = new DataInputStream(Stream);
byte[][ buffer = new byte[size];
din.readFully(buffer);
return new String(buffer, 0, size); // or you may want to use a specific encoding here
Related
I am trying to establish communication between Arduino and Android over Uart. So, while reading buffer on Android side I am not getting data in chunks.
if (uartDevice != null) {
// Loop until there is no more data in the RX buffer.
try {
byte[] buffer = new byte[CHUNK_SIZE];
int read;
while ((read = uartDevice.read(buffer, buffer.length)) > 0) {
data = new String(buffer, StandardCharsets.UTF_8).substring(0, read);
System.out.println(String.format("%020x", new BigInteger(1, data.getBytes(/*YOUR_CHARSET?*/))));
} catch (IOException e) {
Log.w(TAG, "Unable to transfer data over UART", e);
}
Expected output is:
2a3619010101001a0708403031301010011214084030313010100112140845
Instead I am receiving:
2a361a010101001a070840303130101001121408403031
8403031301010011214084030313010100112140845
3031301010011214084030313010100112140845
If you want to write code that only prints the bytes that you get I would try the following:
if (uartDevice != null) {
// Loop until there is no more data in the RX buffer.
try {
byte[] buffer = new byte[CHUNK_SIZE];
int read;
while ((read = uartDevice.read(buffer, buffer.length)) > 0) {
for (int i = 0; i < read; i++) {
System.out.printf("%02x", buffer[i]);
}
}
} catch (IOException e) {
Log.w(TAG, "Unable to transfer data over UART", e);
}
System.out.println(); // Adds a newline after all bytes
}
The following is a method that takes a UartDevice as a parameter, reads from it until the end and returns a single byte array with the whole content. No arbitrary buffer that is guaranteed to hold the whole content is needed. The returned array is exactly as big as it needs to be. Only a small read buffer is used to increase performance. Error handling is ignored.
This assumes that the data is not larger than it fits into memory.
byte[] readFromDevice(UartDevice uartDevice) {
byte[] buffer = new byte[CHUNK_SIZE];
int read;
ByteArrayOutputStream data = new ByteArrayOutputStream();
while ((read = uartDevice.read(buffer, buffer.length)) > 0) {
data.write(buffer, 0, read);
}
return data.toByteArray();
}
The method returns when all data has been read and you can process the returned array at your leasure.
In trying to fulfill a partial range request using chrome as the video playback tool, the video playback start playing back but on reaching halfway, it freezes as if the client is still waiting on more data. At this point the server has already sent the entire request. Please observe the following requests and response and the code used to send the range requested:
Range:bytes=0-
Accept-Ranges:bytes
Content-Length:546827
Content-Range:bytes 0-546827/546828
Content-Type:video/webm
Accept-Ranges:bytes
Content-Length:6155
Content-Range:bytes 540672-546827/546828
Content-Type:video/webm
Accept-Ranges:bytes
Content-Length:1
Content-Range:bytes 546827-546827/546828
Content-Type:video/webm
Is the second response handled correctly? Cause it freezes on this request and starts making multiple requests again until the request times out.
Code handling the request:
private static void copyStream (PullBufferStream inputStream, OutputStream outputStream, BytesStreamObject bytesWrittenObj, int from, int to) throws IOException, CodecNotAvailableException {
//export media stream
BufferedOutputStream bout = new BufferedOutputStream (outputStream);
Buffer b = new Buffer();
long skipped =0;
int byteLength = 0;
do {
inputStream.read(b);
int len = b.getLength();
byte[] data = (byte[]) b.getData();
int offset = b.getOffset();
if (len > 0) {
if(skipped < from) {
// skip bytes until reach from position in inputstream
if(skipped + len <= from) {
// skip all bytes just read
skipped += len;
} else {
// skip only some of bytes read
offset += (from - skipped);
len -= (from - skipped);
skipped = from;
}
}
if(to >= 0) {
if(from + byteLength + len <= to) {
// use the full buffer
} else {
// trim len to needed bytes to be read to prevent exceeding the "to" input parameter
len = (to + 1) - (from + byteLength);
}
}
byteLength+= len;
bytesWrittenObj.bytesWritten = byteLength;
bout.write(data, offset, len);
//bout.write(data, from, end);
}
} while (!b.isEOM());
//fileProperties.setBytesLength(byteLength);
//bout.close();
}
I needed to ensure I flush the outputstream but also made some changes on the actual file size rather than using stream.available().
I coded this packet handler but I can imagine scenarios in which it will get stuck or won't be able to read incomplete data. My questions are:
Should I use two buffers, one for the current incoming data and other to append incomplete data to?
I'm being stupidly over-complicated?
Code:
byte[] buffer;
int bufferLength;
int bytesRead;
buffer = new byte[1024];
while (bluetoothConnected) {
try {
// Wait for packet header
if (mmInStream.available() >= 8) {
bufferLength = mmInStream.read(buffer);
bytesRead = 0;
// Parse every packet
while (true) {
int commandType = ByteBuffer.wrap(buffer, 0, 2).order(ByteOrder.LITTLE_ENDIAN).getShort();
int payloadSize = ByteBuffer.wrap(buffer, 2, 2).order(ByteOrder.LITTLE_ENDIAN).getShort();
int packetSize = PACKET_HEADER_SIZE + payloadSize;
// Break if payload is incomplete
if (bufferLength < (bytesRead + packetSize)) {
// Append to other buffer
break;
}
byte[] packet = new byte[packetSize];
System.arraycopy(buffer, bytesRead, packet, 0, packetSize);
parsePacketSequence(socket, packet);
bytesRead += packetSize;
// Break if all bytes are read
if (bufferLength == bytesRead)
{
break;
}
// Break if more bytes are needed
// Packet header incomplete
if ((bufferLength - bytesRead) < PACKET_HEADER_SIZE)
{
// Append to other buffer
break;
}
}
}
}
catch (IOException e) {
bluetoothConnected = false;
Log.d(TAG, "Error " + e);
break;
}
}
Should I use two buffers, one for the current incoming data and other to append incomplete data to?
No.
I'm being stupidly over-complicated?
Yes.
Here's a simple version using DataInputStream:
DataInputStream din = new DataInputStream(mmInStream);
while (bluetoothConnected) {
try {
// Read packet header
int commandType = swap(din.readShort());
int payloadSize = swap(din.readShort());
int packetSize = PACKET_HEADER_SIZE + payloadSize;
byte[] packet = new byte[packetSize];
din.readFully(packet);
parsePacketSequence(socket, packet);
}
catch (IOException e) {
bluetoothConnected = false;
Log.d(TAG, "Error " + e);
break;
}
}
The swap() method which converts a short in litte-endian byte order to Java byte order is left as an exercise for the reader.
NB I don't see how parsePacketSequence() can work if it doesn't know commandType.
E&OE
Is there a way to read all InputStream values at once without a need of using some Apache IO lib?
I am reading IR signal and saving it from the InputStream into the byte[] array. While debugging, I have noticed that it works only if I put a delay there, so that I read all bytes at once and then process it.
Is there a smarter way to do it?
CODE:
public void run() {
Log.i(TAG, "BEGIN mConnectedThread");
byte[] buffer = new byte[100];
int numberOfBytes;
removeSharedPrefs("mSharedPrefs");
// Keep listening to the InputStream while connected
while (true) {
try {
// Read from the InputStream
numberOfBytes = mmInStream.read(buffer);
Thread.sleep(700); //If I stop it here for a while, all works fine, because array is fully populated
if (numberOfBytes > 90){
// GET AXIS VALUES FROM THE SHARED PREFS
String[] refValues = loadArray("gestureBuffer", context);
if (refValues!=null && refValues.length>90) {
int incorrectPoints;
if ((incorrectPoints = checkIfGesureIsSameAsPrevious(buffer, refValues, numberOfBytes)) < 5) {
//Correct
} else {
//Incorrect
}
}
saveArray(buffer, numberOfBytes);
}else{
System.out.println("Transmission of the data was corrupted.");
}
buffer = new byte[100];
// Send the obtained bytes to the UI Activity
mHandler.obtainMessage(Constants.MESSAGE_READ, numberOfBytes, -1, buffer)
.sendToTarget();
} catch (IOException e) {
Log.e(TAG, "disconnected", e);
connectionLost();
// Start the service over to restart listening mode
BluetoothChatService.this.start();
break;
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Edit:
My old answer is wrong, see EJPs comment! Please don't use it. The behaviour of ByteChannels depend on wether InputStreams are blocking or not.
So this is why I would suggest, you just copy IOUtils.read from Apache Commons:
public static int read(final InputStream input, final byte[] buffer) throws IOException {
int remaining = buffer.length;
while (remaining > 0) {
final int location = buffer.length - remaining;
final int count = input.read(buffer, location, remaining);
if (count == -1) { // EOF
break;
}
remaining -= count;
}
return buffer.length - remaining;
}
Old answer:
You can use ByteChannels and read into a ByteBuffer:
ReadableByteChannel c = Channels.newChannel(inputstream);
ByteBuffer buf = ByteBuffer.allocate(numBytesExpected);
int numBytesActuallyRead = c.read(buf);
This read method is attempting to read as many bytes as there is remaining space in the buffer. If the stream ends before the buffer is fully filled, the number of bytes actually read is returned. See JavaDoc.
The result I'm getting is that files of the same type are returning the same md5 hash value. For example two different jpgs are giving me the same result. However, a jpg vs a apk are giving different results.
Here is my code...
public static String checkHashURL(String input) {
try {
MessageDigest md = MessageDigest.getInstance("MD5");
InputStream is = new URL(input).openStream();
try {
is = new DigestInputStream(is, md);
int b;
while ((b = is.read()) > 0) {
;
}
} finally {
is.close();
}
byte[] digest = md.digest();
StringBuffer sb = new StringBuffer();
for (int i = 0; i < digest.length; i++) {
sb.append(
Integer.toString((digest[i] & 0xff) + 0x100, 16).substring(
1));
}
return sb.toString();
} catch (Exception ex) {
throw new RuntimeException(ex);
}
}
This is broken:
while ((b = is.read()) > 0)
Your code will stop at the first byte of the stream which is 0. If the two files have the same values before the first 0 byte, you'll fail. If you really want to call the byte-at-a-time version of read, you want:
while (is.read() != -1) {}
The parameterless InputStream.read() method returns -1 when it reaches the end of the stream.
(There's no need to assign a value to b, as you're not using it.)
Better would be to read a buffer at a time:
byte[] ignoredBuffer = new byte[8 * 1024]; // Up to 8K per read
while (is.read(ignoredBuffer) > 0) {}
This time the condition is valid, because InputStream.read(byte[]) would only ever return 0 if you pass in an empty buffer. Otherwise, it will try to read at least one byte, returning the length of data read or -1 if the end of the stream has been reached.