Develop Reference

ByteBuffer put method adds 0 after ByteArray

I currently ran into a actually quite weird problem with the put method of java.nio.bytebuffer and I wondered if you might know the answer to it, so let's get to it.
My goal is it to concatenate some data to a bytebuffer. The problem is after I call the put method it always adds a 0 after the byte array.
Here is the method which has those side effects:
public ByteBuffer toByteBuffer() {
ByteBuffer buffer = ByteBuffer.allocateDirect(1024));
buffer.put(type.name().getBytes()); // 0 added here
data.forEach((key, value) -> {
buffer.putChar('|');
buffer.put(key.getBytes()); // 0 added here
buffer.putChar('=');
buffer.put(value.getBytes()); // 0 added here
});
buffer.flip();
return buffer;
}
Expected Output should look something like this:
ClientData|OAT=14.9926405|FQRIGHT=39.689075|.....
Actual Ouptut where _ represents the 0s:
ClientData_|OAT_=14.9926405_|FQRIGHT_=39.689075_|.....
The documentation doesn't say anything about this side effect.
Also the put method only puts a 0 in between the byte arrays but not at the very end of the buffer.
I assume the method might be wrong or at least not properly documented, but I really have no clue why it would behave that way.

I think you may be slightly misinterpreting what is happening here. I note your comment about "but not at the very end of the buffer".
The \0 is actually coming from the putChar(char) calls, not the put(byte[]) calls. As it says in the docs (emphasis added):
Writes two bytes containing the given char value, in the current byte order
The default byte order is big endian; given that the chars you are writing are in the 7-bit ASCII range, this means "the byte you want" is going to be preceded by 0x00.
If you want to write a byte, use put(byte):
buffer.put((byte) '|');

Reading in Partial InputStream

I'm looking to read an InputStream in sections because I need the first n bytes of the file and last m bytes as well as the contents between.
byte[] beginning = inputStream.readNBytes(16);
This works just fine, but to get the last m bytes, I tried the following:
byte[] middle = inputStream.readNBytes(inputStream.available() - 32);
byte[] end = inputStream.readNBytes(inputStream.available());
The end variable looks how I expect it to but not the middle variable, which ends up cutting out part of the stream.
I'm also a bit confused why the buf parameter size in the input stream doesn't seem to be equal to the byte array size when converting one to the other.
Anyway, I assume this isn't working how I want it to because (inputStream.available() - 32) is not adding up to a value compatible with readNBytes, so part of the stream is lost.
Is there a way to go about doing this?
EDIT: What I ended up doing which seemed to work(mostly) is when creating the file, to prepend both pieces I will later be extracting instead of prepending one and appending the other. That way I can just call inputStream.readAllBytes() on the last piece.
I also had to change where I'm writing to the file. I was writing to a CipherOutputStream when I should've been writing to the FileOutputStream and using that to create the Cipher OS.
Even after doing this I still have an extra 16 bytes at the end of the file, which confuses me, but I can easily ignore that last bit if I can't figure out why it's doing that.

Read all bytes from socket Stops at 52964 bytes

I'm making a Server that gets packages at 64 kb size.
int length = 65536;
byte[] bytes = new byte[length];
int pos = 0;
while(pos < length -1)
{
System.out.println("Before read");
pos += dis.read(bytes, pos, length-pos);
System.out.println(""+pos+" >> "+ length);
}
This is the code I use to read all bytes from the socket. Dis is a InputStream.
When I run the code 1 out of n goes wrong. The code only receives 52964 bytes instead of 65536 bytes.
I also checked the C code and it says it send 65536 bytes.
Does someone know what I'm doing wrong?

This is yet another case where Jakarta Commons IOUtils is a better choice than writing it yourself. It's one line of code, and it's fully tested. I recommend IOUtils.readFully() in this case.
If it does not read the entire buffer, then you know that you're not sending all the content. Perhaps you're missing a flush on the server side.

InputStream.read() returns the number of bytes read or -1 if the end of the stream has been reached. You need to check for that error condition. Also, I suspect your while(..) loop is the problem. Why are you calling it pos as in position? You may be terminating prematurely. Also, ensure that your C code, whatever it is doing, is sending properly. You can examine the network traffic with a tool like Wireshark to be sure.

What do you mean it "goes wrong"? What is the output? It can't be exiting the loop before reading the full 64 KB, so what really happens?
Also, it's better to save the return value of the I/O call separately and inspect it, before assuming the I/O was successful. If that's DataInputStream.read(), it returns -1 on error.

Your code is incorrect as it doesn't check for -1.
This is a case for using DataInputStream.readFully() rather than coding it yourself and getting it wrong.

Parsing a TCP packet

I'm having some trouble to parse a TCP packet from a socket...
In my protocol, my messages are like this:
'A''B''C''D''E'.........0x2300
'A''B''C''D''E' --> start message pattern
0x2300 --> two bytes end message
But due to the Nagle's algorithm, sometimes my messages are concatenated like:
'A''B''C''D''E'.........0x2300'A''B''C''D''E'.........0x2300'A''B''C''D''E'.........0x2300
I already tried to setNoDelay() to true but the problem persists.
I have the message in a byte[].
How could I split my messages to be parsed individually?
PS: For now I am able to get the first message but the others are lost...

Just loop through you received data and check for end-markers. When found set a start index to the next package and continue searching. Something like this:
int packageStart = 0;
for(int i = 0; i < data.length - 1; i++) {
if(data[i] == 0x23 && data[i + 1] == 0x00) {
// Found end of package
i++;
processPackage(data, packageStart, i);
packageStart = i;
}
// At this point: from packageStart till data.length are unprocessed bytes...
As noted, there might be some left over data (if data did not end with the end-marker). You might want to keep it, so you can prepend it to the next batch of received data. And thus preventing data-loss due to chopped up TCP/IP packages.

You have to think of it as parsing a continuous stream of bytes. Your code needs to identify the start and end of a message.
Due to the way packets get sent, you may have a complete message, multiple messages, a partial message, etc. You code needs to identify when a message has begun and keep reading until it has found the end of a message or in some instance, when you've read more bytes than your max message size and you need to resync.
I've seen some comm managers drop and reestablish the connection (start over) and others throw away data until they can get back in sync. Then you get into the fun of whether you need guaranteed delivery and retransmission.
The best protocols are the simple ones. Create a message header which contains say an SOH byte, a two byte message length (or whatever is appropriate), a 2 byte message type and 1 byte message subtype. You can also end the message with any number of bytes. Look at an ASCII chart, there's a number of Hex bytes 00-1F that are pretty standard since the terminal days.
No point in reinventing the wheel here. Makes it easier, because you know how long this message should be instead of looking for patterns in the data.

It sounds like you need to treat it like a Byte Stream and buffer the packets until you see your EOF code 0x2300.

how can I write to the byte buffer half way in java?

Suppose each time the buffer of the input-steam read 1000 bytes. There are some start signs and the video name at the beginning of the buffer, before the actual video content, like 100 byte. I don't want to write them into the result buffer. So the first time write 101-999 to the buffer. And the second time I hope to write 1000-1999. Currently, it write 0-999 again, and the result video has an 900 extra bytes before the actually video contents.
Is there anyway to write the buffer skipping the first buffer length? thanks!

I use this code for skipping bytes from a ByteBuffer:
import java.nio.ByteBuffer;
public class Utility {
public static void skip(ByteBuffer bb, int skip) {
bb.position( bb.position() + skip);
}
}

Sophia, you really do need to include example code so people can help, but I see from your tags you are likely asking about NIO's ByteBuffer.
What you want to do is skip the content you don't want by way of the ByteBuffer.position(int) method - there is no magic in the ByteBuffer impl, it is a backing data store (either a byte[] or direct memory reference to OS) and a series of int pointers that refer to conceptual positions in the buffer (start, end, limit, etc.) -- you just want to make sure you "skip" the bytes you don't want, which can be done by moving the position beyond it so the next operation to write out the entire buffer will start from position and go to limit.