I'm working on the server for a game right now. The server's packet reading loop is blocking, and typically waits until a packet is received to continue through the loop. However, if the client disconnects, the DataInputStream returns a single byte (-1) and the loop is executed in rapid succession, as is expected. However, I don't use the DataInputStream's read() method to read one byte at a time, I use the read(byte[]) method to read them all at once into a byte array. As such, I can't easily detect if the stream is returning a single byte valued at -1.
Possible Solution: I could check if the first byte of the array is -1, and if so loop through the array to see if the rest of the array is nothing but zeroes. Doing this seems extremely inefficient however, and I feel that it would affect performance as client count increases.
Here's a simplified version of my packet-reading loop:
while (!thread.isInterrupted() && !isDisconnected())
{
try
{
byte[] data = new byte[26];
data = new byte[26];
input.read(data);
//Need to check if end of stream here somehow
Packet rawPacket = Packet.extractPacketFromData(data); //Constructs packet from the received data
if(rawPacket instanceof SomePacket)
{
//Do stuff with packet
}
}
catch(IOException e)
{
disconnectClient(); //Toggles flag showing client has disconnected
}
}
Your understanding of read(byte[]) is incorrect. It doesn't set a value in your array to -1.
The Javadoc says:
Returns:
the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached.
You need to check the return value:
int bytesRead = input.read(data);
if (bytesRead == -1)
{
// it's end of stream
}
As a side note, even when just reading data normally, you do need to check the number of bytes read was the number of bytes you requested. The call to read is not guaranteed to actually fill your array.
You should take a look at readFully() which does read fully, and throws an EOFException for end of stream:
Reads some bytes from an input stream and stores them into the buffer array b. The number of bytes read is equal to the length of b.
This method blocks until one of the following conditions occurs:
b.length bytes of input data are available, in which case a normal return is made.
End of file is detected, in which case an EOFException is thrown.
An I/O error occurs, in which case an IOException other than EOFException is thrown.
Related
I've read the java docs and a number of related questions but am unsure if the following is guaranteed to work:
I have a DataInputStream on a dedicated thread that continually reads small amounts of data, of known byte-size, from a very active connection. I'd like to alert the user when the stream becomes inactive (i.e. network goes down) so I've implemented the following:
...
streamState = waitOnStreamForState(stream, 4);
int i = stream.readInt();
...
private static int
waitOnStreamForState(DataInputStream stream, int nBytes) throws IOException {
return waitOnStream(stream, nBytes, STREAM_ACTIVITY_THRESHOLD, STREAM_POLL_INTERVAL)
? STREAM_STATE_ACTIVE
: STREAM_STATE_INACTIVE;
private static boolean
waitOnStream(DataInputStream stream, int nBytes, long timeout, long pollInterval) throws IOException {
int timeWaitingForAvailable = 0;
while( stream.available() < nBytes ){
if( timeWaitingForAvailable >= timeout && timeout > 0 ){
return false;
}
try{
Thread.sleep(pollInterval);
}catch( InterruptedException e ){
Thread.currentThread().interrupt();
return (stream.available() >= nBytes);
}
timeWaitingForAvailable += pollInterval;
}
return true;
}
The docs for available() explain:
Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next caller of a method for this input stream. The next caller might be the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes.
Does this mean it's possible the next read (inside readInt()) might only, for instance, read 2 bytes, and the subsequent read to finish retrieving the Integer could block? I realize readInt() is a method of the stream 'called next' but I presume it has to loop on a read call until it gets 4 bytes and the docs don't mention subsequent calls. In the above example is it possible that the readInt() call could still block even if waitOnStreamForState(stream, 4) returns STREAM_STATE_ACTIVE?
(and yes, I realize my timeout mechanism is not exact)
Does this mean it's possible the next read (inside readInt()) might only, for instance, read 2 bytes, and the subsequent read to finish retrieving the Integer could block?
That's what it says. However at least the next read() won't block.
I realize readInt() is a method of the stream 'called next' but I presume it has to loop on a read call until it gets 4 bytes and the docs don't mention subsequent calls. In the above example is it possible that the readInt() call could still block even if waitOnStreamForState(stream, 4) returns STREAM_STATE_ACTIVE?
That's what it says.
For example, consider SSL. You can tell that there is data available, but you can't tell how much without actually decrpyting it, so a JSSE implementation is free to:
always return 0 from available() (this is what it used to do)
always return 1 if the underlying socket's input stream has available() > 0, otherwise zero
return the underlying socket input stream's available() value and rely on this wording to get it out of trouble if the actual plaintext data is less. (However the correct value might still be zero, if the cipher data consisted entirely of handshake messages or alerts.)
However you don't need any of this. All you need is a read timeout, set via Socket.setSoTimeout(), and a catch for SocketTimeoutException. There are few if any correct uses of available(): fewer and fewer as time goes on, it seems to me. You should certainly not waste time calling sleep().
The client reads the status byte in the loop while it is equal to 0x01:
do {
input.read(magicWord); // ALWAYS THE SAME BYTE AFTER THE SERVER IS GONE
if (magicWord[0] == (byte) 0xFF)
break;
// retrieve the progress
byte[] cur = new byte[4];
input.read(cur);
// and set the progress bar
progressBar.setValue(ByteBuffer.wrap(cur).getInt());
} while (!isCancelled());
But if the server crashes (for example by SIGKILL) while the client is in this loop, the client keeps getting the last magickWord sent by the server over and over again (and gets into the endless loop). Even setting SO_TIMEOUT doesn't solve the problem. How should the client detect a server failure?
It doesn't read anything at all. input.read(magicWord) return -1. You 0xFF byte in your buffer since previous read (read method doesn't clear buffer).
That call to read returns the number of bytes read. You do not check that result.
And you do not clear your read buffer - thus you keep seeing the same bytes all the time.
Your code is wrong.
You must check the return value of the read call. Look it up in the documentation and write your code accordingly.
Why does this part of my client code is always zero ?
InputStream inputStream = clientSocket.getInputStream();
int readCount = inputStream.available(); // >> IS ALWAYS ZERO
byte[] recvBytes = new byte[readCount];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int n = inputStream.read(recvBytes);
...
Presumably it's because no data has been received yet. available() tries to return the amount of data available right now without blocking, so if you call available() straight after making the connection, I'd expect to receive 0 most of the time. If you wait a while, you may well find available() returns a different value.
However, personally I don't typically use available() anyway. I create a buffer of some appropriate size for the situation, and just read into that:
byte[] data = new byte[16 * 1024];
int bytesRead = stream.read(data);
That will block until some data is available, but it may well return read than 16K of data. If you want to keep reading until you reach the end of the stream, you need to loop round.
Basically it depends on what you're trying to do, but available() is rarely useful in my experience.
From the java docs
Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream. The next invocation might be the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes.
Note that while some implementations of InputStream will return the total number of bytes in the stream, many will not. It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream.
A subclass' implementation of this method may choose to throw an IOException if this input stream has been closed by invoking the close() method.
The available method for class InputStream always returns 0.
This method should be overridden by subclasses.
Here is a note to understand why it returns 0
In InputStreams, read() calls are said to be "blocking" method calls. That means that if no data is available at the time of the method call, the method will wait for data to be made available.
The available() method tells you how many bytes can be read until the read() call will block the execution flow of your program. On most of the input streams, all call to read() are blocking, that's why available returns 0 by default.
However, on some streams (such as BufferedInputStream, that have an internal buffer), some bytes are read and kept in memory, so you can read them without blocking the program flow. In this case, the available() method tells you how many bytes are kept in the buffer.
According to the documentation, available() only returns the number of bytes that can be read from the stream without blocking. It doesn't mean that a read operation won't return anything.
You should check this value after a delay, to see it increasing.
There are very few correct uses of available(), and this isn't one of them. In this case no data had arrived so it returned zero, which is what it's supposed to do.
Just read until you have what you need. It will block until data is available.
For good or bad I have been using code like the following without any problems:
ZipFile aZipFile = new ZipFile(fileName);
InputStream zipInput = aZipFile.getInputStream(name);
int theSize = zipInput.available();
byte[] content = new byte[theSize];
zipInput.read(content, 0, theSize);
I have used it (this logic of obtaining the available size and reading directly to a byte buffer)
for File I/O without any issues and I used it with zip files as well.
But recently I stepped into a case that the zipInput.read(content, 0, theSize); actually reads 3 bytes less that the theSize available.
And since the code is not in a loop to check the length returned by zipInput.read(content, 0, theSize); I read the file with the 3 last bytes missing
and later the program can not function properly (the file is a binary file).
Strange enough with different zip files of larger size e.g. 1075 bytes (in my case the problematic zip entry is 867 bytes) the code works fine!
I understand that the logic of the code is probably not the "best" but why am I suddenly getting this problem now?
And how come if I run the program immediately with a larger zip entry it works?
Any input is highly welcome
Thanks
From the InputStream read API docs:
An attempt is made to read as many as len bytes, but a smaller number
may be read.
... and:
Returns: the total number of bytes read into the buffer, or -1 if
there is no more data because the end of the stream has been reached.
In other words unless the read method returns -1 there is still more data available to read, but you cannot guarantee that read will read exactly the specified number of bytes. The specified number of bytes is the upper bound describing the maximum amount of data it will read.
Using available() does not guarantee that it counted total available bytes to the end of stream.
Refer to Java InputStream's available() method. It says that
Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream. The next invocation might be the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes.
Note that while some implementations of InputStream will return the total number of bytes in the stream, many will not. It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream.
An example solution for your problem can be as follows:
ZipFile aZipFile = new ZipFile(fileName);
InputStream zipInput = aZipFile.getInputStream( caImport );
int available = zipInput.available();
byte[] contentBytes = new byte[ available ];
while ( available != 0 )
{
zipInput.read( contentBytes );
// here, do what ever you want
available = dis.available();
} // while available
...
This works for sure on all sizes of input files.
The best way to do this should be as bellows:
public static byte[] readZipFileToByteArray(ZipFile zipFile, ZipEntry entry)
throws IOException {
InputStream in = null;
try {
in = zipFile.getInputStream(entry);
return IOUtils.toByteArray(in);
} finally {
IOUtils.closeQuietly(in);
}
}
where the IOUtils.toByteArray(in) method keeps reading until EOF and then return the byte array.
I have a socketChannel configured as blocking, but when reading byte buffers of 5K from this socket, I get an incomplete buffer sometimes.
ByteBuffer messageBody = ByteBuffer.allocate(5*1024);
messageBody.mark();
messageBody.order(ByteOrder.BIG_ENDIAN);
int msgByteCount = channel.read(messageBody);
Ocasionally, messageBody is not completely filled and channel.read() does not return -1 or an exception, but the actual number of bytes read (which is less than 5k).
Has anyone experienced a similar problem?
That's how reads work. The SocketChannel documentation says:
A read operation might not fill the buffer, and in fact it might not read any bytes at all. [...] It is guaranteed, however, that if a channel is in blocking mode and there is at least one byte remaining in the buffer then this method will block until at least one byte is read [emphasis added].
When you use sockets you must anticipate that the socket might transfer fewer bytes than you expect. You must loop on the .read method to get the remainder of the bytes.
This is also true when you send bytes through a socket. You must check how many bytes were sent, and loop on the send until all bytes have been sent.
This behavior is due to the network layers splitting the messages into multiple packets. If your messages are short, then you are less likely to encounter this. But you should always code for it.
With 5k bytes per buffer you are very likely to see the sender's message spit into multiple packets. Each read operation will receive one packet, which is only part of your message.
TCP/IP sends the information in packets, they are not always all available when you do the read, therefore you must do the read in a loop.
char [] buffer = new char[1024];
int chars_read;
try
{
while((chars_read = from_server.read(buffer)) != -1)
{
to_user.write(buffer,0,chars_read);
to_user.flush();
}
}
catch(IOException e)
{
to_user.println(e);
}
See this post