Develop Reference

How to deal with a slow consumer in traditional Java NIO?

So, I've been brushing up my understanding of traditional Java non-blocking API. I'm a bit confused with a few aspects of the API that seem to force me to handle backpressure manually.
For example, the documentation on WritableByteChannel.write(ByteBuffer) says the following:
Unless otherwise specified, a write operation will return only after
writing all of the requested bytes. Some types of channels,
depending upon their state, may write only some of the bytes or
possibly none at all. A socket channel in non-blocking mode, for
example, cannot write any more bytes than are free in the socket's
output buffer.
Now, consider this example taken from Ron Hitchens book: Java NIO.
In the piece of code below, Ron is trying to demonstrate how we could implement an echo response in a non-blocking socket application (for context here's a gist with the full example).
//Use the same byte buffer for all channels. A single thread is
//servicing all the channels, so no danger of concurrent access.
private ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
protected void readDataFromSocket(SelectionKey key) throws Exception {
var channel = (SocketChannel) key.channel();
buffer.clear(); //empty buffer
int count;
while((count = channel.read(buffer)) > 0) {
buffer.flip(); //make buffer readable
//Send data; don't assume it goes all at once
while(buffer.hasRemaining()) {
channel.write(buffer);
}
//WARNING: the above loop is evil. Because
//it's writing back to the same nonblocking
//channel it read the data from, this code
//can potentially spin in a busy loop. In real life
//you'd do something more useful than this.
buffer.clear(); //Empty buffer
}
if(count < 0) {
//Close channel on EOF, invalidates the key
channel.close();
}
}
My confusion is on the while loop writing into output channel stream:
//Send data; don't assume it goes all at once
while(buffer.hasRemaining()) {
channel.write(buffer);
}
It really confuses me how NIO is helping me here. Certainly the code may not be blocking as per the description of the WriteableByteChannel.write(ByteBuffer), because if the output channel cannot accept any more bytes because its buffer is full, this write operation does not block, it just writes nothing, returns, and the buffer remains unchanged. But --at least in this example-- there is no easy way to use the current thread in something more useful while we wait for the client to process those bytes. For all that matter, if I only had one thread, the other requests would be piling up in the selector while this while loop wastes precious cpu cycles “waiting” for the client buffer to open some space. There is no obvious way to register for readiness in the output channel. Or is there?
So, assuming that instead of an echo server I was trying to implement a response that needed to send a big number of bytes back to the client (e.g. a file download), and assuming that the client has a very low bandwidth or the output buffer is really small compared to the server buffer, the sending of this file could take a long time. It seems as if we need to use our precious cpu cycles attending other clients while our slow client is chewing our file download bytes.
If we have readiness in the input channel, but not on the output channel, it seems this thread could be using precious CPU cycles for nothing. It is not blocked, but it is as if it were since the thread is useless for undetermined periods of time doing insignificant CPU-bound work.
To deal with this, Hitchens' solution is to move this code to a new thread --which just moves the problem to another place--. Then I wonder, if we had to open a thread every time we need to process a long running request, how is Java NIO better than regular IO when it comes to processing this sort of requests?
It is not yet clear to me how I could use traditional Java NIO to deal with these scenarios. It is as if the promise of doing more with less resources would be broken in a case like this. What if I were implementing an HTTP server and I cannot know how long it would take to service a response to the client?
It appears as if this example is deeply flawed and a good design of the solution should consider listening for readiness on the output channel as well, e.g.:
registerChannel(selector, channel, SelectionKey.OP_WRITE);
But how would that solution look like? I’ve been trying to come up with that solution, but I don’t know how to achieve it appropriately.
I'm not looking for other frameworks like Netty, my intention is to understand the core Java APIs. I appreciate any insights anyone could share, any ideas on what is the proper way to deal with this back pressure scenario just using traditional Java NIO.

NIO's non-blocking mode enables a thread to request reading data from a channel, and only get what is currently available, or nothing at all, if no data is currently available. Rather than remain blocked until data becomes available for reading, the thread can go on with something else.
The same is true for non-blocking writing. A thread can request that some data be written to a channel, but not wait for it to be fully written. The thread can then go on and do something else in the meantime.
What threads spend their idle time on when not blocked in IO calls, is usually performing IO on other channels in the meantime. That is, a single thread can now manage multiple channels of input and output.
So I think you need to rely on the design of the solution by using a design pattern for handling this issue, maybe **Task or Strategy design pattern ** are good candidates and according to the framework or the application you are using you can decide the solution.
But in most cases you don't need to implement it your self as it's already implemented in Tomcat, Jetty etc.
Reference : Non blocking IO

Why does my in.read(); function inconsistently only when positioned after a bin.readLine(), even though both have consistent input?

So I've been having some real problems with a multi-client server application I've been coding. After lots of fiddling around I came to realise that although an in.read() method that I'd positioned after a bin.readLine usually receives its information, occationally it just hangs as though the information had never been sent. This is even though all information has been sent in exactly the same way in the same order. To simplify things I just wrote a very simple client server program to test this situation.
Both the client and the server use while (true){ to loop continually, and both have System.out.println(here1) or (here2) etc. to allow me to see where in the code any unexpected behavior has arisen. So this is the while loop on the client side. The in.read(); at the bottom of the loop is only there to block until the server's while loop also reaches the bottom, so that the client doesn't loop faster than the server.
while (true){
System.out.println("here1");
pout.println("Hello everybody");
System.out.println("here2");
out.write(150);
System.out.println("here3");
in.read();
}
And this is the server's while loop.
while (true){
System.out.println("here1");
String a = bin.readLine();
System.out.println("here2");
int b = in.read();
System.out.println("here3");
System.out.println(a);
System.out.println(b);
out.write(1);
}
Some when I run the applictions, everything behaves as expected, both looping round and printing a continually repeating list of here1..2..3. And on the server side here1..2..3. followed by Hello Everybody, 150. However all of a sudden after a second or two (sometimes less, sometimes more - it's very inconsistent) the server blocks just after here2, and so it seems that it is waiting for read() to receive information. However on the client side you can see that everything was sent as normal and the client is waiting just after here3.
I don't understand this at all, particularly the inconsistency. If I replace pout.println() with out.write(), and replace bin.readLine() with in.read(), this problem never happens and the two programs loop around forever like no ones business. I would really like to know why this happens!

The buffered reader 'bin' is buffering data that you are trying to read via 'in'. You can't do that: you can't mix buffered and unbuffered I outnumber the same underlying source.

Using the java BufferedRead in Matlab with an TCP/IP Ethernet connection - accessing the most recent data

I am currently writing a MATLAB code to control instruments mounted on the end of a robotic arm. I am accessing the robot's positional data over a TCP/IP ethernet connection. The coms are working. The data is transmitted in XML file packets. I have written a parsing script to extract the correct data from the stream.
However, when I try to add more commands (for example to control the instruments) my code does not work correctly. The issue is that the buffer fills up with a certain (large) amount of data from the robot and when I call the in.readLine command, it reads the next line of the buffer, not the most recent line received over the connection. If the program has been busy doing other things (simulated by the pause(0.01) command, then the positional data displayed can be quite behind where the robot actually is. If the robot stays stationary, the data eventually catches up though. There is no command to flush a BufferedReader, and I cannot find a way of using the reset and mark commands to work as this seems to reset to an earlier string of data in the buffer. Is there a way to access the most recent data in the buffer which is constantly being updated from the ethernet communications?
I cannot delete the buffer and create a new one as this breaks the connection to the robot and causes its program to fail.
Apologies for the code below, it's a work in progress so excuse its rough-and-readiness. The code that is commented out using the mark and reset commands is just one attempt at using them I tried. I also tried playing around with the size of the buffer so it would constantly overflow and "flush" itself but even with the buffer size set to its smallest of 1, the problem still persisted.
clear all; close all; clc;
import java.net.*;
import java.io.*;
import java.lang.System;
PORT = 6008;
try
server_socket = ServerSocket(PORT);
disp('Server started, awaiting connections...')
client_socket = server_socket.accept;
in = BufferedReader(InputStreamReader(client_socket.getInputStream));
% m = in.markSupported;
% if(m == 1)
% disp('Mark supported');
% end
% in.mark(8192);
while 1
XMLdata = in.readLine; %Reads the next line in the buffer
[test, bin]=strtok(char(XMLdata)); %
if strcmp(test, '<RIst') == 1 %Tests to see if it is a line with positional data
rsi_parse %If so, parses the data into an array
% in.reset;
% in.mark(8192);
end
pause(0.01) %Pause to simulate other commands which cause the buffer to fill up
end
catch
disp('Error. Closing connection.')
closeConnections(server_socket, client_socket, in);
end

Currently you are using MATLAB's ability to run Java as a scripting language, which is a neat feature. While doing that you inherit MATLAB's "only" major limitation, it is single threaded. There are workarounds like timers, but at it's core there is only one thread.
In order the get the latest data from your robot, you seem to be forced to continuously read from the data stream. This can be achieved by creating a pure Java class that spans a thread, that continuously reads from your socket and updates a class property with the latest data.
The Java class must be compiled using javac in order to generate a class file that can be instantiated as an object in MATLAB. Apart from a start and stop method it should provide a getLatestDate() method that's called from the MATLAB script in order to get the latest data.

It sounds like the question is: "how can I insure that I am reading the most recent data from my input stream, discarding data if needed".
If so, then you will want to expand your data ingestion portion of the code with an inner while loop:
while 1
%
% Read until we find the end of the buffer
%
XMLdata = in.readLine; %Reset current line
foundData = nan; %Reset value found
while ~isempty(XMLdata) %Loop to the end of the buffer
[test, bin]=strtok(char(XMLdata)); %Line testing, as before
if strcmp(test, '<RIst') == 1
foundData = rsi_parse %Store the found data, we may get a more recent version
end
XMLdata = in.readLine; %Try and read another line
end
%Do something with foundData
%
%Go and perform other work
%
pause(0.01) %Go and do something else
end
Related to your comment that you cannot "purge" a BufferedReader, I think that you can, like this:
charactersSkipped = in.skip(1000000000);
(If you were writing directly in Java you should use in.skip(java.lang.Long.MAX_VALUE);, but that breaks the Matlab/Java interface, and is sent to Java as a negative number, so you just need to pick the largest number you are comfortable typing.)
But after you have purged it there is no clear way to back up until you find the character pattern you are looking for.

How to reliably tell when finished reading from a socket in Java?

We are integrating with an external product that requires us to communicate with it using Java sockets. We've been able to read small responses from the server with no issues, but larger responses are causing some headaches.
I made some changes to the socket handling logic and now we seem to be able to read large responses 90% of the time. It does still fail occasionally. Failure, in this case, means that the Java client stops reading from the socket before the entire response has been read. The client thinks that the read operation is finished, and stops normally - there are no exceptions or timeouts involved.
Here's what the current logic looks like:
StringWriter response = new StringWriter();
PrintWriter writer = new PrintWriter(response);
char[] buf = new char[4096];
int readChars;
do {
readChars = responseBufferedReader.read(buf);
writer.write(buf, 0, readChars);
} while(readChars != -1 && responseBufferedReader.ready());
responseBufferedReader is a BufferedReader wrapped around an InputStreamReader wrapped around the Socket's InputStream.
This code works most of the time, but it seems like checking for readChars != -1 and ready() are not reliable enough to indicate if we've read all of the content from the server. Comparing the number of read characters to the buffer size is also not reliable, since the server seems to be a little slow at sending the response back causing these numbers to differ.
I've tried changing the size of the character buffer; it helped, but it's still not working 100% of the time.
Is there a better and more reliable way to read entirely from a Socket without knowing the size of the expected response? I've been doing some research on SocketChannels, but I'm not sure if there's any benefit to be had by switching.
In case it helps, we're making a single, blocking Socket connection to the server. The Socket is configured for a 100 second timeout

You shouldn't be checking whether the BufferedReader is ready() to tell if you're done. It's possible that bytes are still being read off of the wire and the BufferedReader has nothing for you, but the Socket is not yet closed.
I'm not sure what value the BufferedReader is getting you (well, it might help your performance). I would expect the following to work better:
StringWriter response = new StringWriter();
PrintWriter writer = new PrintWriter(response);
char[] buf = new char[4096];
int readChars;
do {
readChars = inputStreamReader.read(buf);
writer.write(buf, 0, readChars);
} while(readChars != -1);
I think that you're probably dropping out of the loop when the network is the bottleneck - the loop processes the results fast enough so that the Reader isn't ready yet and you assume that you're done reading (even though you're not).

See the answers to the question at Java blocking socket returning incomplete ByteBuffer
If the network breaks the message into packets, you will drop out of your loop before reading all packets.

See my answer to this question. The same answer applies whenever you are reading from a stream, and the only reliable way to know when you're done if EOF doesn't apply (as in this case) is to encode the end-of-message in the protocol somehow -- how you do it is up to you.
Edit:
If you can't modify the protocol to include message boundaries, you might investigate a timeout-based approach. You have to decide how long to wait before detecting the end of a message... but then you have to handle the case where you get two messages in the same stream. Can your code handle this situation now? If so, then your answer is to buffer the input and use that logic to detect record boundaries.

Java NIO: Sending large messages quickly leads to truncated packets and data loss

I've got this nasty problem where sending multiple, large messages in quick succession from a Java (NIO) server (running Linux) to a client will lead to truncated packets. The messages have to be large and sent very rapidly for the problem to occur. Here's basically what my code is doing (not actual code, but more-or-less what's happening):
//-- setup stuff: --
Charset charset = Charset.forName("UTF-8");
CharsetEncoder encoder = charset.newEncoder();
String msg = "A very long message (let's say 20KB)...";
//-- inside loop to handle incoming connections: --
ServerSocketChannel ssc = (ServerSocketChannel)key.channel();
SocketChannel sc = ssc.accept();
sc.configureBlocking(false);
sc.socket().setTcpNoDelay(true);
sc.socket().setSendBufferSize(1024*1024);
//-- later, actual sending of messages: --
for (int n=0; n<20; n++){
ByteBuffer bb = encoder.encode(CharBuffer.wrap(msg+'\0'));
sc.write(bb);
bb.rewind();
}
So, if the packets are long enough and sent as quickly as possible (i.e. in a loop like this with no delay), then on the other end it often comes out something like this:
[COMPLETE PACKET 1]
[COMPLETE PACKET 2]
[COMPLETE PACKET 3]
[START OF PACKET 4][SOME OR ALL OF PACKET 5]
There is data loss, and the packets start to run together, such that the start of packet 5 (in this example) arrives in the same message as the start of packet 4. It's not just truncating, its running the messages together.
I imagine that this is related to the TCP buffer or "window size", or that the server here is just providing data faster than the OS, or network adapter, or something, can handle it. But how do I check for, and prevent it from happening? If I reduce the length of message per use of sc.write(), but then increase the repetitions, I'll still run into the same problem. It seems to simply be an issue with the amount of data in a short amount of time. I don't see that sc.write() is throwing any exceptions either (I know that in my example above I'm not checking, but have in my tests).
I'd be happy if I could programmatically check if it is not ready for more data yet, and put in a delay, and wait until it is ready. I'm also not sure if "sc.socket().setSendBufferSize(1024*1024);" has any effect, or if I'd need to adjust this on the Linux side of things. Is there a way to really "flush" out a SocketChannel? As a lame workaround, I could try to explicitly force a complete send of anything that is buffered any time I'm trying to send a message of over 10KB, for example (which is not that often in my application). But I don't know of any way to force a send of the buffer (or wait until it has sent). Thanks for any help!

There are many reasons why sc.write() would not send some or all of the data. You have to check the return value and/or the number of bytes remaining in the buffer.
for (int n=0; n<20; n++){
ByteBuffer bb = encoder.encode(CharBuffer.wrap(msg+'\0'));
if(sc.write(bb) > 0 && bb.remaining() == 0) {
// all data sent
} else {
// could not send all data.
}
bb.rewind();
}

You are not checking the return value of:
sc.write(bb);
This returns the number of bytes written, which might be less than the data available in your buffer. Because of how nio works you can probably just call remaining() on your bytebuffer to see if there are any left.

I haven't done any NIO programming, but according to the Javadocs, sc.write() will not write the entire ByteBuffer if the SocketChannel is in non-blocking mode (as yours is) and the socket's output buffer is full.
Because you are writing so quickly, it is very likely that you are flooding your connection and your network or receiver cannot keep up.
I'd be happy if I could programmatically check if it is not ready for more data yet
You need to check the return value of sc.write() to find out whether your output buffer is full.

Don't assume you have any control over what data ends up in which packet.
More here: What's the best way to monitor a socket for new data and then process that data?

You are using the non-blocking mode: sc.configureBlocking(false);
Set blocking to true and your code should work as it is. Suggestions made by others here to check send count and loop will also work.