Develop Reference

How to consume a process' stdout as a stream, without blocking?

In Java (or clojure) I would like to spin up an external process and consume its stdout as a stream. Ideally, I would like to consume the process' output stream every time that the external process flushes it, but am not sure how that can be accomplished, and how it can be accomplished without blocking.
Going around consuming a Java ProcessPipeInputStream for a shelled out process (for example a Unix ProcessPipeInputStream), I find the inherited InputStream methods a bit low-level to work with, and am not sure if there's a non-blocking way to consume from the stream every time the producer-side flushes or otherwise in a non-blocking fashion.
Many code examples block on the output stream in an infinite loop, thereby hogging a thread for the listening. My hope is this blocking behavior can be avoided altogether.
Bottom line:
Is there a non-blocking way to be notified on an input stream, every time that the producing side of it flushes?

You need to create a separate Thread that would consume from such a stream allowing the rest of your program to do whatever is meant to be do doing in parallel.
class ProcessOutputReader implements Runnable {
private InputStream processOutput;
public ProcessOutputReader(final InputStream processOutput) {
this.processOutput = processOutput;
}
#Override
public void run() {
int nextByte;
while ((nextByte = processOutput.read()) != -1) {
// do whatever you need to do byte-by-byte.
processByte(nextByte);
}
}
}
class Main {
public static void main(final String[] args) {
final Process proc = ...;
final ProcessOutputReader reader = new ProcessOutputReader(proc.getInputStream());
final Thread processOutputReaderThread = new Thread(reader);
processOutputReaderThread.setDaemon(true); // allow the VM to terminate if this is the only thread still active.
processOutputReaderThread.start();
...
// if you wanna wait for the whole process output to be processed at some point you can do this:
try {
processOutputReaderThread.join();
} catch (final InterruptedException ex) {
// you need to decide how to recover from if your wait was interrupted.
}
}
}
If instead of processing byte-by-byte you want to deal with each flush as a single piece... I'm not sure there is 100% guaranteed to be able tocapture each process flush. After all the process own's IO framework software (Java, C, Python, etc.) may process the "flush" operation differently and perhaps what you end up receiving is multiple blocks of bytes for any given flush in that external process.
In any case you can attempt to do that by using the InputStream's available method like so:
#Override
public void run() {
int nextByte;
while ((nextByte = processOutput.read()) != -1) {
final int available = processOutput.available();
byte[] block = new byte[available + 1];
block[0] = nextByte;
final int actuallyAvailable = processOutput.read(block, 1, available);
if (actuallyAvailable < available) {
if (actuallyAvailable == -1) {
block = new byte[] { nextByte };
} else {
block = Arrays.copyOf(block, actuallyAvailable + 1);
}
}
// do whatever you need to do on that block now.
processBlock(block);
}
}
I'm not 100% sure of this but I think that one cannot trust that available will return a guaranteed lower bound of the number of bytes that you can retrieve without being block nor that the next read operation is going to return that number of available bytes if so requested; that is why the code above checks on the actual number of bytes read (actuallyAvailable).

Does this non-standard Java synchronization pattern work?

Let's say I have two threads running like this:
Thread A which performs computation while updating pixels of a shared image
Thread B periodically reads the image and copies it to the screen
Thread A performs work quickly, say 1 million updates per second, so I suspect it would be a bad idea to lock and unlock on a lock/mutex/monitor that often. But if there is no lock and no way of establishing a happens-before relation from thread A to thread B, then by the Java memory model (JMM spec) thread B is not guaranteed at all to see any of A's updates to the image.
So I was thinking that the minimum solution is for threads A and B to both synchronize periodically on the same shared lock, but not actually perform any work while inside the synchronized block - this is what makes the pattern non-standard and dubious. To illustrate in half-real half-pseudo code:
class ComputationCanvas extends java.awt.Canvas {
private Object lock = new Object();
private int[] pixels = new int[1000000];
public ComputationCanvas() {
new Thread(this::runThreadA).start();
new Thread(this::runThreadB).start();
}
private void runThreadA() {
while (true) {
for (1000 steps) {
update pixels directly
without synchornization
}
synchronized(lock) {} // Blank
}
}
private void runThreadB() {
while (true) {
Thread.sleep(100);
synchronized(lock) {} // Blank
this.repaint();
}
}
#Override
public void paint(Graphics g) {
g.drawImage(pixels, 0, 0);
}
}
Does adding empty synchronization blocks in this way correctly achieve the effect of transferring data from thread A to thread B? Or is there some other solution I failed to imagine?

Yes it works. But it works horribly.
Happens before only works when the release of the writer happens before the acquire of the reader. Your implementation assumes that whatever you're writing will complete before the subsequent reading/updating from ThreadB. Causing your data to be flushed all the time by synchronized will cause performance problems, although to what extent I cannot say for sure. Sure, you've made your synchronization finer grained, have you tested it yet?
A better solution might use a singleton/transfer SPSC (single producer/single consumer) queue to store the current snapshot of the writing thread and use that whenever you update.
int[] data = ...
Queue<int[]> queue = new ...
// Thread A
while (true) {
for (1000 iterations or so) {
...
}
queue.add(data);
}
// Thread B
while (true) {
int[] snapshot = queue.take();
this.repaint();
}
The advantage of this is that you don't need to busywait, you can just wait for the queue to block or until the next write. You can skip writes that you don't have time to update. You don't need to depend on the arbitrary thread scheduler to plan data flushes for you.
Remember that thread-safe data structures are great for passing data between threads.
Edit: oops, forgot to say that depending on how your updates go, you might want to use an array copy to prevent your data from being garbled from random writes that aren't cached.

Jetty - Possible memory leak when using websockets and ByteBuffer

I'm using Jetty 9.3.5.v20151012 to deliver a large number of events to clients using websockets. The events consist of 3 parts: a number, an event type and a timestamp and each event is serialized as byte[] and sent using ByteBuffer.
After a certain number of hours/days, depending on the number of clients, I notice an increase in heap memory and without any possibility for the GC to recover it.
When the heap (set to 512MB) is almost full, the memory used by the jvm is about 700-800 MB and the CPU is at 100% (it seams like the GC is trying very often to clean up). At the beginning, when I start Jetty, the memory is at about 30MB when calling the GC but after some time, this number increases more and more. Eventually the process is killed.
I'm using jvisualvm as profiler for memory leak debug and I've attached some screenshots of the head dump:
Here is the main code that handles the message sending using ByteBuffer:
I basically have a method that creates a byte[] (fullbytes) for all events that need to be sent in one message:
byte[] numberBytes = ByteBuffer.allocate(4).putFloat(number).array();
byte[] eventBytes = ByteBuffer.allocate(2).putShort(event).array();
byte[] timestampBytes = ByteBuffer.allocate(8).putDouble(timestamp).array();
for (int i = 0; i < eventBytes.length; i++) {
fullbytes[i + scount*eventLength] = eventBytes[i];
}
for (int i = 0; i < numberBytes.length; i++) {
fullbytes[eventBytes.length + i + scount*eventLength] = numberBytes[i];
}
for (int i = 0; i < timestampBytes.length; i++) {
fullbytes[numberBytes.length + eventBytes.length + i + scount*eventLength] = timestampBytes[i];
}
And then another method (called in a separate thread) that sends the bytes on the websockets
ByteBuffer bb = ByteBuffer.wrap(fullbytes);
wsSession.getRemote().sendBytesByFuture(bb);
bb.clear();
As I've read on a few place (in documentation or here and here), this issue should not appear, since I'm not using direct ByteBuffer. Could this be a bug related to Jetty / websockets?
Please advise!
EDIT:
I've made some more tests and I have noticed that the problem appears when sending messages to a client that is not connected, but jetty has not received the onClose event (for ex. a user puts his laptop in standby). Because the on close event is not triggered, the server code doesn't unregister the client and keeps trying to send the messages to that client. I don't know why but the close event is received after 1 or 2 hours. Also, sometimes (in don't know the context yet) although the event is received and the client (socket) is unregistered, a reference to a WebSocketSession object (for that client) still hangs. I haven't found out why this happens yet.
Until then, I have 2 possible workarounds, but I have no idea how to achieve them (that have other good uses as well):
Always detect when a connection is not open (or closed temporarily, for ex. user puts laptop in standby). I tried using sendPing() and implementing onFrame() but I couldn't find a solution. Is there a way to do this?
Periodically "flush" the buffer. How can I discard the messages that were not sent to the client so they don't keep on queuing?
EDIT 2
This may be pointing the topic to another direction so I made another post here.
EDIT 3
I've done some more tests regarding the large number of messages/bytes sent and I found out why "it seamed" that the memory leak only appeared sometimes: when sending bytes async on a different thread than the one used when sevlet.configure() is called, after a large build-up, the memory is not being released after the client disconnects. Also I couldn't simulate the memory leak when using sendBytes(ByteBuffer), only with sendBytesByFuture(ByteBuffer) and sendBytes(ByteBuffer, WriteCallback).
This seams very strangem but I don't believe I'm doing something "wrong" in the tests.
Code:
#Override
public void configure(WebSocketServletFactory factory) {
factory.getPolicy().setIdleTimeout(1000 * 0);
factory.setCreator(new WebSocketCreator() {
#Override
public Object createWebSocket(ServletUpgradeRequest req,
ServletUpgradeResponse resp) {
return new WSTestMemHandler();
}
});
}
#WebSocket
public class WSTestMemHandler {
private boolean connected = false;
private int n = 0;
public WSTestMemHandler(){
}
#OnWebSocketClose
public void onClose(int statusCode, String reason) {
connected = false;
connections --;
//print debug
}
#OnWebSocketError
public void onError(Throwable t) {
//print debug
}
#OnWebSocketConnect
public void onConnect(final Session session) throws InterruptedException {
connected = true;
connections ++;
//print debug
//the code running in another thread will trigger memory leak
//when to client endpoint is down and messages are still sent
//because the GC will not cleanup after onclose received and
//client disconnects
//if the "while" loop is run in the same thread, the memory
//can be released when onclose is received, but that would
//mean to hold the onConnect() method and not return. I think
//this would be bad practice.
new Thread(new Runnable() {
#Override
public void run() {
while (connected) {
testBytesSend(session);
try {
Thread.sleep(4);
} catch (InterruptedException e) {
}
}
//print debug
}
}).start();
}
private void testBytesSend(Session session) {
try {
int noEntries = 200;
ByteBuffer bb = ByteBuffer.allocate(noEntries * 14);
for (int i = 0; i < noEntries; i++) {
n+= 1.0f;
bb.putFloat(n);
bb.putShort((short)1);
bb.putDouble(123456789123.0);
}
bb.flip();
session.getRemote().sendBytes(bb, new WriteCallback() {
#Override
public void writeSuccess() {
}
#Override
public void writeFailed(Throwable arg0) {
}
});
//print debug
} catch (Exception e) {
e.printStackTrace();
}
}
}

Your ByteBuffer use is incredibly inefficient.
Don't create all of those minor/tiny ByteBuffers just to get a byte array, and then toss it out. ick.
Note: you don't even use the .array() call correctly, as not all ByteBuffer allocations have a backing array you can access like that.
The byte array's numberBytes, eventBytes, timestampBytes, and fullbytes should not exist.
Create a single ByteBuffer, representing the entire message you intend to send, allocate it it to be either the size you need, or larger.
Then put the individual bytes you want into it, flip it, and give the Jetty implementation that single ByteBuffer.
Jetty will use the standard ByteBuffer information (such as position and limit) to determine what part of that ByteBuffer should actually be sent.

Consumer(s)-Producer issue in webserver streaming an array of data

Producer-Consumer blog post states that:
"2) Producer doesn't need to know about who is consumer or how many consumers are there. Same is true with Consumer."
My problem is that I have an array of data that I need to get from the Webserver to clients as soon as possible. The clients can appear mid-calculation. Multiple clients at different times can request the array of data. Once the calculation is complete it is cached and then it can simply be read.
Exmaple Use Case: While the calculation is occurring I want to serve each and every datum of the array as soon as possible. I can't use a BlockingQueue because say if a second client starts to request the array while the first one has already used .take() on the first half of the array. Then the second client missed half the data! I need a BlockingQueue where you don't have to take(), but you could instead just read(int index).
Solution? I have a good amount of writes on my array, so I wouldn't want to use CopyOnWriteArrayList? The Vector class should work but would be inefficient?
Is it preferable to use a ThreadSafeList like this and just add a waitForElement() function? I just don't want to reinvent the wheel and I prefer crowd tested solutions for multi-threaded problems...

As far as I understand you need to broadcast data to subscribers/clients.
Here are some ways that I know for approaching it.
Pure Java solution, every client has a BlockingQueue and every time you broadcast a message you put it every queue.
for(BlockingQueue client: clients){
client.put(msg);
}
RxJava provides a reactive approach. Clients will be subscribers and ever time you emit a message, subscribers will be notified and they can choose to cancel their subscription
Observable<String> observable = Observable.create(sub->{
String[] msgs = {"msg1","msg2","msg3"};
for (String msg : msgs) {
if(!sub.isUnsubscribed()){
sub.onNext(msg);
}
}
if (!sub.isUnsubscribed()) { // completes
sub.onCompleted();
}
});
Now multiple subscribers can choose to receive messages.
observable.subscribe(System.out::println);
observable.subscribe(System.out::println);
Observables are a bit functional, they can choose what they need.
observable.filter(msg-> msg.equals("msg2")).map(String::length)
.subscribe(msgLength->{
System.out.println(msgLength); // or do something useful
});
Akka provides broadcast routers

This is not exactly a trivial problem; but not too hard to solve either.
Assuming your producer is an imperative program; it generates data chunk by chunk, adding each chunk to the cache; the process terminates either successfully or with an error.
The cache should have this interface for the produce to push data in it
public class Cache
public void add(byte[] bytes)
public void finish(boolean error)
Each consumer obtains a new view from the cache; the view is a blocking data source
public class Cache
public View newView()
public class View
// return null for EOF
public byte[] read() throws Exception
Here's a straightforward implementation
public class Cache
{
final Object lock = new Object();
int state = INIT;
static final int INIT=0, DONE=1, ERROR=2;
ArrayList<byte[]> list = new ArrayList<>();
public void add(byte[] bytes)
{
synchronized (lock)
{
list.add(bytes);
lock.notifyAll();
}
}
public void finish(boolean error)
{
synchronized (lock)
{
state = error? ERROR : DONE;
lock.notifyAll();
}
}
public View newView()
{
return new View();
}
public class View
{
int index;
// return null for EOF
public byte[] read() throws Exception
{
synchronized (lock)
{
while(state==INIT && index==list.size())
lock.wait();
if(state==ERROR)
throw new Exception();
if(index<list.size())
return list.get(index++);
assert state==DONE && index==list.size();
return null;
}
}
}
}
It can be optimized a little; most importantly, after state=DONE, consumers should not need synchronized; a simple volatile read is enough, which can be achieved by a volatile state

Handle Java socket concurrency

I am building a server that sends data via a single TCP socket for each user every 2 seconds and on a separate thread. There are also special events occasionally sent along side with the regular data. Sometimes, data in multiple packets would mix up so I created a queue to make sure it does not happen. However, the issue is still there, is my approach not correct or is there something wrong with my code?
protected void sendData (byte[] data) {
if (isSendingData) {
dataQueue.add(data);
return;
}
isSendingData = true;
Thread sendThread = new Thread() {
public void run () {
try {
BufferedOutputStream outStream = new BufferedOutputStream(connectionSocket.getOutputStream());
outStream.write(data);
outStream.flush();
// check queue, if there are data, send
byte[] moreData = null;
if (dataQueue.size() > 0) {
moreData = dataQueue.remove(0);
}
isSendingData = false;
if (moreData != null) {
sendData(moreData);
}
}
catch (Exception e) {
System.out.println ("Error sending data to peripheral: " + e);
isSendingData = false;
}
}
};
sendThread.start ();
}

The proper idiom to remove concurrency issues using a queue is to have a long-lived thread run an infinite loop which takes elements from the queue and processes them. Typically you'll use a blocking queue so that on each iteration the thread goes to sleep until there is an item to process.
Your solution deviates from the above in many aspects. For example:
if (isSendingData) {
dataQueue.add(data);
return;
}
isSendingData = true;
—if this method is called concurrently, this will result in a race condition: both threads can read isSendingData as false, then concurrently proceed to sending data over the network. If isSendingData isn't volatile, you've also got a data race on it (entirely separate from the race condition explained above).
if (dataQueue.size() > 0) {
moreData = dataQueue.remove(0);
}
—this is another race condition: after you read size as zero, the other thread can add an item to the queue. Now that item will possibly never be processed. It will linger in the queue until another such thread is started.
The more obvious way your solution is not correct is that the thread you start has no loops and is intended to just process one message, plus possibly one extra message in the queue. This should be reworked so that there are no special cases and sendData always, unconditionally, submits to a queue and never does any sending on its own.

I would do this completely differently. You don't want arbitrarily long queues in your application.
Have your hearbeat thread synchronize on the socket when sending the heartbeat.
Don't have it sending anything else.
Get rid of the queue, isSendingData, etc.
Have your main application synchronize on the socket when it wants to send, and just send whenever it needs to.
Use the same BufferedOutputStream or BufferedWriter for all sending, and flush it after each send.