Does Netty 4(or 5) continue reading while the handler chain is invoked ? Like reading in the background and filling up a huge queue and processing all reads after the handler chain is completely ready ?
I'm asking because I want to save incoming buffers to a file but files are blocking and if there are more incoming messages than the hard drive can handle do those messages queue up or does blocking the event simply stops the event queue from reading more messages ? On the sending side I use FileRegion so flow control for the writer is given.
I looked into the source but I'm not entirely sure how those ChannelInvokers work, it seems that if no one is specified a default one is created using eventLoop.next().
Would be nice if someone could help me to make sure that I'm not introducing a potential flow control bug...
I saw the option setAutoRead but I'm pretty sure this does something different.
If you want to do something like this you should specify a custom DefaultEventExecutorGroup when adding the handler that writes to the FS. This way you will hand-over the work from the EventLoop and so not block anything here.
Related
Maybe there is some "integration-pattern" here I miss...
I have a proccess (a thread from an TaskExecutor) that is some cases need to stop and wait for an additional data to continue.
I was thinking about blocking in a receive method, but I don't find how to send, from a different thread a message to that channel (a temporal one, isn't it?) to unblock this thread, only this.
The component responsible about unblock should receive a message from some kind of messagin platform (redis,rabbit,...) and then "notify" the blocked execution.
An ugly implementation could be a wait/notify but of course I don't want to use that having a full "message-oriented" solution.
Is there any component/solution for this problem?
Maybe a subscriber with some topic I can use to be sure only that thead ir running again, but I cannot block in a publishsubscribe channel, can I?
thanks a lot,
that is some cases need to stop and wait for an additional data to continue.
Looks like this is indeed the use-case for the Thread Barrier component.
Another way to do something similar is an Aggregator for the releaseStrategy as 2 messages by size.
Anyway the correlationKey is a key entity in both use-cases.
Hi guys am getting following error am using Websocket and Tomcat8.
java.lang.IllegalStateException: The remote endpoint was in state [TEXT_FULL_WRITING] which is an invalid state for called method
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$StateMachine.checkState(WsRemoteEndpointImplBase.java:1092)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$StateMachine.textStart(WsRemoteEndpointImplBase.java:1055)
at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendString(WsRemoteEndpointImplBase.java:186)
at org.apache.tomcat.websocket.WsRemoteEndpointBasic.sendText(WsRemoteEndpointBasic.java:37)
at com.iri.monitor.webSocket.IRIMonitorSocketServlet.broadcastData(IRIMonitorSocketServlet.java:369)
at com.iri.monitor.webSocket.IRIMonitorSocketServlet.access$0(IRIMonitorSocketServlet.java:356)
at com.iri.monitor.webSocket.IRIMonitorSocketServlet$5.run(IRIMonitorSocketServlet.java:279)
You are trying to write to a websocket that is not in a ready state. The websocket is currently in writing mode and you are trying to write another message to that websocket which raises an error. Using an async write or as not such good practice a sleep can prevent this from happening. This error is also normally raised when a websocket program is not thread safe.
Neither async or sleep can help.
The key problem is the send-method can not be called concurrently.
So it's just about concurrency, you can use locks or some other thing. Here is how I handle it.
In fact, I write a actor to wrap the socketSession. It will produce an event when the send-method is called. Each actor will be registered in an Looper which contains a work thread and an event queue. Meanwhile the work thread keeps sending message.
So, I will use the sync-send method inside, the actor model will make sure about the concurrency.
The key problem now is about the number of Looper. You know, you can't make neither too much or too few threads. But you can still estimate a number by your business cases, and keep adjusting it.
it is actually not a concurrency issue, you will have the same error in a single-threaded environment. It is about asynchronous calls that must not overlap.
You should use session.get**Basic**Remote().sendText instead of session.get**Async**Remote().sendText() to avoid this problem. Should not be an issue as long as the amount of data you are writing stays reasonable small.
I want to generate some text string that is going to be sent via TCP socket . I have accomplished it within few minutes.
However I want a producer consumer pattern.I dont care if it failed or not.
Should I create a Blocking Queque at application for this ? Should I create a service ?
Note that I want a single thread to manage this job.
In the case it's a short task (like you commented), I'd recommend putting it within an AsyncTask as a background thread. You can control anything about this separately, which will help you also debugging it. Services are more intended for long executing tasks, so I'd not recommend it at this scope (it's a bit harder even to communicate with other Activity's. Here you'll find the AsyncTask's documentation, and here a good example.
The Blocking structure depends on your needs - but I don't think you'll need that in your case. Anyway, if you would need that, there're lots of thread-safe data structures you may use, you might find this helpful.
Create a LinkedBlockingQueue where your producer adds data. Create a Timer that fires every second or so. The task of the Timer would be to send the messages over the wire.
For this, both the producer (the one generating the messages) and consumer (Timer) should have access to the LinkedBlockingQueue. The Timer will remove the first element of the LinkedBlockingQueue and then send it.
Sounds good ?
Is there a way to immediately print the message received from the client without using an infinite loop to check whether the input stream is empty or not?
Because I found that using infinite loop consumes a lot of system resources, which makes the program running so slow. And we also have to do the same (infinite loop) on the client side to print the message on the screen in real time.
I'm using Java.
You should be dealing with the input stream in a separate Thread - and let it block waiting for input. It will not use any resources while it blocks. If you're seeing excessive resource usage while doing this sort of thing, you're doing it wrong.
I think you can just put your loop in a different thread and have it sleep a bit (maybe for half a second?) between iterations. It would still be an infinite loop, but it would not consume nearly as many resources.
You don't you change your architecture a little bit to accommodate WebSockets. check out Socket.IO . It is a cross browser WebSockets enabler.
You will have to write controllers (servlets for example in java) that push data to the client. This does not follow the request-response architecture.
You can also architect it so that a "push servlet" triggers a "request" from the client to obtain the "response".
Since your question talks about Java, and if you are interested in WebSockets, check this link out.
If you're using Sockets, which you should be for any networking.
Then you can use the socket's DataInputStream which you can get using socket.getInputStream() (i think that's the right method) and do the following:
public DataInputStream streamIn;
public Socket soc;
// initialize socket, etc...
streamIn = soc.getInputStream();
public String getInput() {
return (String) streamIn.readUTF(); // Do some other casting if this doesn't work
}
streamIn.readUTF() blocks until data is available, meaning you don't have to loop, and threading will let you do other processing while you wait for data.
Look here for more information on DataInputStream and what you can do with it: http://docs.oracle.com/javase/6/docs/api/java/io/DataInputStream.html
A method that does not require threads would involve subclassing the input stream and adding a notify type method. When called this method would alert any interested objects (i.e. objects that would have to change state due to the additions to the stream) that changes have been made. These interested objects could then respond in anyway that is desired.
Objects writing to the buffer would do their normal writing, and afterward would call the notify() method on the input stream, informing all interested objects of the change.
Edit: This might require subclassing more than a couple of classes and so could involve a lot of code changes. Without knowing more about your design you would have to decide if the implementation is worth the effort.
There are two approaches that avoid busy loops / sleeps.
Use a thread for each client connection, and simply have each thread call read. This blocks the thread until the client sends some data, but that's no problem because it doesn't block the threads handling other clients.
Use Java NIO channel selectors. These allow a thread to wait until one of set of channels (in this case sockets) has data to be read. There is a section of the Oracle Java Tutorials on this.
Of these two approaches, the second one is most efficient in terms of overall resource usage. (The thread-per-client approach uses a lot of memory on thread stacks, and CPU on thread switching overheads.)
Busy loops that repeatedly call (say) InputStream.available() to see if there is any input are horribly inefficient. You can make them less inefficient by slowing down the polling with Thread.sleep(...) calls, but this has the side effect of making the service less responsive. For instance, if you add a 1 second sleep between each set of polls, the effect that each client will see is that the server typically delays 1 second before processing each request. Assuming that those requests are keystrokes and the responses echo them, the net result is a horribly laggy service.
I'm writing a chess program in Java. The GUI should be able to communicate with a chess engine supporting the Chess Engine Communication Protocol. But I'm having some difficulties reconciling the protocol with Java's I/O facilities.
Because engines that predate protocol version 2 do not send "feature", xboard uses a timeout mechanism: when it first starts your engine, it sends "xboard" and "protover N", then listens for feature commands for two seconds before sending any other commands.
It seems that Java's facilities for interrupting I/O operations are limited. The only option I can find is NIO's InterruptibleChannel, which closes itself when interrupted.
I don't want the stream to close when the timeout occurs -- I just want to interrupt the read. Does anyone know a solution?
I think you may be overthinking the problem. You don't need to abort the read() call after 2 seconds, you just need your backing logic to understand that after 2 seconds it should not expect to receive any "feature" commands. Then your implementation can write the next command, and your read() will return the byte(s) from the response to that command.
That's how I'd approach it anyways, by having generic code that reads in bytes and passes them further up the chain where context-specific processing can be done. Then you don't need to interrupt the read, the upstream code just needs to understand that the data it eventually gets back may be a "feature" command, or it may not be.
It's not clear to me that you need to do anything much. What you have quoted is the timeout behaviour of the board. You don't have to implement that, it is done, at the board, which is the peer, i.e. the other end.