I have a very peculiar problem. In android I am using FileInputStream to read from the serial (ttySx/COM) port. I am using this to decide which of the known devices is connected (if any at all). What I basically do is:
Are you device 1? No...
Are you device 2? No...
Are you device 3? Yes...
Great lets do some stuff...
And this works great. If there is any incoming data to read (response from device), everything is fine. However, if there is no device connected to ttySx there is nothing to respond to my write. That means nothing to read.
Now, FileInputStream.read() is a blocking call. When I call it in the thread, thread is effectively frozen. I cannot interrupt the thread because for that I would have to read something first. So far everything makes perfect sense.
As there is no response from the port for quite some time I decide that there is nothing connected and want to stop reading and dispose of the thread(actually I do not want to bother with the port anymore because with nothing connected, it is useless to me at this moment). As mentioned earlier interrupt itself is no good. What should be working, is to close() the FileInputStream (read() will throw an exception and hooray!). The close() works... As long as the read() read anything ever (like when I had an answering device connected, then disconnect it -> read() is stuck - because no data to read - but close() works).
However if there was not a thing connected to the port when the read() started (equals: I haven't read a single byte), the close() method does nothing. It does not close the stream. Nor does work the closing of FileInputStream channel.
I could create a workarround: Store the FileInputStream somewhere and when I want to read from the port again later, use the same instance. That would work for me. Unfortunately I would quite unnecessarily block the port itself. No other process (for example another application) could read from the port because it is stuck in "uninterruptable" read...
Any ideas why this is happening and how to make it right? Or some other way to detect if there is anything connected to the ttySx port?
Thanks.
EDIT1: The library used for communication with serial port is https://github.com/cepr/android-serialport-api
In the end we used FileInputStream::available().
First time we tried it, it was like:
Check if anything is available.
Read (regardless of availability)
Of course, when we checked the available, there was nothing to read yet. Then the read call blocked and waited for input. When we checked again, there was nothing available already, because read had cleared the port.
Therefore this suggestion Java close FileInputStream before reading anything from M. Prokhorov was the correct one for my situation.
If anyone would wonder about the behavior in question:
From researching it, it seems that reading streams was not designed for ports/sockets in first place. It was designed for regular files. You read, reach the end of document and close the stream. The exceptions are designed for wrong sequential usage of a stream (you open it, close id and then try to read).
If you enter blocking mode, it will block until it reads at least a byte. No way around it. Close initializes the "closing state" similarly to setting the interrupt state of a thread.
Related
I am writing to and reading from a Linux file in java, which in reality is a communication port to a hardware device. To do this I use RandomAccessFile (I'll explain why later) and it works well in most cases. But sometimes a byte is lost and then my routine blocks indefinitely since there is no timeout on the read method.
To give some more details on the file: it is a USB receipt printer that creates a file called /dev/usb/lp0 and though I can use a cups driver to print, I still need the low level communication through this file to query the status of the printer.
The reason I use RandomAccessFile is that I can have the same object for both reading and writing.
I tried to make a version with InputStream and OutputStream instead (since that would allow me to use the available() method to implement my timeout). But when I first open the InputStream and then the OutputStream I get an exception when opening the OutputStream since the file is occupied.
I tried writing with the OutputStream and then closing it before opening the InputStream to read, but then I lose some or all of the reply before it has opened the InputStream.
I tried switching to channels instead (Files.newByteChannel()). This also allows me to have just one object, and the documentation says it only reads the bytes available and returns the count (which also allows me to implement a timeout). But it blocks in the read method anyway when there is nothing to read, despite what the documentation says.
I also tried a number of ways to implement timeouts on the RandomAccessFile using threads.
The first approach was to start a separate thread at the same time as starting to read, and if the timeout elapsed in the thread I closed the file from the thread, hoping that this would unlock the read() operation with an exception, but it didn't (it stayed blocked).
I also tried to do the read in a separate thread and brutally kill it with the deprecated Thread.stop() once the time had elapsed. This worked one time, but it was not possible to reopen the file again after that.
The only solution I have made work is to have a separate thread that continuously calls read, and whenever it gets a byte it puts it in a LinkedBlockingQueue, which I can read from with a timeout. This approach works, but the drawback is that I can never close the file (again for the same reasons explained above, I can't unblock a blocked read). And my application requires that I sometimes close this connection to the hardware.
Anyone who can think of a way to read from a file with timeout that would work in my case (that allows me to have both a read and a write access open to the file at the same time)?
I am using Java8 by the way.
For an exercise, we are to implement a server that has a thread that listens for connections, accepts them and throws the socket into a BlockingQueue. A set of worker threads in a pool then goes through the queue and processes the requests coming in through the sockets.
Each client connects to the server, sends a large number of requests (waiting for the response before sending the next request) and eventually disconnects when done.
My current approach is to have each worker thread waiting on the queue, getting a socket, then processing one request, and finally putting the (still open) socket back into the queue before handling another request, potentially from a different client. There are many more clients than there are worker threads, so many connections queue up.
The problem with this approach: A thread will be blocked by a client even if the client doesn't send anything. Possible pseudo-solutions, all not satisfactory:
Call available() on the inputStream and put the connection back into the queue if it returns 0. The problem: It's impossible to detect if the client is still connected.
As above but use socket.isClosed() or socket.isConnected() to figure out if the client is still connected. The problem: Both methods don't detect a client hangup, as described nicely by EJP in Java socket API: How to tell if a connection has been closed?
Probe if the client is still there by reading from or writing to it. The problem: Reading blocks (i.e. back to the original situation where an inactive client blocks the queue) and writing actually sends something to the client, making the tests fail.
Is there a way to solve this problem? I.e. is it possible to distinguish a disconnected client from a passive client without blocking or sending something?
Short answer: no. For a longer answer, refer to the one by EJP.
Which is why you probably shouldn't put the socket back on the queue at all, but rather handle all the requests from the socket, then close it. Passing the connection to different worker threads to handle requests separately won't give you any advantage.
If you have badly behaving clients you can use a read timeout on the socket, so reading will block only until the timeout occurs. Then you can close that socket, because your server doesn't have time to cater to clients that don't behave nicely.
Is there a way to solve this problem? I.e. is it possible to distinguish a disconnected client from a passive client without blocking or sending something?
Not really when using blocking IO.
You could look into the non-blocking (NIO) package, which deals with things a little differently.
In essence you have a socket which can be registered with a "selector". If you register sockets for "is data ready to be read" you can then determine which sockets to read from without having to poll individually.
Same sort of thing for writing.
Here is a tutorial on writing NIO servers
Turns out the problem is solvable with a few tricks. After long discussions with several people, I combined their ideas to get the job done in reasonnable time:
After creating the socket, configure it such that a blocking read will only block for a certain time, say 100ms: socket.setSoTimeout(100);
Additionally, record the timestamp of the last successful read of each connection, e.g. with System.currentTimeMillis()
In principle (see below for exception to this principle), run available() on the connection before reading. If this returns 0, put the connection back into the queue since there is nothing to read.
Exception to the above principle in which case available() is not used: If the timestamp is too old (say, more than 1 second), use read() to actually block on the connection. This will not take longer than the SoTimeout that you set above for the socket. If you get a TimeoutException, put the connection back into the queue. If you read -1, throw the connection away since it was closed by the remote end.
With this strategy, most read attempts terminate immediately, either returning some data or nothing beause they were skipped since there was nothing available(). If the other end closed its connection, we will detect this within one second since the timestamp of the last successful read is too old. In this case, we perform an actual read that will return -1 and the socket's isClosed() is updated accordingly. And in the case where the socket is still open but the queue is so long that we have more than a second of delay, it takes us aditionally 100ms to find out that the connection is still there but not ready.
EDIT: An enhancement of this is to change "last succesful read" to "last blocking read" and also update the timestamp when getting a TimeoutException.
No, the only way to discern an inactive client from a client that didn't shut down their socket properly is to send a ping or something to check if they're still there.
Possible solutions I can see is
Kick clients that haven't sent anything for a while. You would have to keep track of how long they've been quiet for, and once they reach a limit you assume they've disconnected .
Ping the client to see if they're still there. I know you asked for a way to do this without sending anything, but if this is really a problem, i.e you can't use the above solution, this is probably the best way to do it depending on the specifics(since it's an exercise you might have to imagine the specifics).
A mix of both, actually this is probably better. Keep track of how long they've been quiet for, after a bit send them a ping to see if they still live.
I have an Android app that acts as server and feeds over TCP some data from sensors with arbitrary intervals (within 5-60 seconds). Client apps occasionally send small chunks of data over the same connection. Data must be sent and received without any delays.
All samples and tutorials (like this one http://adblogcat.com/asynchronous-java-nio-for-dummies/ ) show more or less same scenario - when reading is finished, switch to OP_WRITE. when writing is done switch to OP_READ and so on.
Obviously it won't work for my case. I tried yo enable both reads and writes at once like this
serverChannel.register(selector, SelectionKey.OP_READ|SelectionKey.OP_WRITE);
But it makes selector cycle constantly which loads CPU pretty much.
I'm sure this question is not really correct so i'll be glad even if someone give me totally different but working idea or point where i'm wrong. I did not post the code since it is nearly identical to that of aforementioned tutorial.
Your question and the samples you cite are based on a fallacy. There is no such thing as a 'mode' in NIO. You can read and write whenever you like, but they can both do nothing if done at the wrong time.
OP_READ firing means that a read will return with data or end of stream, i.e. that there is data or a FIN in the socket receive buffer. This is normally false, except when the peer has sent some data or closed his end of the connection.
OP_WRITE firing means that a write will transfer some data, i.e. that there is room in the socket send buffer. This is normally true, and this in turn is why selecting on it normally smokes the CPU.
Normally a channel should only be registered for OP_READ.
When you have something to write, write it.
If and only if write() returns zero, register the channel for OP_WRITE, remember the buffer you were writing, and return to the select loop.
When OP_WRITE fires for this channel, repeat the write, and if it completes, deregister the channel for OP_WRITE.
There is a lot of junk on the Internet and this is particularly true where NIO is concerned. Among the numerous problems in your citation (seen over and over again in this kind of material):
select() is not asynchronous;
'only one of those can happen at a time' is false;
you do not need the Selector's 'permission' to write;
registering OP_WRITE and waiting for it to fire when you have something to write and don't already know that the socket send buffer is full is just an elaborate and pointless waste of time;
you can't change a channel that has been registered for OP_ACCEPT to read/write;
closing a channel cancels the key;
closing either the channel or the socket closes both;
finishConnect() can return false;
write() can return zero, or less than the amount of data that was supplied;
OP_CONNECT can only fire if isConnectionPending() is true.
I am running 2 threads in my applciation. One to check for incoming packets and one to process and send packets. They both do it on the SAME STREAM.
Example for 1:
while (connection open) {
in.readObject() instanceof ...
}
Example for 2:
while (connection open) {
processPacket(in)
}
I'm pretty sure EOFException is when the threads try and use the stream at the same time. It's not a constant EOF but only like every 1 second I get an EOF the rest works fine. So that's why I suspect that they overlap and try to use the stream at the same time.
If that is the problem, anyone know how do I synchronize them to do it after another while still keeping the current update speed and using two threads?
I need two threads because the check for incoming waits in a line until a packet gets recived and I need the server to constantly send process and check for packets.
How do I fix the EOFException?
If your getting an EOFException, it typically means the other side hung up. You usually get these on the read side.
Here's a similar SO question
Edit 1: The question is really why is the socket closed. It can be for any number of reasons, a programmable timer on the server side checking for no data within X minutes, a firewall closing the connection, a network interruption, etc..
Both threads shouldn't be reading the same Stream.
You should read the objects and put them in a ConcurrentLinkedQueue, then from the second thread you can check the queue for objects ready to process.
EOFException is 'normal'. It happens on one thread too. Your architecture of reading in two threads simultaneously cannot possibly work, but it isn't the cause of this problem. The cause is that the peer closed the connection. This is going to happen. Unless your application protocol contains message counts or a close notify or some other means of predicting EOS, it is going to get EOFExceptions, or readLine() returning null, or read() returning -1, depending which read methods you are calling.
I'm programming a simple Java NIO server and have a little headache: I get normal InputStreams i need to pipe to my clients. I have a single thread performing all writes, so this creates a problem: if the InputStream blocks, all other connection writing will be paused.
I can use InputStream.available() to check if there are any incoming data I can read without blocking, but if I've reached end-of-stream it seems I must call read() to know.
This creates a major headache for me, but I can't possibly believe I'm the first to have this problem.
The only options I've come up with so far:
Have a separate thread for each InputStream, however that's just silly since I'm using non-blocking I/O in the first place. I could also have a thread pool doing this but then again that limits the amount of simultaneous clients I can pipe the InputStream to.
Have a separate thread reading these streams with a timeout (using another thread to interrupt if reading has lasted longer than a certain amount of time), but that'll most certainly choke the data flow should I have many open InputStreams not delivering data.
Of course, if there was a magic InputStream.isEof() or isClosed() then this wouldn't be any problem at all :'(
".....Have a separate thread for each InputStream, however that's just silly since I'm using non-blocking I/O in the first place...."
It's not silly at all. First you need to check whether you can retrieve a SelectableChannel from your InputStream implementation. If it does you are lucky and you can just register it with a selector and do as usual. But chances are that your InputStream may have a channel that's not a SelectableChannel, in which case "Have a separate thread for each InputStream" is the obvious thing to do and probably the right thing to do.
Note that there is a similar problem discussed in SO about not able to get a SelectableChannel from an inputstream. Unfortunately you are stuck.
I have a single thread performing all
writes
Have you stopped to consider whether that is part of the problem rather than part of the solution?