For an exercise, we are to implement a server that has a thread that listens for connections, accepts them and throws the socket into a BlockingQueue. A set of worker threads in a pool then goes through the queue and processes the requests coming in through the sockets.
Each client connects to the server, sends a large number of requests (waiting for the response before sending the next request) and eventually disconnects when done.
My current approach is to have each worker thread waiting on the queue, getting a socket, then processing one request, and finally putting the (still open) socket back into the queue before handling another request, potentially from a different client. There are many more clients than there are worker threads, so many connections queue up.
The problem with this approach: A thread will be blocked by a client even if the client doesn't send anything. Possible pseudo-solutions, all not satisfactory:
Call available() on the inputStream and put the connection back into the queue if it returns 0. The problem: It's impossible to detect if the client is still connected.
As above but use socket.isClosed() or socket.isConnected() to figure out if the client is still connected. The problem: Both methods don't detect a client hangup, as described nicely by EJP in Java socket API: How to tell if a connection has been closed?
Probe if the client is still there by reading from or writing to it. The problem: Reading blocks (i.e. back to the original situation where an inactive client blocks the queue) and writing actually sends something to the client, making the tests fail.
Is there a way to solve this problem? I.e. is it possible to distinguish a disconnected client from a passive client without blocking or sending something?
Short answer: no. For a longer answer, refer to the one by EJP.
Which is why you probably shouldn't put the socket back on the queue at all, but rather handle all the requests from the socket, then close it. Passing the connection to different worker threads to handle requests separately won't give you any advantage.
If you have badly behaving clients you can use a read timeout on the socket, so reading will block only until the timeout occurs. Then you can close that socket, because your server doesn't have time to cater to clients that don't behave nicely.
Is there a way to solve this problem? I.e. is it possible to distinguish a disconnected client from a passive client without blocking or sending something?
Not really when using blocking IO.
You could look into the non-blocking (NIO) package, which deals with things a little differently.
In essence you have a socket which can be registered with a "selector". If you register sockets for "is data ready to be read" you can then determine which sockets to read from without having to poll individually.
Same sort of thing for writing.
Here is a tutorial on writing NIO servers
Turns out the problem is solvable with a few tricks. After long discussions with several people, I combined their ideas to get the job done in reasonnable time:
After creating the socket, configure it such that a blocking read will only block for a certain time, say 100ms: socket.setSoTimeout(100);
Additionally, record the timestamp of the last successful read of each connection, e.g. with System.currentTimeMillis()
In principle (see below for exception to this principle), run available() on the connection before reading. If this returns 0, put the connection back into the queue since there is nothing to read.
Exception to the above principle in which case available() is not used: If the timestamp is too old (say, more than 1 second), use read() to actually block on the connection. This will not take longer than the SoTimeout that you set above for the socket. If you get a TimeoutException, put the connection back into the queue. If you read -1, throw the connection away since it was closed by the remote end.
With this strategy, most read attempts terminate immediately, either returning some data or nothing beause they were skipped since there was nothing available(). If the other end closed its connection, we will detect this within one second since the timestamp of the last successful read is too old. In this case, we perform an actual read that will return -1 and the socket's isClosed() is updated accordingly. And in the case where the socket is still open but the queue is so long that we have more than a second of delay, it takes us aditionally 100ms to find out that the connection is still there but not ready.
EDIT: An enhancement of this is to change "last succesful read" to "last blocking read" and also update the timestamp when getting a TimeoutException.
No, the only way to discern an inactive client from a client that didn't shut down their socket properly is to send a ping or something to check if they're still there.
Possible solutions I can see is
Kick clients that haven't sent anything for a while. You would have to keep track of how long they've been quiet for, and once they reach a limit you assume they've disconnected .
Ping the client to see if they're still there. I know you asked for a way to do this without sending anything, but if this is really a problem, i.e you can't use the above solution, this is probably the best way to do it depending on the specifics(since it's an exercise you might have to imagine the specifics).
A mix of both, actually this is probably better. Keep track of how long they've been quiet for, after a bit send them a ping to see if they still live.
Related
I am building a server in java that communicates with several clients at the same time, the initial approach we had is the the server listens to connections from the clients, once a connection is received and a socket is created, a new thread is spawned to handle the communication with each client, that is read the request with an ObjectInputStream, do the desired operation (fetch data from the DB, update it, etc.), and send back a response to the client (if needed). While the server itself goes back to listen to more connections.
This works fine for the time being, however this approach is not really scalable, it works great for a small amount of clients connected at the same time, however since every client spawns another thread, what will happen when there are a too many clients connected at once?
So my next idea was to maintain a list of sorts that will hold all connected clients (the socket object and some extra info), use a ThreadPool for to iterate through them and read anything they sent, if a message was received then put it in a queue for execution by another ThreadPool of worker threads, and once the worker has finished with its task if a response is required then send it.
The 2 latter steps are pretty trivial to implement, the problem is that with the original thread per client implementation, I use ObjectInputStream.readObject() to read the message, and this method blocks until there is something to read, which is fine for this approach, but I can't use the same thing for the new approach, since if I block on every socket, I will never get to the ones that are further down the list.
So I need a way to check if I have anything to read before I call readObject(), so far I tried the following solutions:
Solution 1:
use ObjectInputStream.available() to check if there is anything available to read, this approach failed since this method seems to always return 0, regardless of whether there is an object in the stream or not. So this does not help at all.
Solution 2:
Use PushbackInputStream to check for the existence of the first unread byte in the stream, if it exists then push it back and read the object using the ObjectInputStream, and if it doesn't move on:
boolean available;
int b = pushbackinput.read();
if (b==-1)
available = false;
else
{
pushbackinput.unread(b);
available = true;
}
if (available)
{
Object message= objectinput.readObject();
// continue with what you need to do with that object
}
This turned out to be useless too, since read() blocks also if there is no input to read. It seems to only return the -1 option if the stream was closed. If the stream is still open but empty it just blocks, so this is no different than simply using ObjectInputStream.readObject();
Can anyone suggest an approach that will actually work?
This is a good question, and you've done some homework.... but it involves going through some history to get things right. Note, your issue is actually more to do with the socket-level communication rather than the ObjectInputStream:
The easiest way to do things in the past was to have a separate thread per socket. This was scalable to a point but threads were expensive and slow to create.
In response, for large systems, people created thread pools and would service the sockets on threads when there was work to do. This was complicated.
The Java language was then changed with the java.nio package which introduced the Selector together with non-blocking IO. This created a reliable (although sometimes confusing) way to service multiple sockets with fewer threads. In your case through, it would not help fully/much because you want to know when a full Object is ready to be read, not when there's just 'some' object.
In the interim the 'landscape' changed, and Java is now able to more efficiently create and manage threads. 'Current' thinking is that it is better/faster and easier to allocate a single thread per socket again.... see Java thread per connection model vs NIO
In your case, I would suggest that you stick with the thread-per-socket model, and you'll be fine. Java can scale and handle more threads than sockets, so you'll be fine.
I am running 2 threads in my applciation. One to check for incoming packets and one to process and send packets. They both do it on the SAME STREAM.
Example for 1:
while (connection open) {
in.readObject() instanceof ...
}
Example for 2:
while (connection open) {
processPacket(in)
}
I'm pretty sure EOFException is when the threads try and use the stream at the same time. It's not a constant EOF but only like every 1 second I get an EOF the rest works fine. So that's why I suspect that they overlap and try to use the stream at the same time.
If that is the problem, anyone know how do I synchronize them to do it after another while still keeping the current update speed and using two threads?
I need two threads because the check for incoming waits in a line until a packet gets recived and I need the server to constantly send process and check for packets.
How do I fix the EOFException?
If your getting an EOFException, it typically means the other side hung up. You usually get these on the read side.
Here's a similar SO question
Edit 1: The question is really why is the socket closed. It can be for any number of reasons, a programmable timer on the server side checking for no data within X minutes, a firewall closing the connection, a network interruption, etc..
Both threads shouldn't be reading the same Stream.
You should read the objects and put them in a ConcurrentLinkedQueue, then from the second thread you can check the queue for objects ready to process.
EOFException is 'normal'. It happens on one thread too. Your architecture of reading in two threads simultaneously cannot possibly work, but it isn't the cause of this problem. The cause is that the peer closed the connection. This is going to happen. Unless your application protocol contains message counts or a close notify or some other means of predicting EOS, it is going to get EOFExceptions, or readLine() returning null, or read() returning -1, depending which read methods you are calling.
My java application has to send messages(multithreaded) to a socket server. The application can send around 100-200 messages a second.
I want to know which is a better approach to do this ?
Open a single client socket and send the message from all threads though this one socket.
Disadvantages: Have to handle the reconnection logic on connection failure,may lose many messages when reconnection is in process.Thread safety, blocking ??
Create a new client socket connection for each thread and close it after sending.
Disadvantages: Even though I close the socket, the ports will wait till the TIME_WAIT period.
Which is a better practical approach ?
I would propose 3. :
Open an socket per thread, and reuse threads (for example via thread pool). Then handle reconnection inside thread, or just dispose it properly and create new one. This way you can avoid blocking and synchronisation issues
100-200 messages per second isn't that much. I wouldn't re-connect every time as this is expensive. If you re-use your connection, it will be much faster.
If you are worried about losing messages, you can send a batch of messages or one at a time and wait for a confirmation from the server they have been received. You can still send thousands of messages per second this way.
I have a socket tcp connection between two java applications. When one side closes the socket the other side remains open. but I want it to be closed. And also I can't wait on it to see whether it is available or not and after that close it. I want some way to close it completely from one side.
What can I do?
TCP doesn't work like this. The OS won't release the resources, namely the file descriptor and thus the port, until the application explicitly closes the socket or dies, even if the TCP stack knows that the other side closed it. There's no callback from kernel to user application on receipt of the FIN from the peer. The OS acknowledges it to the other side but waits for the application to call close() before sending its FIN packet. Take a look at the TCP state transition diagram - you are in the passive close box.
One way to detect a situation like this without dedicating a thread to each socket is to use the select/poll/epoll/kqueue family of functions. The socket being passively closed will be signaled as readable and read attempt will return the EOF.
Hope this helps.
Both sides have to read from the connection, so they can detect when the peer has closed. When read returns -1 it will mean the other end closed the connection and that's your clue to close your end.
If you are still reading from your socket, then you will detect the -1 when it closes.
If you are no longer reading from your socket, go ahead and close it.
If it's neither of these, you are probably having a thread wait on an event. This is NOT the way you want to handle thousands of ports! Java will start to get pukey at around 3000 threads in windows--much less in Linux (I don't know why).
Make sure you are using NIO. Use a single thread to manage all your ports (connection pool). It should just grab the data from a thread, forward it to a queue. At that point I think I'd have a thread pool take the data out of the queues and process it because actually processing the data from a port will take some time.
Attaching a thread to each port will NOT work, and is the biggest reason NIO was needed.
Also, having some kind of a "Close" message as part of your stream to trigger closing the port may make things work faster--but you'll still need to handle the -1 to cover the case of broken streams
The usual solution is to let the other side know you are going to close the connection, before actually closing it. For instance, in the case of the SMTP protocol, the server will send '221 Bye' before it closes the connection.
You probably want to have a connection pool.
In some circumstances I wish to send an error message from a server to client using non-blocking I/O (SocketChannel.write(ByteBuffer)) and then disconnect the client. Assuming I write the full contents of the message and then immediately disconnect I presume the client may not receive this message as I'm guessing that the OS hasn't actually sent the data at this point.
Is this correct, and if so is there a recommended approach to dealing with this situation?
I was thinking of using a timer whereby if I wish to disconnect a client I send a message and then close their connection after 1-2 seconds.
SocketChannel.write will in non-blocking mode return the number of bytes which could immediately be sent to the network without blocking. Your question makes me think that you expect the write method to consume the entire buffer and try asynchronously to send additional data to the network, but that is not how it's working.
If you really need to make sure that the error message is sent to the client before disconnecting the socket, I would simply enable blocking before calling the write method. Using non-blocking mode, you would have to call write in a loop, counting the number of bytes being sent by each invocation and exit the loop when you've succeeded to pass the entire message to the socket (bad solution, I know, unnecessary code, busy wait and so on).
you may be better off launching a thread and synchronously write data to the channel. the async api is more geared toward "one thread dispatching multiple channels" and not really intended for fire and forget communications.
The close() method of sockets makes sure, everything sent using write before is actually sent before the socket is really closed. However this assumes that your write() was able to copy all data to the tcp stacks output window, which will not always work. For solutions to this see the other answers.