I have a simple client-server application using sockets for the communication. One possibility is to close the socket every time the client has sent something to the server.
But my idea is to keep the connection always open, i.e. if a client contacts the server the connection should be put into a queue (e.g. LinkedBlockingQueue) and kept open, this would increase the performance.
How can I check in the server if there is new data available in a socket in the queue? The only thing I can imagine is to constantly iterate over the whole queue and check every socket if it has new data. But this would be very inefficient because if I have several threads working on the queue, the queue gets blocked when one thread is scanning over it.
Or is there a possibility to register a callback function on the socket, so that the socket informs the threads that data is ready?
But my idea is to keep the connection always open, i.e. if a client contacts the server the connection should be put into a queue (e.g. LinkedBlockingQueue) and kept open, this would increase the performance.
Keeping connections open will improve performance, though there are scaling issues: an open socket uses kernel resources. (I wouldn't use a queue though ...)
How can I check in the server if there is new data available in a socket in the queue?
If you have a number of sockets to different clients, and you want to process data in (roughly) the order that it arrives, there are two common techniques:
Create a thread per socket, and have each thread simply do a read. This will (naturally) block the thread until data becomes available.
Use the NIO channel selector mechanism (see Selector) which allows you to find out which of a group of I/O channels is ready for a read or write.
Thread per socket tends to be resource hungry (thread stacks), and does not scale well at all if you have multiple threads that are active simultaneously. (Too many context switches, too much load on the thread scheduler.)
By contrast, selectors map onto native syscalls provided by the host operating system, and thus they are efficient and responsive ... if used intelligently.
(You could also obtain non-blocking channels for the sockets, and poll them round-robin fashion. But that isn't going to be either efficient or responsive.)
As you can see, none of these ideas work with a queue. Either you have a number of threads each dealing with one socket, or you have one thread dealing with an array or (array) list of sockets. The queue abstraction is not designed for indexing or iterating.
Or is there a possibility to register a callback function on the socket, so that the socket informs the threads that data is ready?
See #Lolo's answer.
A practical solution would be to use NIO2 AsynchronousSocketChannels to perform asynchronous read operations with a callback that you can specify as a CompletionHandler.
Related
How to use TBufferedTransport of TThreadedSelectorServer in java?
in Python client:
self.tsocket= TSocket.TSocket(self.host, self.port)
self.transport = TTransport.TBufferedTransport(self.tsocket)
protocol = TBinaryProtocol(self.transport)
client = Handler.Client(protocol)
self.transport.open()
in Java Server
TNonblockingServerSocket serverTransport = new TNonblockingServerSocket(port);
TProcessor tprocessor = new ExecutionService.Processor<ExecutionService.Iface>(handler);
TThreadedSelectorServer.Args tArgs = new TThreadedSelectorServer.Args(serverTransport);
tArgs.processor(tprocessor);
tArgs.protocolFactory(new TBinaryProtocol.Factory());
this.server = new TThreadedSelectorServer(tArgs);
The Python client uses TBufferedTransport, and the Java server uses TFramedTransport. Causes an exception:
AbstractNonblockingServer$FrameBuffer Read an invalid frame size of -2147418111. Are you using TFramedTransport on the client side?
For some reasons, the client cannot be modified, so I want to modify the java server to TBufferedTransport.
How to use TBufferedTransport of TThreadedSelectorServer in java?
thanks！！！
The TThreadedSelectorServer requires TFramedTransport (reference):
A Half-Sync/Half-Async server with a separate pool of threads to handle non-blocking I/O. Accepts are handled on a single thread, and a configurable number of nonblocking selector threads manage reading and writing of client connections. ... Like TNonblockingServer, it relies on the use of TFramedTransport.
This applies for the other non-blocking server classes deriving from TNonblockingServer (reference):
A nonblocking TServer implementation. This allows for fairness amongst all connected clients in terms of invocations. This server is inherently single-threaded. If you want a limited thread pool coupled with invocation-fairness, see THsHaServer. To use this server, you MUST use a TFramedTransport at the outermost transport, otherwise this server will be unable to determine when a whole method call has been read off the wire. Clients must also use TFramedTransport.
If you cannot use TFramedTransport on the client side, you therefore have to use a blocking server, i.e. TThreadPoolServer (reference):
Server which uses Java's built in ThreadPool management to spawn off a worker pool that deals with client connections in blocking way.
Your code would then look like this:
TServerSocket serverTransport = new TServerSocket(9090);
TThreadPoolServer.Args tArgs = new TThreadPoolServer.Args(serverTransport);
tArgs.processor(processor);
tArgs.protocolFactory(new TBinaryProtocol.Factory());
TThreadPoolServer server = new TThreadPoolServer(tArgs);
To detail the differences between the blocking and the non-blocking servers (for general reference, apologies if the difference is already clear to you): Blocking means that when data is read from a socket, no other operation can be done while reading. So when the data arrives partially, the current thread waits until the remaining data arrives. So when a blocking server only has a single thread, only one client can be handled at a time. The time spend waiting for further data from a client cannot be used to serve other clients.
To support multiple clients, multiple threads can be added (as done for TThreadPoolServer). Each thread can only handle one client at a time as before, so the number of clients that can be served simultaneously is limited by the number of threads. You could of course spawn many threads, but this does not scale well: The threads used by the Java ThreadPool which backs the TThreadPoolServer are system-level threads, so they come with some resource over-head for creation and switching between threads. So creating a large number of threads to serve a large number of clients means more time is spent with OS book-keeping of the tasks.
Non-blocking servers (deriving from TNonblockingServer) are meant to solve this problem by utilizing the time spend waiting for data from one client by reading data from other clients. This way a single thread can handle multiple clients, reading from whichever client currently has available data. A non-blocking server can of course also have multiple threads, each handling multiple clients. This way the number of threads does not have to scale with the number of clients. Instead, the number of threads can be chosen proportionally to the number of CPU cores, and then each thread running on a core can read as much data as the I/O band-width and CPU speed allows. For this reason, a non-blocking server scales better with high-client numbers.
For this reason, if you have to handle a large number of clients simultaneously , using TNonblockingServer would be preferable and it would be better to find a way to switch the client to use the TFramedTransport. If your use-case is handling only a limited number of clients, then using TThreadPoolServer without modifying the client should be fine, even if each client produces a lot of data.
I am working on a application which should be very light weight and use minimum number of threads.
I need a socket server in my application for heartbeat monitor.
ServerSocket listener= new ServerSocket(port);
while (true) {
Socket socket = listener.accept();
Runnable thread = new HBClient(this, socket);
thread.run();
}
Problem here is I have to use one thread per one client.
Is there a way to do this without using threads? Maybe an event driven approach to identify when a client is connected or a non blocking method to accept clients.(I already checked java.nio but it seems even that cannot be used without threads)
Using NIO (for New IO, not Non-blocking IO) you can use a Selector on a single thread to handle multiple channels whereas with basic IO you have one thread responsible for one task (accepting connections or doing communication on a connection).
The basic premise is that you have resources and the single selector will "spin around" and choose one of them to process for whatever needs to be done (connect, read, write). Once that's done, another resource will be selected and so on. Of course a resource won't be selected unless there's actually something to do, and the channels inform that with SelectionKey flags to indicate which operations can be done.
However using non-blocking IO is a lot harder to program to than basic IO, and if you're not handling a lot of resources it won't be that much of an [improvement](NIO Performance Improvement compared to traditional IO in Java
) either. Even if you do want to use NIO it's recommended that unless you do NIO for learning purposes, use an existing framework like Netty that will make it a lot easier for you to concentrate on the functionality of the program and not the intricacies of getting NIO to work properly.
If you do want to spend time with NIO, there are plenty of questions on SO that discuss it like Java NIO Server
No. Even if you try to implement an event driven approach, someone should still listen to the socket to throw an event. So it is basically impossible to do this with a single thread.
But, you can break the infinite loop when you notify a connected client. You won't be accepting new clients but you'll be in a single thread.
Since channel is not thread safe, I can either synchronize th channel instance before publish or I create a channel each time I need and close it.
But in my opinion neither of them have a good performance due to cost of locking or create and destory channels.
So how should I publish message to rabbitmq with high tps? Any good pratise on this?
So, first thing first. A channel is not a connection. In RabbitMQ, Channels are the same thing as application sessions, whereas a connection represents an underlying TCP session with the server. This answer explains some of that nicely.
TCP sessions are expensive to create, so they tend to have a lifetime outside the lifetime of any particular worker thread. Channels are extremely cheap to create - all the server does is assign you an integer for your channel identifier and you have a new channel.
Most operations on RabbitMQ close a channel when they fail. This is done because there is no practical consequence of doing so. Would they close the underlying connection, that would cause a lot of problems for the app.
Design Guidance
Pooling would be appropriate for connections, if you really have a lot of processing going on. A discussion on how to do this is really beyond what I can provide in a short answer.
Pooling is absolutely not appropriate for channels. A channel is a lightweight construct that is designed to have a transient lifetime. If it lasts longer than one or two publishes, great. But you should expect that every time you try an operation, there is a possibility it will fail and close the channel. This does not close the underlying connection, but a new channel will have to be reestablished to do anything with the broker.
Consumer lifetimes are tied to channels. When the channel closes, the attached consumer is closed too. Design your consumer objects (worker threads) to be able to get a connection and create a new channel when this happens, and then re-subscribe themselves.
Avoid sharing a channel across threads. One thread = one channel.
While I don't have any particular experience with the Java client, I don't believe locks should be necessary, and I would certainly hope that the implementation doesn't do something screwy to make Channels anything other than lightweight.
If you're writing your own protocol implementation library (not necessary, but also not a bad idea if you need fine-grained control), allocate one thread to manage each connection. Don't do reads and writes to the TCP socket in parallel, or you'll break the protocol.
Regarding the Java client, I think you can assume that channel operations (reads and writes, etc.) are NOT thread-safe, which is why you want to stick to one thread/one channel paradigm. I think you can assume that creating channels (a connection operation) is thread-safe.
You should use a pool.
For instance, use Apache's Generic Object Pool and provide an implementation for opening, closing and checking connections. When you need to publish a message, you borrow a channel from the pool, use it, and return it.
I want to write a chat application in java which can handle many users simultaneously. I read about sockets and threadpools to limit thread number, but I can't imagine how to handle e.g. 100 socket connections at the same time and do not create 100 new threads. Idea is that client connects at the beginning and his connection stays opened until he leaves the chat. He can send data to server as well as receive other users messages.
Read from socket is blocking operation, so I would need to check all user's sockets in loop with some timeout if new data is available in particular socket connection? My first idea was to create e.g. 3 threads for handling input from all connected users and 3 threads for outcomming communication from server to clients, but how can I achieve that? Is there any async API for sockets in Java where can I define threadpools for in/out communication?
Make a Client class that extends Thread. Write all the methods and in the void run() method, write the code you want executed when the client connection is made.
On the Server side, listen for new connections. Accept a new connection, get the information about the connection, pass it in the constructor to create a new Client object, and add it to an ArrayList to keep track of all ongoing connections and execute the start() method. So, all the Client objects are in an Arraylist, and the they keep running at the same time.
I had made such a chat application about an year ago. And do not forget to close the connection once the Client disengages, orelse all the objects pile up and slow up the application. I learnt that the hard way.
Use Netty as it provides an NIO framework (non-blocking IO) so that you do not need 1 thread per connection. It is a little bit (or a lot..) more complicated to write a server using non-blocking IO, but there are performance gains in regards to not requiring one thread per connection.
However, 100 threads is not so many, so you could still create your server using standard IO and one thread per connection, it just depends on how much you need to scale.
For a server setup using Netty, you create a channel to which new connections are assigned. This channel is an ordered series of handlers which process incoming (and outgoing) messages from a connection / client. The handlers themselves all need to be asynchronous such that when a handler needs to return a message to the client it writes it asynchronously (non-blockingly) to the channel and receives a future back to which it can attach actions for when the message is actually written.
There is a little bit of a learning curve, but it is not that steep and the overall design of your application will be much better if built the Netty way vs using standard blocking IO.
I am studying the structure for my server-client communications in my multiplayer game.
I came to the conclusion that UDP is the best choice due to the "shoot and forget" way of using it that will not block the application if a packet is lost.
I will also use TCP to send reliability needing packets, like during login procedures and exchange of informations like change of server, change of map, updates etc. It will also run an IRC based chat. (all the commands actually are IRC-style custom messages).
I was wondering what is the best way to send the interaction messages (moves, spells, objects, actions etc) between server and clients.
Reading some documentation I came to the MulticastSocket.
My question is:
Is better to send a continuous flow of information to all the clients starting a thread for each player (as I do in TCP communications) where each DatagramSockets will listen to a queue sending each new message to its client. This will mean that all the maps and all the movements (supposing there can be 50 players all-over the maps) will be sent to all the players, and each packet has to be larger to contain all those informations.
Or is better to use a thread for each map, active only if some player are inside that specific map, using a multicast communication, sending a message to only the players that are inside that map, listening with a MulticastSocket.
I read about problems with firewall or routers using multicast, but I can't figure out what those problems can be (different from normal UDP).
The application should be used by anyone with few configuration problems.
Looking at your scenario above you need to decided if your application absolutely needs TCP connection as TCP connection requires one thread per TCP connection, no exceptions (unless using nio).
Now to target the UDP section of the program, you have two basic choices:
a) You spawn one thread for receiving datagram packets for all players.
In this case, all players send their datagram packets to a single receiver which then decides what to do with the data. This data may be sent to various queues for other threads for processing. Data can be sent back to all players using a single thread or multiple threads (per player).
PROS:
Low resource usage
Low program (synchronization) overhead.
CONS:
Possible network slowness (due to masses of packets going towards the same socket)
Higher chance of packet drop (again due to masses of packets going to the same socket)
Serial processing
Disconnect events are messy and hard to deal with
b) You spawn one thread per player and listen on a different port per player.
In this case, all players get their own handler threads which may process the data directly or send it to a central processing queue. By doing this, data can be processed in parallel, allowing for faster processing speeds with a higher resource usage. Synchronization will also require special attention, uses of atomics and re-entrant read/write locks may be needed. Writing back out to the socket should generally occur on another "per player thread".
PROS:
Parallel Processing
Modular (have all the handling code per player in one thread, start thread on player join)
Disconnects are easier to handle and don't cause problems with other players.
Fast network response, concurrent packet receiving.
CONS:
High resource usage (a lot more objects)
High synchronization overhead
High thread count (may be as many as 2 ~ 4x threads to players ratio)
In either case, by using TCP you will need at least one thread per player. The question is are you willing to use a lot more resources for a smoother, swifter response from the server.