NIO and TCP make a great pair for many connections. Since a new connection needs to be opened for each new client, each of these clients would typically need their own thread for blocking I/O operations. NIO addresses that problem by allowing data to be read when it can, rather than blocking until it is available. But what about UDP?
I mean, connectionless UDP does not have the blocking nature of TCP associated with it because of how the protocol is designed (send it and forget it, basically). If I decide to send some data to some address, then it will do so, without delays (on server-end). Likewise, if I want to read data, I can just receive individual packets from different sources. I don't need to have many connections to many places using many threads to deal with each of them.
So, how does NIO and selectors enhance UDP? More specifically, when would one prefer to use UDP with NIO rather than the ol' java.net package?
Well the DatagramSocket.receive(...) method is documented as a blocking operation. So for instance if you had one thread that is trying to handle packets from N different sockets, you would need to use NIO and selectors. Similarly, if the thread had to multiplex checking for new packets with other activities, you might do this.
If you don't have these or similar requirements, then selectors won't help. But that's no different to the TCP case. You shouldn't use selectors with TCP if you don't need them, because it potentially adds an extra system call.
(On Linux, in the datagram case, you'd do a select syscall followed by a recv ... instead of just a recv.)
But if you're only dealing with one DatagramSocket, wouldn't the receive method read packets immediately as they arrive, regardless of the fact that they're from a different computer?
If you are listening on one socket for datagrams from "everyone" then yes. If you have different sockets for different computers then no.
And for the TCP comment, sometimes the use of a selector is justified simply by the fact that it is very resource demanding to have thousands of threads, as it would be required by a blocking TCP server.
We weren't discussing that case. But yes, that is true. And the same is true if you have thousands of threads blocking on UDP receives.
My point was that it you don't have lots of threads, or if it doesn't matter if a thread blocks, then NIO doesn't help. In fact, it may reduce performance.
NIO removes the necessity for threads altogether. It lets you handle all your clients in one thread, including both TCP and UDP clients.
connectionless UDP does not have the blocking nature of TCP associated with it
That's not true. Receives still block, and so can sends, at least in theory.
Related
I just work with an new project backed by RabbitMQ, and there are multiple consumer instances created listening to the same queue when the application starts. Howerver they shares the same connections with different channels.
The messages from the queue are massive(millions messages for one single producing behavior ) so I guess the very first code author is trying to do something to make consuming faster.
I am trying to find some posts discussing on this but I can't find a very certain answer.
What I get so far is:
Each channel will have a separate dispatch thread
The operation commands on the same channel is serialized even though they are called in multiple thread
So
creating multiple consumers thus multiple channels will have multiple dispatch threads, but I don't think it provided a better performance to message dispatching since the dispatch should far from enough with one single thread.
The operation of ack will can be paralized in different channels, I am not quite sure this will give any better performances.
Since more channels consume more system resources I wonder is this practice good?
There seem to be a few things going on here, so let's try to look at this scenario from a holistic perspective.
For starters, it sounds like the original designer of this code understood some basics about RabbitMQ (or learned a few things by trial and error), but may have had trouble putting all the pieces together- hopefully I can help.
RabbitMQ connections are, in reality, AMQP-over-TCP connections (and thus are somewhere around the session layer of the OSI model). TCP connections are supposed to be opened up and used until some sort of network interruption or application shutdown closes them (and for this reason, AMQP has trouble with firewalls and other smart network devices). Using a single TCP connection for message processing activities for a single logical process is a good idea, as creating and destroying TCP connections is usually an expensive process for the computer, which leads to
RabbitMQ channels are used to multiplex communication streams in the AMQP-Over-TCP connection (and are defined in the AMQP Protocol Spec). All they do is specify an integer value (I can't remember the number of bytes, but it doesn't matter anyway) used to preface the subsequent command or response on a TCP connection. Most AMQP operations are channel-specific. For the purposes of higher-level operations, channels are treated similar to connections, as they are application-level constructs.
Now, where I think the question starts to go off the rails a bit is here:
The messages from the queue are massive(millions messages for one
single producing behavior ) so I guess the very first code author is
trying to do something to make consuming faster.
A fundamental assumption about a system which uses queues is that messages are consumed at approximately the same rate that they are produced. Queues exist to buffer uneven producing activities. The mathematics and statistics of how queues work are quite interesting, and assuming the production of messages is done in response to some real-world stimulus, your system is virtually guaranteed to behave in a predictable manner. Therefore, your design goal is to ensure that there are enough consumers to process the messages that are produced, and to respond to changing conditions as needed. Your goal should not be to "speed up" the consumers (unless they have some specific issue), but rather to have enough consumers to process the total load.
Further, the average number of items in the queue at any time should approach zero. It is usually a good idea to have overcapacity so that you don't wind up with an unstable situation where messages start accumulating in the queue (and the queue ends up looking like the Stack Overflow Close Vote Queue).
And that brings us to an attempt to answer your fundamental question, which seems to deal with threading and possibly detailed implementation of the Java client, which I will readily admit I have not used (I'm a .NET guy).
Here are some design guidelines for your software:
Ensure that a single thread uses no more than one channel.
Use one TCP connection per logical consuming process.
Balance the number of logical processes on a single physical machine such that resource contention is not a problem (you don't want to starve your consumers of computer resources).
Try to use BASIC.GET as opposed to a push-based consumer. Use of consumers is difficult in practice, and there is no performance benefit at the protocol level over a BASIC.GET. Note I do not know if the Java library has implemented these differently such that it does cause a performance difference- stranger things have been known to happen.
If you do use consumers, make sure pre-fetch is set to 0 (disabled) and that AutoAck is set to false if reliable processing is important (most applications require reliable processing). Along with this, make sure you are acknowledging messages upon completion of processing!
Periodically reboot your consuming threads, channels, and processors - or do a BASIC.Recover. There are degrees of randomness that will result in unacknowledged messages accumulating over time, and this will deal with it.
Again, if you prefer to use consumers, generally speaking to share consumers across channels is a bad idea. Each consumer should get its own channel.
At the moment I have a project where we develop a Java Texas Holdem Application. Of course this Application is based on a client server socket system. I am saving all joined clients (I'm getting them with socketServer.accept() method) in an ArrayList. At the moment I make one thread for each joined client, which permanently checks if the client send any data to the server. My classmate told me it would be way better if I create one big Thread, that iterates through the whole Client ArrayList and checks every Client inputstreamreader. Should I trust him?
Creating a thread per Socket isn't a good idea if your application will have a lot of clients.
I'd recommend into looking into external libraries and how they handle their connonections. Example: http://netty.io/, https://mina.apache.org/
Both approaches are not feasible. Having a thread per connection will quickly exhaust resources in any loaded system. Having one thread pinging all connections in a loop will produce a terrible performance.
The proper way is to multiplex on the sockets - have a sane number of threads (16, why not), distribute all sockets between those 16 threads and multiplex on those sockets using select() variant - whatever is available in Java for this.
I am currently writing a server for a fast paced multiplayer game, which should be run in UDP (I read TCP was inappropriate due to the way it handles packet drops, in applications requiring timely delivered data, please correct me if TCP is more useful) The game is not mass multiplayer, and should be hosted primarily by the players themselves, probably on local dedicated servers. While I am pretty sure that the client needs to be in NIO, to avoid game lag from network problems, I do not know how to write the server yet. Here is a list of possibilities I considered:
Using separate thread for each player with a socket bound to each
Using one single socket to loop through every player
Doing the same, but with NIO
using one socket to broadcast messages to all clients simultaneously
using one socket for receiving, and one for sending on a separate thread
Which of these is the most appropriate approach to handling time critical data with a relatively low number of clients? (no more than 16)
For 16 clients, unless your hardware sucks REALLY badly, or you do some outrageous processing, you'll be fine performance-wise either way you go. That means that you can write it any way it makes it simpler for you to write and maintain.
A few words about your options though:
Using separate thread for each player with a socket bound to each
This approach makes more sense for a low number of TCP sockets. If you're using UDP, you only have one socket, so there's no need for that. It is however useful to have a thread per client if your game requires some parallel processing per-client.
Using one single socket to loop through every player
This is probably the easiest option, unless your game requires really special. As I've said earlier, you should be fine performance-wise. So if it makes writing the code easier do it. Avoid premature optimizations.
using one socket for receiving, and one for sending on a separate
thread
Unless you have a good reason for doing this, I'd avoid this idea. It might (needlessly) complicate things network-wise since the clients will be sending packets to port X, and getting replies from port Y. NAT will be a serious issue.
using one socket to broadcast messages to all clients simultaneously
First of all, that would only work in one direction - from the server to the clients. Second, that would only work inside a LAN since routers don't forward broadcast packets. If you're fine with both limitations and find yourself designing a system in which the server sends some packets to all players (at least to those in the same network), that might not be a bad idea. As far as performance goes, unless you're sending a large number of packets, you'll probably won't notice a difference between sending each player a separate copy, and sending a broadcast.
TCP has more overheads, leading to higher bandwidth required to transmit the same information. TCP does however have the advantage of being resent if it doesn't arrive. Make sure the messages you send over the network are losable if you use UDP. for example, if you send movements as additions and alterations then it'll lose sync quickly with the clients. You'll need to overwrite values when transmitting.
You'll need a thread per player for listening to their outputs such as keystrokes or actions, as you have to wait for data on each thread and therefore it would block on player 1 until they make an action and player 2 would not be able to make action until player 1 had.
Broadcasting game states might make sense - but i have no experience with that or something as time critical as fast paced multiplayer games
I'm creating a client/server pair in Java that, for now, only supports interlaced text communication via PrintWriters and BufferedReaders wrapped around both server and client's IO streams.
I would like to implement a function that uses Image[Input/Output]Stream to send a BufferedImage from the server to the client at a set interval.
The problem is that I want the BufferedImages to be sent/received in separate threads so that the client/server can still send/receive text commands.
Can I create multiple streams or sockets? If so, is that the best way?
One way to accomplish this with a single socket is multiplexing the individual streams over a single bytestream connected to the socket, a good implementation of this is BEEP.
Yes, sure you can create as many threads and sockets as you need. Just be careful: do not forget to close sockets and keep process of threads creation under control: to many threads do not improve your performance and may even cause your system to halt.
Probably you should use thread pool. But it depends on your application. Take a look on java.util.concurrency package.
If you have more specific questions do not hesitate to ask them.
A multiplexing stream should maintain multiple buffers.
A reader should be given it's own buffer by the multiplexing stream. The multiplexing stream should grow every buffer during a write operation, and shrink the desired buffer during a read operation.
A single rewind buffer is harder to manage, since the readers need to be stateful, but is generally more scalable, if not performant.
The specific connection protocol used is an implementation detail. Network sockets are just buffers, and can be used to implement a multiplexing stream. The network becomes the bottleneck in this case.
I am writing a server-client application. I have a server that holds several sockets that I have got from the accept() method of ServerSocket. I want to read from these sockets but I don't necesserally know which socket is ready to be read from. I need some kind of selector that will select one of the sockets that are ready to be read from, so I can read the data it sends.
Thanks.
You have basically two options to make it work:
Have dedicated thread per accepted socket. This is because the 'regular' socket I/O is blocking. You can not selectively handle multiple sockets using a single thread. And as there is no 'peeking' functionality, you will always take a risk of getting blocked when you invoke read. By having a thread per each socket you are interested in reading, blocking reads will not block any other operations (threads).
Use NIO. NIO allows for asynchronous I/O operations, and basically exactly what you asked for - a Selector.
If you do decide to go NIO-way, I would recommend checking out MINA and Netty. I've found them much easier to work with than plain NIO. Not only will you get a nicer API to work with, but at least MINA had workarounds for some nasty NIO bugs, too.