I have a Java application that require communication between different process. Process could run in same JVM or different JVM, but runs on the same machine.
My application need to submit "messages" to another process (same or different JVM) and forgot about it. similar to messaging queue like IBM "MQ", but simple, and only use memory, no IO to hard disk for performance gains.
I'm not sure what is the best approach from Performance prescriptive.
I wonder if RMI is efficient in terms of Performance, I think it require some overhead.
What about TCP/IP socket using local host?
any other thought?
I wonder if RMI is efficient in terms of Performance, I think it require some overhead.
RMI is efficient for what it does. It does much more than most people need, but is usually more than faster enough. You should be able to get of the order of 1-3 K messages per second with a latency around 1 milli-second.
What about TCP/IP socket using local host?
That is always an option but with plain Java Serialization this will not be a lot faster than using RMI. How you do the serialization and deserialization is critical for high performance.
An important note is that much of the time is spent serializing and deserilizing the message, something most transports don't help you with, so if you want maximum performance you have to consider an efficient marshalling strategy. Most transport protocols only benchmark raw bytes.
Ironically if you are willing to use disk, it can be faster than TCP or UDP (like ZeroMQ) plus you get persistence for "free".
This library (I am the author) can perform millions of messages per second between processes with latency as low as 100 nano-second (350x lower than ZeroMQ) https://github.com/peter-lawrey/Java-Chronicle Advantages are
ultra fast serialization and deserialization, something most transport benchmarks avoid including this as it often takes much longer than the transport costs.
is that you can monitor what is happening between queues any time after the message was sent.
replay all messages.
the producer can be any amount of data ahead of your consumer to handle micro-burst gracefully up to the size of your disk space. e.g. the consumer can be TBs behind.
supports replication over TCP.
restart of either the consumer or producer is largely transparent.
If you are developing server application try to consider ZeroMQ. It has great performance, allow to build interprocess communication easier, allow to build asynchronous API.
ZeroMQ declare fantastic performance with InterProcess communication. Even better than TCP sounds great. We are consider this solution for our clusterisation schema.
Pieter Hintjens give the great answer for performance comparison between different Message Broker.
Related
I'm going to develop a little Android game with multiplayer feature. I've made a server framework in C++ using eNet library and I would like to use this framework for make the server.
So, there is any networking library like eNet compatible with Java and C++? I know that exist jEnet (but is very out of date Java-enet-wrapper (https://github.com/csm/java-enet-wrapper), it's immature.
Check out https://github.com/julienr/libenet-android.
ENet is much more advisable than UDT in your case as UDT can be processor intensive and a gaming service would at least hope for many connections. The difference is from UDTs implementation of congestion control which has relatively high CPU demand. UDT is awesome, but designed more for large, high bandwith transfers over long distances, rather than small, high latency transactions which are desired in gaming.
Also note that main stream congestion control algorithms do not do well with small transactions. They work by monitoring RTT of each packet in a transaction and/or by monitoring packet loss rate within a transaction, which is moot when each transation is only 1 - 2 packets on avg. The additional demands of the congestion control protocol will effect latency even though the congestion control itself is not likely to ever be engaged if transfers are kept small.
You might try out UDT: http://udt.sourceforge.net/
I have used it before with good success to communicate between Java and C++ processes.
I would like to ask what would be more appropriate to choose when developing a server similar to SmartFoxServer. I intend to develop a similar yet different server. In the benchmarks made by the ones that developed the above server they had something like 10000 concurrent clients.
I made a bit of research regarding the cost of using too many threads(>500) but cannot decide which way to go. I once made a server in java but that was for a small application and had nothing to do with heavy loads.
Thanks
Take a look at Apache Mina. They've done alot of the heavy lifting required to use NIO effectively in a networking application. Whether or not NIO increases your ability to process concurrent connections really depends on your implementation, but the performance boosts in Tomcat, JBoss and Jetty are plenty evidence to you already in the positive.
i'm not familiar with smartfoxserver, so i can only speak generically (which is not always good :P but here i go)
i think those are 2 different questions. on one hand, the io performance when using native java sockets vs. native sockets written in c (like tomcat).
the other question is how to scale up to that kind of concurrency level. other than that, i'd always choose native sockets (i.e: c).
now, how to scale: it's not a good idea to have a lot of threads running at the same time (os constraints, etc), so i'd choose to scale horizontally, meaning to add a load balancer that can send the requests to different servers that can be linked by using messages (using jms, like rabbitmq or activemq, or even using a protocol like stomp or amqp).
other solution, a cloud environment that allows you to grow your installation as you need
In most benchmarks which test 10K or 100K connections, the server is doing no work and unless your server does next to nothing, these test are unrealistic.
You need to take a clear idea of mow many concurrent connections you want to support.
If you have less than 1K connection, using a thread per connection will work ok. This is the simplest approach to take. Using a dispatcher model with NIO will work better if your request are very simple. Otherwise it won't matter much.
If you have more than 1K connections it is likely you want to use more than one server as each connection is getting less than 1% of a core and the cost of a basic server is relatively cheap these days.
Please suggest java library, that implements reliable udp. It will be used for a game server to communicate to clients and to other servers.
PS Maybe you can suggest tech that will be more productive to work with for such task(game server)? But this must work on linux.
Edit: It's an action type game, so it needs to talk to server as fast as possible.
Edit 2: I found Enet which was used for a FPS game, but it's C++, will there be an overhead if I call it many times a second?
These are the libraries/frameworks I know of that implement something like reliable UDP:
Mobile Reliable UDP (MR-UDP)
MR-UDP aims at providing reliable communication based on UDP from/to mobile nodes (MNs), with least possible overhead. It extends a Reliable UDP (R-UDP) protocol with mobility-tolerating features, such as the ability to handle intermit-tent connectivity, Firewall/NAT traversal and robustness to switching of IP addresses or network interfaces (e.g. Cellular to WiFi, and vice-versa).
UDT-Java
Java implementation of UDP-based Data Transfer (UDT)
UDT is a reliable UDP based application level data transport protocol for distributed data intensive applications over wide area high-speed networks. UDT uses UDP to transfer bulk data with its own reliability control and congestion control mechanisms. The new protocol can transfer data at a much higher speed than TCP does. UDT is also a highly configurable framework that can accommodate various congestion control algorithms.
JNetRobust
Fast, reliable & non-intrusive message-oriented virtual network protocol for the JVM 1.6+.
It resides between the transport and the application layer.
Characteristics:
reliability of transmitted data
received, unvalidated data is available immediately
the package is bigger than UDP's package, but smaller than TCP's package
no flow control
no congestion control
Disclaimer: I'm the author of JNetRobust, it's new and still in alpha.
There is an java implementation of RUDP (Reliable UDP) protocol (RFC908, RFC1151)
http://sourceforge.net/projects/rudp/?source=dlp
You may find you don't need reliable messaging for all message types. For example, if you are repeatedly sending the status of things like players, and a few packets are lost it may not even matter.
There are reliable high performance UDP based libraries which support Java. One of these is 29West's LBM. It is not cheaper because it is very hard to get this right. Even with a professional product you may need a dedicated network for UDP to minimize loss.
For the purpose of a game I suggest you use a JMS service like ActiveMQ which runs wherever you can run Java. You should be able send 10K messages per second with a few milli-seconds latency.
When people say something must be as fast as possible, this can mean just about anything. For some people this means 10 ms, 1 ms, 100 us, 10 us, 1 us is acceptable. Some network routers support passing packets with a 600 ns latency. The lower the latency the greater the cost and the greater the impact on the design. Assuming you need more speed than you need can impact the design and cost unnecessarily.
You have to be realistic seeing that you have a human interface. A human cannot respond faster than about 1/20 of a second or about 50 ms. If you keep the messaging to less than 5 ms, a human will not be able to tell the difference.
Libjitsi has SCTP over UDP, which breaks everything up into packets like UDP but guarantees reliable delivery, like TCP. See https://github.com/jitsi/libjitsi/blob/master/src/org/jitsi/sctp4j/Sctp.java
UDP is by definition not a reliable service. It does not guarantee a high quality of service. You can, however, use TCP.
I've read several posts about java.net vs java.nio here on StackOverflow and on some blogs. But I still cannot catch an idea of when should one prefer NIO over threaded sockets. Can you please examine my conclusions below and tell me which ones are incorrect and which ones are missed?
Since in threaded model you need to dedicate a thread to each active connection and each thread takes like 250Kilobytes of memory for it's stack, with thread per socket model you will quickly run out of memory on large number of concurrent connections. Unlike NIO.
In modern operating systems and processors a large number of active threads and context switch time can be considered almost insignificant for performance
NIO throughoutput can be lower because select() and poll() used by asynchronous NIO libraries in high-load environments is more expensive than waking up and putting to sleep threads.
NIO has always been slower but it allows you to process more concurrent connections. It's essentially a time/space trade-off: traditional IO is faster but has a heavier memory footprint, NIO is slower but uses less resources.
Java has a hard limit per concurrent threads of 15000 / 30000 depending on JVM and this will limit thread per connection model to this number of concurrent connections maximum, but JVM7 will have no such limit (cannot confirm this data).
So, as a conclusion, you can have this:
If you have tens of thousands concurrent connections - NIO is a better choice unless a request processing speed is a key factor for you
If you have less than that - thread per connection is a better choice (given that you can afford amount of RAM to hold stacks of all concurrent threads up to maximum)
With Java 7 you may want to go over NIO 2.0 in either case.
Am I correct?
That seems right to me, except for the part about Java limiting the number of threads – that is typically limited by the OS it's running on (see How many threads can a Java VM support? and Can't get past 2542 Threads in Java on 4GB iMac OSX 10.6.3 Snow Leopard (32bit)).
To reach that many threads you'll probably need to adjust the stack size of the JVM.
I still think the context switch overhead for the threads in traditional IO is significant. At a high level, you only gain performance using multiple threads if they won't contend for the same resources as much, or they spend time much higher than the context switch overhead on the resources.
The reason for bringing this up, is with new storage technologies like SSD, your threads come back to contend on the CPU much quicker
There is not a single "best" way to build NIO servers, but the preponderance of this particular question on SO suggests that people think there is! Your question summarizes the use cases that are suited to both options well enough to help you make the decision that is right for you.
Also, hybrid solutions are possible too! You could hand the channel off to threads when they are going to do something worthy of their expense, and stick to NIO when it is better.
I would say start with thread-per-connection and adapt from there if you run into problems.
If you really need to handle a million connections you should consider writing (or finding) a simple request broker in C (or whatever) that will use far less memory per connection than any java implementation can. The broker can receive requests asynchronously and queue them to backend workers written in your language of choice.
The backends thus only need a thread per active request, and you can just have a fixed number of them so the memory and database use is predetermined to some degree. When large numbers of requests are running in parallel the requests are made to wait a bit longer.
Thus I think you should never have to resort to NIO select channels or asynchronous I/O (NIO 2) on 64-bit systems. The thread-per-connection model works well enough and you can do your scaling to "tens or hundreds of thousands" of connections using some more appropriate low-level technology.
It is always helpful to avoid premature optimization (i.e. writing NIO code before you really have massive numbers of connections coming in) and don't reinvent the wheel (Jetty, nginx, etc.) if possible.
What most often is overlooked is that NIO allows zero copy handling. E.g. if you listen to the same multicast traffic from within multiple processes using old school sockets on one single server, any multicast packet is copied from the network/kernel buffer to each listening application. So if you build a GRID of e.g. 20 processes, you get memory bandwidth issues. With nio you can examine the incoming buffer without having to copy it to application space. The process then copies only parts of the incoming traffic it is interested in.
another application example:
see http://www.ibm.com/developerworks/java/library/j-zerocopy/ for an example.
I'm pretty new to web programming and I'm currently developing a web back end for a mobile application. Currently I have the users log in using servlet interactions and once they have full access to the application I need to open a Socket Connection so that I can provide server pushes. Now the problem I'm running into is how people handle thousands of concurrent socket connections. I've run into people talking about ThreadPools which seems pretty easy to implement and NIO. Is there some framework that I can work with to ensure my servers are handling at least 20-30k concurrent connections. I could also forget TCP connections and go for Long-polling but from my understanding TCP is best option resource wise.
#Steve - I'm looking at the former: One serversocket with thousands of connections.
I would look into clustering the web end immediately and use that as your primary scaling mechanism. 30k connections is quite a lot and you don't have much room for growth before you hit a server limit of some kind. If the I/O itself isn't onerous I would just use lots of threads and servers with lots of horsepower and memory. Get it working that way so you can ship, and have a fallback plan to switch to multiplexed NIO if performance or scaling becomes a problem, but be warned that it's a radical overhaul and about ten times as complex to program as java.net. After several years' consideration I am more and more wondering whether NIO to economize on threads is really worth it: it adds several new problems of its own such as a need for push parsing; synchronization issues with the selector if there are worker threads that need to change the registration state of channels; lots of ways to get the code wrong; and the fact that the scheduling overhead moves out of the OS into your application, where you only have linear set-iterator data structures to deal with it unless you engage in yet another level of complexity. It's worth remembering that select() was invented for Unix to allow economizing on processes, which are expensive. Threads are pretty cheap really, and provide a very simple programming model with built-in context for handling a single connection. NIO barely manages this at all except via disciplined use of selection key attachments, much less naturally.