I am just trying to understand how to write a thread-per-request TCP server in Java.
I have already written a thread-per-connection server, that runs serverSocket.accept() and creates a new thread each time a new connection comes in.
How could this be modified into a thread-per-request server?
I suppose the incoming connections could be put into some sort of queue, but how would you know which one has issued a request & is ready for service?
I am suspecting that NIO is necessary here, but not sure.
Thanks.
[edit]
To be clear - The original "server" is just a loop that I have written that waits for a connection and then passes it to a new thread.
The lecturer has mentioned "thread-per-request" architecture, and I was wondering how it worked "under the hood".
My first idea about how it works, may be completely wrong.
You can use a Selector to achieve your goal. Here is a good example you can refer.
You can use plain IO, or blocking NIO, (OR non-blocking NIO, or async NIO2) You can have any multiple threads per connection (or a shared worker thread pool) but unless these are waiting for slow services like databases, this might be any faster (it can be much slower if you want low latency)
Related
I am designing architecture for android app -> server communication via TCP. It will be custom application protocol based on the TCP ... I am familiar with network programming, but I did not work a lot with the java (even, this is more general question then Java question) - and I did not work a lot with applications where you have undefined number of clients (in this case, in dependency of how many android app users you have). Hence I have few doubts and questions.
Scenario:
Users are going to install android application and login
After login, they will establish TCP connection with the server
Obviously, server side implementation needs to process requests in paralel
Consider that, both client and server side will be implemented using Java
Lets consider that application is really successfull - and it have 3mil + installations - which means a lot of users (yeah right :) )
Question:
What is the best way (best practice) to implement server side in order to handle client connections for this type of application?
Based on my research, you have only three approaches possible:
Using threadpool
Using one thread per tcp connection
Using threadpool and non blocking async approach in java (similar to what nodejs is doing with libuv)
EDIT:
Elaboration:
1. Using just a threadpool here seems "weird", due to the fact that we would need to have a huge threadpool in order to be able to assign thread per tcp connection. Using threadpool in order to serve e.g. http request (where tcp connection will be closed after request is completed), seems more then great idea.. but not for the tcp connections which are going to be used for a longer time)
2. Creating one thread per tcp connection every single time - seems limited as well (Why is creating a Thread said to be expensive?)
Java threads are implemented as native threads and huge numbers of threads is the wrong way to write a practical Java application.
I suppose that this depends, of course, and what type of application you have in general.
3. Using threadpool and non blocking async approach in java (similar to what nodejs is doing with libuv).
Based on what I read (and suggested so far as well), this seems like the best approach. Maybe its just because I have more experience with this type of applications (nodejs non blocking single threaded workers) - but seems like the best solution.
Maybe there are some ways - practices I am not familiar with - which can make this process more efficient?
Can you suggest any resources (books or similar) for this type of applications?
NOTE: Please make a note that I understand, that I am able to make this process more efficient with couple of methods I am already familiar with - for example, closing tcp connection when app goes in background - and reconnecting/establishing when user is using application again and similar (but this depends from the application itself of course).
I am wondering, am I missing something here - or it is simply as it is. If you want to have a lot of users - and lot of tcp connections - you will need to use one thread for every single user (or other approaches I mentioned above).
Other resources I went through:
Max number of threads - Linux
Increase number of threads - JVM
Max number of threads allowed to run
Runnable vs Thread
... and other external resources
At the moment I have a project where we develop a Java Texas Holdem Application. Of course this Application is based on a client server socket system. I am saving all joined clients (I'm getting them with socketServer.accept() method) in an ArrayList. At the moment I make one thread for each joined client, which permanently checks if the client send any data to the server. My classmate told me it would be way better if I create one big Thread, that iterates through the whole Client ArrayList and checks every Client inputstreamreader. Should I trust him?
Creating a thread per Socket isn't a good idea if your application will have a lot of clients.
I'd recommend into looking into external libraries and how they handle their connonections. Example: http://netty.io/, https://mina.apache.org/
Both approaches are not feasible. Having a thread per connection will quickly exhaust resources in any loaded system. Having one thread pinging all connections in a loop will produce a terrible performance.
The proper way is to multiplex on the sockets - have a sane number of threads (16, why not), distribute all sockets between those 16 threads and multiplex on those sockets using select() variant - whatever is available in Java for this.
Friends,
I am building a JAVA TCP listener, where it can handle 6000 incoming request at a time. I am creating a socket connection and accepting data, after accepting data, i am doing some operation over it by creating a thread, but i am not killing this thread, since device will send data in every two minutes, so i am only making thread to sleep mode for 30 seconds.
But after running system for five minute, my application which is running under tomcat6.0 giving error - "The web application appears to have started a thread named [Thread-214] but has failed to stop it. This is very likely to create a memory leak."
Please help me to understand where i am doing wrong?
Thanks in advance.
If you have many sockets, instead of using thread per channel.
Try using One thread that go over all sockets.
look at the Java Selector
http://www.exampledepot.com/egs/java.nio/NbClient.html
You should be aware that your operating system can not handle that much threads. Moreover, memory is allocated for each thread, so you will fill up your heap very quickly.
As I don't know what your trying to achieve, I am only guessing that you have a design flaw in your application, usually threads are reused to handle requests.
I think Selector may help. You might want to read a short introduction about Selector in this link http://tutorials.jenkov.com/java-nio/selectors.html
"A Selector is a Java NIO component which can examine one or more NIO Channel's, and determine which channels are ready for e.g. reading or writing. This way a single thread can manage multiple channels, and thus multiple network connections."
If you need to handle many TCP connections with Java - you should use NIO. But programming bare NIO (Selector) is hard - so use Netty, it's designed specially for such tasks. Also Netty works fine inside Tomcat.
may be you should use thread pool
http://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html
I need to create a relatively simple Java tcp/ip server and I'm having a little trouble determining if I should use something like Netty or just stick with simple ServerSocket and InputStream/OutputStream.
We really just need to listen for a request, then pass the new client Socket off to some processing code in a new thread. That thread will terminate once the processing is complete and the response is sent.
I like the idea of pipelines, decoders, etc. in Netty, but for such a simple scenario it doesn't seem worth the added up front development time. It seems like a bit overkill for our initial requirements, but I'm a little nervous that there are lots of things I'm not considering. What, if any, are the benefits of Netty for such simple requirements? What am I failing to consider?
The main advantage of Netty over simply reading from and writing to sockets using streams is that Netty supports non-blocking, asynchronous I/O (using Java's NIO API); when you use streams to read and write from sockets (and you start a new thread for each connected accepted from a ServerSocket) you are using blocking, synchronous I/O.
The Netty approach scales much better, which is important if your system needs to be able to handle many (thousands) of connections at the same time. If your system does not need to scale to many simultaneous connections, it might not be worth the trouble to use a framework like Netty.
Some more background information: Threads are relatively expensive resources in an operating system. Each thread needs memory for the stack (which can be for example 2 MB in size). When you create thousands of threads, this is going to cost a lot of memory; also, operating systems have limits on the number of threads that can be created. So you don't want to start a new thread for each accepted connection. The idea of asynchronous I/O is to decouple the threads from the connections (no one-to-one relation). There can be many more connections than threads, and whenever some event happens on one of the connections (for example, data is received), a thread from a thread pool is temporarily used to handle the event.
I think that the benefits of using netty are not immediate but actually come later when requirements change and maintenance becomes more complex for your project. Netty brings built in understanding of the HTTP protocol so that you can provide simple RESTful web services. Also you have the option of utilizing asynchronous request processing that netty provides as a framework so that you can potentially get better performance and service several orders of magnitude more concurrent requests.
First, write the logic of your service so that it's independent of your communication layer.
As Victor Sorokin said, there's a learning advantage to doing it yourself. So it ought to be worthwhile to write it with sockets. It will involve less effort to get started, and if it works well enough then you're off to the races.
If you find that you need more scalability/robustness later, you can switch to netty. Just write a new netty layer that communicates for your service logic layer and swap them out.
I am writing a server-client application. I have a server that holds several sockets that I have got from the accept() method of ServerSocket. I want to read from these sockets but I don't necesserally know which socket is ready to be read from. I need some kind of selector that will select one of the sockets that are ready to be read from, so I can read the data it sends.
Thanks.
You have basically two options to make it work:
Have dedicated thread per accepted socket. This is because the 'regular' socket I/O is blocking. You can not selectively handle multiple sockets using a single thread. And as there is no 'peeking' functionality, you will always take a risk of getting blocked when you invoke read. By having a thread per each socket you are interested in reading, blocking reads will not block any other operations (threads).
Use NIO. NIO allows for asynchronous I/O operations, and basically exactly what you asked for - a Selector.
If you do decide to go NIO-way, I would recommend checking out MINA and Netty. I've found them much easier to work with than plain NIO. Not only will you get a nicer API to work with, but at least MINA had workarounds for some nasty NIO bugs, too.