At the moment I have a project where we develop a Java Texas Holdem Application. Of course this Application is based on a client server socket system. I am saving all joined clients (I'm getting them with socketServer.accept() method) in an ArrayList. At the moment I make one thread for each joined client, which permanently checks if the client send any data to the server. My classmate told me it would be way better if I create one big Thread, that iterates through the whole Client ArrayList and checks every Client inputstreamreader. Should I trust him?
Creating a thread per Socket isn't a good idea if your application will have a lot of clients.
I'd recommend into looking into external libraries and how they handle their connonections. Example: http://netty.io/, https://mina.apache.org/
Both approaches are not feasible. Having a thread per connection will quickly exhaust resources in any loaded system. Having one thread pinging all connections in a loop will produce a terrible performance.
The proper way is to multiplex on the sockets - have a sane number of threads (16, why not), distribute all sockets between those 16 threads and multiplex on those sockets using select() variant - whatever is available in Java for this.
Related
I am designing architecture for android app -> server communication via TCP. It will be custom application protocol based on the TCP ... I am familiar with network programming, but I did not work a lot with the java (even, this is more general question then Java question) - and I did not work a lot with applications where you have undefined number of clients (in this case, in dependency of how many android app users you have). Hence I have few doubts and questions.
Scenario:
Users are going to install android application and login
After login, they will establish TCP connection with the server
Obviously, server side implementation needs to process requests in paralel
Consider that, both client and server side will be implemented using Java
Lets consider that application is really successfull - and it have 3mil + installations - which means a lot of users (yeah right :) )
Question:
What is the best way (best practice) to implement server side in order to handle client connections for this type of application?
Based on my research, you have only three approaches possible:
Using threadpool
Using one thread per tcp connection
Using threadpool and non blocking async approach in java (similar to what nodejs is doing with libuv)
EDIT:
Elaboration:
1. Using just a threadpool here seems "weird", due to the fact that we would need to have a huge threadpool in order to be able to assign thread per tcp connection. Using threadpool in order to serve e.g. http request (where tcp connection will be closed after request is completed), seems more then great idea.. but not for the tcp connections which are going to be used for a longer time)
2. Creating one thread per tcp connection every single time - seems limited as well (Why is creating a Thread said to be expensive?)
Java threads are implemented as native threads and huge numbers of threads is the wrong way to write a practical Java application.
I suppose that this depends, of course, and what type of application you have in general.
3. Using threadpool and non blocking async approach in java (similar to what nodejs is doing with libuv).
Based on what I read (and suggested so far as well), this seems like the best approach. Maybe its just because I have more experience with this type of applications (nodejs non blocking single threaded workers) - but seems like the best solution.
Maybe there are some ways - practices I am not familiar with - which can make this process more efficient?
Can you suggest any resources (books or similar) for this type of applications?
NOTE: Please make a note that I understand, that I am able to make this process more efficient with couple of methods I am already familiar with - for example, closing tcp connection when app goes in background - and reconnecting/establishing when user is using application again and similar (but this depends from the application itself of course).
I am wondering, am I missing something here - or it is simply as it is. If you want to have a lot of users - and lot of tcp connections - you will need to use one thread for every single user (or other approaches I mentioned above).
Other resources I went through:
Max number of threads - Linux
Increase number of threads - JVM
Max number of threads allowed to run
Runnable vs Thread
... and other external resources
I'm creating a Java server for a quite big simulation and I have a couple of high level design questions.
Some background:
The server will run a simulation.
Clients will connect to the server via TCP connections from mobile devices and interact with data structures in the simulation. Initially I will try to use a simple polling scheme in the clients. I find it hard to maintain long-lived TCP connections between mobile devices and the server and I'm not yet sure whether the clients will try to keep an open TCP connection or whether they will set it up and tear it down for each transmission.
When a client is active on a mobile device, I would like to have the client poll the server at least a few times a minute.
The simulation will keep running regardless of whether clients are connected or not.
The total number of existing clients could get very large, many thousands.
Clients mostly poll the server for simulation state, but also sometimes issues control commands to the simulation.
All messages are small in size.
I expect the server to run under Linux on multi-core CPU server hardware.
Currently I have the following idea for threading model in the server:
The simulation logic is executed by a few threads. The simulation logic threads both read and write from/to the simulation data structures.
For each client there is a Java thread performing a blocking read call to the socket for that client. When a poll command is received from a client, the corresponding client thread reads info from the simulation data structures (one client poll would typically be interested in a small subset of the total data structures) and sends a reply to the client on the client's socket. Thus, access to the data structures would need to be synchronized between the client threads and the simulation threads (I would try to have the locks on smaller subsets of the data). If a control command is received from the client, the client thread would write to the data structures.
For small number of clients, I think this would work fine.
Question 1: Would this threading model hold for a large number (thousands) of connected clients? I'm not familiar with what memory/CPU overhead there would be in such a Java implementation.
Question 2: I would like to avoid having the server asynchronously send messages to the clients but in certain scenarios I may need to have the server send "update yourself now" messages asynchronously to some or many clients and I'm not quite sure how to do that. Having the simulation logic thread(s) send those messages doesn't seem right... maybe some "client notification thread pool" concept?
You ask two questions; I'll answer the first.
I've previously written an application that involved thousands of threads in one application. We did once run into a problem with the maximum number of threads on the Linux server; for us, I think the limit was about 1000 threads. This affected our Java application because Java threads use native threads. We set the limit higher, and the application scaled to about 2000 threads, which was what we needed without an issue; I don't know what would have happened had we needed to scale it much higher.
The fact that the default maximum number of threads was 1000 suggests that it might not be wise to run too many thousands of threads on a single Linux server. I believe the primary issue is that sufficient memory for a stack needs to be allocated for each thread.
Our intended long term fix was to change to an architecture where threads from a thread pool each serviced multiple sockets. This really isn't too much of an issue; for each socket, the thread just has to process any pending messages before going on to the next socket. You would have to be careful about synchronizing memory access, but your application already needs to do that since your simulation interacts with multiple threads already so that part would not be a huge change.
I am writing a program to demonstrate the distance vector routing algorithm. This program creates multiple routers and their routing table. Each router is its own thread using a two-dimensional array. I need to have the threads send their tables to one another. I am not sure how to communicate between these threads. I looked into sockets but am not listening on any ports. I am not sure how to use pipes or if this is the best option or not. Any suggestions would be helpful.
If you run two different processes, you can use Sockets, JMS or files to share information.
If you run just two threads in one process, you should create some thread safe storage, say ConcurrentLinkedQueue for this. Here is more of them http://javarevisited.blogspot.com/2013/02/concurrent-collections-from-jdk-56-java-example-tutorial.html
Have each Runnable or Thread that implements your routing thread expose (say) a java.util.concurrent.TransferQueue<YourTableType> and deliver your tables to it. The routing thread can pull the tables from the transfer queue at it's leisure.
The java.util.concurrent package is extremely well documented, so it's worth having a good look around in it.
Why don't create a server/client and make the call from the client to the server and then the server send the answer to the other client ?
You'll need to create 2 files, server.java and client.java,
Make the connection client/server via TCP,
Messages from client to client via UDP;
If you need any help just ask, I've made a few projects like that in university.
I am just trying to understand how to write a thread-per-request TCP server in Java.
I have already written a thread-per-connection server, that runs serverSocket.accept() and creates a new thread each time a new connection comes in.
How could this be modified into a thread-per-request server?
I suppose the incoming connections could be put into some sort of queue, but how would you know which one has issued a request & is ready for service?
I am suspecting that NIO is necessary here, but not sure.
Thanks.
[edit]
To be clear - The original "server" is just a loop that I have written that waits for a connection and then passes it to a new thread.
The lecturer has mentioned "thread-per-request" architecture, and I was wondering how it worked "under the hood".
My first idea about how it works, may be completely wrong.
You can use a Selector to achieve your goal. Here is a good example you can refer.
You can use plain IO, or blocking NIO, (OR non-blocking NIO, or async NIO2) You can have any multiple threads per connection (or a shared worker thread pool) but unless these are waiting for slow services like databases, this might be any faster (it can be much slower if you want low latency)
I am designing a application where many clients connect to a central server. This server keeps these connections, sending keep-alives every half-hour. The server has a embedded HTTP server, which provides a interface to the client connections (ex. http://server/isClientConnected?id=id). I was wondering what is the best way to go about this. My current implementation is in Java, and I just have a Map with ID's as the key, but a thread is started for each connection, and I don't know if this is really the best way to do this. Any pointers would be appreciated.
Thanks,
Isaac Waller
Use the java.nio package, as described on this page: Building Highly Scalable Servers with Java NIO. Also read this page very carefully: Architecture of a Highly Scalable NIO-Based Server.
Personally I'd not bother with the NIO internals and use a framework like Apache MINA or xSocket. NIO is complicated and easy to get wrong in very obscure ways. If you want it to "just work", then use a framework.
With a single thread per connection you can usually scale up to about 10,000 connections on a single machine. For a Windows 32 machine, you probably will hit a limit around 1,000 connections.
To avoid this, you can either change the design of your program, or you can scale out (horizontal). You have to weight the cost of development with the cost of hardware.
The single thread per user, with a single continuous connection is usually the easiest programming model. I would stick with this model until you reach the limits of your current hardware. At that point, I would decide to either change the code, or add more hardware.
If the clients will be connected for long periods of time, allocating a thread per client can be problematic. Each thread on the server requires a certain amount of resources (memory for the stack, for example).
You could use Jetty Continuations to handle the client request with fewer threads by using asynchronous servlets.
Read more about the the Reactor pattern. There is an implementation for that in Java (it uses channels instead of thread for client).
It is easy to implement and very efficient.