I am developing a webserver in java that will provide websocket communication to its' clients. I have been proposed to use a thread pool when dealing with many clients because it is a lot more time efficient than to use one thread per client.
My question is simply, will Javas ExecutorService, newFixedThreadPool be able to handle a queue of runnable tasks with thread blocking methods being called inside of them?
In other words i guess i am wondering if this thread pool is asynchronous?
The reason i am asking is that i have tried using a newFixedThreadPool with, lets say, 2 threads. Then when i connect 3 clients to the server, i can only receive commands from the first two. But i guess i could be doing something wrong, thats why i am asking.
The runnable tasks are also in an infinite while loop (only ends when client disconnects).
Well, it depends on your implementation. The easiest case is having clients keeping their thread active until the disconnect (or get kicked out because of a timeout). In this case, your thread pool isn't very efficient. I'll only re-use disconnected users' threads instead of creating new one (which is good, but not really relevant).
The second case would be activating the threads only when needed (let's say when a client sends or receives a messages). In this case, you need to remember the server-side (keeping an id for example), in order to be able to sever the thread connection when they don't need them, and re-establish it when they do. In order to do that, you must keep the sockets somewhere, but unbound to any specific thread.
I actually didn't code that myself but I don't see why it would work as this is the mechanism used for websites (i.e. HTTP protocol)
Related
I am creating a distributed system with many machines for learning. I need to send and receive data between machines, and I am using java.nio to create that network. In one machine, I use one thread for serversocketchannel to receive data from other machines, and use for each package of data I create new thread to send it. It means that one thread for receiving and multiple threads for sending in one machines.
But I face a problem that since one thread handles receiving, many client will be pending when connecting.
Should I change it to one thread handles receiving and one thread handles sending?
Thank you
P/s: I don't want to use any 3rd party framework.
If there are more senders than receivers in your network, then obviously some of the senders will end up waiting. If you have more receivers than senders, then obviously some of the receivers will be idle, since nearly all of the time a sender will probably be connected to a receiver, one to one.
I cannot judge on what you "should do" as I don't know what you're trying to accomplish.
Anyway, the two common patterns used on the receiver side are:
One thread handles all
One thread handles just accepting the connection and opening the stream, then delegates the actual work with the stream to another thread (usually from a thread pool to prevent resource exhaustion that could happen if a new thread was created for every connection)
I'm trying to understand how to deal with threads within a Java client that connects to HornetQ. I'm not getting a specific error but fail to understand how I'm expected to deal with threads in the first place (with respect to the HornetQ client and specifically MessageHandler.onMessage() -- threads in general are no problem to me).
In case this is relevant: I'm using 'org.hornetq:hornetq-server:2.4.7.Final' to run the server embedded into my application. I don't intend this to make a difference. In my situation, that's just more convenient from an ops perspective than running a standalone server process.
What I did so far:
create an embedded server: new EmbeddedHornetQ(),
.setConfiguration()
create a server locator: HornetQClient.createServerLocator(false, new TransportConfiguration(InVMConnectorFactory.class.getName()))
create a session factory: serverLocator.createSessionFactory()
Now it seems obvious to me that I can create a session using hornetqClientSessionFactory.createSession(), create a producer and consumer for that session, and deal with messages within a single thread using .send() and .receive().
But I also discovered consumer.setMessageHandler(), and this tells me that I didn't understand threading in the client at all. I tried to use it, but then the consumer calls messageHandler.onMessage() in two threads that are distinct from the one that created the session. This seems to match my impression from looking at the code -- the HornetQ client uses a thread pool to dispatch messages.
This leaves me confused. The javadocs say that the session is a "single-thread object", and the code agrees -- no obvious synchronization going on there. But with onMessage() being called in multiple threads, message.acknowledge() is also called in multiple threads, and that one just delegates to the session.
How is this supposed to work? How would a scenario look in which MessageHandler does NOT access the session from multiple threads?
Going further, how would I send follow-up messages from within onMessage()? I'm using HornetQ for a persistent "to-do" work queue, so sending follow-up messages is a typical use case for me. But again, within onMessage(), I'm in the wrong thread for accessing the session.
Note that I would be okay with staying away from MessageHandler and just using send() / receive() in a way that allows me to control threading. But I'm convinced that I don't understand the whole situation at all, and that combined with multi-threading is just asking for trouble.
I can answer part of your question, although I hope you've already fixed the issue by now.
Form the HornetQ documentation on ClientConsumer (Emphasis mine):
A ClientConsumer receives messages from HornetQ queues.
Messages can be consumed synchronously by using the receive() methods which will block until a message is received (or a timeout expires) or asynchronously by setting a MessageHandler.
These 2 types of consumption are exclusive: a ClientConsumer with a MessageHandler set will throw HornetQException if its receive() methods are called.
So you have two choices on handling message reception:
Synchronize the reception yourself
Do not provide a MessageListener to HornetQ
In your own cunsumer Thread, invoke .receive() or .receive(long itmeout) at your leisure
Retrieve the (optional) ClientMessage object returned by the call
Pro: Using the Session you hopefully carry in the Consumer you can forward the message as you see fit
Con: All this message handling will be sequential
Delegate Thread synchronization to HornetQ
Do not invoke .receive() on a Consumer
Provide a MessageListener implementation of onMessage(ClientMessage)
Pro: All the message handling will be concurrent and fast, hassle-free
Con: I do not think it possible to retrieve the Session from this object, as it is not exposed by the interface.
Untested workaround: In my application (which is in-vm like yours), I exposed the underlying, thread-safe QueueConnection as a static variable available application-wide. From your MessageListener, you may invoke QueueSession jmsSession = jmsConnection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE); on it to obtain a new Session and send your messages from it... This is probably alright as far as I can see because the Session object is not really re-created. I also did this because Sessions had a tendency to become stale.
I don't think you should want so much to be in control of your Message execution threads, especially transient Threads that merely forward messages. HornetQ has built-in Thread pools as you guessed, and reuses these objects efficiently.
Also as you know you don't need to be in a single Thread to access an object (like a Queue) so it doesn't matter if the Queue is accessed through multiple Threads, or even through multiple Sessions. You need only make sure a Session is only accesed by one Thread, and this is by design with MessageListener.
So I've been given a school exercise, where I am to make a chatserver in java. I've done it in TCP, but I could just aswell have done it in UDP.
I'm starting to do some thread implementations, but now I'm not really sure how I should approach it, and how many threads to make.
So fare, this is my approach:
Server needs 1 thread for running, 1 thread for receiving messages, and 1 thread to send messages. Furthermore, I've made a thread for each Client connected, which the server puts in a ClientThread[], which is then used for messaging each client. This comes to a total of 13 threads ( 10 clients max )
Furthermore, I guess each local client needs a local thread, for sending and receiving messages aswell.
Is this the right approach here? Will it be problematic to have a server running 13 threads?
Thanks in advance!
Your approach looks solid, but you don't really want to handle an array of so much client threads. You should use Threadpools
You store in memory a list of already initialized threads, you only open them at startup and close them at shutdown. every time a client/server needs to send a message, you will use a thread, then return it to the pool (instead of closing it). you can also configure a pool to grow on demand
I have a java server that handles logins from multiple clients. The server creates a thread for each tcp/ip socket listener. Database access is handled by another thread that the server creates.
At the moment the number of clients I have attaching to the server is quite low (<100) so I have no real performance worries, but I am working out how I should handle more clients in the future. My concern is that with lots of clients my server and database threads will get bogged down by constant calls to their methods from the client threads.
Specifically in relation to the database: At the moment each client thread accesses the public database thread on its server parent and executes a data access method. What I think I should do is have some kind of message queue that a client thread can put its data request on and the database thread will do it when it gets round to it. If there is data to be returned from the data access call then it can put it on a queue for the client thread to pick up. All of this wouldn't hit the main server code or any other client threads.
I therefore think that I want to implement an asynchronous message queue that client threads can put a message on and the database thread will pick up from. Is that the right approach? Any thoughts and links to somewhere I can read up about implementation would be appreciated.
I would not recommend this approach.
JMS was born for this sort of thing. It'll be better than any implementation you'll write from scratch. I'd recommend using a Java EE app server that has JMS built in or something like ActiveMQ or RabbitMQ that you can add to a servlet engine like Tomcat.
I would strongly encourage you to investigate these before writing your own.
What you are describing sounds like an ExecutorCompletionService. This is essentially an asynch task broker that accepts requests (Runnables or Callables) from one thread, returning a "handle" to the forthcoming result in the form of a Future. The request is then executed in a thread pool (which could be a single thread thread pool) and the result of the request is then delivered back to the calling thread through the Future.
In between the time that the request is submitted and response is supplied, your client thread will simply wait on the Future (with an optional timeout).
I would advise, however, that if you're expecting a big increase in the number of clients (and therefore client threads), you should evaluate some of the Java NIO Server frameworks out there. This will allow you to avoid allocating one thread per client, especially since you expect all these threads to spend some time waiting on DB requests. If this is the case, I would suggest looking at MINA or Netty.
Cheers.
//Nicholas
It sounds like what you want to do is limit the number of concurrent requests to the database you want to allow. (To stop it being overloaded)
I suggest you have a limited size connection pool. When too many threads want to use the database they will have to wait until a connection is free. A simple way to do this is with a BlockingQueue with all the connections created in advance.
private final BlockingQueue<Connection> connections = new ArrayBlockingQueue<Connection>(40); {
// create connections
}
// to perform a query.
Connection conn = connections.get();
try {
// do something
} finally {
connections.add(conn);
}
This way you can keep your thread design much the same as it is and limit the number of concurrent queries to the database. With some tweaking you can create the connections as needed and provide a time out if a database connection cannot be obtained quickly.
A little help please.
I am designing a stateless server that will have the following functionality:
Client submits a job to the server.
Client is blocked while the server tries to perform the job.
The server will spawn one or multiple threads to perform the job.
The job either finishes, times out or fails.
The appropriate response (based on the outcome) is created, the client is unblocked and the response is handed off to the client.
Here is what I have thought of so far.
Client submits a job to the server.
The server assigns an ID to the job, places the job on a Queue and then places the Client on an another queue (where it will be blocked).
Have a thread pool that will execute the job, fetch the result and appropriately create the response.
Based on ID, pick the client out of the queue (thereby unblocking it), give it the response and send it off.
Steps 1,3,4 seems quite straight forward however any ideas about how to put the client in a queue and then block it. Also, any pointers that would help me design this puppy would be appreciated.
Cheers
Why do you need to block the client? Seems like it would be easier to return (almost) immediately (after performing initial validation, if any) and give client a unique ID for a given job. Client would then be able to either poll using said ID or, perhaps, provide a callback.
Blocking means you're holding on to a socket which obviously limits the upper number of clients you can serve simultaneously. If that's not a concern for your scenario and you absolutely need to block (perhaps you have no control over client code and can't make them poll?), there's little sense in spawning threads to perform the job unless you can actually separate it into parallel tasks. The only "queue" in that case would be the one held by common thread pool. The workflow would basically be:
Create a thread pool (such as ThreadPoolExecutor)
For each client request:
If you have any parts of the job that you can execute in parallel, delegate them to the pool.
And / or do them in the current thread.
Wait until pooled job parts complete (if applicable).
Return results to client.
Shutdown the thread pool.
No IDs are needed per se; though you may need to use some sort of latch for 2.1 / 2.3 above.
Timeouts may be a tad tricky. If you need them to be more or less precise you'll have to keep your main thread (the one that received client request) free from work and have it signal submitted job parts (by flipping a flag) when timeout is reached and return immediately. You'll have to check said flag periodically and terminate your execution once it's flipped; pool will then reclaim the thread.
How are you communicating to the client?
I recommend you create an object to represent each job which holds job parameters and the socket (or other communication mechanism) to reach the client. The thread pool will then send the response to unblock the client at the end of job processing.
The timeouts will be somewhat tricky, and will have hidden gotcha's but the basic design would seem to be to straightforward, write a class that takes a Socket in the constructor. on socket.accept we just do a new socket processing instantiation, with great foresight and planning on scalability or if this is a bench-test-experiment, then the socket processing class just goes to the data processing stuff and when it returns you have some sort of boolean or numeric for the state or something, handy place for null btw, and ether writes the success to the Output Stream from the socket or informs client of a timeout or whatever your business needs are
If you have to have a scalable, effective design for long-running heavy-haulers, go directly to nio ... hand coded one-off solutions like I describe probably won't scale well but would provide fundamental conceptualizing basis for an nio design of code-correct work.
( sorry folks, I think directly in code - design patterns are then applied to the code after it is working. What does not hold up gets reworked then, not before )