Do the instances of Netty's channel handlers such as (SimpleChannelInboundHandler, ChannelInboundHandlerAdapter, etc.) share the same thread and stack or do each have it’s own thread and stack? I ask because I instantiate numerous channel handlers and I need them to communicate with each other and I must decide between using thread communication or non- threaded communication.
Thank you for your answer
As a general rule, if your handler has state, then it's one handler per channel (pipeline). Otherwise, annotate your handler with #ChannelHandler.Sharable and use the same instance for each channel.
The answer: it depends
I assume that you must be building a server, some of what I say might not apply.
According to https://netty.io/4.0/api/io/netty/channel/ChannelHandler.html, one of the factors that determines which thread your channel handler runs on is how you add it to the pipeline. Netty allows the capability to use a single instance of your handler for all pipelines (i.e. Connections to your server), in which case you must accommodate for different threads.
In contrast, if you use a channel initializer to add handlers to the pipeline, then no, you do not need to communicate between threads because each connection uses a different instance of the handler.
This is also given that you are using more than one worker thread to run your channel handlers. You must also account for which channel your handlers are communicating with, if you don't store a state variable using a channel initialized handler, then you must accommodate for inter-thread communication.
Your best bet is to debug the current thread in each handler before making a decision, netty's threading behavior and how it interacts with your program is highly subjective on your implementation and where you are moving your data.
Related
In my understanding of Netty's, incoming message passed to eventLoop(only one eventLoop, one thread). Next, EventLoop doesn't process it, but pass it to ExecutorService(it holds multiple threads in pool) for execution.
All this happens with the help of NIO. EventLoop waits for incoming messages and pass it by selectors, keys, channels etc.
Am I right?
Netty 4 is used
As far as i know Netty uses the EvenLoopGroups for handeling incoming and outgoing data as well as incoming connections.
That shouln't be so interesting when you start using Netty as the way the data goes through different classes. When a message is inbound the first interfacce you can intercept it is the decoder (ByteToMessageDecoder) where your encrypted ByteBuf is available. Then its making its way through the handler (ChannelInboundHandler).
Since channel is not thread safe, I can either synchronize th channel instance before publish or I create a channel each time I need and close it.
But in my opinion neither of them have a good performance due to cost of locking or create and destory channels.
So how should I publish message to rabbitmq with high tps? Any good pratise on this?
So, first thing first. A channel is not a connection. In RabbitMQ, Channels are the same thing as application sessions, whereas a connection represents an underlying TCP session with the server. This answer explains some of that nicely.
TCP sessions are expensive to create, so they tend to have a lifetime outside the lifetime of any particular worker thread. Channels are extremely cheap to create - all the server does is assign you an integer for your channel identifier and you have a new channel.
Most operations on RabbitMQ close a channel when they fail. This is done because there is no practical consequence of doing so. Would they close the underlying connection, that would cause a lot of problems for the app.
Design Guidance
Pooling would be appropriate for connections, if you really have a lot of processing going on. A discussion on how to do this is really beyond what I can provide in a short answer.
Pooling is absolutely not appropriate for channels. A channel is a lightweight construct that is designed to have a transient lifetime. If it lasts longer than one or two publishes, great. But you should expect that every time you try an operation, there is a possibility it will fail and close the channel. This does not close the underlying connection, but a new channel will have to be reestablished to do anything with the broker.
Consumer lifetimes are tied to channels. When the channel closes, the attached consumer is closed too. Design your consumer objects (worker threads) to be able to get a connection and create a new channel when this happens, and then re-subscribe themselves.
Avoid sharing a channel across threads. One thread = one channel.
While I don't have any particular experience with the Java client, I don't believe locks should be necessary, and I would certainly hope that the implementation doesn't do something screwy to make Channels anything other than lightweight.
If you're writing your own protocol implementation library (not necessary, but also not a bad idea if you need fine-grained control), allocate one thread to manage each connection. Don't do reads and writes to the TCP socket in parallel, or you'll break the protocol.
Regarding the Java client, I think you can assume that channel operations (reads and writes, etc.) are NOT thread-safe, which is why you want to stick to one thread/one channel paradigm. I think you can assume that creating channels (a connection operation) is thread-safe.
You should use a pool.
For instance, use Apache's Generic Object Pool and provide an implementation for opening, closing and checking connections. When you need to publish a message, you borrow a channel from the pool, use it, and return it.
Do we need to create individual channels for each thread or use the same channel for all threads? Also the same question about connection. Do we need to use different connections for each thread or a single connection?
What is the difference when we use one channel across all threads and individual channels for each thread?
Connection:
According to the java doc (https://www.rabbitmq.com/releases/rabbitmq-java-client/v3.6.5/rabbitmq-java-client-javadoc-3.6.5/):
Current implementations are thread-safe for code at the client API level, and in fact thread-safe internally except for code within RPC calls.
Channel:
According to the doc (https://www.rabbitmq.com/api-guide.html):
Channel instances must not be shared between threads. Applications should prefer using a Channel per thread instead of sharing the same Channel across multiple threads. While some operations on channels are safe to invoke concurrently, some are not and will result in incorrect frame interleaving on the wire. Sharing channels between threads will also interfere with * Publisher Confirms.
I'm trying to understand how to deal with threads within a Java client that connects to HornetQ. I'm not getting a specific error but fail to understand how I'm expected to deal with threads in the first place (with respect to the HornetQ client and specifically MessageHandler.onMessage() -- threads in general are no problem to me).
In case this is relevant: I'm using 'org.hornetq:hornetq-server:2.4.7.Final' to run the server embedded into my application. I don't intend this to make a difference. In my situation, that's just more convenient from an ops perspective than running a standalone server process.
What I did so far:
create an embedded server: new EmbeddedHornetQ(),
.setConfiguration()
create a server locator: HornetQClient.createServerLocator(false, new TransportConfiguration(InVMConnectorFactory.class.getName()))
create a session factory: serverLocator.createSessionFactory()
Now it seems obvious to me that I can create a session using hornetqClientSessionFactory.createSession(), create a producer and consumer for that session, and deal with messages within a single thread using .send() and .receive().
But I also discovered consumer.setMessageHandler(), and this tells me that I didn't understand threading in the client at all. I tried to use it, but then the consumer calls messageHandler.onMessage() in two threads that are distinct from the one that created the session. This seems to match my impression from looking at the code -- the HornetQ client uses a thread pool to dispatch messages.
This leaves me confused. The javadocs say that the session is a "single-thread object", and the code agrees -- no obvious synchronization going on there. But with onMessage() being called in multiple threads, message.acknowledge() is also called in multiple threads, and that one just delegates to the session.
How is this supposed to work? How would a scenario look in which MessageHandler does NOT access the session from multiple threads?
Going further, how would I send follow-up messages from within onMessage()? I'm using HornetQ for a persistent "to-do" work queue, so sending follow-up messages is a typical use case for me. But again, within onMessage(), I'm in the wrong thread for accessing the session.
Note that I would be okay with staying away from MessageHandler and just using send() / receive() in a way that allows me to control threading. But I'm convinced that I don't understand the whole situation at all, and that combined with multi-threading is just asking for trouble.
I can answer part of your question, although I hope you've already fixed the issue by now.
Form the HornetQ documentation on ClientConsumer (Emphasis mine):
A ClientConsumer receives messages from HornetQ queues.
Messages can be consumed synchronously by using the receive() methods which will block until a message is received (or a timeout expires) or asynchronously by setting a MessageHandler.
These 2 types of consumption are exclusive: a ClientConsumer with a MessageHandler set will throw HornetQException if its receive() methods are called.
So you have two choices on handling message reception:
Synchronize the reception yourself
Do not provide a MessageListener to HornetQ
In your own cunsumer Thread, invoke .receive() or .receive(long itmeout) at your leisure
Retrieve the (optional) ClientMessage object returned by the call
Pro: Using the Session you hopefully carry in the Consumer you can forward the message as you see fit
Con: All this message handling will be sequential
Delegate Thread synchronization to HornetQ
Do not invoke .receive() on a Consumer
Provide a MessageListener implementation of onMessage(ClientMessage)
Pro: All the message handling will be concurrent and fast, hassle-free
Con: I do not think it possible to retrieve the Session from this object, as it is not exposed by the interface.
Untested workaround: In my application (which is in-vm like yours), I exposed the underlying, thread-safe QueueConnection as a static variable available application-wide. From your MessageListener, you may invoke QueueSession jmsSession = jmsConnection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE); on it to obtain a new Session and send your messages from it... This is probably alright as far as I can see because the Session object is not really re-created. I also did this because Sessions had a tendency to become stale.
I don't think you should want so much to be in control of your Message execution threads, especially transient Threads that merely forward messages. HornetQ has built-in Thread pools as you guessed, and reuses these objects efficiently.
Also as you know you don't need to be in a single Thread to access an object (like a Queue) so it doesn't matter if the Queue is accessed through multiple Threads, or even through multiple Sessions. You need only make sure a Session is only accesed by one Thread, and this is by design with MessageListener.
In netty, events that flow through a channel pipeline occur in order as each channel is effectively only assigned to one thread and each handler calls each other in turn. This makes sense and alleviates many syncronisation issues.
However if you use an IdleStateHandler, from my reading of the source, it appears that the channelIdle event will be 'processed' in the context of the Timers thread (the thread the HashedWheelTime is using for example).
Is this case or have I missed something? If it is the case, does this mean that it is possible for an idlestate event and an IO event (example a messageRecieved event) to be executing on the same channel at the same time?
Also, as I could save the ChannelHandler ctx and use it in a different thread to 'write' to a channel for example, thus also have an event going downstream in one thread and upstream in another thread at the same time on the same channel?
Finally which thread do ChannelFutures execute in?
All of these use cases are perfectly acceptable and not a criticism of Netty at all, I am actually very fond of the library and make use of it everywhere. Its just that as I try to do more and more complex things with it, I want to understand more about how it works under the hood so that I can ensure I am using the proper and just the right amount (no more, no less) of syncronisation (locking).
Yes the channelIdle event and downstream / upstream event could be fired at the same time. So if your handler does implement at least two of the three you need to add proper synchronization. I guess we should make it more clear in the javadocs..
Now the other questions..
You can call Channel.write(...) from every thread you want too.. So its possible to just store it somewhere and call write whenever you want. This also gives the "limitation" that you need have proper synchronization for "downstream" handlers.
ChannelFutures are executed from within the worker thread.
However if you use an IdleStateHandler, from my reading of the source, it appears that the channelIdle event will be 'processed' in the context of the Timers thread (the thread the HashedWheelTime is using for example).
Is this case or have I missed something? If it is the case, does this mean that it is possible for an idlestate event and an IO event (example a messageRecieved event) to be executing on the same channel at the same time?
Yes, you have missed the concept of ordered event execution available in the Netty, If you are not haveing ExecutionHandler with OrderedMemoryAwareThreadPoolExecutor in the pipeline, you can not have the ordered event execution, specially channel state events from IdleStateHandler.
Also, as I could save the ChannelHandler ctx and use it in a different thread to 'write' to a channel for example, thus also have an event going downstream in one thread and upstream in another thread at the same time on the same channel?
Yes its correct.
More over, If you want to have ordered event execution in the downstream, you have have an downstream execution handler impl with OrderedMemoryAwareThreadPoolExecutor.
Finally which thread do ChannelFutures execute in?
The are executed by Oio/Nio Worker threads,
But if your down stream handler is consuming some type of event and firing another type of event to below the downstream, then your downstream handler can optionally handle the future execution. this can be done by get the future form downstream event and calling
future.setSuccess()