I am trying to understand the use case of using Queue.
My understanding:
Queue means one-to-one. The only use case(if not rare, very few) would be: Message is intended for only one consume.
But even in those cases, I may want to use Topic (just to be future safe). The only extra caution would be to make subscriptions durable. Or, in special situations, I would use bridging / dispatcher mechanism.
Given above, I would always (or in most cases) want to publish to a topic. Subscriber can be either durable topic(s) or dispatched queue(s).
Please let me know what I am missing here or I am missing the original intent?
The design requirements on when to use queues are simple if you think in terms of real-world examples:
Submit online order (exactly-once processing to avoid charging credit
card twice)
Private peer-to-peer chat (exactly one receiver for each message)
Parallel task distribution (distribute tasks amongst many workers in a networked system)
...and examples for when to use topics...
News broadcast to multiple subscribers; notification service, stock ticker, etc.
Email client (unique durable subscriber; you still get emails when you're disconnected)
You said...
But even in those cases, I may want to use Topic (just to be future
safe). The only extra case I would have to do is to make (each)
subscription durable. Or, I special situations, I would use bridging /
dispatcher mechanism.
You're over-engineering the design. It's true, you can achieve exactly-once processing using a topic and durable subscriber, but you'd be limited to a single durable subscriber; the moment you start another subscriber for that topic, you'll get duplicate processing for the same message, not to mention, a single durable subscriber is hardly a solution that scales; it would be a bottleneck in your system for sure. With a queue, you can deploy 1000 receivers on 100 nodes for the same queue, and you'd still get exactly-once processing for a single message.
You said...
Give above, I would always (or in most cases) want to publish to a
topic. Subscriber can be either durable topic(s) or dispatched
queue(s).
Using a dispatched queue with a topic subscriber is sort of redundant. You basically get asynchronous dispatching when using queues, so why not just use a queue?...no reason to put a topic in front of it.
Queues and Topics in JMS represent two different models - point to point and publish/subscribe. Topics will keep a message until all clients receive them, all subscribers handling them. Queues will wait for the first consumer to pull the message, and consider it read at that point.
You are probably missing that both queues and topics can have multiple subscribers. A queue will deliver the message to one of potentially many subscribers, while a topic will deliver the message to all subscribers.
If you in your case are sure that there is only one subscriber, then a queue subscriber and a durable topic subscriber will behave similarly. I would rather look at such a scenario as a "special case".
Related
I have a question about RabbitMQ queue. I would like to send two types of messages on just one queue.
I know that, I can create two different queues, and use routing key to send different messages to different queue.
But I would like to have two consumers on one Queue, and somehow bind Consumer with type of message. It's event's driven via rabbit queue, when client and core are publishers and consumers.
Is it possible, or should I use different Queues?
Data exchange
Like #kendavidson said, there is a possibility to use only one queue, to exchange different messages, but it is terrible idea, because it's not efficient, so you should use it only if it's truly nesesery.
I found comment #Петр Александров useful and I created separate queue for every consumer to fix my problem, and it's something you probably looking for.
I have two instances of my app subscribing to a topic. Since there are two instances(i.e two subscribers), two events(messages) will be generated and written to a queue. (Now i have duplicate message in the queue and one by one each is going to be processed) But I want to have a solution where only one event is processed/or only one message is written to the queue. How can i achieve that?. I must have two subscribers instead one goes down
JMS topics follow publish-subscribe semantics where every subscriber will get the message. However, JMS queues follow point-to-point semantics where only 1 of the connected consumers will receive the message. Therefore, if you want the message to be consumed by only one client then all of your consumers should be connected to a JMS queue rather than a topic.
In my system, there are many users who write the blogs. I need to subscribe to different users. There is no centralized system(it's a swing application).
I am using JMS.
The user may follow one user, two users or 100 users.
m_destination1 = m_session.createQueue("USER.DEVID");
m_consumer1 = m_session.createConsumer(m_destination1);
m_destination2 = m_session.createQueue("USER.HARRY");
m_consumer2 = m_session.createConsumer(m_destination2);
Is there any generic way to write the above lines of code for unknown no. of users ? Like one consumer can receive message from many users.
Here wildcard will not work.
The best thing you can use is Mirrored Queue feature of activeMQ,
you can read the documentation here
http://activemq.apache.org/mirrored-queues.html
What mirrored queue basically does is,it forwards all the messages sent on queue to a similar named topic, this topic can then be subscribed by multiple consumers.
If you use mirrored queue, you will need your consumers to subscribe to different topics.
Your design cries out for publish-subscribe(topic) domain rather than a point-to-point architecture(i.e queue).As you already would be having an architecture which generates a queue for different people writing blogs,change to that system wont be required but your requirement will be catered.
I addition to this,if 2 consumers listen on a queue then they will pick up messages parallely from queue i.e If there are 2 messages on queue then both consumers will process 1 message independently,I don't think that's what you want.
Hope this helps!
Good luck!
#Vihar's answer is right that you should be using the publish-subscribe paradigm by using a topic, to allow multiple consumers to both be notified of new blog posts. It sounds like your primary pain point is that you've got one destination per author and users that want to consume messages have to subscribe to each of them individually.
Instead, have all new-post messages published into a single topic (let's call it NewPostNotificationTopic). Clients can then subscribe to all messages but immediately check them against the list of authors they care about and immediately stop processing any notification for an author they're not following. (This puts the filtering into the message handler rather than into the ActiveMQ network.) This does mean that each message will be passed to each client, but as long as the messages are small and your network is fast and your users are usually connected to the network, this might be a workable solution. But if you can't afford the network bandwidth of sending all messages to all clients, or if your consumers will be offline for long periods of time and you can't afford to hold a copy of all messages till they come back online, this may not work for you.
Alternatively, publish all messages into that same topic, but set the author's ID as a header on the message and use message selectors to tell ActiveMQ to only deliver messages matching a given author ID. This will be more efficient, but you're back to needing to explicitly tell ActiveMQ which authors you care about, either with a single subscription with a selector that contains ORs or with one subscription per author. The latter is cleaner but gets you back to your problem of one subscription per author per reader; the former results in only one subscription but it has to be updated each time you add/remove an author for a reader, and you'll need to make sure you handle the race conditions inherent in removing the subscription and adding another one. I'd go with the first solution I proposed (doing the filtering in the message handler instead of in the ActiveMQ subscriptions) if the performance concerns I raised there aren't a problem; otherwise I'd probably go with one subscription per author per reader, rather than having a single subscription with an ORed selector and needing to redo the subscription each time something changed.
I am begining to implement an ActiveMQ based messaging service to send worker tasks to various servers, however I am noticing that in the default mode, if no one is "listening" to a producer's topic, any message from that producer will be lost.
I.e.,
If Producer Senders Message with a live broker
But No Consumer is there to listen
Message goes no where
I would like instead for the Broker to hold on to messages until at least one listener receives it.
I am trying a couple ways of implementing this, but not sure on the most optimal/right way way:
Implement a Message Acknowledgement feature
(Caveat to this is I need the producer to wait on its listener after every message which seems very, very clunky and last resort...)
Implement the Session Transaction
(I am having trouble with this one, it sounds like the right thing to use here because of the word transaction, but I think it has more to do with the producer-broker interaction, not the producer-consumer)
Ideally, there is a mode to send a (or a set of) messages, and after sending a Boolean is returned stating if the message(s) were listened by at least one consumer.
Transactions and acknowlegdement conflict somehow with the general idea of a JMS topic.
Just use a queue instead of a topic. Access this queue using CLIENT_ACKNOWLEDGE or a transacted session. A worker task is to be processed by one worker only anyway, so the queue solves another problem.
If there was a special reason to use topics, you could consider a message driven bean (MDB) on the same host like the JMS provider (you could achieve this by using JBoss with its integrated HornetQ for example), but this is still not really correct.
Another possibility is to have both a topic and a queue. The latter is only for guaranteed delivery of each message.
This isn't really a typical messaging pattern. Typically, you have one receiver and a durable queue or multiple receivers with durable subscriptions to a topic. in either situation, each receiver will always receive the message. i don't really understand a use case where "at least one" receiver should receive it.
and yes, transactions only deal with the interactions between client and broker, not between client and eventual receiver(s).
I am trying to find an answer on of how to notify an EMS Publisher in case of a Subscriber failure.
In a case of Publisher->EMS server->Subscriber, if a Subscriber fails, I need to inform Publisher to take a corrective action.I am not bothered about durabilty/PERSIETENCE, my significance is of time. In Trading systems, If I send an market order to a Subscriber who in turn sends it to an exchange, if it fails, I need to make my Publisher publish the messages on a different topic to another Sunscriber(another exchange).
Any ideas is appreciated.
The tibjmsadmin.jar library contains methods to detect when subscribers disconnect. Easier than writing code, you can:
if you have Hawk, use the
tibjmsadmin.hma to write a Hawk rule
in the event a subscriber
disconnects, or
listen on the monitor
topic
$sys.monitor.connection.disconnect -
the body of the message tells you
which subscriber disconnected.
However these "monitoring" approaches to failing over the publisher have a significant problem - in the time it takes for you to detect the subscriber failure and redirect the publisher, some messages may get through and get stuck in the defunct queue. You don't wat this happening to any $10M trades!
EMS knows when subscribers are connected or not and you should take advantage of this.
Use a "distributed queue" and there shouldn't be any need to code logic into your application to switch to a new subscriber when it fails. This happens without message loss and maintains the order of messages. It is also good architectural practice to keep load balancing and failover logic out of your code and in the administration setup of your JMS provider.
Basically you setup multiple subscribers to a queue (each exchange represented by a subscriber). The default action will be for EMS to load-balance messages across your subscribers in a round-robin fashion. But you can set the queue to "exclusive" so that messages go to only one subscriber at a time. Then if that active subscriber fails, the messages are forwarded to another subscriber.
See the EMS manual for more details on all these topics.
Not sure if you have access, you could try looking at the ReceiverCount or ConsumerCount in either QueueInfo or TopicInfo - I believe you need the tibjms.admin package. May be you can query this before you publish and then selectively publish? Not sure what the overhead is.
Because of the nature of JMS, AFAIK no transaction states (unless you use a XA transaction - with all of that overhead) or acknowledgements will propagate through the EMS broker. I.e.acks are always between publisher and broker and consumer and broker.
Failing the above, you could try a separate ack topic for which the roles are reversed, but then the failure case is a timeout - I'm not sure this is sensible.
If you don't really care which exchange the order goes to - why not make the topic/queue exclusive and make both consumers attempt to consume - the first one to succeed will process all of the messages - if it dies, then the second one (which could be periodically retrying - may successfully connect).. Alternatively allow both to process orders off the queue - remember a message will only be ever processed by a single consumer...
I really cannot see the advantage of a decoupled messaging bus in your order flow... makes no sense to me..