Develop Reference

Kafka behaviour if consumer fail

I've looked up throught a lot of different articles about Apache Kafka transactions, recovery and exactly-once new features. Still don't understand an issue with consumer recovery. How to be sure that every message from queue will be processed even if one of consumers dies?
Let's say we have a topic partition assigned to consumer. Consumer polls a message and started to work on it. And shutted down due to power failure without commit. What will happens? Will any other consumer from the same group repoll this message?

Consumers periodically send heartbeats, telling the broker that they are alive. If broker does not receive heartbeats from the consumer, it considers the consumer dead and reassigns its partitions. So, if consumer dies, its partitions will be assigned to another consumer from the group and uncommitted messages will be sent to the newly assigned consumer.

Is it possible to have JMS queue wait until it has received the acknowledgement of a message before dispatching the next message

I have setup a JMS queue that is fed by a single producer, and consumed by 8 different consumers.
I would like to configure my queue/broker so that one message being delivered to a consumer blocks the queue until the consumer is done processing the message. During the processing of this first message, the following messages may not be delivered to another consumer. It doesn't matter which consumer processes which message, and it is acceptable for the same consumer to consume many messages in a row as long as when it dies another consumer is able to pick up the rest of the unprocessed messages.
In order to do this, I have configured all of my consumers to use the CLIENT acknowledgement mode, and I have coded them so that message.acknowledge() is called only at the end of the message processing.
My understanding was that this should be sufficient to satisfy my requirements.
However I am apparently wrong, because it looks like my brojer (OpenMQ) is delivering the messages to consumers as fast as possible, without waiting for the consumer acknowledgement. As a result, I get multiple messages processed in parallel, one for each consumer.
I'm obviously doing something wrong, but I can't figure out what.
As a workaround, I figure I could create a durable subscription with a fixed client ID shared between all my consumers. It would probably work by only allowing one consumer to even connect to the broker, but I can't shake the feeling that this is a rather ugly workaround.
Does anyone have an idea of how I should configure my Broker and/or my Client to make this possible?

Delete messages from specific durable Tibco EMS subscription

I have Tibco EMS server, some topics and number of durable subscriptions to this topics(more than one to every topic).
My task is to delete(by receiving them with appropriate acknowledge mode) messages for specific durable subscriber.
My question: is it possible to manage subscriber's pending messages by "substitute" it with my own subscriber(with the same name, id)? And it's important not to affect topic's pending messages, in other words, delete some messages from one topic subscription, but remain those messages in other topic(the same topic) subscription.

Well, I've found the answer, just forgot to post it before.
As mentioned above, under question itself, there is no way to delete messages from topic. But I had little different task: to delete messages under specific durable subscription. And this is real(with some conditions).
Lets say, you have to delete messages from durable subscription "MySubscr". To do so you should create connection and create Durable Subscriber with same name "MySubscr". But it's not enough. If you just do so, then another durable subscriber will be created with the same name, but under connection with different ClientID . And it will operate as standalone durable connection without any impact to required "MySubscr" durable(actually, they will look like MySubscr:123 and MySubscr:567 durable subscription, where 123 and 567 are the ClientIDs, at least for TibcoEMS). To fix it, you should set ClientID explicitly to your connection by connection.setClientID() method, but you can do it only if initial connection is not connected(that's why I noted about durable subscriber, it can accumulate messages without subscriber connected).
So you should wait until subscriber will disconnect by itself(isConnected() method for TibcoEMS, I didn't see similar method in JMS API, but suppose most of implementations have something like this) or to destroy connection(with certain ClientID) manually(TibjmsAdmin.destroyConnection() method from TibcoEMS). And after this set the ClientID to your connection and get access to the messages of this subscriber. You can read messages by consuming them with Acknowledge mode Client(then they will remain in the topic), or with the mode Auto(then they will be deleted).
Important note: you can't consume some certain message, all the messages are consumed like in queue, so you can do it only one-by-one. If you found some unwanted message and wish to delete it(by consuming with autoacknowledge mode or by calling acknowledge() method on the message) then you'll lost all prior messages with it. AFAIK, there is no way to delete message without deleting prior ones.
Another important note: while you doing your messages magic it is important for the initial client not to connect again till your connection isn't closed, because it will get DublicateClientIDException(if it is using certain ClientID) or it will create another Durable Subscription which will have no access to the prior messages from the subscription.

Make sure the broker holds messages until at least one consumer gets it

I am begining to implement an ActiveMQ based messaging service to send worker tasks to various servers, however I am noticing that in the default mode, if no one is "listening" to a producer's topic, any message from that producer will be lost.
I.e.,
If Producer Senders Message with a live broker
But No Consumer is there to listen
Message goes no where
I would like instead for the Broker to hold on to messages until at least one listener receives it.
I am trying a couple ways of implementing this, but not sure on the most optimal/right way way:
Implement a Message Acknowledgement feature
(Caveat to this is I need the producer to wait on its listener after every message which seems very, very clunky and last resort...)
Implement the Session Transaction
(I am having trouble with this one, it sounds like the right thing to use here because of the word transaction, but I think it has more to do with the producer-broker interaction, not the producer-consumer)
Ideally, there is a mode to send a (or a set of) messages, and after sending a Boolean is returned stating if the message(s) were listened by at least one consumer.

Transactions and acknowlegdement conflict somehow with the general idea of a JMS topic.
Just use a queue instead of a topic. Access this queue using CLIENT_ACKNOWLEDGE or a transacted session. A worker task is to be processed by one worker only anyway, so the queue solves another problem.
If there was a special reason to use topics, you could consider a message driven bean (MDB) on the same host like the JMS provider (you could achieve this by using JBoss with its integrated HornetQ for example), but this is still not really correct.
Another possibility is to have both a topic and a queue. The latter is only for guaranteed delivery of each message.

This isn't really a typical messaging pattern. Typically, you have one receiver and a durable queue or multiple receivers with durable subscriptions to a topic. in either situation, each receiver will always receive the message. i don't really understand a use case where "at least one" receiver should receive it.
and yes, transactions only deal with the interactions between client and broker, not between client and eventual receiver(s).

TIBCO EMS notification to Publisher in case of Subscriber failure

I am trying to find an answer on of how to notify an EMS Publisher in case of a Subscriber failure.
In a case of Publisher->EMS server->Subscriber, if a Subscriber fails, I need to inform Publisher to take a corrective action.I am not bothered about durabilty/PERSIETENCE, my significance is of time. In Trading systems, If I send an market order to a Subscriber who in turn sends it to an exchange, if it fails, I need to make my Publisher publish the messages on a different topic to another Sunscriber(another exchange).
Any ideas is appreciated.

The tibjmsadmin.jar library contains methods to detect when subscribers disconnect. Easier than writing code, you can:
if you have Hawk, use the
tibjmsadmin.hma to write a Hawk rule
in the event a subscriber
disconnects, or
listen on the monitor
topic
$sys.monitor.connection.disconnect -
the body of the message tells you
which subscriber disconnected.
However these "monitoring" approaches to failing over the publisher have a significant problem - in the time it takes for you to detect the subscriber failure and redirect the publisher, some messages may get through and get stuck in the defunct queue. You don't wat this happening to any $10M trades!
EMS knows when subscribers are connected or not and you should take advantage of this.
Use a "distributed queue" and there shouldn't be any need to code logic into your application to switch to a new subscriber when it fails. This happens without message loss and maintains the order of messages. It is also good architectural practice to keep load balancing and failover logic out of your code and in the administration setup of your JMS provider.
Basically you setup multiple subscribers to a queue (each exchange represented by a subscriber). The default action will be for EMS to load-balance messages across your subscribers in a round-robin fashion. But you can set the queue to "exclusive" so that messages go to only one subscriber at a time. Then if that active subscriber fails, the messages are forwarded to another subscriber.
See the EMS manual for more details on all these topics.

Not sure if you have access, you could try looking at the ReceiverCount or ConsumerCount in either QueueInfo or TopicInfo - I believe you need the tibjms.admin package. May be you can query this before you publish and then selectively publish? Not sure what the overhead is.
Because of the nature of JMS, AFAIK no transaction states (unless you use a XA transaction - with all of that overhead) or acknowledgements will propagate through the EMS broker. I.e.acks are always between publisher and broker and consumer and broker.
Failing the above, you could try a separate ack topic for which the roles are reversed, but then the failure case is a timeout - I'm not sure this is sensible.
If you don't really care which exchange the order goes to - why not make the topic/queue exclusive and make both consumers attempt to consume - the first one to succeed will process all of the messages - if it dies, then the second one (which could be periodically retrying - may successfully connect).. Alternatively allow both to process orders off the queue - remember a message will only be ever processed by a single consumer...
I really cannot see the advantage of a decoupled messaging bus in your order flow... makes no sense to me..