I need to listen to a Rabbit Queue in the Flume Custom Source which I have developed.This requirement may seem awkward in Flume.But this is how its needed.
As I am using Spring AMQP to listen to the queue for simplicity,I am just not able to understand how to invoke the OnMessage() method within the Flume lifecycle Start() method,So that the messages can be posted onto the Flume channel.
I have looked at the Spring MessageListenerAdapter concept but I have not been able to find any example to implement the same.
onMessage() is a part of MessageListener pattern. It is some active component, which is initiated by the external system (from big height). And it works each time by that remote command, so you can't use it as a passive componet to be initiated by the user call.
Since you have "Flume lifecycle Start()" from other side and SimpleMessageListenerContainer has the same from its side, I'd say you have to correlate their lifecycles to work in tandem.
From here you should to provide for the SimpleMessageListenerContainer some inline MessageListener implementation, which invokes the desired method to "post onto the Flume channel".
HTH
UPDATE
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory);
....
container.setMessageListener(new MessageListener() {
public void onMessage(Message message) {
sendMessageToFlumeChannel(message);
}
});
Where the sendMessageToFlumeChannel is a method of the holding class.
Of course it can be any POJO instead of MessageListener implementation, but the main goal to delegate listener resul to some method.
Related
Our configuration is: 1...n Message receivers with a shared database.
Messages should only be processed once.
#RabbitListener(bindings = #QueueBinding(
value = #Queue(value = "message-queue", durable = "true"),
exchange = #Exchange(value = TOPIC_EXCHANGE, type = "topic", durable = "true"),
key = MESSAGE_QUEUE1_RK)
)
public void receiveMessage(CustomMessage message) throws InterruptedException {
System.out.println("I have been received = " + message);
}
We want to to guarantee messages will be processed once, we have a message store with id's of messages already processed.
Is it possible to hook in this check before receiveMessage?
We tried to look at a MessagePostProcessor with a rabbitTemplate but didn't seem to work.
any advice on how to do this?
We tried with a MethodInterceptor and this works, but is pretty ugly.
Thanks
Solution found - thanks to Gary
I created a MessagePostProcessorInjector which implements SmartLifecycle
and on startup, I inspect each container and if it is a AbstractMessageListenerContainer add a customer MessagePostProccesser
and a custom ErrorHandler which looks for certain type of Exceptions and drops them (other forward to defaultErrorHandler)
Since we are using DLQ I found throwing exceptions or setting to null wouldn't really work.
I'll make a pull request to ignore null Messages after a MPP.
Interesting; the SimpleMessageListenerContainer does have a property afterReceivePostProcessors (not currently available via the listener container factory used by the annotation, but it could be injected later).
However, those postprocessors won't help because we still invoke the listener.
Please feel free to open a JIRA Improvement Issue for two things:
expose the afterReceivePostProcessors in the listener container factories
if a post processor returns null, skip calling the listener method.
(correction, the property is indeed exposed by the factory).
EDIT
How it works...
During context initialization...
For each annotation detected by the bean post processor the container is created and registered in the RabbitListenerEndpointRegistry
Near the end of context initialization, the registry is start()ed and it starts all containers that are configured for autoStartup (default).
To do further configuration of the container before it's started (e.g. for properties not currently exposed by the container factories), set autoStartup to false.
You can then get the container(s) from the registry (either as a collection or by id). Simply #Autowire the registry in your app.
Cast the container to a SimpleMessageListenerContainer (or alternatively a DirectMessageListenerContainer if using Spring AMQP 2.0 or later and you are using its factory instead).
Set the additional properties (such as the afterReceiveMessagePostProcessors); then start() the container.
Note: until we enhance the container to allow MPPs that return null, a possible alternative is to throw an AmqpRejectAndDontRequeueException from the MPP. However, this is probably not what you want if you have DLQs configured.
Throwing an exception extending from ImmediateAcknowledgeAmqpException from postProcessMessage() of DuplicateChecking MPP when message is duplicate will also not pass the message to the rabbit Listener.
If I have more than one #JmsListener method in a Spring Boot application is there a way to explicitly declare either listener to consume all its messages from its queue before the other?
In other words, if I have
#JmsListener(destination = "queueOne")
public void processOrder1(String message) {. . .}
#JmsListener(destination = "queueTwo")
public void processOrder2(String message) {. . .}
is there configuration available to have processOrder1() run to completion, shut it down, then have processOrder2() run? Or will processOrder2() always execute after processOrder1()(which is what the debugger suggests)? Or are they somehow running separately independent of each other?
If they cannot be configured in such a way I'd like to know why not.
Not with configuration - each listener gets its own independent asynchronous listener container.
You could configure the container factory with autoStartup set to false and start each container programmatically - get a reference to the container from the container registry using the listener's id attribute and start()/stop() it.
One difficulty is how you would "know" that the first one has completed its work.
My project publishes RESTful/SOAP services. One of these sends messages to a JMS queue on a Websphere application server. The application runs on the same application server. What I need is to define a listener to this queue. How can I activate this listener without a direct call from the service?
The project structure looks like this:
Project:
-ejb
-rest
-soap
The user calls methods on the service, which calls the EJB component, so I dont have any main method where I can init the listener.
I need a solution which activates a permanent listener to the queue.
I already have the source code I just don't know how to initialize the listener.
Not sure where you have the issues:
Do something like:
define the JMS resources in WebSphere
inject the javax.jms.Queue as a Resource (or maybe using CDI? Not sure if CDI supports this) in a EJB
use this Queue to send messages
define a MDB (#MessageDriven) to listen for messages
WebSphere MDB with a lot of configuration it works!!!! But look at this:
#MessageDriven(activationConfig={
#ActivationConfigProperty(propertyName="destination", propertyValue="myDestination"),
#ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue")
})
public class MsgBean implements javax.jms.MessageListener {
public void onMessage(javax.jms.Message msg) {
String receivedMsg = ((TextMessage) msg).getText();
System.out.println("Received message: " + receivedMsg);
}
}
I am very new to Camel, and have been struggling to understand how to use camel in a specific scenario.
In this scenario, there is a (Java-based) agent that generates actions from time to time. I need an event-driven consumer to get notified of these events. These events will be routed to a 'file' producer (for the time being).
In the camel book, the example is for a polling consumer. I could not find a generic solution for an event-driven consumer.
I came across a similar implementation for JMX :
public class JMXConsumer extends DefaultConsumer implements NotificationListener {
JMXEndpoint jmxEndpoint;
public JMXConsumer(JMXEndpoint endpoint, Processor processor) {
super(endpoint, processor);
this.jmxEndpoint = endpoint;
}
public void handleNotification(Notification notification, Object handback) {
try {
getProcessor().process(jmxEndpoint.createExchange(notification));
} catch (Throwable e) {
handleException(e);
}
}
}
Here, the handleNotification is invoked whenever a JMX notification arrives.
I believe I have to do something similar to get my consumer notified whenever the agent generates an action. However, the above handleNotification method is specific to JMX. The web page says: " When implementing your own event-driven consumer, you must identify an analogous event listener method to implement in your custom consumer."
I want to know: How can I identify an analogous event listener, so that my consumer will be notified whenever my agent has an action.
Any advice/link to a web page is very much appreciated.
I know this is an old question, but I've been struggling with it and just thought I would document my findings for anyone else searching for an answer.
When you create an Endpoint class (extending DefaultEndpoint) you override the following method for creating a consumer:
public Consumer createConsumer(Processor processor)
In your consumer then, you have access to a Processor - calling 'process' on this processor will create an event and trigger the route.
For example, say you have some Java API that listens for messages, and has some sort of Listener. In my case, the Listener puts incoming messages onto a LinkedBlockingQueue, and my Consumer 'doStart' method looks like this (add your own error handling):
#Override
protected void doStart() throws Exception {
super.doStart();
// Spawn a new thread that submits exchanges to the Processor
Runnable runnable = new Runnable() {
#Override
public void run() {
while(true) {
IMessage incomingMessage = myLinkedBlockingQueue.take();
Exchange exchange = getEndpoint().createExchange();
exchange.getIn().setBody(incomingMessage);
myProcessor.process(exchange);
}
}
};
new Thread(runnable).start();
}
Now I can put the Component that creates the Endpoint that creates this Consumer in my CamelContext, and use it like this:
from("mycomponent:incoming").to("log:messages");
And the log message fires every time a new message arrives from the Java API.
Hope that helps someone!
Event driven is what camel is.
Any route is actually an event listener.
given the route:
from("activemq:SomeQueue").
bean(MyClass.class);
public class MyBean{
public void handleEvent(MyEventObject eventPayload){ // Given MyEventObject was sent to this "SomeQueue".
// whatever processing.
}
}
That would put up an event driven consumer. How to send events then? If you have camel embedded in your app and access to the CamelContext from your event action generator, then you could grab a Producer Template from it and just fire of your event to whatever endpoint you defined in Camel, such as "seda:SomeQueue".
Otherwise, if your Camel instance is running in another server or instance than your application, then you should use some other transport rather than SEDA. Preferably JMS, but others will do as well, pick and choose. ActiveMQ is my favourite. You can start an embedded activemq instance (intra JVM) easily and connect it to camel by:
camelContext.addComponent("activemq", activeMQComponent("vm://localhost"));
I have recently discovered message selectors
#ActivationConfigProperty(
propertyName="messageSelector",
propertyValue="Fragile IS TRUE")
My Question is: How can I make the selector dynamic at runtime?
Lets say a consumer decided they wanted only messages with the property "Fragile IS FALSE"
Could the consumer change the selector somehow without redeploying the MDB?
Note: I am using Glassfish v2.1
To my knowledge, this is not possible. There may be implementations that will allow it via some custom server hooks, but it would be implementation dependent. For one, it requires a change to the deployment descriptor, which is not read after the EAR is deployed.
JMS (Jakarta Messaging) is designed to provide simple means to do simple things and more complicated things to do more complicated but less frequently needed things. Message-driven beans are an example of the first case. To do some dynamic reconfiguration, you need to stop using MDBs and start consuming messages using the programmatic API, using an injected JMSContext and topic or queue. For example:
#Inject
private JMSContext context;
#Resource(lookup="jms/queue/thumbnail")
Queue thumbnailQueue;
JMSConsumer connectListener(String messageSelector) {
JMSConsumer consumer = context.createConsumer(logTopic, messageSelector);
consumer.setMessageListener(message -> {
// process message
});
return consumer;
}
You can call connectListener during startup, e.g. in a CDI bean:
public void start(#Observes #Initialized(ApplicationScoped.class) Object startEvent) {
connectListener("Fragile IS TRUE");
}
Then you can easily reconfigure it by closing the returned consumer and creating it again with a new selector string:
consumer.close();
consumer = connectListener("Fragile IS FALSE");