I'm implementing a listener class which listens for some events and then processes them. If processing of this event goes well then this event is never notified again, but in case of any exception event will be again notified to MyBeanImplementation class after some time and it can again try processing it.
Following code works fine but since this event processing may take some time,
1. I want to have multiple listeners.
2. Limit number of calls to service, may be using threadpool.
How can I have multiple listeners which will process each event differently? I'm new to Spring and don't have much idea if this is even possible or not.
Heres is an example:
// Spring Configuration:
<bean id="MyBean" class="MyBeanImplementation">
// Sample Class
public class MyBeanImplementation implements EventListener {
#override
public processEvent(Event event) throws EventProcessFailureException {
try {
// Validate event
validateEvent(event);
// Call another service to store part of information from this event
// This service takes some time to return success
boolean success = makeCallToServiceAndStoreInfo(event);
if(!success) {
throw new EventProcessFailureException("Error storing event information!");
}
} catch (Exception e) {
throw new EventProcessFailureException(e);
}
}
}
Thanks
Basically, you can use Strategy pattern to introduce different listeners that re-act in a different way to a shared event.
<bean id="strategy1Listener" />
<bean id="strategy2Listener" />
<bean id="strategy3Listener" />
Then, you can introduce a composite listener to iterate through other listeners and pass through the event and allow them to process the event:
<bean id="compositeStrategyListener">
<property name="listeners">
<list>
<ref bean="strategy1Listener" />
<ref bean="strategy2Listener" />
<ref bean="strategy3Listener" />
</list>
</property>
</bean>
On the other side of the story, you have an object that generates/publishes the events:
<bean id="eventGenerator">
<property name="eventListener" ref="compositeStrategyListener" />
</bean>
So, now eventGenerator publishes the generated event to a compositeStrategyListner which iterates over the listeners it has and allows them to process the event in their own different way.
You can also take advantage of Spring Task Execution to configure how you need to run the task of event processing.
Why not using google guava eventbus library with spring together? It will handle every thing needed for listening events.
Related
I have written a simple Kafka app with spring integration kafka 3.2.1.RELEASE and kafka-clients 2.5 to learn kafka transactions.
It recieves the messages from a topic and sends them to another topic. The beans.xml file is as follows
<int-kafka:message-driven-channel-adapter
listener-container="container"
auto-startup="true"
send-timeout="30000"
channel="channelA"/>
<bean id="container" class="org.springframework.kafka.listener.KafkaMessageListenerContainer" parent="kafkaMessageListenerContainerAbstract">
<constructor-arg>
<bean class="org.springframework.kafka.listener.ContainerProperties">
<constructor-arg
name="topics"
value="test"/>
<property name="transactionManager" ref="KafkaTransactionManager"/>
</bean>
</constructor-arg>
</bean>
.
.
.
<int-kafka:outbound-channel-adapter kafka-template="kafkaTemplate"
auto-startup="true"
channel="channelB"
topic="output"/>
<bean id="dbsenderTemplate" class="org.springframework.kafka.core.KafkaTemplate">
<constructor-arg>
<bean class="org.springframework.kafka.core.DefaultKafkaProducerFactory">
<constructor-arg>
<map>
<entry key="value.serializer" value="org.apache.kafka.common.serialization.StringSerializer"/>
<entry key="key.serializer" value="org.apache.kafka.common.serialization.StringSerializer"/>
<entry key="bootstrap.servers" value="localhost:9092"/>
</map>
</constructor-arg>
<property name="transactionIdPrefix" value="mytest-"/>
<property name="producerPerConsumerPartition" value="false"/>
</bean>
</constructor-arg>
</bean>
The code that starts the app is as follows:
GenericXmlApplicationContext tempContext = new GenericXmlApplicationContext("beans.xml");
tempContext.close();
//POINT A.
try {
Thread.sleep(60000);
} catch (InterruptedException e) {
e.printStackTrace();
}
GenericXmlApplicationContext context = new GenericXmlApplicationContext();
context.load("beans.xml");
context.refresh();
//POINT B
At POINT A I just closed the context to check which beans are closed, and put a 60 seconds sleep to have time to check the JMX console. I noticed that even though the context is closed but the producer is still registered in JMX. After that I traced the code and noticed that on context closing the KafkaTemplate calls the following code:
public void flush() {
Producer<K, V> producer = getTheProducer();
try {
producer.flush();
}
finally {
closeProducer(producer, inTransaction());
}
}
protected void closeProducer(Producer<K, V> producer, boolean inTx) {
if (!inTx) {
producer.close(this.closeTimeout);
}
}
It means it creates a producer but because it is transactional it will not be closed.
This behaviour makes that runnning the context again on POINT B and sending the message cause the javax.management.InstanceAlreadyExistsException: kafka.producer:type=app-info,id=producer-mytest-0 Exception.
Why the KafkaTemplate does not close these producers?
And another question is what happens to these producers when a new KafkaTemplate is created on POINT B?
The last question is if I change the producerPerConsumerPartition property to true the mentioned app still registers producer Mbean with producer-mytest-0 and does not follow the groupid.topic.partition pattern in naming. Is it a correct behaviour?
UPDATES:
I understood when the KafkaTemplate executeInTransaction is called. At the finally block it calls the close on the producer and as it is a logical close, the following code is called on the CloseSafeProducer and put it in the cache:
if (!this.cache.contains(this)
&& !this.cache.offer(this)) {
this.delegate.close(closeTimeout);
}
This makes when the context is closed the destroy method of DefaultKafkaProducerFactory clears the cache and closes the producer physically. But in my situation application context is created but before consume and producing any message the context is closed, only the flush method of KafkaTemplate is called internally that force it to create a transactional producer but does not put it in the cache. As I didn't start a producer and KafkaTemplate do it on flush, is not it good that DefaultKafkaProducerFactory put them in cache before using them?
The producer cannot be closed if this template operation is participating in a transaction that was started outside of the template.
Even when closed, it is only "logically" closed - cached for reuse by another operation.
Is it a correct behaviour?
Yes, for producer-initiated transactions; the alternative name is used when a consumer initiates the transaction.
The InstanceAlreadyExistsException problem is simply because you are creating two application contexts with identical configuration. Why are you doing that?
There is the following Spring Integration configuration with Retry:
<int:chain input-channel="sdCreationChannel" output-channel="debugLogger">
<int:poller fixed-delay="500" />
<int:filter ref="sdIntegrationExistingRequestSentFilter" method="filter"/>
<int:service-activator ref="sdCreationServiceImpl" method="processMessage">
<int:request-handler-advice-chain>
<ref bean="retryAdvice"/>
</int:request-handler-advice-chain>
</int:service-activator>
</int:chain>
<bean id="retryAdvice" class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice" >
<property name="retryTemplate">
<bean class="org.springframework.retry.support.RetryTemplate">
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="${integration.retry.initial.delay}"/>
<property name="multiplier" value="${integration.retry.backoff.multiplier}"/>
</bean>
</property>
<property name="retryPolicy">
<bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="${integration.retry.max.attempts}" />
</bean>
</property>
</bean>
</property>
</bean>
The simplified Java code is as follows:
#Component("sdCreationServiceImpl")
public class SDCreationServiceImpl implements SDCreationService {
#Autowired
private NotifySD notifySD;
#Override
public void processMessage(IntegrationPayload integrationPayload) {
List<ConfirmationCode> sdConfCodes = findCodesFromPayLoad(integrationPayload);
notifySD.activateConfirmationCodes(sdConfCodes);
}
The problem with retrying this code is that the List sdConfCodes can be partially processed at every Retry, so every time we need to send for processing the less amount of elements. What is best way to organize this code?
Following the Artem Bilan suggestion (Thanks!) I created the 2nd method with variable list in SDCreationServiceImpl, i.e. activateConfirmationCodes and then in the XML specification pointed to this method as a method for sdCreationServiceImpl.
#Component("sdCreationServiceImpl")
public class SDCreationServiceImpl implements SDCreationService {
#Autowired
private NotifySD notifySD;
List<ConfirmationCode> sdConfCodes = new ArrayList<ConfirmationCode()>;
#Override
public void processMessage(IntegrationPayload integrationPayload) {
sdConfCodes = findCodesFromPayLoad(integrationPayload);
}
public void activateConfirmationCodes()
{
notifySD.activateConfirmationCodes(sdConfCodes);
}
And then the XML spec for service-activator is as follows:
<int:service-activator ref="sdCreationServiceImpl" method="activateConfirmationCodes">
<int:request-handler-advice-chain>
<ref bean="retryAdvice"/>
</int:request-handler-advice-chain>
</int:service-activator>
Yes, this method activateConfirmationCodes is invoked in Retry but the 1st method processMessage is not invoked at all. Is it possible to specify one method for invoking at the first try and other method for retrying?
Second with this design the list becomes the singleton and this can give problems in multithreading, correct?. Can this list be associated with a bean only for a particular message?
From big it isn't clear where is your problem. From other side let me share some my thoughts, perhaps I will guess your goal.
Having the List<ConfirmationCode> as a payload allows us to modify it at any time. So, let's assume we have list as 10 elements. On the first attempt we have processed 3 of them. The fourth has failed. We have to go to retry throwing some appropriate exception.
But we come back to the beginning of the retry-aware method, so with the same arguments. If we remove those successful items from the collection, the next retry iteration won't process them at all.
From one side you can achieve that distinguishing findCodesFromPayLoad() service and activateConfirmationCodes(), applying retry for the last one.
From other side you can mark items as processed in the activateConfirmationCodes(), so the next findCodesFromPayLoad(integrationPayload) won't return them.
In other words there are enough ways to modify collection without changing messages.
I'm trying to modify my spring-amqp project such that if a certain exception type is thrown, always requeue the message. Otherwise, retry x number of times then reject.
Here is the relevant configuration xml
<rabbit:connection-factory id="rabbitMqConnectionFactory" host="localhost" port="5672" />
<rabbit:template id="rabbitTemplate" connection-factory="rabbitMqConnectionFactory"
exchange="my.exchange" routing-key="foo"/>
<bean id="myListenerContainer" class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
<property name="connectionFactory" ref="rabbitMqConnectionFactory" />
<property name="messageListener" ref="myMessageListener" />
<property name="concurrentConsumers" value="2" />
<property name="queueNames" value="work.q" />
<property name="adviceChain" ref="myRetryInterceptor" />
</bean>
And here is the relevant code
#Bean(name="myRetryInterceptor")
public MethodInterceptor getInterceptor() {
return RetryInterceptorBuilder.stateless()
.retryPolicy(getRetryPolicy())
.recoverer(new RejectAndDontRequeueRecoverer())
.build();
}
private RetryPolicy getRetryPolicy() {
return new SimpleRetryPolicy(5, Collections.EMPTY_MAP) {
#Override
public boolean canRetry(RetryContext context) {
Throwable t = context.getLastThrowable();
if ((t!=null) && (t.getCause() instanceof com.test.MessageRetryException)) {
return true;
}
return super.canRetry(context);
}
};
}
I notice that when I deliberately throw a com.test.MessageRetryException in myMessageListener, the same thread is given the message over and over again. This is in contrast to an implementation where no spring classes are modified. In this case, the message is given alternately to one consumer thread then another. Am I doing something wrong?
#Bean(name="myRetryInterceptor")
//all consumers are given the message here
public MethodInterceptor getInterceptor() {
return RetryInterceptorBuilder.stateful()
.maxAttempts(5)
.recoverer(new RejectAndDontRequeueRecoverer())
.build();
}
No; you're doing nothing wrong; stateless retry is done within the scope of the original delivery (on the same thread). Message rejection does not occur until the retries are exhausted.
Stateful retry, on the other hand, rejects the delivery on each attempt and it will be resubmitted and (possibly) handled by a different thread.
Since rabbitmq now requeues rejected message at the head of the queue, there is not really any benefit of switching to stateful recovery and it requires a message id header to work (so the state can be determined for the message).
EDIT: Hmmm - I see you are using stateful retry; so I would expect the redeliveries to happen on alternate threads.
If you can post a DEBUG log on a Gist or some place; I can take a look. But, as I said, stateless retry is simpler anyway.
I am using the HDIV Web Application Security Framework for a java web application. Every new web-page-request generates hdiv-internal security information that is cached and used for security checks.
I have the following szenario:
I have one order page that pops up a confirmation-page for 2 seconds when something was added to or removed from the cart.
after 50 popups the the underlaying order page is removed from the cache and therefor an error occurs in the app.
does anybody know how to influence the hdiv cache-removal strategy to keep the basepage alive?
One way around is to increase org.hdiv.session.StateCache.maxSize from 50 to 500.
but this would only cure the symptoms not the underlying cause.
Update:
using #rbelasko solution
I succeded to use the original org.hdiv.session.StateCache to change the maxSize to 20 and verified in the debug-log that the cachentries are dismissed after 20 entries.
When I changed it to use my own implementation it didn-t work
Bean definition
<bean id="cache" class="com.mycompany.session.StateCacheTest" singleton="false"
init-method="init">
<property name="maxSize">
<value>20</value>
</property>
</bean>
My own class
public class StateCacheTest extends StateCache
{
private static final Log log = LogFactory.getLog(StateCacheTest.class);
public StateCacheTest()
{
log.debug("StateCache()");
}
#Override
public void setMaxSize(final int maxSize)
{
super.setMaxSize(maxSize);
if (log.isDebugEnabled())
{
log.debug("setMaxSize to " + maxSize);
}
}
}
In the debug-log were no entries from StateCacheTest
Any ideas?
Update 2:
While i was not able to load a different IStateCache implementation via spring i was able to make this error less likely using
<hdiv:config ... maxPagesPerSession="200" ... />
the bean-settings definition
<property name="maxSize">
<value>20</value>
</property>
had no effect on the cachesize in my system.
You could create a custom IStateCache interface implementation.
Using the HDIV explicit configuration (not using HDIV's new custom schema) this is the default configuration for "cache" bean:
<bean id="cache" class="org.hdiv.session.StateCache" singleton="false"
init-method="init">
<property name="maxSize">
<value>200</value>
</property>
</bean>
You could create your own implementation and implement the strategy that fits your requirements.
Regards,
Roberto
I'm reading Spring Framework reference, chapter about JMS integration. There are some examples for sending text messages and asynchronously receiving them (by listeners). And there is also an example for JmsTemplate function convertAndSend which converts given object to a message. The reference says:
By using the converter, you and your application code can focus on the business object that is being sent or received via JMS and not be concerned with the details of how it is represented as a JMS message.
But there is no example for receiving such messages. They mention function receiveAndConvert but, unfortunately, it receives synchronously.
So how am I to receive it asynchronously? Must I be aware that when I convertAndSend a Map, the resulting message will be a MapMessage, and just check in my listener for this type of message and handle it? But they promised I'm not to be concerned with the details of how it is represented as a JMS message.
So is there a better way?
I know it's been a while since this was asked, but I had the same problem, solved it and wanted to give an explicit code example here.
Here's my MessageListener. This implements the onMessage(Message) method to intercept messages asynchronously.
package com.package.amqp;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessageListener;
import org.springframework.amqp.support.converter.JsonMessageConverter;
import com.package.model.User;
public class TestListener implements MessageListener {
public void onMessage(Message message) {
JsonMessageConverter jmc = new JsonMessageConverter();
User u = (User)jmc.fromMessage(message);
System.out.println("received: " + u.getFirstName());
}
}
The messages are then converted using the standard JsonMessageConvertor in my case as this is the messageConvertor I plugged into my rabbitTemplate bean.
<bean id="rabbitConnectionFactory" class="org.springframework.amqp.rabbit.connection.SingleConnectionFactory">
<constructor-arg value="10.10.1.2"/>
<property name="username" value="guest"/>
<property name="password" value="guest"/>
</bean>
<bean class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
<property name="connectionFactory" ref="rabbitConnectionFactory"/>
<property name="queueName" value="queue.helloWorld"/>
<property name="messageListener" ref="someListener"/>
</bean>
<bean id="someListener" class="com.package.amqp.TestListener"></bean>
<bean id="rabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate">
<property name="connectionFactory" ref="rabbitConnectionFactory"/>
<property name="messageConverter">
<bean class="org.springframework.amqp.support.converter.JsonMessageConverter"/>
</property>
</bean>
Hope this helps someone!
Owen
While JmsTemplate provides basic synchronous receive methods, asynchronous reception is a whole lot more complicated, and is beyond the scope of JmsTemplate.
Asynchronous reception of JMS messages is done in Spring using Message Listener Containers, which asynchronously take messages from the JMS destination and pass them to your application. You can plug a MessageConverter in to your message listener container via a MessageListenerAdapter (plug the converter into the adapter, plug your application's listener into the adapter, then plug the adapter into the listener container).