I just want to understand that what is the scope of #kafkaListener, either prototype or singleton. In case of multiple consumers of a single topic, is it return the single instance or multiple instances. In my case, I have multiple customers are subscribed to single topic and get the reports. I just wanted to know, what would happen, if
multiple customers wants to query for the report on the same time. In
my case, I am closing the container after successful consumption of
messages but at the same time if some other person wants to fetch
reports, the container should be open.
how to change the scope to prototype (if it is not) associated with Id's of
container, so that each time a separate instance can be generated.
#KafkaListener(id = "id1", topics = "testTopic" )
public void listen() {
// code goes here
}
A Single Listener Instance is invoked for all consuming Threads.
The annotation #KafkaListener is not Prototype scoped, and it is not possible with this annotation either.
4.1.10. Thread Safety
When using a concurrent message listener container, a single listener instance is invoked on all consumer threads. Listeners, therefore, need to be thread-safe, and it is preferable to use stateless listeners. If it is not possible to make your listener thread-safe or adding synchronization would significantly reduce the benefit of adding concurrency, you can use one of a few techniques:
Use n containers with concurrency=1 with a prototype scoped MessageListener bean so that each container gets its own instance (this is not possible when using #KafkaListener).
Keep the state in ThreadLocal<?> instances.
Have the singleton listener delegate to a bean that is declared in SimpleThreadScope (or a similar scope).
To facilitate cleaning up thread state (for the second and third items in the preceding list), starting with version 2.2, the listener container publishes a ConsumerStoppedEvent when each thread exits. You can consume these events with an ApplicationListener or #EventListener method to remove ThreadLocal<?> instances or remove() thread-scoped beans from the scope. Note that SimpleThreadScope does not destroy beans that have a destruction interface (such as DisposableBean), so you should destroy() the instance yourself.
By default, the application context’s event multicaster invokes event listeners on the calling thread. If you change the multicaster to use an async executor, thread cleanup is not effective.
https://docs.spring.io/spring-kafka/reference/html/
=== Edited ===
Lets take their 3rd option (Delcaring a SimpleThreadScope and delegating to it)
Register SimpleThreadScope . It is not picked up automatically. You need to register it like below:
#Bean
public static BeanFactoryPostProcessor beanFactoryPostProcessor() {
return new BeanFactoryPostProcessor() {
#Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
beanFactory.registerScope("thread", new SimpleThreadScope());
}
};
}
Create a component with scopeName = "thread"
#Component
#Scope(scopeName = "thread", proxyMode = ScopedProxyMode.TARGET_CLASS)
public class KafkaDelegate{
public void handleMessageFromKafkaListener(String message){
//Do some stuff here with Message
}
}
Create a #Service
public class KafkaListenerService{
#Autowired
private KafkaDelegate kafkaDelegate;
#KafkaListener(id = "id1", topics = "testTopic" )
public void listen(String message) {
kafkaDelete.handleMessageFromKafkaListener(message);
}
}
Another example: How to implement a stateful message listener using Spring Kafka?
See this answer for an example of how to use a prototype scoped #KafkaListener bean.
Related
Suppose I am writing a service, which needs some executor service/separate thread. I give ability to use factory method to not worry about executor service, but still want to allow passing existing executor service (dependency injection).
How can I manage for executorService.shutdown()?
Example code:
public class ThingsScheduler {
private final ExecutorService executorService;
public ThingsScheduler(ExecutorService executorService) {
this.executorService = executorService;
}
public static ThingsScheduler createDefaultSingleThreaded() {
return new ThingsScheduler(Executors.newSingleThreadExecutor());
}
public scheduleThing() {
executorService.submit(new SomeTask());
}
// implement Closeable?
// #PreDestory?
// .shutdown() + JavaDoc?
}
There are several problems
We should have ability to shutdown internally created executor, or in best case handle it automatically (Spring #PreDestory, or in worst case finalize())
We shold rather not shutdown executor if it's externally managed (injected)
We could create some attribute stating if executor is created by our class or if it's injected, and then on finalize/#PreDestroy/shutdown hook we could shut down it, but it not feels elegant for me.
Maybe we should completely resign from factory method and always require injection pushing executor lifecycle management to the client?
You may crate an instance of anonymous sub-inner class from your default factory as shown below. The class will define the close/#PreDestroy method which shall be called by your DI container.
e.g.
public class ThingsScheduler {
final ExecutorService executorService;
public ThingsScheduler(ExecutorService executorService) {
this.executorService = executorService;
}
/**
* assuming you are using this method as factory method to make the returned
* bean as managed by your DI container
*/
public static ThingsScheduler createDefaultSingleThreaded() {
return new ThingsScheduler(Executors.newSingleThreadExecutor()) {
#PreDestroy
public void close() {
System.out.println("closing the bean");
executorService.shutdown();
}
};
}
}
I would say that solution in fully up to you. Third-party libraries like spring widely use a dedicated attribute to understand who should release a particular resource depending on its creator. mongoInstanceCreated in SimpleMongoDbFactory, localServer in SimpleHttpServerJaxWsServiceExporter, etc. But they do it because these classes are created only for external usage. If your class is only used in your application code than you can either inject executorService and don't care about its releasing or create and release it inside the class which uses it. This choice depends on your class/application design (does your class work with any executorService, whether executorService is shared and used by other classes, etc). Otherwise i don't see other option than the dedicated flag.
More "elegant" solution would be to extend your ExecutorService and in it override shutdown method (whichever you choose). In case of injection, you would return that extended type and it would have it's own shutdown logic. In case of factory - you still have original logic.
After some more thinking I came up with some conclusions:
do not think about shutting it down if it's injected - someone else created it, someone else will manage it's lifecycle
an executor factory could be injected instead of Executor, then we create instance using factory and manage closing it by ourself as we manage the lifecycle (and in such case responses from other users applies)
I have two Spring based async thread pools and methods in the same Spring bean. The doWork() uses the default Spring thread pool and holdAndReprocess() uses its own Spring thread pool.
I currently have my class setup like below where doWork processes some work and then if a failure occurs it parks the thread in the holdAndReprocess() "queue" thread pool and then waits and reprocesses the thread by calling the doWork(). With my current setup, the call to holdAndReprocess() and then the call back to doWork() is synchronous. Any ideas on how to wire this such that all communication between the doWork() and holdAndReprocess is asynchronous?
I'm using xml backed configuration and not pure annotation driven Spring beans.
public class AsyncSampleImpl implements AsyncSample {
#Async
public void doWork(){
holdAndReprocess();
}
#Async("queue")
#Transactional(propagation = Propagation.REQUIRED)
public void holdAndReprocess(){
//sleeps thread for static amount of time and then reprocesses
doWork();
}
}
Read https://stackoverflow.com/a/4500353/516167
As you're calling your #Async method from another method in the same object, you're probably bypassing the async proxy code and just calling your plain method, ie within the same thread.
Split this bean into two beans and invoke holdAndReprocess() from separate bean.
This rules apply also to #Transactional annotations.
Read about this: https://stackoverflow.com/a/5109419/516167
From Spring Reference Documentation Section 11.5.6, “Using #Transactional”
In proxy mode (which is the default), only 'external' method calls coming in through the proxy will be intercepted. This means that 'self-invocation', i.e. a method within the target object calling some other method of the target object, won't lead to an actual transaction at runtime even if the invoked method is marked with #Transactional!
Draft
public class AsyncSampleImpl implements AsyncSample {
public void doWork(){
reprocessor.holdAndReprocess();
}
public void holdAndReprocess(){
//sleeps thread for static amount of time and then reprocesses
worker.doWork();
}
}
Does CDI allows pooling in some way?Because I thought this is a feature of EJB beans but Adam Bien says in this screencast that container chooses whether to create new instance of class through reflection or use the existing one. So if I have for example these two beans
#RequestScoped
public class RequestBean {
public void doIt() {
}
}
#SessionScoped
public class SessionBean {
#Inject
private RequestBean bean;
public void doSomething() {
bean.doIt();
}
}
the question is - is there always new instance of RequestBean created upon calling doSomething or does CDI container somehow manage instances in pool?
The first one is scoped to the request, so a new instance is created for each request. The second one is scoped to the session, so a new one is created for each session.
CDI doesn't pool and recycle the objects, because it has no idea if the objects are stateful or not, and you don't want, in a request, to get back the state that a bean had in a previous request. That would ruin the whole point of the request/session scope.
Unless beans are really costly to create (because they start a new connection or something like that), pooling them doesn't bring any advantage. Short-lived objects are very fast to create and garbage collect nowadays. And if the bean is really expensive to create, then it should probably be a singleton.
We're developing a Web Frontend using JSF 2 and Weld Cdi on Tomcat.
Now I've a problem executing multiple webservices in parallel to optimize the request time.
The user may select mutliple items form a list.
For each selected item, the process gathers it's information from one webservice using the list key as parameter.
My current approach is using a Producer, that returns the webservice port interface, which is injected into the bean. The bean calls this webservie in a loop for each selected key.
#Inject
private WSAnzeigeAssetsummen serviceAccess;
:
for ( Integer pfNr : sessionKeys.getPfnrList() ) {
summaryTable = serviceAccess.execute(snr, pfnr, requestType, "", desiredRows, userName);
processResult(summaryTable):
}
To get faster, I tried to use a ExecutorService and as many workers as needed, which are returning Futures.
The problem of this construct is, that I can't inject the service port into the worker, cause the worker is not managed. Creating the service port by hand, works but is not appreciated, cause it ignores the producer class.
Also when testing, it's not possible to inject a dummy service port, which delivers predefined result sets.
Since I did not find anything, about parallel execution in a tomcat-weld enviroment, there must be something wrong with my approach.
What is the correct approach to solve such a situation ?
Edit: To be more clear what I tried...
public class DataCollector implements ISumRequest<Integer, Integer, String, FutureResult> {
ExecutorService pool = Executors.newCachedThreadPool();
#Inject
SessionBean sessionBean;
public Future<FutureResult> collectInformation(Integer snr, Integer pfnr, String requestType) {
CollectWorker worker = new CollectWorker (snr,pfnr,requestType,sessionBean.getUserName());
return pool.submit(worker);
}
}
When doing like this, the worker is not managed.
You can wrap your created worker in a CDI creational context, something like this:
#Inject
private BeanManager beanManager;
public <T extends Object> T performInjection(final T obj) {
if (this.beanManager != null) { // only do injection if the bean manager is present.
// Create a creational context from the BeanManager
final CreationalContext creationalContext = this.beanManager.createCreationalContext(null);
// Create an injection target with the Type of the instance we need to inject into
final InjectionTarget injectionTarget = this.beanManager.createInjectionTarget(this.beanManager.createAnnotatedType(obj.getClass()));
// Perform injection into the instance
injectionTarget.inject(obj, creationalContext);
// Call PostConstruct on instance
injectionTarget.postConstruct(obj);
}
return obj;
}
In spring it is possible to do this. Does anybody have code samples?
If you want to notify a bean about something, simply call a method:
#Service
public class Notifier {
#Autowired
private Notified notified;
public void something() {
notified.notify(..);
}
}
But event handling is usually asynchronous. In that case you will have to create a new Thread (or use the executors framework since Java 5), pass a reference to / inject the target bean, and let it notify it.
And if instead you want to notify multiple beans, without knowing which exactly, then use the event mechanism that spring provides as an implementation of the observer pattern.
You can use Spring Integration for messaging between beans in your context. Look at MessageChannel and ServiceActivator. You can route, filter, split messages to your beans how ever you need.