Develop Reference

How can I acknowledge a JMS message on another thread and request for redelivery of unacknowledged messages?

Step 1: I need to receive message by one thread.
Step 2: Process and sending ack and redelivery request (throwing exception) by another thread.
Sample code:
List<Message> list=new ArrayList();
#JmsListener(destination = "${jms.queue-name}", concurrency = "${jms.max-thread-count}")
public void receiveMessage(Message message) throws JMSException,UnsupportedEncodingException {
list.add(message)
}
void run() {
foreach(Message message:list) {
//need to send ack or throw exception for redeliver if error
}
}
Now another thread will start and process the list which contains data then how can I send an acknowledgement or throw an exception for redelivery?

Typically you'd let your framework (e.g. Spring) deal with concurrent message processing. This is, in fact, one of the benefits of such frameworks. I don't see any explicit benefit to dumping all the messages into a List and then manually spawning a thread to process it. Spring is already doing this for you via the #JmsListener by invoking receiveMessage in a thread and providing configurable concurrency.
Furthermore, if you want to trigger redelivery then you'll need to use a transacted JMS session and invoke rollback() but JMS sessions are not threadsafe so you'll have to control access to it somehow. This will almost certainly make your code needlessly complex.

Handle multiple amqp messages concurrently through one consumer inside one spring-rabbit service

EDIT
Just found out how to run multiple consumers inside one service:
#Bean
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory, MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(RENDER_QUEUE);
container.setConcurrentConsumers(concurrentConsumers); // setting this in env
container.setMessageListener(listenerAdapter);
return container;
}
#Bean
MessageListenerAdapter listenerAdapter(RenderMessageConsumer receiver) {
return new MessageListenerAdapter(receiver, "reciveMessageFromRenderQueue");
}
Now the only question that remains is: how can I have a global limit? So how do multiple instances of the AMQP receiver share the total number of consumers? So I want to set a global number of concurrentConsumers to 10, run 2 instances of the consumerSerivce and have each instance run around 5 consumers. Can this be managed by rabbitMq?
I have a Spring service that consumes AMQP messages and calls a http resource for each message.
After the http call completes another queue is called to either report error or done. Only then will message handling complete and the next message be taken from the queue.
// simplified
#RabbitListener(queues = RENDER_QUEUE)
public void reciveMessageFromRenderQueue(String message) {
try {
RenderMessage renderMessage = JsonUtils.stringToObject(message, RenderMessage.class);
String result = renderService.httpCallRenderer(renderMessage);
messageProducer.sendDoneMessage(result);
} catch (Exception e) {
logError(type, e);
messageProducer.sendErrorMessage(e.getMessage());
}
}
There are at times hundreds or thousands of render messages in the queue but the http call is rather long running and not doing much. This becomes obvious as I can improve the message handling rate by running multiple instances of the service thus adding more consumers and calling the http endpoint multiple times. One instance has exactly one consumer for the channel so the number of instances is equal to the number of consumers. However that heavily increases memory usage (since the service uses spring) for just forwarding a message and handling the result.
So I thought, I'd do the http call asynchronously and return immediatly after accepting the message:
.httpCallRendererAsync(renderMessage)
.subscribeOn(Schedulers.newThread())
.subscribe(new Observer<String >() {
public void onNext(String result) {
messageProducer.sendDoneMessage(result);
}
public void onError(Throwable throwable) {
messageProducer.sendErrorMessage(throwable.getMessage());
}
});
That however overloads the http endpoint which cannot deal with 1000 or more simultanous requests.
What I need is for my amqp service to take a certain amount of messages from the queue, handle them in separate threads, make the http call in each of them and return with "message handled". The amount of messages taken from the queue however needs to be shared between multiple instances of that service, so if maximum is 10, message consumption is round robin, the first 5 odd messages should be handled by instance one and the first 5 even messages by instance 2 and as soon as one instance finishes handling the message it should take another one from the queue.
What I found are things like prefetch with limts by consumer and by channel as described by rabbitmq. And the spring-rabbit implementation which uses prefetchCount and the transactionSize described here. That however does not seem to do anything for a single running instance. It will not spawn additional threads to handle more messages concurrently. And of course it will not reduce the number of messages handled in my async scenario since those messages are immediatly considered "handled".
#Bean
public RabbitListenerContainerFactory<SimpleMessageListenerContainer> prefetchContainerFactory(ConnectionFactory rabbitConnectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(rabbitConnectionFactory);
factory.setPrefetchCount(5);
factory.setTxSize(5);
return factory;
}
// and then using
#RabbitListener(queues = RENDER_QUEUE, containerFactory = "prefetchContainerFactory")
The most important requirement for me seems to be that multiple messages should be handled in one instance while the maximum of concurrently handled messages should be shared between instances.
Can that be done using rabbitMq and spring? Or do I have to implemenent something in between.
In an early stage it might be acceptable to just have concurrent message handling in one instance and not share that limit. Then I'll have to configure the limit manually using environment variables while scaling the number of instances.

Now the only question that remains is: how can I have a global limit? So how do multiple instances of the AMQP receiver share the total number of consumers? So I want to set a global number of concurrentConsumers to 10, run 2 instances of the consumerSerivce and have each instance run around 5 consumers. Can this be managed by rabbitMq?
There is no mechanism in either RabbitMQ or Spring to support such a scenario automatically. You can, however, change the concurrency at runtime (setConcurrentConsumers() on the container) so you could use some external agent to manage the concurrency on each instance.

WLPs MicroProfile (FaultTolerance) Timeout Implementation does not interrupt threads?

I'm testing the websphere liberty's fault tolerance (microprofile) implementation. Therefore I made a simple REST-Service with a ressource which sleeps for 5 seconds:
#Path("client")
public class Client {
#GET
#Path("timeout")
public Response getClientTimeout() throws InterruptedException {
Thread.sleep(5000);
return Response.ok().entity("text").build();
}
}
I call this client within the same application within another REST-service:
#Path("mpfaulttolerance")
#RequestScoped
public class MpFaultToleranceController {
#GET
#Path("timeout")
#Timeout(4)
public Response getFailingRequest() {
System.out.println("start");
// calls the 5 seconds-ressource; should time out
Response response = ClientBuilder.newClient().target("http://localhost:9080").path("/resilience/api/client/timeout").request().get();
System.out.println("hello");
}
}
Now I'd expect that the method getFailingRequest() would time out after 4 ms and throw an exception. The actual behaviour is that the application prints "start", waits 5 seconds until the client returns, prints "hello" and then throws an "org.eclipse.microprofile.faulttolerance.exceptions.TimeoutException".
I turned on further debug information:
<logging traceSpecification="com.ibm.ws.microprofile.*=all" />
in server.xml. I get these information, that the timeout is registered even bevor the client is called! But the thread is not interrupted.
(if someone tells me how to get the stacktrace pretty in here... I can do that.)
Since this a very basic example: Am I doing anything wrong here? What can I do to make this example run properly.
Thanks
Edit: Running this example on WebSphere Application Server 18.0.0.2/wlp-1.0.21.cl180220180619-0403) auf Java HotSpot(TM) 64-Bit Server VM, Version 1.8.0_172-b11 (de_DE) with the features webProfile-8.0, mpFaultTolerance-1.0 and localConnector-1.0.
Edit: Solution, thanks to Andy McCright and Azquelt.
Since the call cannot be interrupted I have to make it asynchronous. So you got 2 threads: The first an who invoke the second thread with the call. The first thread will be interrupted, the second remains until the call finishes. But now you can go on with failure handling, open the circuit and stuff like that to prevent making further calls to the broken service.
#Path("mpfaulttolerance")
#RequestScoped
public class MpFaultToleranceController {
#Inject
private TestBase test;
#GET
#Path("timeout")
#Timeout(4)
public Response getFailingRequest() throws InterruptedException, ExecutionException {
Future<Response> resp = test.createFailingRequestToClientAsynch();
return resp.get();
}
}
And the client call:
#ApplicationScoped
public class TestBase {
#Asynchronous
public Future<Response> createFailingRequestToClientAsynch() {
Response response = ClientBuilder.newClient().target("http://localhost:9080").path("/resilience/api/client/timeout").request().get();
return CompletableFuture.completedFuture(response);
}
}

It does interrupt threads using Thread.interrupt(), but unfortunately not all Java operations respond to thread interrupts.
Lots of things do respond to interrupts by throwing an InterruptedException (like Thread.sleep(), Object.wait(), Future.get() and subclasses of InterruptableChannel) but InputStreams and Sockets don't.
I suspect that you (or the library you're using to make the request) is using a Socket which isn't interruptible so you don't see your method return early.
It's particularly unintuitive because Liberty's JAX-RS client doesn't respond to thread interrupts as Andy McCright mentioned. We're aware it's not a great situation and we're working on making it better.

I had the same problem. For some URLs I consume, the Fault Tolerance timeout doesn't work.
In my case I use RestClient. I solved my problem using the readTimeout() of the RestClientBuilder:
MyRestClientClass myRestClientClass = RestClientBuilder.newBuilder().baseUri(uri).readTimeout(3l, TimeUnit.SECONDS) .build(MyRestClientClient.class);
One advantage of using this Timeout control is that you can pass the timeout as a parameter.

Business Logic in Netty?

I'm developing a server based on the Netty libraby and I'm having a problem with how to structure the application with regards to business Logic.
currenty I have the business logic in the last handler and that's where I access the database. The thing I can't wrap my head around is the latency of accessing the database(blocking code). Is it advisable to do it in the handler or is there an alternative? code below:
public void channelRead(ChannelHandlerContext ctx, Object msg)
throws Exception {
super.channelRead(ctx, msg);
Msg message = (Msg)msg;
switch(message.messageType){
case MType.SIGN_UP:
userReg.signUp(message.user);// blocking database access
break;
}
}

you should execute the blocking calls in DefaultEventExecutorGroup or your custom threadpool that can be added to when the handler is added
pipeline.addLast(new DefaultEventExecutorGroup(50),"BUSSINESS_LOGIC_HANDLER", new BHandler());
ctx.executor().execute(new Runnable() {
#Override
public void run() {
//Blocking call
}});

Your custom handler is initialized by Netty everytime the Server receives a request, hence one instance of the handler is responsible for handling one Client.
So, it is perfectly fine for issuing blocking calls in your handler. It will not affect other Client's, as long as you don't block it indefinitely (or atleast not for very long time), thereby not blocking Netty's Thread for long and you do not get too much load on your server instance.
However, if you want to go for asynchronous design, then there can be more than a few design patterns that you can use.
For eg. with Netty, if you can implement WebSockets, then perhaps you can make the blocking calls in a separate Thread, and when the results are available, you can push them to the client through the WebSocket already established.

how to implement an event-drive consumer in camel

I am very new to Camel, and have been struggling to understand how to use camel in a specific scenario.
In this scenario, there is a (Java-based) agent that generates actions from time to time. I need an event-driven consumer to get notified of these events. These events will be routed to a 'file' producer (for the time being).
In the camel book, the example is for a polling consumer. I could not find a generic solution for an event-driven consumer.
I came across a similar implementation for JMX :
public class JMXConsumer extends DefaultConsumer implements NotificationListener {
JMXEndpoint jmxEndpoint;
public JMXConsumer(JMXEndpoint endpoint, Processor processor) {
super(endpoint, processor);
this.jmxEndpoint = endpoint;
}
public void handleNotification(Notification notification, Object handback) {
try {
getProcessor().process(jmxEndpoint.createExchange(notification));
} catch (Throwable e) {
handleException(e);
}
}
}
Here, the handleNotification is invoked whenever a JMX notification arrives.
I believe I have to do something similar to get my consumer notified whenever the agent generates an action. However, the above handleNotification method is specific to JMX. The web page says: " When implementing your own event-driven consumer, you must identify an analogous event listener method to implement in your custom consumer."
I want to know: How can I identify an analogous event listener, so that my consumer will be notified whenever my agent has an action.
Any advice/link to a web page is very much appreciated.

I know this is an old question, but I've been struggling with it and just thought I would document my findings for anyone else searching for an answer.
When you create an Endpoint class (extending DefaultEndpoint) you override the following method for creating a consumer:
public Consumer createConsumer(Processor processor)
In your consumer then, you have access to a Processor - calling 'process' on this processor will create an event and trigger the route.
For example, say you have some Java API that listens for messages, and has some sort of Listener. In my case, the Listener puts incoming messages onto a LinkedBlockingQueue, and my Consumer 'doStart' method looks like this (add your own error handling):
#Override
protected void doStart() throws Exception {
super.doStart();
// Spawn a new thread that submits exchanges to the Processor
Runnable runnable = new Runnable() {
#Override
public void run() {
while(true) {
IMessage incomingMessage = myLinkedBlockingQueue.take();
Exchange exchange = getEndpoint().createExchange();
exchange.getIn().setBody(incomingMessage);
myProcessor.process(exchange);
}
}
};
new Thread(runnable).start();
}
Now I can put the Component that creates the Endpoint that creates this Consumer in my CamelContext, and use it like this:
from("mycomponent:incoming").to("log:messages");
And the log message fires every time a new message arrives from the Java API.
Hope that helps someone!

Event driven is what camel is.
Any route is actually an event listener.
given the route:
from("activemq:SomeQueue").
bean(MyClass.class);
public class MyBean{
public void handleEvent(MyEventObject eventPayload){ // Given MyEventObject was sent to this "SomeQueue".
// whatever processing.
}
}
That would put up an event driven consumer. How to send events then? If you have camel embedded in your app and access to the CamelContext from your event action generator, then you could grab a Producer Template from it and just fire of your event to whatever endpoint you defined in Camel, such as "seda:SomeQueue".
Otherwise, if your Camel instance is running in another server or instance than your application, then you should use some other transport rather than SEDA. Preferably JMS, but others will do as well, pick and choose. ActiveMQ is my favourite. You can start an embedded activemq instance (intra JVM) easily and connect it to camel by:
camelContext.addComponent("activemq", activeMQComponent("vm://localhost"));