I have created bean for inbound channel with acknowledge property as manual, and chain method for publishing the output message ,
<int-amqp:inbound-channel-adapter channel="InputChannel"
queue-names="Input" connection-factory="connectionFactory" concurrent-consumers="1" message-converter="Converter"
acknowledge-mode="MANUAL" prefetch-count="5"/>
<int:chain input-channel="InputChannel" output-channel="OutputChannel">
<int:transformer method = "transform" >
<bean class="com.sampleconverter" />
</int:transformer>
<int:service-activator method="transform">
<bean class="com.Transformer" />
</int:service-activator>
<int:object-to-string-transformer />
</int:chain>
Can you please help me with the way to acknowledge messages processed with the manual acknowledge mode,
Thanks in advance.
The Reference Manual has dedicated paragraph on the matter:
Setting the mode toMANUAL allows user code to ack the message at some other point during processing. To support this, with this mode, the endpoints provide the Channel and deliveryTag in the amqp_channel and amqp_deliveryTag headers respectively.
#ServiceActivator(inputChannel = "foo", outputChannel = "bar")
public Object handle(#Payload String payload, #Header(AmqpHeaders.CHANNEL) Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) Long deliveryTag) throws Exception {
// Do some processing
if (allOK) {
channel.basicAck(deliveryTag, false);
// perhaps do some more processing
}
else {
channel.basicNack(deliveryTag, false, true);
}
return someResultForDownStreamProcessing;
}
Related
So I have application that sends message to activemq queue with spring integration.
<int-feed:inbound-channel-adapter id="feedAdapter"
channel="feedChannel"
auto-startup="${auto.startup:true}"
url="https://stackoverflow.com/feeds/question/49479712">
<int:poller fixed-rate="10000"/>
</int-feed:inbound-channel-adapter>
<int:channel id="feedChannel"/>
<int:transformer id="transformer" input-channel="feedChannel"
expression="payload.title + payload.author + '#{systemProperties['line.separator']}'"
output-channel="feedOutputChannel"/>
<int:channel id="feedOutputChannel"/>
<jms:outbound-gateway id="jmsOutGateway"
request-destination="inputQueue"
request-channel="feedOutputChannel"
requires-reply="false"/>
But now I want to create different application which consumes message from that queue and just prints it out to console with spring integration. I have made this:
<jms:message-driven-channel-adapter id="JMSInboundAdapter" destination="inputQueue"/>
<bean id="inputQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="input.queue"/>
</bean>
It works when I run application that sends message to queue. But it doesnt when I run message consume application.
Error I get : Dispatcher has no subscribers for channel 'application.JMSInboundAdapter'.
How do I need to configure my message consumer application?
If there is no channel on the adapter, the id becomes the channel name.
You need something to subscribe to that channel (e.g. a <service-activator inputChannel="JMSInboundAdapter" ... />).
Currently we are using Spring Integration 2.1.0 Release(Due to legacy application can not switch on latest version ) in our application.
Application flow is as below:
All the configuration details are defined in a configuration file, like host name, port number, terminator etc
Get the message from TCP using tcp-inbound-channel-adapter via channel.
Pass it to splitter for further flow.
Here issue is if message has terminator other than, which is defined in configuration file,message does not come to class defined for splitter, if terminator is same, it is working fine.
Requirement is if terminator value is different it should show a error message on same channel using tcp-outbound-channel-adapter(inbound and outbound is used due asynchronous call).
I have enabled the application and spring logging at Trace level but not able to understand why and where message is stuck.
Code for Configuration file is
<Config>
<host>localhost</host>
<port>8888</port>
<mode>server</mode>
<terminator>10</terminator>
<msgLength>65535</msgLength>
<inChannel>telnetInboundCustomChannel</inChannel>
</Config>
XML for connection details
<beans:bean id="serverCustomSerializer"
class="com.core.serializer.CustomSerializer">
<beans:property name="terminatingChar" value="${server.terminator}"/>
<beans:property name="maxLength" value="${server.msgLength}"/>
</beans:bean>
<beans:bean id="serverFactoryTaskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<beans:property name="corePoolSize" value="5" />
<beans:property name="queueCapacity" value="0" />
</beans:bean
<int:channel id="telnetLandingChannel" />
<ip:tcp-connection-factory id="serverFactory" type="server"
host="${server.host}" port="${server.port}" single-use="false"
serializer="${server.serializer}" deserializer="${server.serializer}" task-
executor="serverFactoryTaskExecutor"/>
<ip:tcp-inbound-channel-adapter id="serverInboundAdpater"
channel="telnetLandingChannel" connection-factory="serverFactory"
error-channel="errorChannel" auto-startup="false"/>
<ip:tcp-outbound-channel-adapter id="serverOutboundAdapter"
channel="serverReplyChannel"
connection-factory="serverFactory"
auto-startup="true"/>
XML for Channel details and flow are:
<int:channel id="telnetInboundCustomChannel" />
<int:splitter id="messageSplitter"
input-channel="telnetInboundCustomChannel" ref="telnetCustomMessageSplitter"
method="splitCustomMessageStream"
outputchannel="base24CustomSplitterChannel" />
<int:filter id="messageFilter" input-
channel="base24CustomSplitterChannel"
output-channel="base24CustomCoreMessageChannel"
ref="telnetCustomMessageFilter"
method="customMessageFilter" />
<!--Other code to get data from filer and pass it to correct router -->
If somehow message is visible in filter class, I can apply the logic to written error code on TCP connection.
I have applied the break points on run() of TcpNetConnection class as well. I am not able to understand Spring Integration internal flow. How message is coming even till splitter.
I have noticed one more thing if I send message with correct terminator, after sending with wrong terminator, Spring will append new message with old message.
Looks like without correct terminator spring is not able to cut the frame and it is stuck in telnetInboundCustomChannel.
Please guide how to fix this issue and reason of issue for better understanding.
It's not clear how you can detect a bad terminator. By definition the deserializer needs to know a message is complete before returning. You could detect a socket close (bite < 0) and n>0 and return a special message but I don't see how else you can emit a message unless you know what invalid terminator(s) to look for.
EDIT
If you mean check for another "special" (non-printable) character, then you can use something like...
if (n > 0 && (bite == bytes.byteValue())) {
break;
}
else (if bite < 0x20) {
return ("Bad terminator for" + new String(buffer, 0, n)).getBytes();
}
The requirement is strictly meaningless. There is no such thing as a message in TCP, and no such thing as a message with an undefined terminator in any protocol.
I have an inbound RabbitMQ channel adapter that successfully processes 3000 messages per day, however very occasionally I see a unacked message count of 1 in the RabbitMQ admin console. This seems to remain like that.
I do have a re-try advice chain to re-try 3 times and then move to a DLQ via a dead letter routing key, this has worked fine for most exceptions.
The unacked has happened twice in the last few weeks, and on one of the occasions I was able to take a thread dump and see that the int-http:outbound-gateway call was stuck waiting for a http response getStatusCode()
I have a receive-timeout="59000" on the int-amqp:inbound-channel-adapter which I was hoping would timeout the thread anywhere it exceeds the timeout ?
I now notice there is a reply-timeout attribute on the int-http:outbound-gateway should I be setting that ?
Any ideas appreciated ?
<int-amqp:inbound-channel-adapter id="amqpInCdbEvents" channel="eventsAMQPChannel" channel-transacted="true" transaction-manager="transactionManager"
queue-names="internal.events.queue" connection-factory="connectionFactory"
receive-timeout="59000" concurrent-consumers="${eventsAMQPChannel.concurrent-consumers}"
advice-chain="retryChain" auto-startup="false" />
<int:channel id="eventsAMQPChannel" />
<!-- CHAIN of processing for Asynch Processing of Events from intermediate Queue -->
<int:chain id="routeEventChain" input-channel="eventsAMQPChannel">
<int:json-to-object-transformer type="xx.xx.xx.json.Event" object-mapper="springJacksonObjectMapper"/>
<int:header-enricher>
<int:header name="originalPayload" expression="payload" overwrite="true"/>
<int:header name="message_id" expression="payload.id" overwrite="true"/>
</int:header-enricher>
<int:router expression="payload.eventType">
<int:mapping value="VALUE" channel="valueEventChannel"/>
<int:mapping value="SWAP" channel="swapEventChannel"/>
</int:router>
</int:chain>
<int:channel id="valueEventChannel" />
<int:channel id="swapEventChannel" />
<int:chain id="valueEventChain" input-channel="valueEventChannel" output-channel="nullChannel">
<int:transformer ref="syncValuationTransformer" />
<int:object-to-json-transformer object-mapper="springJacksonObjectMapper" />
<int:header-enricher>
<int:header name="contentType" value="application/json;charset=UTF-8" overwrite="true"/>
</int:header-enricher>
<int-http:outbound-gateway id="httpOutboundGatewayValuationServiceFinalValuation"
expected-response-type="java.lang.String"
http-method="POST" charset="UTF-8"
extract-request-payload="true"
url="${value.service.uri}/value"/>
</int:chain>
reply-timeout is a timeout when sending the reply to the reply channel (if it can block - e.g. a bounded queue channel that's full).
int-http:outbound-gateway call was stuck waiting for a http response getStatusCode()
You set the client timeout (readtimeout) on a ClientHttpRequestFactory that you can configure into the outbound adapter...
/**
* Create a new instance of the {#link RestTemplate} based on the given {#link ClientHttpRequestFactory}.
* #param requestFactory HTTP request factory to use
* #see org.springframework.http.client.SimpleClientHttpRequestFactory
* #see org.springframework.http.client.HttpComponentsClientHttpRequestFactory
*/
public RestTemplate(ClientHttpRequestFactory requestFactory) {
this();
setRequestFactory(requestFactory);
}
I have a producers( write in CPP) that send to “spring-integration server” the binary-data.
And it works correctly as :
spring integration time out clients
Now I have to send a reply (like an ACK) to the producer.
I have read about the gateway, but actually I’m confused.
My configuration is:
<int-ip:tcp-connection-factory id="serverTcpConFact"
type="server"
port="5566"
using-nio="true"
single-use="false"
task-executor="myTaskExecutor"
deserializer="serializer"
serializer="serializer"/>
<int-ip:tcp-inbound-channel-adapter id="tcpInboundAdapter"
channel="tcpInbound"
connection-factory="serverTcpConFact" />
<int:channel id="tcpInbound" />
<int:service-activator
output-channel="tcpOutbound"
input-channel="tcpInbound"
ref="importService"
method="handler" />
<bean id="importService" class="my.ImportService" />
<int:channel id="tcpOutbound" />
<int:gateway id="mygateway"
service-interface="my.IpMyGatway"
default-request-channel="tcpInbound"
default-reply-channel="tcpOutbound"
default-reply-timeout="6000"/>
I also have an custom serializator, the problem is that the my spring integration server doesn’t send the reply.
I need that the reply executes:
#Override
public void serialize(MyMessage arg0, OutputStream outputStream) throws IOException {
// TODO Auto-generated method stub
logger.info("serialize messages");
// here I have to write my ACK ! ( .. or not?)
}
And then sends the message to the producer for each message.
Thank you.
I wonder why <int-ip:tcp-inbound-gateway> isn't enough for you...
There is just enough to generate a reply message from service and gateway will send it ot the client as response.
The simple sample:
<ip:tcp-inbound-gateway id="gatewaySerializedNio"
connection-factory="connectionFactory"
request-channel="serviceChannel" />
<channel id="serviceChannel" />
<service-activator input-channel="serviceChannel"
ref="service" method="process"/>
<beans:bean id="service" class="com.my.proj.MyService" />
The return value from MyService#process method will be serialized to the TCP socket.
I am testing Spring-AMQP with Spring-Integration support, I've following configuration and test:
<rabbit:connection-factory id="connectionFactory" />
<rabbit:queue name="durableQ"/>
<int:channel id="consumingChannel">
<int:queue capacity="2"/> <!-- Message get Acked as-soon-as filled in Q -->
</int:channel>
<int-amqp:inbound-channel-adapter
channel="consumingChannel"
queue-names="durableQ"
connection-factory="connectionFactory"
concurrent-consumers="1"
acknowledge-mode="AUTO"
/>
public static void main(String[] args) {
System.out.println("Starting consumer with integration..");
AbstractApplicationContext context = new ClassPathXmlApplicationContext(
"classpath:META-INF/spring/integration/spring-integration-context-consumer.xml");
PollableChannel consumingChannel = context.getBean("consumingChannel",
PollableChannel.class);
int count = 0;
while (true) {
Message<?> msg = consumingChannel.receive(1000);
System.out.println((count++) + " \t -> " + msg);
try { //sleep to check number of messages in queue
Thread.sleep(50000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
In this configuration it was evident that as soon as message arrives at consumingChannel they are Acked and hence removed from queue. I validated this by placing a high sleep after receive and check queue-size. There are no further control on it.
Now if I set acknowledge-mode=MANUAL, there are no ways seems to do manual ack via spring integration.
My need is to process message and after processing do a manual-ack so till ack message remains persisted at durableQ.
Is there any way to handle MANUAL ack with spring-amqp-integration? I want to avoid passing ChannelAwareMessageListener to inbound-channel-adapter since I want to have control of consumer's receive.
Update:
It even doesn't seems to be possible when using own listener-container with inbound-channel-adapter:
// Below creates a default direct-channel (spring-integration channel) named "adapter", to receive poll this channel which is same as above
<int-amqp:inbound-channel-adapter id="adapter" listener-container="amqpListenerContainer" />
<bean id="amqpListenerContainer" class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory" />
<property name="queueNames" value="durableQ" />
<property name="acknowledgeMode" value="MANUAL" />
// messageListener not allowed when using with adapter, so no way of having own ChannelAwareMessageListener, so no channel exposed onMessage, hence no way to ack
<property name="messageListener" ref="listener"/>
</bean>
<bean id="listener" class="com.sd.springint.rmq.MsgListener"/>
Above configuration throws error as messageListener property is not allowed, see inline comment on tag. So purpose of using listner-container got defeated (for exposing channel via ChannelAwareMessageListener).
To me spring-integration cannot be used for manual-acknowledgement (I know, this is a hard saying!), Can anyone help me in validating this or Is there any specific approach/configuration required for this which I am missing?
The problem is because you are using async handoff using a QueueChannel. It is generally better to control the concurrency in the container (concurrent-consumers="2") and don't do any async handoffs in your flow (use DirectChannels). That way, AUTO ack will work just fine. Instead of receiving from the PollableChannel subscribe a new MessageHandler() to a SubscribableChannel.
Update:
You normally don't need to deal with Messages in an SI application, but the equivalent of your test with a DirectChannel would be...
SubscribableChannel channel = context.getBean("fromRabbit", SubscribableChannel.class);
channel.subscribe(new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
System.out.println("Got " + message);
}
});
MANUAL Ack is allowed only via Channel.basicAck(). So, you should have an access to the Channel, on which your message was received.
Try to play with advice-chain of <int-amqp:inbound-channel-adapter>:
Implement some Advice as MethodBeforeAdvice
The advice-chain on Container is applied for ContainerDelegate#invokeListener
The first argument of that method is exactly a Channel
Suppose you can place to the MessageProperties.headers that Channel within that Advice
And configure <int-amqp:inbound-channel-adapter> with mapped-request-headers to that Channel.
And in the end try to invoke basicAck() on that Channel header from Spring Integration Message in the any place of your downstream flow.