I'm new to Spring Integration and Spring Integration AMQP.
I have the following code:
<bean id="enricher" class="soft.Enricher"/>
<amqp:inbound-channel-adapter queue-names="QUEUE1" channel="amqpInboundChannel"/>
<int:channel id="amqpInboundChannel">
<int:interceptors>
<int:wire-tap channel="logger"/>
</int:interceptors>
</int:channel>
<int:header-enricher input-channel="amqpInboundChannel" output-channel="routingChannel">
<int:header name="store" value="sj" />
</int:header-enricher>
<int:channel id="routingChannel" />
<int:header-value-router input-channel="routingChannel" header-name="store">
<int:mapping value="sj" channel="channelSJ" />
<int:mapping value="jy" channel="channelJY" />
</int:header-value-router>
<amqp:outbound-channel-adapter channel="channelSJ" exchange-name="ex_store" routing-key="sj" amqp-template="rabbitTemplate"/>
<amqp:outbound-channel-adapter channel="channelJY" exchange-name="ex_store" routing-key="jy" amqp-template="rabbitTemplate"/>
<int:channel id="channelSJ" />
<int:channel id="channelJY" />
<int:logging-channel-adapter id="logger" level="ERROR" />
The setup is the following:
Everything is working fine except that headers are lost when a message is picked up by the inbound-channel-adapter.
Likewise the header being enriched called "store" is also lost when the message is being send to the exchange using the outbound-channel-adapter.
This is how a message is looking before being picked up by the inbound-channel-adapter:
This is how the same message is looking after the whole process (notice no headers)
I think your problem is described here:
"By default only standard AMQP properties (e.g. contentType) will be copied to and from Spring Integration MessageHeaders. Any user-defined headers within the AMQP MessageProperties will NOT be copied to or from an AMQP Message unless explicitly identified via 'requestHeaderNames' and/or 'replyHeaderNames' properties of this HeaderMapper. If you need to copy all user-defined headers simply use wild-card character ''.*"
So you need to define your own custom instance of DefaultAmqpHeaderMapper and configure the inbound-channel-adapter with it. See here.
It might look something like this:
<bean id="myHeaderMapper" class="org.springframework.integration.amqp.support.DefaultAmqpHeaderMapper">
<property name="requestHeaderNames" value="*"/>
<property name="replyHeaderNames" value="*"/>
</bean>
<amqp:inbound-channel-adapter queue-names="QUEUE1" channel="amqpInboundChannel"
header-mapper="myHeaderMapper"/>
Related
I am trying to do the following using spring integration.
I would like to read from an input channel an incoming Message that goes throws a series of Steps and produces a final message.Each step can throw an Exception. This could be either due to a DB call or ExternalCall etc. Each step is considered as a service-activator. If all is well a successful a final Message is generated.I need to handle each Exception thrown by each Step as a separate case and then produce the appropriate final Message.
Using the below xml I have written a test code that passes when sucessfull but when an error occurs although I see a Final message generated in the postSend (sent=true) on channel 'bean 'finalChannel' log but the process does not return to the final queue.
Why does this happen?
<?xml version="1.0" encoding="UTF-8"?>
<context:component-scan base-package="com.demo.">
</context:component-scan>
<header-enricher input-channel="inputChannel"
output-channel="enrichedChannel">
<header name="initialMessage" expression="getPayload()" />
</header-enricher>
<!-- first-step -->
<service-activator input-channel="enrichedChannel"
output-channel="firstStep" ref="validateOne" />
<gateway id="validateOne" default-request-channel="ch1"
error-channel="errors1" />
<chain input-channel="ch1">
<service-activator id="mockServiceOneActivator"
ref="mockServiceOne" method="register" />
</chain>
<!-- enrich-header with payload to be available down the line -->
<header-enricher input-channel="firstStep"
output-channel="secondStep">
<header name="initialInfo" expression="getPayload()" />
</header-enricher>
<!-- second-step -->
<service-activator input-channel="secondStep"
output-channel="enrichWithTwoChannel" ref="validateTwo" />
<gateway id="validateTwo" default-request-channel="ch2"
error-channel="errors2" />
<chain input-channel="ch2">
<service-activator id="mockServiceTwoActivator"
ref="mockServiceTwo" method="callExternal" />
</chain>
<!-- enrich-header with payload to be available down the line -->
<header-enricher input-channel="enrichWithTwoChannel"
output-channel="eligibilityCheck">
<header name="serviceTwoInfo" expression="getPayload()" />
</header-enricher>
<!-- final-step -->
<service-activator input-channel="eligibilityCheck"
output-channel="finalChannel" id="mockServiceFinalActivator"
ref="mockServiceFinal" method="submit" />
<!-- error handling -->
<service-activator input-channel="errors1"
output-channel="finalChannel" ref="traceErrorHandler"
method="handleFailedTrace" />
<service-activator input-channel="errors2"
output-channel="finalChannel" ref="traceErrorHandler2"
method="handleFailedTrace" />
<channel id="finalChannel">
<queue />
</channel>
<service-activator input-channel="errorChannel"
ref="globalExceptionHandler" method="handleError" />
My handler code looks like this ..
#Component("traceErrorHandler")public class TraceErrorHandler {
public Message<FinalMessage> handleFailedTrace(Message<?> errorMessage) {
MessagingException payload = (MessagingException) errorMessage.getPayload();
InitialMessage im = (InitialMessage) payload.getFailedMessage().getHeaders().get("initialMessage");
ServiceOneException error = (ServiceOneException) payload.getCause();
FinalMessage v = new FinalMessage(im.getFrom(), im.getTo(), error.getErrorTemplate());
Message<FinalMessage> finalMessage = MessageBuilder.withPayload(v).copyHeaders(payload.getFailedMessage().getHeaders()).build();
return finalMessage;
}}
I am not sure if this is the correct approach in regards to error-handling. The initial chains are simple activators but further down the line there will be more logic.
Should we I always use chains to handle separate error channels
Even if its a single activator service called.
Only when there are Multiple service-activator/transforms that encapsulate a single point of error?
EDIT 1
I added a request-handler-advice-chain reference a bean.
<int:service-activator
id="serviceOneActivator" input-channel="enrichedChannel"
ref="serviceOne" method="register" output-channel="firstStep">
<int:request-handler-advice-chain>
<ref bean="myclass" />
</int:request-handler-advice-chain></int:service-activator>
The reference bean is at first a default definition in xml
<bean id="myclass"
class="org.springframework.integration.handler.advice.ExpressionEvaluatingRequestHandlerAdvice">
<property name="failureChannel" ref="finalChannel" />
<property name="onFailureExpression" value="#payload"/>
<property name="returnFailureExpressionResult" value="true" />
</bean>
if I add the <property name="onFailureExpression" value="#payload"/> property it fails with:
Cannot convert value of type 'java.lang.String' to required type 'org.springframework.expression.Expression' for property 'onFailureExpression'
What i would like to do is convert the exception message to a final message object? but all expression i have added seem to fail on load.
A chain with one component makes no sense.
A chain is syntactic sugar only, each component still stands alone at runtime.
If you want to handle errors for individual endpoints, consider using ExpressionEvaluatingRequestHandlerAdvices instead.
See https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/messaging-endpoints.html#message-handler-advice-chain
I need to pass the file to service layer which i am receiving in SFTP path.
below is configuration and i am seeing the message receiving in my service-activator like
GenericMessage [payload=com.jcraft.jsch.ChannelSftp$2#322906a2,
headers={closableResource=org.springframework.integration.sftp.session.SftpSession#379662a7,
id=704c58e7-1d93-3bef-0330-233c0f9af55c, file_remoteDirectory=/tmp/charge/,
file_remoteFile=Charge.txt, timestamp=1594158522576}]
<bean id="sftpSessionFactory"
class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
<property name="host" value="hostname"/>
<property name="port" value="22"/>
<property name="user" value="vkp"/>
<property name="password" value="1234"/>
<property name="allowUnknownKeys" value="true"/>
</bean>
<int-sftp:inbound-streaming-channel-adapter id="sftpAdapterAutoCreate"
session-factory="sftpSessionFactory"
filename-pattern="*.txt"
channel="receiveChannel"
remote-directory="/tmp/charge/">
</int-sftp:inbound-streaming-channel-adapter>
<int:poller fixed-rate="25000" max-messages-per-poll="1" id="shippingChargePoller" default="true"/>
<int:channel id="receiveChannel">
<int:queue/>
</int:channel>
<int:stream-transformer id="withCharset" charset="UTF-8" input-
channel="receiveChannel" />
<int:service-activator id="FeedListener" input-channel="receiveChannel" method="onMessage">
<bean class="com.listener.ChargeFeedListener"/>
</int:service-activator>
public void onMessage(Message<?> message){
System.out.println(message.toString());
System.out.println( " Received File is "+message.getPayload());
}
But i am not receiving the file in my java class . What i need to do to get the file ?
Please, read documentation: https://docs.spring.io/spring-integration/docs/current/reference/html/sftp.html#sftp-streaming. The <int-sftp:inbound-streaming-channel-adapter> is not about files. It does open an InputStream for a remote entry (probably file on SFTP) and let you to do with this stream whatever you want. For example (also according that docs), there is a StreamTransformer which let's you to read that stream into a byte[] or string if you provide a charset. If you really want to deal with files, then you need to consider to switch to the <int-sftp:inbound-channel-adapter>. That one pull the remote entry and store its content into a local file. Then that java.io.File is sent to the channel for your consideration.
I think we had a chat with you on the matter in other your question: Spring SFTP Integration is not polling the file.
Please, let us know what is wrong with our docs that confuses you so you have to raise questions like this over here.
I have a problem with spring integration basically I'am trying to create chain which will invoke 1 of several services and returns result as reply to chain gateway for further processing.
<int:chain input-channel="inputChannel" output-channel="outputChannel">
<int:header-enricher>
<int:header name="temporalPayload" expression="payload"/>
<int:header name="matcher" value="other" />
</int:header-enricher>
<int:gateway request-channel="routerChannel" reply-channel="replyChannel" error-channel="errorChannel"/>
<int:transformer expression="headers.temporalPayload"/>
</int:chain>
<int:router input-channel="routerChannel" expression="headers.matcher.matches('(test)|(test2)')?headers.matcher:'other'" >
<int:mapping value="test1" channel="channel1"/>
<int:mapping value="test2" channel="channel2"/>
<int:mapping value="other" channel="channel3" />
</int:router>
<int:service-activator input-channel="channel1" output-channel="replyChannel" ref="service1" method="handle"/>
<int:service-activator input-channel="channel2" output-channel="replyChannel" ref="service2" method="handle"/>
<int:service-activator input-channel="channel3" output-channel="replyChannel" ref="service3" method="handle"/>
Problem that gateway does not receive reply to channel "replyChannel" from service-activator. What I'am doing wrong?
You can turn on DEBUG logging level for org.springframework.integration category and observe how your message travels.
On the other hand, in most cases you don’t need that reply-Channel on the gateway and just rely on the replyChannel header populated by the gateway. In this case you even have to remove the output-Channel on the service-activators.
There might be some subscriber for your repjyChannel who steals messages for the Gateway.
You can try without reply-Channel or investigate logs as I suggested above.
I have 2 spring integration context files using similar file-based integration pattern. Both scan directory looking for a message and both work if deployed by themselves. If I include both modules into another spring context they are loading without issues. However only second one is working, and the first one is getting: MessageDeliveryException: Dispatcher has no subscribers. I've attempted to combine them into a single context file with no positive gain. We are currently on version 2.1.3 of Spring Integration and version 2.1 of Spring Integration File. Any ideas are greatly appreciated!
inpayment-context.xml:
<!-- START of in-bound message implementation -->
<int:channel id="file-inpayment-channel" datatype="java.io.File" />
<bean id="xmlPatternFileListFilter" class="org.springframework.integration.file.filters.SimplePatternFileListFilter">
<constructor-arg value="*.xml" />
</bean>
<task:executor id="batchInBoundExecuter" pool-size="1-1" queue-capacity="20" rejection-policy="CALLER_RUNS" />
<int-file:inbound-channel-adapter directory="file:${inpayment.inbox}" filter="xmlPatternFileListFilter"
channel="file-inpayment-channel">
<int:poller id="inPaymentrPoller" fixed-delay="1000" task-executor="batchInBoundExecuter" default="true" />
</int-file:inbound-channel-adapter>
<bean id="inPaymentService" class="com.somepackage.InPaymentBootstrapService" />
<int:service-activator id="batchJobLaunchService" ref="inPaymentService" input-channel="file-inpayment-channel"
method="schedule" />
<!-- START of out-bound message implementation -->
<int:channel id="inpayment-file-out-channel" datatype="java.io.File" />
<int:gateway id="inboundPaymentGateway" service-interface="com.somepackage.InboundPaymentGateway"
default-request-channel="inpayment-file-out-channel" />
<int-file:outbound-channel-adapter directory="file:${inpayment.inprocess}" channel="inpayment-file-out-channel"
auto-create-directory="true" delete-source-files="true" />
<!-- END of out-bound message implementation -->
scheduler-context.xml:
<!-- START of in-bound message implementation -->
<int:channel id="scheduler-file-in-channel" datatype="java.io.File" />
<bean id="simplePatternFileListFilter" class="org.springframework.integration.file.filters.SimplePatternFileListFilter">
<constructor-arg value="*.xml" />
</bean>
<task:executor id="batchJobRunExecuter" pool-size="1-1" queue-capacity="20" rejection-policy="CALLER_RUNS"/>
<int-file:inbound-channel-adapter directory="file:${scheduler.inbox}" filter="simplePatternFileListFilter"
channel="scheduler-file-in-channel">
<int:poller id="schedulerPoller" fixed-delay="5000" task-executor="batchJobRunExecuter" default="true" />
</int-file:inbound-channel-adapter>
<bean id="launchService" class="com.somepackage.BatchJobLaunchService" />
<int:service-activator id="batchJobLaunchService" ref="launchService" input-channel="scheduler-file-in-channel"
method="schedule" />
<!-- END of in-bound message implementation -->
<!-- START of out-bound message implementation -->
<int:channel id="scheduler-file-out-channel" datatype="java.io.File" />
<int:channel id="scheduler-xml-out-channel" datatype="com.somepackage.ScheduledJob" />
<int:gateway id="batchJobSchedulerGateway" service-interface="com.innovation.customers.guideone.scheduler.integration.SchedulerGateway"
default-request-channel="scheduler-xml-out-channel" />
<int:transformer input-channel="scheduler-xml-out-channel" output-channel="scheduler-file-out-channel" ref="schedulerFileTransformer"
method="transformToFile" />
<int-file:outbound-channel-adapter directory="file:${scheduler.completed}" channel="scheduler-file-out-channel"
auto-create-directory="true" delete-source-files="true" />
<!-- END of out-bound message implementation -->
Common Spring Context:
<context:component-scan base-package="com.somepackage" />
<import resource="classpath:g1-scheduler-context.xml"/>
<import resource="classpath:g1-inpayment-context.xml"/>
EDIT
2014-08-27 11:01:01,530 ERROR [batchJobRunExecuter-1][:] org.springframework.integration.handler.LoggingHandler : org.springframework.integration.MessageDeliveryException: Dispatcher has no subscribers. at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:108) at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:101)
I see the issue in your config:
<int:service-activator id="batchJobLaunchService" ref="inPaymentService" input-channel="file-inpayment-channel"
method="schedule" />
and
<int:service-activator id="batchJobLaunchService" ref="launchService" input-channel="scheduler-file-in-channel"
method="schedule" />
They assume to be different services, but at the same time use the same id - batchJobLaunchService.
Spring by default allows to do that, but only the last bean definition with the same id wins. That's why the <service-activator> for the launchService hasn't been pupolated and hence the EventDrivenConsumer bean hasn't been subscribed to the scheduler-file-in-channel.
Be careful and use unique id for all your beans.
It isn't so easy to throw expection on the duplication case, but if you switch on INFO for the org.springframework category you'll the message that one bean overrrides another.
I am trying to use spring integration to do a Gateway --> Splitter-->ServiceActivator --> Aggregator Pattern in an event driven fashion backed by JMS . I Expect the service activator to be multi-threaded and any of the end points can be executed on a cluster and not necessarily the originating server . I could get this working in a single JVM without using JMS ( Using SI Channels ) but I understand that SI Channels will not help me scale horizontally i.e multiple VMs .
Here's the configuration I have so far
<int:gateway id="transactionGateway" default-reply-channel="transaction-reply"
default-request-channel="transaction-request" default-reply-timeout="10000"
service-interface="com.test.abc.integration.service.ProcessGateway">
</int:gateway>
<int-jms:outbound-gateway id="transactionJMSGateway"
correlation-key="JMSCorrelationID" request-channel="transaction-request"
request-destination="transactionInputQueue" reply-channel="transaction-reply"
reply-destination="transactionOutputQueue" extract-reply-payload="true"
extract-request-payload="true">
<int-jms:reply-listener
max-concurrent-consumers="20" receive-timeout="5000"
max-messages-per-task="1" />
</int-jms:outbound-gateway>
<!-- Inbound Gateway for Splitter -->
<int-jms:inbound-gateway id="splitterGateWay"
request-destination="transactionInputQueue" request-channel="splitter-input"
reply-channel="splitter-output" concurrent-consumers="1"
default-reply-destination="processInputQueue"
max-concurrent-consumers="1" extract-reply-payload="true"
correlation-key="JMSCorrelationID" extract-request-payload="true" />
<!-- Inbound Gateway Invokes Service Activator and Sends response back to
the channel -->
<int-jms:inbound-gateway id="seriveActivatorGateway"
request-destination="processInputQueue" request-channel="process-input"
reply-channel="process-output" concurrent-consumers="1"
default-reply-destination="processOutputQueue"
max-concurrent-consumers="1" extract-reply-payload="true"
correlation-key="JMSCorrelationID" extract-request-payload="true"
max-messages-per-task="1" />
<int-jms:inbound-gateway id="aggregatorGateway"
request-destination="processOutputQueue" request-channel="aggregator-input"
reply-channel="aggregator-output" concurrent-consumers="1"
default-reply-destination="transactionOutputQueue"
max-concurrent-consumers="1" extract-reply-payload="true"
extract-request-payload="true" max-messages-per-task="1"
correlation-key="JMSCorrelationID" />
<int:splitter id="transactionSplitter" input-channel="splitter-input"
ref="processSplitter" output-channel="splitter-output">
</int:splitter>
<int:service-activator id="jbpmServiceActivator"
input-channel="process-input" ref="jbpmService" requires-reply="true"
output-channel="process-output">
</int:service-activator>
<int:aggregator id="transactionAggregator"
input-channel="aggregator-input" method="aggregate" ref="processAggregator"
output-channel="aggregator-output" message-store="processResultMessageStore"
send-partial-result-on-expiry="false">
</int:aggregator>
Before using gateway I tried using JMS backed Channels and that approach was't successful either . The problem I am facing now is that the Splitter now reply back to the transactionOutputQueue . I tried playing around with jms:header-enricher without much success . I feel that my approach to the problem /SI might have fundamental flaw . Any help /guidance is highly appreciated .
Also , in the code snippet I have provided above use a simple in memory aggregator , I understand that If I need to get this working across the cluster I might need a JDBC backed Aggregator but for the for now , I am trying to get this pattern working on a single VM
Here's the updated working configuration based on Gary's Comment
<bean id="processOutputQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="test.com.abc.process.output" />
</bean>
<bean id="transactionOutputQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="test.com.abc.transaction.result" />
</bean>
<bean id="transactionInputQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="test.com.abc.transaction.input" />
</bean>
<int:gateway id="transactionGateway"
default-request-channel="transaction-request" default-reply-timeout="10000"
default-reply-channel="aggregator-output"
service-interface="com.test.abc.integration.service.ProcessGateway">
</int:gateway>
<int:splitter id="transactionSplitter" input-channel="transaction-request"
ref="processSplitter" output-channel="splitter-output">
</int:splitter>
<int-jms:outbound-gateway id="splitterJMSGateway"
correlation-key="JMSCorrelationID" request-channel="splitter-output"
request-destination="processInputQueue" reply-channel="aggregator-input"
reply-destination="processOutputQueue" extract-request-payload="true"
extract-reply-payload="true">
<int-jms:reply-listener
max-concurrent-consumers="20" receive-timeout="5000" />
</int-jms:outbound-gateway>
<!-- Inbound Gateway Invokes Service Activator and Sends response back to
the channel -->
<int-jms:inbound-gateway id="seriveActivatorGateway"
request-destination="processInputQueue" request-channel="process-input"
reply-channel="process-output" default-reply-destination="processOutputQueue"
concurrent-consumers="5" max-concurrent-consumers="10"
extract-reply-payload="true" correlation-key="JMSCorrelationID"
extract-request-payload="true" max-messages-per-task="1" />
<int:service-activator id="jbpmServiceActivator"
input-channel="process-input" ref="jbpmService" requires-reply="true"
output-channel="process-output">
</int:service-activator>
<int:aggregator id="transactionAggregator"
input-channel="aggregator-input" ref="processAggregator"
output-channel="aggregator-output" message-store="processResultMessageStore"
send-partial-result-on-expiry="false">
</int:aggregator>
<bean id="processResultMessageStore"
class="org.springframework.integration.store.SimpleMessageStore" />
<bean id="processResultMessageStoreReaper"
class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="processResultMessageStore" />
<property name="timeout" value="5000" />
</bean>
<task:scheduled-tasks>
<task:scheduled ref="processResultMessageStoreReaper"
method="run" fixed-rate="1000" />
</task:scheduled-tasks>
<int:logging-channel-adapter id="logger"
level="DEBUG" log-full-message="true" />
<int-stream:stdout-channel-adapter
id="stdoutAdapter" channel="logger" />
I limited the JMS pipeline only to the Service Activator , which is what I originally wanted .
The only question I have based on the above approach is that do I need to have my Aggregator backed by a database even if I use this across multiple VMS ( Since the JMS gateway in front of it make sure that it receives only the messages that have valid correlation ID ?)
Regards ,
You probably don't need to use JMS between every component. However we have lots of test cases for chained gateways like this, and all works fine.
Something must be wired up incorrectly. Since you didn't show your full configuration, it's hard to speculate.
Be sure to use the latest version (2.2.4) and turn on DEBUG logging and follow a message through the flow; as long as your message payload is identifiable across JMS boundaries, it should be easy to figure out where things go awry.