I have an inbound RabbitMQ channel adapter that successfully processes 3000 messages per day, however very occasionally I see a unacked message count of 1 in the RabbitMQ admin console. This seems to remain like that.
I do have a re-try advice chain to re-try 3 times and then move to a DLQ via a dead letter routing key, this has worked fine for most exceptions.
The unacked has happened twice in the last few weeks, and on one of the occasions I was able to take a thread dump and see that the int-http:outbound-gateway call was stuck waiting for a http response getStatusCode()
I have a receive-timeout="59000" on the int-amqp:inbound-channel-adapter which I was hoping would timeout the thread anywhere it exceeds the timeout ?
I now notice there is a reply-timeout attribute on the int-http:outbound-gateway should I be setting that ?
Any ideas appreciated ?
<int-amqp:inbound-channel-adapter id="amqpInCdbEvents" channel="eventsAMQPChannel" channel-transacted="true" transaction-manager="transactionManager"
queue-names="internal.events.queue" connection-factory="connectionFactory"
receive-timeout="59000" concurrent-consumers="${eventsAMQPChannel.concurrent-consumers}"
advice-chain="retryChain" auto-startup="false" />
<int:channel id="eventsAMQPChannel" />
<!-- CHAIN of processing for Asynch Processing of Events from intermediate Queue -->
<int:chain id="routeEventChain" input-channel="eventsAMQPChannel">
<int:json-to-object-transformer type="xx.xx.xx.json.Event" object-mapper="springJacksonObjectMapper"/>
<int:header-enricher>
<int:header name="originalPayload" expression="payload" overwrite="true"/>
<int:header name="message_id" expression="payload.id" overwrite="true"/>
</int:header-enricher>
<int:router expression="payload.eventType">
<int:mapping value="VALUE" channel="valueEventChannel"/>
<int:mapping value="SWAP" channel="swapEventChannel"/>
</int:router>
</int:chain>
<int:channel id="valueEventChannel" />
<int:channel id="swapEventChannel" />
<int:chain id="valueEventChain" input-channel="valueEventChannel" output-channel="nullChannel">
<int:transformer ref="syncValuationTransformer" />
<int:object-to-json-transformer object-mapper="springJacksonObjectMapper" />
<int:header-enricher>
<int:header name="contentType" value="application/json;charset=UTF-8" overwrite="true"/>
</int:header-enricher>
<int-http:outbound-gateway id="httpOutboundGatewayValuationServiceFinalValuation"
expected-response-type="java.lang.String"
http-method="POST" charset="UTF-8"
extract-request-payload="true"
url="${value.service.uri}/value"/>
</int:chain>
reply-timeout is a timeout when sending the reply to the reply channel (if it can block - e.g. a bounded queue channel that's full).
int-http:outbound-gateway call was stuck waiting for a http response getStatusCode()
You set the client timeout (readtimeout) on a ClientHttpRequestFactory that you can configure into the outbound adapter...
/**
* Create a new instance of the {#link RestTemplate} based on the given {#link ClientHttpRequestFactory}.
* #param requestFactory HTTP request factory to use
* #see org.springframework.http.client.SimpleClientHttpRequestFactory
* #see org.springframework.http.client.HttpComponentsClientHttpRequestFactory
*/
public RestTemplate(ClientHttpRequestFactory requestFactory) {
this();
setRequestFactory(requestFactory);
}
Related
I'm new in Spring Integration and I'm working with two process:
1.- Read files from sftp, save information. I use method "mget".
2.- After of save information, rename files processed in sftp server with method "mput".
this is my configuration:
<bean id="filePrinter" class="com.maven.integration.FilePrinter" />
<int:channel id="outboundChannel"></int:channel>
<int:channel id="aggregateResultsChannel"/>
<int:channel id="toGet"></int:channel>
<int:gateway id="simpleGateway" service-interface="com.maven.integration.FileGateway"
default-request-channel="ftpChannel" />
<int-sftp:outbound-gateway
session-factory="sftpClientFactory"
request-channel="ftpChannel"
command="mget"
command-options="-R -P"
expression="payload"
filename-regex="(new|.*.csv)"
mode="IGNORE"
local-directory-expression="#targetDir.get() + #remoteDirectory"
reply-channel="outboundChannel"
>
<int-sftp:request-handler-advice-chain>
<int:retry-advice />
</int-sftp:request-handler-advice-chain>
</int-sftp:outbound-gateway>
<int:splitter input-channel="outboundChannel" output-channel="toGet"/>
<bean id="targetDir" class="java.util.concurrent.atomic.AtomicReference">
<constructor-arg value="${target}"/>
</bean>
<int:service-activator ref="filePrinter" method="print"
input-channel="toGet"/>
<int-sftp:outbound-gateway
session-factory="sftpClientFactory"
request-channel="toGet"
command="mput"
command-options="-R"
local-directory="src/test/"
expression="payload"
filename-regex="(new|.*.csv)"
mode="IGNORE"
remote-filename-generator-expression="payload.getName() + '.ack'"
remote-directory-expression="#remoteDirectory"
reply-channel="aggregateResultsChannel"
>
<int-sftp:request-handler-advice-chain>
<int:retry-advice />
</int-sftp:request-handler-advice-chain>
</int-sftp:outbound-gateway>
<int:aggregator input-channel="aggregateResultsChannel"/>
currently only the first outbound-gateway with mget command is executed, but the second outbound-gateway is not executed, how could the second process run?
Because your toGet is a simple DirectChannel with the "only one per message subscriber" logic. At the same time your mput gateway uses that as its request-channel="toGet" sharing this channel with the
<int:service-activator>. And since this is earlier in the config, it is a first subscriber in thetoGetand therefore only this one process the message sent to thetoGet` by the splitter.
I think what you need to do (if the story is still about mput) is exactly opposite with what you have in mget. So, you should do that logic alongside with the splitting, not after. Therefore you should send MGET result to the splitter and to the MPUT in parallel. For this purpose I suggest you to change your outboundChannel to the <publish-subscribe-channel> and use this outboundChannel in the mput gateway for its request-channel.
when a nested chain fails and reaches the error channel flow, the task executor threads block and are not returned to pool. Is there any way to indicate that the end of flow has reached and they need to be returned to pool.
for example the splitter breaks the payload to 3 messages. The messages are served as -
message 1 - fileChannelTaskExecutor1
message 2 - fileChannelTaskExecutor2
if "nested-chain" gateway call is successful, the 3rd message is served to any of these executor thread which gets freed up earlier.
However if the "nested-chain" gateway call fails and reaches errChannel, both the above executor thread block and are not returned to pool. As a result of which successive messages(message 3) are not processed as no thread is available in pool.
<bean id="fileChannelTaskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="2"/>
<property name="daemon" value="false"/>
</bean>
<int:channel id="splitterResponseChannel">
<int:dispatcher task-executor="fileChannelTaskExecutor"/>
</int:channel>
<int:splitter input-channel="splitterRequestChannel" output-channel="splitterResponseChannel" >
<int:chain input-channel="splitterResponseChannel">
<int:gateway request-channel="nested-chain" error-channel="errChannel"/>
</int:chain>
<int:chain input-channel="errChannel" output-channel="nullChannel">
.....
</int:chain>
The problem here is around one-way nullChannel and the <int:gateway> request/reply nature.
Even if you send Exception to the error-channel you should should re-throw it to the call or return some compensation message from the error flow.
Other wise you end up with the hanging on the gateway which waits for the reply forever. By default, of course!
You can adjust reply-timeout on the matter and release your Thread to the pool over some time in case of that one-way error process.
I am using spring integration to make a client socket (java code), which must send messages to server socket (flash client). Basically, i want to push messages to flash through socket communication without caring for any response from it. The message which should be sent is coming from GW upon send method is called.
I am able to push a message, but the problem is that my client socket is waiting for a response and if it doesn't get one it times out at some point. Here is my configuration:
<int:gateway id="gw"
service-interface="integration.MessageGateway"
default-request-channel="input"/>
<int-ip:tcp-connection-factory id="client"
type="client"
host="localhost"
port="6767"
serializer="clientSerializer"
single-use="true" so-keep-alive="true"
so-timeout="10000"/>
<bean id="clientSerializer" class="org.springframework.integration.ip.tcp.serializer.ByteArrayCrLfSerializer" />
<int:channel id="input" />
<int-ip:tcp-outbound-gateway id="outGateway"
request-channel="input"
connection-factory="client"
request-timeout="10000"
reply-timeout="10000"/>
Consider to use one-way <int-ip:tcp-outbound-channel-adapter>: http://docs.spring.io/spring-integration/docs/latest-ga/reference/html/ip.html#tcp-adapters
I have a producers( write in CPP) that send to “spring-integration server” the binary-data.
And it works correctly as :
spring integration time out clients
Now I have to send a reply (like an ACK) to the producer.
I have read about the gateway, but actually I’m confused.
My configuration is:
<int-ip:tcp-connection-factory id="serverTcpConFact"
type="server"
port="5566"
using-nio="true"
single-use="false"
task-executor="myTaskExecutor"
deserializer="serializer"
serializer="serializer"/>
<int-ip:tcp-inbound-channel-adapter id="tcpInboundAdapter"
channel="tcpInbound"
connection-factory="serverTcpConFact" />
<int:channel id="tcpInbound" />
<int:service-activator
output-channel="tcpOutbound"
input-channel="tcpInbound"
ref="importService"
method="handler" />
<bean id="importService" class="my.ImportService" />
<int:channel id="tcpOutbound" />
<int:gateway id="mygateway"
service-interface="my.IpMyGatway"
default-request-channel="tcpInbound"
default-reply-channel="tcpOutbound"
default-reply-timeout="6000"/>
I also have an custom serializator, the problem is that the my spring integration server doesn’t send the reply.
I need that the reply executes:
#Override
public void serialize(MyMessage arg0, OutputStream outputStream) throws IOException {
// TODO Auto-generated method stub
logger.info("serialize messages");
// here I have to write my ACK ! ( .. or not?)
}
And then sends the message to the producer for each message.
Thank you.
I wonder why <int-ip:tcp-inbound-gateway> isn't enough for you...
There is just enough to generate a reply message from service and gateway will send it ot the client as response.
The simple sample:
<ip:tcp-inbound-gateway id="gatewaySerializedNio"
connection-factory="connectionFactory"
request-channel="serviceChannel" />
<channel id="serviceChannel" />
<service-activator input-channel="serviceChannel"
ref="service" method="process"/>
<beans:bean id="service" class="com.my.proj.MyService" />
The return value from MyService#process method will be serialized to the TCP socket.
I have two Java processes, the first one of them produces messages and puts
them onto an ActiveMQ queue. The second process (consumer) uses Spring
Integration to get messages from the queue and processes them in threads.
I have two requirements:
The consumer should have 3 processing threads. If I have 10 messages
coming in through the queue, I want to have 3 threads processing the first 3
messages, and the other 7 messages should be buffered.
When the consumer stops while some messages are not yet processed, it
should continue processing the messages after a restart.
Here's my config:
<bean id="messageActiveMqQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="example.queue" />
</bean>
<int-jms:message-driven-channel-adapter
destination="messageActiveMqQueue" channel="incomingMessageChannel" />
<int:channel id="incomingMessageChannel">
<int:dispatcher task-executor="incomingMessageChannelExecutor" />
</int:channel>
<bean id="incomingMessageChannelExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="daemon" value="false" />
<property name="maxPoolSize" value="3" />
</bean>
<int:service-activator input-channel="incomingMessageChannel"
ref="myMessageProcessor" method="processMessage" />
The first requirement works as expected. I produce 10 messages and 3
myMessageProcessors start processing a message each. As soon as the 1st message has
finished, the 4th message is processed.
However, when I kill the consumer before all messages are processed, those
messages are lost. After a restart, the consumer does not get those messages
again.
I think in the above configuration that's because the threads generated by the
ThreadPoolTaskExecutor queue the messages. So the messages are already removed
from the incomingMessageChannel. Hence I tried setting the queue capacity of
the incomingMessageChannelExecutor:
<property name="queueCapacity" value="0" />
But now I get error messages when I have more than 3 messages:
2013-06-12 11:47:52,670 WARN [org.springframework.jms.listener.DefaultMessageListenerContainer] - Execution of JMS message listener failed, and no ErrorHandler has been set.
org.springframework.integration.MessageDeliveryException: failed to send Message to channel 'incomingMessageChannel'
I also tried changing the message-driven-channel-adapter to an inbound-gateway,
but this gives me the same error.
Do I have to set an error handler in the inbound-gateway, so that the errors go back to the ActiveMQ queue? How do I have to configure the queue so that the messages are kept in the queue if the ThreadPoolTaskExecutor doesn't have a free thread?
Thanks in advance,
Benedikt
No; instead of using an executor channel, you should be controlling the concurrency with the <message-driven-channel-adapter/>.
Remove the <dispatcher/> from the channel and set concurrent-consumers="3" on the adapter.