Scenario : poll the database every 'n' seconds and retrieve list. Do some internal business validations.If validations are success only, send the retrieved list to an external system.Receive an acknowledgment response from that system(After receiving this,poller should send next list).Then perform some business operations like calling other systems where polling is not needed.
can any one tell me how this scenario can be handled?
poller.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:int="http://www.springframework.org/schema/integration"
xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:int-jdbc="http://www.springframework.org/schema/integration/jdbc"
xmlns:context="http://www.springframework.org/schema/context" xmlns:p="http://www.springframework.org/schema/p"
xmlns:task="http://www.springframework.org/schema/task" xmlns:int-http="http://www.springframework.org/schema/integration/http"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-4.1.xsd
http://www.springframework.org/schema/integration
http://www.springframework.org/schema/integration/spring-integration-4.1.xsd
http://www.springframework.org/schema/integration/http
http://www.springframework.org/schema/integration/http/spring-integration-http-4.1.xsd
http://www.springframework.org/schema/integration/jdbc
http://www.springframework.org/schema/integration/jdbc/spring-integration-jdbc-4.1.xsd
http://www.springframework.org/schema/jdbc
http://www.springframework.org/schema/jdbc/spring-jdbc-4.1.xsd">
<import resource="persistence-config.xml" />
<int:channel id="inchannel">
</int:channel>
<int:channel id="outchannel">
<int:dispatcher task-executor="taskExecutor"/>
</int:channel>
<task:executor id="taskExecutor" pool-size="2"/>
<bean id="poller" class="main.java.com.as.poller.PollerService" />
<int:service-activator input-channel="inchannel"
output-channel="outchannel" ref="poller" method="sendMessage" />
<int-jdbc:inbound-channel-adapter
query="select loyalty_id from TBL_RECEIPT where receipt_status=0"
channel="inchannel" data-source="dataSource" max-rows-per-poll="1">
<int:poller fixed-rate="5000">
</int:poller>
</int-jdbc:inbound-channel-adapter>
</beans>
ackage main.java.com.as.poller;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.stereotype.Component;
#Component
public class PollerService{
public void sendMessage()
{
ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext(
"AS-servlet.xml");
DirectChannel directChannel = (DirectChannel) context.getBean("inchannel");
}
}
I added a direct channel to poll the DB.Also added an executer channel..But i am getting the exception
Error creating bean with name 'org.springframework.integration.config.ServiceActivatorFactoryBean#0': Cannot resolve reference to bean 'executerChannel' while setting bean property 'outputChannel'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'executerChannel' is defined
rg.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1477)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1222)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
Can anyone please help with this...I have created only PollerService.java and dont have any idea to implement executer channel and task executer
The <poller> does that for you by default if you use fixed-delay option and your downstream flow is single threaded (only DirectChannels). In that case the next polling task (in your case reading DB using JPA) won't be started until the finish of previous one and some time like fixed-delay. See PeriodicTrigger, BTW.
Regarding your "Then perform some business operations". You should just make the next channel after "Receive an acknowledgment response" as an ExecutorChannel to free the polling Thread for the next poll.
You should really read more in the Spring Integration Reference Manual and especially about the Poller.
Related
In my Spring boot application, I have written below to poll Data from DB .
The data is polled when the Spring boot application is started..
How can I start polling the data on some event?
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-jdbc="http://www.springframework.org/schema/integration/jdbc"
xmlns:jdbc="http://www.springframework.org/schema/jdbc"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/jdbc http://www.springframework.org/schema/integration/jdbc/spring-integration-jdbc.xsd
http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.0.xsd">
<int:channel id="fromdb"/>
<int:channel id="outChannel">
<int:dispatcher task-executor="executorPool"/>
</int:channel/>
<task:executor id="executorPool" pool-size="10"/>
<int-jdbc:inbound-channel-adapter
channel="fromdb" data-source="dataSource"
max-rows-per-poll="100"
query="SELECT * FROM Items WHERE INVENTORY_STATUS = 0"
update="UPDATE Items SET INVENTORY_STATUS = 1">
<int:poller fixed-delay="4000" />
</int-jdbc:inbound-channel-adapter>
<int:bridge input-channel="fromdb" output-channel="outChannel"/>
<int:service-activator input-channel="outChannel"
ref="jdbcMessageHandler" method="onMessage" />
</beans>
Above configuration start polling when app starts. However, I want polling to when below API is invoked
#RestController
public class FileConsumerController {
private final Logger logger = LoggerFactory.getLogger(FileConsumerController.class);
#GetMapping(value = "/inventoryStatus")
public ResponseEntity<String> getInventoryStatus(#RequestParam("accId") String accId){
// ONLY after this API is invoked, I want above int-jdbc:inbound-channel-adapter to start polling
}
}
ONLY after this API is invoked, I want above int-jdbc:inbound-channel-adapter to start polling. How can this be achieved?
See an answer to this question: How to invoke Spring channel adapter from rest api?
The inbound channel adapter does its logic on a schedule and that happens in a cycle independently of the rest application logic. It really sounds like you'd need to get data from DB when that getInventoryStatus is called. So, a JDBC outbound Gateway with request-reply capabilities is what you need.
any inbound channel adapter is an automatic reaction to data changes in the target store: it does nothing with human initiated events.
See more info in docs:
https://www.enterpriseintegrationpatterns.com/patterns/messaging/PollingConsumer.html
https://docs.spring.io/spring-integration/reference/html/core.html#pollable-message-source
I have a Spring application where I am using RabbitMQ for publishing and consuming messages. The connection factory is created in the following way:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-amqp="http://www.springframework.org/schema/integration/amqp"
xmlns:rabbit="http://www.springframework.org/schema/rabbit"
xmlns:int-stream="http://www.springframework.org/schema/integration/stream"
xsi:schemaLocation="http://www.springframework.org/schema/integration/amqp http://www.springframework.org/schema/integration/amqp/spring-integration-amqp.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/stream http://www.springframework.org/schema/integration/stream/spring-integration-stream.xsd
http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.4.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="RabbitMQ" class="com.services.RabbitMqListenerService" />
<rabbit:connection-factory id="connectionFactory"
host="${rabbitmq.host}" port="${rabbitmq.port}"
username="${rabbitmq.username}" password="${rabbitmq.password}" cache-mode="${rabbitmq.cacheMode}"
connection-cache-size="${rabbitmq.connectionCacheSize}"/>
</beans>
But while I am running the tests instead of creating this connection I want to create an in-memory map which will work as a queue for me. This I want to do to make sure all my messages are in sync as the async messages might make the tests flaky.
In spring-boot applications it is very simple to do. In application.java I can add below code and it will work:
public ConnectionFactory connectionFactory() {
if (Boolean.parseBoolean(environment.getProperty("mock.rabbitmq"))) {
return new MockRMQConnectionFactory();
}
com.rabbitmq.client.ConnectionFactory rabbitmqConnectionFactory = new com.rabbitmq.client.ConnectionFactory();
rabbitmqConnectionFactory.setHost(environment.getProperty("rabbitmq.address"));
rabbitmqConnectionFactory.setUsername(environment.getProperty("rabbitmq.user"));
rabbitmqConnectionFactory.setPassword(environment.getProperty("rabbitmq.password"));
rabbitmqConnectionFactory.setConnectionTimeout(
Integer.parseInt(environment.getProperty("rabbitmq.connection.timeout")));
return connectionFactory;
}
I am just not sure how to do the same thing in when I am trying to create rmq beans in the XML file (rabbitmq.xml). Can someone please help me here.
You can have the separate bean configuration file like you showed above for testing.
Annotate your test with following.
#ContextConfiguration(locations = "rabitmqconfiguration.xml")
#RunWith(SpringJUnit4ClassRunner.class)
where rabitmqconfiguration.xml will have bean definitions.
I am using the following spring configuration to transfer a file from local folder to remote SFTP server.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:int="http://www.springframework.org/schema/integration"
xmlns:sftp="http://www.springframework.org/schema/integration/sftp"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration
http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/sftp
http://www.springframework.org/schema/integration/sftp/spring-integration-sftp-2.2.xsd">
<bean id="sftpSessionFactory"
class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
<property name="host" value="xxxxxxx" />
<property name="knownHosts" value = "C:\knownhosts"/>
<property name="user" value="wildfly" />
<property name="password" value="w!ldfly" />
<property name="port" value="22" />
</bean>
<int:channel id="sftpChannel" />
<sftp:outbound-channel-adapter id="triggerFtpOutBound" channel="sftpChannel"
session-factory="sftpSessionFactory" remote-directory="/home/wildfly">
</sftp:outbound-channel-adapter>
I am using the following code to send file.
#Autowired
private MessageChannel sftpChannel;
Function()
{
File f = new File("c:/test.txt");
Message<File> message = MessageBuilder.withPayload(f).build();
sftpChannel.send(message);
}
I am getting null pointer exception at sftpChannel.send(message). How can i autowire sftpChannel in my code?
The following code works. But, i want to Autowire sftpChannel.
ApplicationContext context = new ClassPathXmlApplicationContext("spring/config/spring-sftp.xml");
MessageChannel sftpChannel = context.getBean("sftpChannel", MessageChannel.class);
File f = new File("c:/test.txt");
Message<File> message = MessageBuilder.withPayload(f).build();
sftpChannel.send(message);
In order to use autowired you need to include
<context:annotation-config />
to your configuration file
<beans
//...
xmlns:context="http://www.springframework.org/schema/context"
//...
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-2.5.xsd">
//...
<context:annotation-config />
//...
</beans>
Here, there is a full example
http://www.mkyong.com/spring/spring-auto-wiring-beans-with-autowired-annotation/
Autowiring works the same as any Spring application; the bean that gets the autowired channel must be declared as a bean in the context.
You can always debug Spring bean wiring by enabling DEBUG logging for org.springframework.
The framework puts out lots of information while wiring classes.
I'm not sure in your code, but that looks like you are going to use an #Autowired property from the constructor.
For this purpose you have to use the constructor injection.
Even if your Function is a valid Spring bean, it is a bad idea to send any message from the constructor.
Please, revise your design and read more books about Spring.
I am using one inbound channel for downloading files & then other outbound channel to upload files & i do not need any poller but dont know why i am getting this exception. Can someone please help me out in this issue. I am using spring 4.0.6 Integration and spring core 4.0.6.
Logtrace:
Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sftpInboundAdapter': Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: No poller has been defined for channel-adapter 'sftpInboundAdapter', and no default poller is available within the context.
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1553)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:684)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)
at com.canaldigital.tsi.bank.config.AppMain.main(AppMain.java:35)
Caused by: java.lang.IllegalArgumentException: No poller has been defined for channel-adapter 'sftpInboundAdapter', and no default poller is available within the context.
at org.springframework.util.Assert.notNull(Assert.java:112)
at org.springframework.integration.config.SourcePollingChannelAdapterFactoryBean.initializeAdapter(SourcePollingChannelAdapterFactoryBean.java:157)
at org.springframework.integration.config.SourcePollingChannelAdapterFactoryBean.afterPropertiesSet(SourcePollingChannelAdapterFactoryBean.java:115)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1612)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1549)
ApplicationConfig.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:task="http://www.springframework.org/schema/task"
xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-sftp="http://www.springframework.org/schema/integration/sftp"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-4.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-3.2.xsd
http://www.springframework.org/schema/task
http://www.springframework.org/schema/task/spring-task-4.0.xsd
http://www.springframework.org/schema/integration
http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/sftp
http://www.springframework.org/schema/integration/sftp/spring-integration-sftp.xsd">
<bean id="sftpSessionFactory" class="org.springframework.integration.file.remote.session.CachingSessionFactory">
<constructor-arg ref="defaultSftpSessionFactory" />
</bean>
<int-sftp:inbound-channel-adapter id="sftpInboundAdapter"
session-factory="sftpSessionFactory"
channel="requestSFTPSEDEVChannel"
filename-pattern="*.*"
remote-directory="/home/oracle/IBSTOBANK/Test/Incoming/"
preserve-timestamp="true"
local-directory="C:/Working_Directory/temp/SE-DEV/"
auto-create-local-directory="true"
temporary-file-suffix=".writing"
delete-remote-files="true">
<!-- <int:poller fixed-rate="1000"/> -->
</int-sftp:inbound-channel-adapter>
<int-sftp:outbound-channel-adapter id="sftpOutboundAdapter"
session-factory="sftpSessionFactory"
channel="requestOutBoundChannel"
remote-directory="/home/oracle/IBSTOBANK/Test/Outgoing/">
</int-sftp:outbound-channel-adapter>
<int:channel id="requestSFTPSEDEVChannel">
</int:channel>
<int:channel id="requestOutBoundChannel">
</int:channel>
Well, looks like you must come back to the theory and understand difference between polling and event-listening.
From other side you should agree that SFTP is some kind of remote file system and there is really no any other way to download new files from there, unless polling. The rule is applied for the JDBC, when we'd like to read new records from the table and so on.
So, since we agreed that <int-sftp:inbound-channel-adapter> is exactly Polling Consumer, we must declare <poller>, implicitly or like a global one.
That's from the side of the existing design and your config on the matter.
From other side let us know what you mean with:
i do not need any poller
How are you going to download files from the SFTP?
Yes, actually <int-sftp:outbound-gateway> can do the stuff for you, but that is a bit different story...
I'm working at implementing a buffer, in order to execute various jobs.
I'm working on a Spring-based project.
I decided to use Spring Integration to accomplish my aim. I went through a Cafè sample project in order to understand how SI works.
To demonstrate Spring Integration I implemented a table where I dynamically insert jobs to be executed.
This table is the "gateway". Then I configured a router and various channels.
What I don't fully understand are the poller elements which have to check if new jobs in the "gateway" are present.
Is this correct?
If so, how can I configure the poller?
Thanks in advance!
Here the xml code:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-feed="http://www.springframework.org/schema/integration/feed"
xmlns:int-stream="http://www.springframework.org/schema/integration/stream"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration
http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/stream
http://www.springframework.org/schema/integration/stream/spring-integration-stream.xsd">
<int:gateway id="usersToSend" service-interface="it.stasbranger.spidly.rss.UsersToSend" />
<int:channel id="channel" />
<int:channel id="providers"/>
<int:router input-channel="providers" ref="providerRouter" method="resolveProviderChannel"/>
<int:channel id="twitterChannel">
<int:queue capacity="10"/>
</int:channel>
<int:service-activator input-channel="twitterChannel" ref="twitterService" method="updateStatusByProfile"/>
<int:channel id="facebookChannel">
<int:queue capacity="10"/>
</int:channel>
<int:service-activator input-channel="facebookChannel" ref="facebookService" method="updateStatusByProfile"/>
<int:channel id="linkedinChannel">
<int:queue capacity="10"/>
</int:channel>
<int:service-activator input-channel="linkedinChannel" ref="linkedinService" method="writeSlogan2Linkedin"/>
<bean id="twitterService" class="it.social.TwitterService"/>
<bean id="facebookService" class="it.social.FacebookService"/>
<bean id="linkedinService" class="it.social.LinkedinService"/>
<int:poller id="poller" default="true">
</int:poller>
FB
<gateway/>s are not polled, they are "message-driven" in that the caller "sends" a message into the flow using the gateway.
For a polling scenario, use an <int:inbound-channel-adapter/> which polls a method (on the poller's schedule) looking for work to do.
If the method returns null, the poller goes back to sleep (until the next trigger). If the method returns a value, the message is sent to the channel.