I am spawning a thread which will keep pulling the chunk of records from database and putting them into the queue. This thread will be started on the server load. I want this thread to be active all the time. If there are no records in the database I want it to wait and check again after some time. I was thinking of using spring task scheduler to schedule this but not sure if that is right because I only want my task to be started once. What will be the good way of implementing this in Spring ?
Also, i need to have a boundary check that if my thread goes down (because of any error or exception condition) it should be re-instantiated after some time.
I can do all this in java by using thread communication methods but just trying if there is something available in Spring or Java for such scenarios.
Any suggestions or pointer will help.
You can use the #Scheduled annotation to run jobs. First create a class with a method that is annotated with #Scheduled.
Class
public class GitHubJob {
#Scheduled(fixedDelay = 604800000)
public void perform() {
//do Something
}
}
Then register this class in your configuration files.
spring-context.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:task="http://www.springframework.org/schema/task"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-3.1.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd">
<tx:annotation-driven/>
<task:annotation-driven scheduler="myScheduler"/>
<task:scheduler id="myScheduler" pool-size="10"/>
<bean id="gitHubJob" class="org.tothought.spring.jobs.GitHubJob"/>
</beans>
For more about scheduling visit the Spring Docs.
#Scheduled(fixedDelay=3600000)
private void refreshValues() {
[...]
}
That runs a task once every hour. It needs to be void and accept no arguments. If you use Spring's Java configuration you'll also need to add the annotation #EnableScheduling to one of your #Configuration classes.
You can try using Quartz scheduler. http://quartz-scheduler.org/
That will allow you to specify the duration of time between task executions. You can set a flag (boolean) that says if the class has run before to avoid duplicating code that you only want to run the 'first' time.
Spring has out-ot-the-box support for scheduling tasks. Here are for the docs for the latest 3.2.x version of spring but check the docs for the version you are using. Looks like it uses Quartz under the hood for scheduling tasks.
I think your requirement is just regular senario which quartz or spring scheduling framework supports very well. but you want to create a spececial approach to impl it. my suggestion is to keep it simple and stupid. spring scheudling will take advantage your worker thread by pooling instead of runing it all the time.
As of statistics, it 's easy to check worker class's log. if you want see the statistics in web console, you need to record worker's log in database. I am sure you could make it easily in normal way.
Related
I faced with problem when error in JDBC transaction cause return JMS-message to queue, I guess so. It means that some message are being processed again.
My app has next configuration:
I have configured JtaTransactionManager like this:
...
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jee="http://www.springframework.org/schema/jee" xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/jee
http://www.springframework.org/schema/jee/spring-jee.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd">
<tx:jta-transaction-manager/>
....
And I has configured JMS DefaultMessageListenerContainer like this:
#Bean
public DefaultMessageListenerContainer jmsContainer(ConnectionFactory connectionFactory, PlatformTransactionManager transactionManager, ....) throws NamingException {
DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setSessionTransacted(true);
container.setTransactionManager(transactionManager);
....
return container;
}
In my case, when I receive a JMS message, in my code logic happens org.springframework.dao.DataIntegrityViolationException. I handle exception via try-catch block and I expect everything will be fine. But in this time, as I see, message return to queue and it being processed again. This behavior isn't reproduce, if in my code happens other Exception, like as IllegalStateException/RuntimeException.
And I suppose that error in JDBC transaction causes broke JMS transaction despite the fact I handle exception.
Let me know, is it correct that I use one transaction manager for JPA and JDBC?
Or I need two different implementations for my DataSource and JMS DefaultMessageListenerContainer?
Also, if I can produce more information, I will.
Thanks!
I want to combine Akka, Apache Camel, Spring and do not know the way forward for leveraging the three things in the same project.
I was successfully able to
1. write some working code with akka, akka-camel extension and camel routes(Java DSL)
2. use camel and spring (use java DSL but spring for transactions and etc..)
Now I need to combine 1 and 2. Can anyone suggest me the simplest way to achieve this?
EDIT
Some say AKKA no longer supports Spring due to conflict in object instantiation as per the link below
Why spring integration doc for akka exists only for 1.3.1 but not for next versions
Also a similar question is there without a proper solution being presented but the post is about 2 years old
akka-camel 2.2.1 route definition using Spring XML
In one blog post (which I can't get hold of the link right now) a method has been described which is in summary, the actors are defined and used Akka way and what ever the processing Akka actors does to be wired using Spring. But there wasn't any solid example.
I imagine your #2 looks like this:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ctx="http://www.springframework.org/schema/context"
xmlns:camel="http://camel.apache.org/schema/spring"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd >
<!-- Camel route configuration -->
<camelContext id = "testCamelRouteContext" xmlns="http://camel.apache.org/schema/spring">
<route id="test_data_webservice">
<from uri="jetty:http://localhost:8888/myTestService"/>
<log logName="HTTP LOG" loggingLevel="INFO" message="HTTP REQUEST: ${in.header.testdata}"/>
<process ref="myTestService"/>
</route>
</camelContext>
<context:annotation-config />
<bean class="com.package.service" id="myTestService"/>
<bean id="genericDao" class="com.dao.Impl">
<property name="dataSource" ref="datasource" />
</bean>
<bean id="testingDataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
datasource stuff
</bean>
</beans>
Is it possible that you can get this camel context through Akka? Something like.
Add in your Akka config:
akka.camel.context-provider="myNewContext"
New ContextProvider class:
class myNewContext extends ContextProvider{
override def getContext(system: ExtendedActorSystem): SpringCamelHybridContext
}
I am guessing this is where the bean injection collision between Spring and Akka could occur. I have never used Akka before so my answer is trivial but I wanted to see if I could provide some help to you.
Reviving an old thread.
akka-springctx-camel library is there to make your life painless to integrate Akka, Spring, Camel, CXF etc.
Add artifact :
<dependency>
<groupId>com.github.PuspenduBanerjee</groupId>
<artifactId>akka-springctx-camel</artifactId>
<version>1.0.0</version>
</dependency>
Add Camel Context Provider in Akka config:
akka.camel.context-provider=system.SpringCamelContextProvider
Get hold of ActorSystem :
implicit val system = SpringContextActorSystemProvider.create
Create a custom RouteBuilder[other way could be Akka Consumer]
class CustomRouteBuilder(system: ActorSystem, echoActor: ActorRef)
extends RouteBuilder {
def configure {
from("direct:testEP")
.routeId("test-route")
.to(echoActor)
}
Get Camel(Spring) Context and add routes to it:
val camel = CamelExtension(system)
camel.context.addRoutes(
new CustomRouteBuilder(system, system.actorOf(Props[EchoActor])))
This test case will give you a detailed idea: https://github.com/PuspenduBanerjee/akka-springctx-camel/blob/master/src/test/scala/AkkaSpringCtxTestSpec.scala
I'm trying to use camel-quartz2 component in a cluster mode with JDBCJobStore.
quartz.properties file:
org.quartz.scheduler.instanceId=AUTO
org.quartz.scheduler.instanceName=JobCluster
org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.oracle.OracleDelegate
org.quartz.jobStore.dataSource=dsQuartzTest
org.quartz.jobStore.isClustered=true
org.quartz.threadPool.class=org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount=5
org.quartz.dataSource.dsQuartzTest.driver = oracle.jdbc.driver.OracleDriver
org.quartz.dataSource.dsQuartzTest.URL = jdbc:oracle:thin:pmuser#//10.13.13.10:1521/PDB1
org.quartz.dataSource.dsQuartzTest.user = pmuser
org.quartz.dataSource.dsQuartzTest.password = oracle
org.quartz.dataSource.dsQuartzTest.maxConnections = 10
camel-context.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Configures the Camel Context-->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:camel="http://camel.apache.org/schema/spring"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring
http://camel.apache.org/schema/spring/camel-spring.xsd">
<bean id="quartz2" class="org.apache.camel.component.quartz2.QuartzComponent">
<property name="propertiesFile" value="quartz.properties"/>
</bean>
<bean id="quartzBean" class="com.ubs.rbs.integration.QuartzBean"/>
<camel:camelContext xmlns="http://camel.apache.org/schema/spring">
<camel:route id="quartzRoute">
<camel:from uri="quartz2://tictac?cron=0+0/5+*+1/1+*+?+*"/>
<camel:setBody>
<camel:simple>
${header.triggerName} - ${header.fireTime}
</camel:simple>
</camel:setBody>
<camel:to uri="log:hello"/>
</camel:route>
<!--
<camel:route>
<camel:from uri="timer://updateQuartzRoute?repeatCount=1"/>
<camel:to uri="bean:quartzBean?method=reschedule(*,'quartzRoute')"/>
</camel:route>
-->
</camel:camelContext>
</beans>
When I first start the application, camel-quartz2 component schedules job with appropriate cron expression, and all works just OK. But, when I stop all instances of app, trigger remains in a WAITING state (it's, probably, OK for cluster, as component cannot tell when last instance is stopped, also, I cannot tell if this point is relevant, but without cluster it seems was no issue). So, when I start app next time trigger is already exists and its settings won't apply. Importantly, when I change cron expression and restart all instances, quartz uses old expression from DB, not new one from component's uri.
I found a workaround for this issue, using additional route (commented out in the xml above) to reschedule quartz in the custom bean as below:
public class QuartzBean {
public void reschedule(CamelContext context, String quartzRouteId) throws Exception {
QuartzEndpoint endpoint = (QuartzEndpoint) context.getRoute(quartzRouteId).getEndpoint();
QuartzComponent component = endpoint.getComponent();
Scheduler scheduler = component.getScheduler();
Trigger oldTrigger = scheduler.getTrigger(endpoint.getTriggerKey());
TriggerBuilder tb = oldTrigger.getTriggerBuilder();
Trigger newTrigger = tb.withSchedule(CronScheduleBuilder.cronSchedule(endpoint.getCron())).build();
scheduler.rescheduleJob(oldTrigger.getKey(), newTrigger);
}
}
Alternatively, I can patch camel-quartz2 component as below (org.apache.camel.component.quartz2.QuartzEndpoint#addJobInScheduler):
private void addJobInScheduler() throws Exception {
// Add or use existing trigger to/from scheduler
Scheduler scheduler = getComponent().getScheduler();
JobDetail jobDetail;
boolean triggerExisted = scheduler.getTrigger(triggerKey) != null;
if (triggerExisted) {
ensureNoDupTriggerKey();
}
jobDetail = createJobDetail();
Trigger trigger = createTrigger(jobDetail);
updateJobDataMap(jobDetail);
// Schedule it now. Remember that scheduler might not be started it, but we can schedule now.
Date nextFireDate = triggerExisted ? scheduler.rescheduleJob(triggerKey, trigger) : scheduler.scheduleJob(jobDetail, trigger);
if (LOG.isInfoEnabled()) {
LOG.info("Job {} (triggerType={}, jobClass={}) is scheduled. Next fire date is {}",
new Object[] {trigger.getKey(), trigger.getClass().getSimpleName(),
jobDetail.getJobClass().getSimpleName(), nextFireDate});
}
// Increase camel job count for this endpoint
AtomicInteger number = (AtomicInteger) scheduler.getContext().get(QuartzConstants.QUARTZ_CAMEL_JOBS_COUNT);
if (number != null) {
number.incrementAndGet();
}
jobAdded.set(true);
}
Unlike the current version in the project's repository, my version always creates new trigger and reschedules job if trigger already existed.
My questions are: Am I missing something obvious? How this component supposed to work in such scenario? Should I try to change component itself or is there a better way to change its schedule from outside?
UPD: I tried versions 2.13.1 from maven and 2.14.SNAPSHOT from sources.
Camel versions 2.12.2 and before have known bugs related to Quartz. Give 2.12.3 and upwards a try, if you're not.
I've submitted a bug: https://issues.apache.org/jira/browse/CAMEL-7627 along with patch and unit test.
I'm trying to get a grip using MongoDB with the Spring Data MongoDB framework. I tried severeal approaches to connect to my local DB and insert + retrieve some collections and documents, using the official Spring Reference Documentation and some simple examples like this Hello-World-Demo.
Actually, for the beginning I'm going to use the MongoTemplate to keep it simple. But now I'run into the following problem.
When I use the Spring Configuration with annotations configure the setting needed to connect to my local DB, everything works fine.
Otherwise When I use XML for the configuration, I run into an java.lang.NullPointerException at com.mongodb.DBTCPConnector.getClusterDescription
Here are my configuration files and the example code for connecting to the DB:
Use Case 1 - Spring Configuration with annotations:
//package, imports etc. here
#Configuration
public class MongoConfiguration {
public #Bean MongoDbFactory mongoDbFactory() throws Exception {
return new SimpleMongoDbFactory(new MongoClient(), "Test1");
}
public #Bean MongoTemplate mongoTemplate() throws Exception {
return new MongoTemplate(mongoDbFactory());
}
}
Using the configuration class like this ...
AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext(MongoConfiguration.class);
MongoOperations mongoOperation = (MongoOperations) ctx.getBean("mongoTemplate");
ctx.close();
for (String s : mongoOperation.getCollectionNames()) {
System.out.println(s);
}
.. creates this output:
documents
leute
system.indexes
system.users
Use Case 2 - XML configuration (SpringConfig2.xml):
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:mongo="http://www.springframework.org/schema/data/mongo"
xsi:schemaLocation="http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/data/mongo
http://www.springframework.org/schema/data/mongo/spring-mongo-1.0.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<mongo:mongo host="127.0.0.1" port="27017" />
<mongo:db-factory dbname="Test1" />
<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
<constructor-arg name="mongoDbFactory" ref="mongoDbFactory" />
</bean>
</beans>
Using the configuration file like this ...
GenericXmlApplicationContext ctx = new GenericXmlApplicationContext("SpringConfig2.xml");
MongoOperations mongoOperation = (MongoOperations) ctx.getBean("mongoTemplate");
ctx.close();
for (String s : mongoOperation.getCollectionNames()) {
System.out.println(s);
}
.. results in this error
java.lang.NullPointerException
at com.mongodb.DBTCPConnector.getClusterDescription(DBTCPConnector.java:404)
at com.mongodb.DBTCPConnector.getMaxBsonObjectSize(DBTCPConnector.java:653)
at com.mongodb.Mongo.getMaxBsonObjectSize(Mongo.java:641)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:81)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:66)
at com.mongodb.DB.getCollectionNames(DB.java:510)
at org.springframework.data.mongodb.core.MongoTemplate$13.doInDB(MongoTemplate.java:1501)
at org.springframework.data.mongodb.core.MongoTemplate$13.doInDB(MongoTemplate.java:1499)
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:394)
at org.springframework.data.mongodb.core.MongoTemplate.getCollectionNames(MongoTemplate.java:1499)
at test.main(Test.java:28)
When debugging DBTCPConnector.getClusterDescription, it seems that in the second case the private class variable cluster is for some reason not instantiated, leading to the described error.
What I'd like to know is: am I doing anything wrong within my XML-configuration or when using this config / context? Why does using XML-configuration end in an error, while using annotation configuration just works fine?
Basically (in the end) I just "copy+paste"'d the code examples from the official references for Spring / Spring Data MongoDB.
I'd appreciate any help / suggestions :)
Thanks to the hint from Tushar Mishra in the comments I was able to track down the origin of the error and why it occurs in one case but not in the other.
I'll try to explain in short, maybe it'll save someone some research time (or remember me if I perhaps run into this or a similar error again).
When closing either the AnnotationConfigApplicationContext-object (UC1) or the AnnotationConfigApplicationContext-object (UC2) with ctx.close(), from somewhere
org.springframework.beans.factory.DisposableBean#destroy() is invoked. As far as I understood, within there used singleton beans get destroyed by invoking the destroy()-methods
of each of those beans.
In short: closing the XXApplicationContext-object also destroys the MongoFactoryBean and with it closes the connection to the MongoDB.
So using c**tx.close()** at the described position leads to using a MongoOperations-object on a closed connection at the next line, resulting in the described NullPointerException.
And for why the sample code with Annotations (UC1) runs fine, whereas the sample code configured with XML (UC2) just breaks with an NullPointerException:
In UC1, the org.springframework.beans.factory.DisposableBean#destroy()-call is interupted by some DisposableBeanMethodInterceptor, which tries to redirect to a dispose() method.
But there is no method implemented in MongoFactoryBean, so the Mongo-connection stays alive and can be used even after ctx.close() was invoked. I think the connection will get killed later by the garbage collector, leading to the same error.
I'm not sure what to make of this, yet. If I should implement a dispose()-method myself or something alike. But at least I figured out, why this code sample behave different, although supposed to do the same thing.
Maybe it helps someone. Thanks again, Tushar, for giving a hint to the right direction.
I'm looking for a way to have spring beans registering themselves to a job processor bean who in turn will execute the registered beans on a schedule.
I'm hoping that the bean would just have to implement an interface and by some spring mechanism get registered to the job processor bean. Or alternatively inject the job processor bean into the beans and then somehow the job processor bean can keep track of where it's been injected.
Any suggestions appreciated, it might be that spring is not the tool for this sort of thing?
Use a spring context something like this:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
<!--Scans the classpath for annotated
components #Component, #Repository,
#Service, and #Controller -->
<context:component-scan base-package="org.foo.bar"/>
<!--Activates #Required, #Autowired,
#PostConstruct, #PreDestroy
and #Resource-->
<context:annotation-config/>
</beans>
And define a pojo like this:
#Component
public class FooBar {}
And inject like this:
#Component
public class Baz {
#Autowired private FooBar fooBar;
}
Spring has a powerful abstraction layer for Task Execution and Scheduling.
In Spring 3, there are also some annotations that you can use to mark bean methods as scheduled (see Annotation Support for Scheduling and Asynchronous Execution)
You can let a method execute in a fixed interval:
#Scheduled(fixedRate=5000)
public void doSomething() {
// something that should execute periodically
}
Or you can add a CRON-style expression:
#Scheduled(cron="*/5 * * * * MON-FRI")
public void doSomething() {
// something that should execute on weekdays only
}
Here's the XML code you'll need to add (or something similar):
<task:annotation-driven executor="myExecutor" scheduler="myScheduler"/>
<task:executor id="myExecutor" pool-size="5"/>
<task:scheduler id="myScheduler" pool-size="10"/>
Used together with
<context:component-scan base-package="org.foo.bar"/>
<context:annotation-config/>
as described by PaulMcKenzie, that should get you where you want to go.