As far as I tried, I have to manually change the CRON_TRIGGERS table in DB. Dirty...
Any way to make more like this?:
There are 2 apps running, both have in .properties file schedule defined as "every minute" and so works the job
I stop one instance and reconfigure (change in .properties file), so the schedule is "every hour"
I start the instance. Now I would like that instace to check, that such job is already defined in DB and to update the schedule there. It is not happening now using configuration from site http://www.objectpartners.com/2013/07/09/configuring-quartz-2-with-spring-in-clustered-mode/
Or what is the typical solution?
So I guess that when you say .properties file, you actually mean the spring bean XML file(s).
It does not make any sense that you statically configure identical jobs with different schedules. If for whatever reason, one instance restarts, it will automatically apply its own schedule. If statically configured, your job triggers should be the same on all instances
If you properly set <property name="overwriteExistingJobs" value="true"/> in your SchedulerFactoryBean it should automatically updates the schedule of the job.
You should never modify the database manually. Always update the scheduler through its API.
Try sth like this:
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="jobDetails">
<list>
<ref bean="yourJobDetail" />
</list>
</property>
<property name="triggers">
<list>
<ref bean="yourJobTrigger" />
</list>
</property>
<property name="configLocation" value="file:${HOME}/yourProperties.properties" />
<!-- Commented, because don't work with autocommit = false on spring data source -->
<!-- <property name="dataSource" ref="mainDataSource"/> -->
<property name="transactionManager" ref="mainTransactionManager" />
<property name="autoStartup" value="true" />
<property name="applicationContextSchedulerContextKey" value="applicationContext" />
<property name="jobFactory">
<bean class="FactoryForJobWithInjectionOfSpringBbean" />
</property>
<!-- Will update database cron triggers to what is in this jobs file on each deploy. Replaces all previous trigger and job data that
was in the database. YMMV -->
<!-- dont work properly with cluster -->
<!-- <property name="overwriteExistingJobs" value="true" /> -->
</bean>
Unfortunately i think that:
<property name="overwriteExistingJobs" value="true" />
dosen't work correctly in cluster mode.
Related
It seems that the built in workflow activities are being executed twice. I am testing the checkout workflow and the DecrementInventoryActivity is removing the quantity from the sku twice.
Is this a known bug or am I doing something wrong?
I created the workflow like so:
<!-- Checkout Workflow Configuration -->
<bean id="blCheckoutWorkflow" class="org.broadleafcommerce.core.workflow.SequenceProcessor">
<property name="processContextFactory">
<bean class="org.broadleafcommerce.core.checkout.service.workflow.CheckoutProcessContextFactory"/>
</property>
<property name="activities">
<list>
<bean p:order="6000" id="blDecrementInventoryActivity" class="org.broadleafcommerce.core.checkout.service.workflow.DecrementInventoryActivity">
<property name="rollbackHandler" ref="blDecrementInventoryRollbackHandler" />
</bean>
<bean p:order="7000" id="blCompleteOrderActivity" class="org.broadleafcommerce.core.checkout.service.workflow.CompleteOrderActivity">
<property name="rollbackHandler" ref="blCompleteOrderRollbackHandler" />
</bean>
<bean p:order="9999999" class="com.mycompany.workflow.checkout.NotifyExternalInventorySystem" />
</list>
</property>
<property name="defaultErrorHandler">
<bean class="org.broadleafcommerce.core.workflow.DefaultErrorHandler">
<property name="unloggedExceptionClasses">
<list>
<value>org.broadleafcommerce.core.inventory.service.InventoryUnavailableException</value>
</list>
</property>
</bean>
</property>
</bean>
Starting with Broadleaf 4.0, the DecrementInventoryActivity was added by default to the blCheckoutWorkflow. See the 3.1.10-4.0.0 migration notes at http://www.broadleafcommerce.com/docs/core/4.0/migration-notes/3.1-to-4.0-migration/3.1.10-to-4.0-migration, in the section "Inventory Management".
This also goes for the defaultErrorHandler, and you can remove the blCompleteOrderActivity (that has always been managed in the framework). Basically, your customized blCheckoutWorkflow bean should change to:
<bean id="blCheckoutWorkflow" class="org.broadleafcommerce.core.workflow.SequenceProcessor">
<property name="activities">
<list>
<bean p:order="9999999" class="com.mycompany.workflow.checkout.NotifyExternalInventorySystem" />
</list>
</property>
</bean>
Starting with Broadleaf 3.0, any modifications to the blCheckoutWorkflow bean undergo the Broadleaf XML merging processing (which merges bean ids like blCheckoutWorkflow's list of activities). In your case, since the DecrementInventoryActivity is already defined in the core framework XML file and your definition of blCheckoutWorkflow merges with it, the final result is 2 instances of the DecrementInventoryActivity.
I asked a similar question, but based on the responses, I did a bad job describing what I am after. I have a spring 4 webapp that loads properties from a properties file. We consume those properties both via the "${proper.name"} expressions in spring, as well as by injecting a properties object into some of our classes.
We want to move most of the properties to a database table and make them reloadable. However, a few need to stay in local properties, potentially overriding the database setting. These should also be loaded dynamically after the app is running.
I know that once a particular bean is injected, it won't get reloaded, that doesn't concern me, it's up to that module to handle that. But I am having trouble getting the behavior I want. In particular, I have implemented an AbstractConfiguration from apache commons configuration to get the dual source and overriding I am after. But while it works for injecting the properties object, expressions loaded with "${prop.name}" don't work at all.
How can I get them to work? Did I override the wrong thing? Is it just some config detail?
<bean id="sysProperties" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="databaseConfigurator" />
<property name="targetMethod" value="getProperties"/>
</bean>
<bean id="databaseConfigurator" class="my.util.config.MyDatabaseConfigurator">
<property name="datasource" ref="dataSource" />
<property name="propertyFile" value="/WEB-INF/my.properties" />
<property name="applicationName" value="ThisApp" />
</bean>
<bean id="dbConfigFactory" class="org.apache.commons.configuration.ConfigurationConverter" factory-method="getProperties">
<constructor-arg ref="databaseConfigurator" />
</bean>
I haven't tested this, but I think it might work.
<bean id="sysProperties" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="databaseConfigurator" />
<property name="targetMethod" value="getProperties"/>
</bean>
<bean id="databaseConfigurator" class="my.util.config.MyDatabaseConfigurator">
<property name="datasource" ref="dataSource" />
<property name="propertyFile" value="/WEB-INF/my.properties" />
<property name="applicationName" value="ThisApp" />
</bean>
<bean name="PropertyPlaceholderConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="properties" ref="CommonsConfigurationFactoryBean"/>
</bean>
<bean name="CommonsConfigurationFactoryBean" class="org.springmodules.commons.configuration.CommonsConfigurationFactoryBean">
<constructor-arg ref="databaseConfigurator"/>
</bean>
I believe need help compiling Heritrix decide rules, although I'm open to other Heritrix suggestions: https://webarchive.jira.com/wiki/display/Heritrix/Configuring+Crawl+Scope+Using+DecideRules
I need to scrape an entire copy of a website (in the crawler-beans.cxml seed list), but not scrape any external (off-site) pages. Any external resources needed to render the current website should be downloaded, however not following any links to off-site pages - only the assets for the current page/domain.
For example, CDN content required for the rendering of a page might be hosted on an external domain (maybe AWS or Cloudflare), so I would need to download that content, as well as following all on-domain links, however not follow any links to pages outside of the scope of the current domain.
You could use 3 decide rules:
The first one accepts all non-html pages, using a ContentTypeNotMatchesRegexDecideRule;
The second one accepts all urls in the current domain.
The third one rejects all pages not in the domain and not directly
reached from the domain (the alsoCheckVia option)
So something like that:
<bean id="scope" class="org.archive.modules.deciderules.DecideRuleSequence">
<property name="rules">
<list>
<!-- Begin by REJECTing all... -->
<bean class="org.archive.modules.deciderules.RejectDecideRule" />
<bean class="org.archive.modules.deciderules.ContentTypeNotMatchesRegexDecideRule">
<property name="decision" value="ACCEPT"/>
<property name="regex" value="(?i)html|wml"/>
</bean>
<bean class="org.archive.modules.deciderules.surt.SurtPrefixedDecideRule">
<property name="decision" value="ACCEPT"/>
<property name="surtsSource">
<bean class="org.archive.spring.ConfigString">
<property name="value">
<value>
http://(org,yoursite,
</value>
</property>
</bean>
</property>
</bean>
<bean class="org.archive.modules.deciderules.surt.NotSurtPrefixedDecideRule">
<property name="decision" value="REJECT"/>
<property name="alsoCheckVia" value="true"/>
<property name="surtsSource">
<bean class="org.archive.spring.ConfigString">
<property name="value">
<value>
http://(org,yoursite,
</value>
</property>
</bean>
</property>
</bean>
</list>
</property>
</bean>
I asked a related question in Crawling rules in heritrix, how to load embedded content? and came up with a solution there. Later I found this post as well. I am submitting my solution here as well:
Note: I know the question is old so it was most likely made for an older heritrix version. I am using 3.4
<bean id="scope" class="org.archive.modules.deciderules.DecideRuleSequence">
<property name="rules">
<list>
<bean class="org.archive.modules.deciderules.AcceptDecideRule" />
<bean class="org.archive.modules.deciderules.NotMatchesListRegexDecideRule">
<property name="decision" value="REJECT"/>
<property name="regexList">
<list>
<value>.*site\.domain/path/.*</value>
</list>
</property>
</bean>
<bean class="org.archive.modules.deciderules.HopsPathMatchesRegexDecideRule">
<property name="decision" value="ACCEPT"/>
<property name="regex" value="(E|X)" />
</bean>
<!-- Below are some of the "standard" rules set up on a fresh job, it behaves the same with and without them when it comes to not loading embedded stuff -->
<bean class="org.archive.modules.deciderules.TooManyHopsDecideRule">
<!-- <property name="maxHops" value="20" /> -->
</bean>
<!-- ...and REJECT those with suspicious repeating path-segments... -->
<bean class="org.archive.modules.deciderules.PathologicalPathDecideRule">
<!-- <property name="maxRepetitions" value="2" /> -->
</bean>
<!-- ...and REJECT those with more than threshold number of path-segments... -->
<bean class="org.archive.modules.deciderules.TooManyPathSegmentsDecideRule">
<!-- <property name="maxPathDepth" value="20" /> -->
</bean>
<!-- ...but always ACCEPT those marked as prerequisitee for another URI... -->
<bean class="org.archive.modules.deciderules.PrerequisiteAcceptDecideRule">
</bean>
<!-- ...but always REJECT those with unsupported URI schemes -->
<bean class="org.archive.modules.deciderules.SchemeNotInSetDecideRule">
</bean>
</list>
</property>
</bean>
Adjust <value>.*site\.domain/path/.*</value> to match you site, and path if any.
You can also adjust <property name="regex" value="(E|X)" /> where E|X can be just E if you just want the known included things in the page, like images, css etc. X is a bit experimental for trying things found in javascript files as well.
I'm creating a Quartz job with the following Spring XML configuration:
<bean class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail">
<bean class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="name" value="The job"/>
<property name="group" value="The group"/>
<property name="jobClass" value="com.example.myapp.MorningJob"/>
<property name="jobDataMap">
<util:map>
<entry key="key1"
value="val1"/>
<entry key="key2"
value="val2"/>
</util:map>
</property>
</bean>
</property>
<property name="cronExpression" value="0 0 6 * * ? *"/>
<property name="misfireInstruction"
value="#{T(org.quartz.CronTrigger).MISFIRE_INSTRUCTION_FIRE_ONCE_NOW}"/>
<property name="timeZone" ref="timezone"/>
</bean>
My job looks like this
#Configurable
#DisallowConcurrentExecution
#PersistJobDataAfterExecution
public class MorningJob implements Job { ... }
However, misfire instruction set has no effect at all. After long app downtime, when the trigger missed multiple times, Quartz tries to start the job many times.
When I'd tried to check context.getTrigger().getMisfireInstruction() from MorningJob.execute() it gives 0, while CronTrigger.MISFIRE_INSTRUCTION_FIRE_ONCE_NOW is 1.
Any ideas why misfire instruction is not set?
The problem was actually in my scheduler configuration. In the declaration of org.springframework.scheduling.quartz.SchedulerFactoryBean I had a property
<property name="overwriteExistingJobs" value="false"/>
Initially, for testing purposes I've configured my trigger to run each few seconds just to see it fires properly. overwriteExistingJobs meant the trigger saved in database wasn't actually updated after change of cron expression and misfireInstruction wasn't actually applied as well. To update the trigger I need either to run scheduler.clear() once or set the mentioned property to true for a while.
I'm sure this should be mentioned more clearly in the documentation as it could be really frustrating to not have a trigger configuration updated after each change. false is the default value of overwriteExistingJobs.
I'm looking the way how to control java properties during installation procedure.
I'm have Spring Data JPA + Hibernate in my application. I've configured jpa properties in bean. And move it to property file.
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceXmlLocation" value="classpath*:META-INF/persistence.xml"></property>
<property name="persistenceUnitName" value="hibernatePersistenceUnit"/>
<property name="dataSource" ref="dataSource"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
</property>
<property name="jpaPropertyMap">
<map>
<entry key="hibernate.hbm2ddl.auto" value="${hibernate.hbm2ddl}"/>
<entry key="hibernate.show_sql" value="${hibernate.show.sql}" />
<entry key="hibernate.dialect" value="${hibernate.dialect}" />
</map>
</property>
</bean>
property file: prop-local-override.properties where local is environment variable.
hibernate.hbm2ddl=update
#hibernate.dialect=org.hibernate.dialect.HSQLDialect
hibernate.show.sql=true
So, everything works fine. But I want to change hibernate.hbm2ddl during installation process. I use rpm package for installation my app. For example, I want to set this property to "create" when app is installing. And return to update after each restart.
Looks like -Dhibernate.hbm2ddl=update works fine. But it's manual work. Does anyone has some idea how to make it automatically, without big tools like Puppet?
Thanks,
Sergii K.
Never tried changing this dynamically before, but you can access that property map from your entity manager:
entityManager.getEntityManagerFactory().getProperties()