Develop Reference

Spring config properties from database and properties

I asked a similar question, but based on the responses, I did a bad job describing what I am after. I have a spring 4 webapp that loads properties from a properties file. We consume those properties both via the "${proper.name"} expressions in spring, as well as by injecting a properties object into some of our classes.
We want to move most of the properties to a database table and make them reloadable. However, a few need to stay in local properties, potentially overriding the database setting. These should also be loaded dynamically after the app is running.
I know that once a particular bean is injected, it won't get reloaded, that doesn't concern me, it's up to that module to handle that. But I am having trouble getting the behavior I want. In particular, I have implemented an AbstractConfiguration from apache commons configuration to get the dual source and overriding I am after. But while it works for injecting the properties object, expressions loaded with "${prop.name}" don't work at all.
How can I get them to work? Did I override the wrong thing? Is it just some config detail?
<bean id="sysProperties" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="databaseConfigurator" />
<property name="targetMethod" value="getProperties"/>
</bean>
<bean id="databaseConfigurator" class="my.util.config.MyDatabaseConfigurator">
<property name="datasource" ref="dataSource" />
<property name="propertyFile" value="/WEB-INF/my.properties" />
<property name="applicationName" value="ThisApp" />
</bean>
<bean id="dbConfigFactory" class="org.apache.commons.configuration.ConfigurationConverter" factory-method="getProperties">
<constructor-arg ref="databaseConfigurator" />
</bean>

I haven't tested this, but I think it might work.
<bean id="sysProperties" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="databaseConfigurator" />
<property name="targetMethod" value="getProperties"/>
</bean>
<bean id="databaseConfigurator" class="my.util.config.MyDatabaseConfigurator">
<property name="datasource" ref="dataSource" />
<property name="propertyFile" value="/WEB-INF/my.properties" />
<property name="applicationName" value="ThisApp" />
</bean>
<bean name="PropertyPlaceholderConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="properties" ref="CommonsConfigurationFactoryBean"/>
</bean>
<bean name="CommonsConfigurationFactoryBean" class="org.springmodules.commons.configuration.CommonsConfigurationFactoryBean">
<constructor-arg ref="databaseConfigurator"/>
</bean>

Override xml-defined spring bean in java-based configuration

I'm extending a complete product called Hippo CMS with my own REST interface. Hippo CMS is using Apache CXF for rest and acquires resources definitions from a spring bean defined somewhere in Hippo CMS sources. This definition look like this:
<bean id="jaxrsRestPlainResourceProviders" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetClass" value="org.apache.commons.collections.ListUtils" />
<property name="targetMethod" value="union" />
<property name="arguments">
<list>
<ref bean="customRestPlainResourceProviders" />
<ref bean="defaultRestPlainResourceProviders" />
</list>
</property>
</bean>
<bean id="defaultRestPlainResourceProviders" class="org.springframework.beans.factory.config.ListFactoryBean">
<property name="sourceList">
<list>
</list>
</property>
</bean>
<!-- Default empty list of custom plain resource providers to be overriden. -->
<bean id="customRestPlainResourceProviders" class="org.springframework.beans.factory.config.ListFactoryBean">
<property name="sourceList">
<list>
</list>
</property>
</bean>
I need to override customRestPlainResourceProviders bean with my own bean. It works fine from XML configuration looking like this:
<bean id="customRestPlainResourceProviders" class="org.springframework.beans.factory.config.ListFactoryBean">
<property name="sourceList">
<list>
<bean class="org.apache.cxf.jaxrs.lifecycle.SingletonResourceProvider">
<constructor-arg>
<bean class="com.xxx.rest.FolderStructureResource"/>
</constructor-arg>
</bean>
</list>
</property>
</bean>
But it doesn't work if I define a bean in Java configuration class (which in the case of other beans works completely fine):
#Bean(name = "customRestPlainResourceProviders")
public ListFactoryBean customRestPlainResourceProviders() {
ListFactoryBean listFactoryBean = new ListFactoryBean();
listFactoryBean.setSourceList(
Lists.newArrayList(
new SingletonResourceProvider(
new FolderStructureResource(repository())
)
)
);
return listFactoryBean;
}
Is there a way to override a bean defined in XML configuration with a bean created in Java configuration class?

What version of spring are you using? I believe this issues is addressed in 4.2.

Heritrix single-site scrape, including required off-site assets

I believe need help compiling Heritrix decide rules, although I'm open to other Heritrix suggestions: https://webarchive.jira.com/wiki/display/Heritrix/Configuring+Crawl+Scope+Using+DecideRules
I need to scrape an entire copy of a website (in the crawler-beans.cxml seed list), but not scrape any external (off-site) pages. Any external resources needed to render the current website should be downloaded, however not following any links to off-site pages - only the assets for the current page/domain.
For example, CDN content required for the rendering of a page might be hosted on an external domain (maybe AWS or Cloudflare), so I would need to download that content, as well as following all on-domain links, however not follow any links to pages outside of the scope of the current domain.

You could use 3 decide rules:
The first one accepts all non-html pages, using a ContentTypeNotMatchesRegexDecideRule;
The second one accepts all urls in the current domain.
The third one rejects all pages not in the domain and not directly
reached from the domain (the alsoCheckVia option)
So something like that:
<bean id="scope" class="org.archive.modules.deciderules.DecideRuleSequence">
<property name="rules">
<list>
<!-- Begin by REJECTing all... -->
<bean class="org.archive.modules.deciderules.RejectDecideRule" />
<bean class="org.archive.modules.deciderules.ContentTypeNotMatchesRegexDecideRule">
<property name="decision" value="ACCEPT"/>
<property name="regex" value="(?i)html|wml"/>
</bean>
<bean class="org.archive.modules.deciderules.surt.SurtPrefixedDecideRule">
<property name="decision" value="ACCEPT"/>
<property name="surtsSource">
<bean class="org.archive.spring.ConfigString">
<property name="value">
<value>
http://(org,yoursite,
</value>
</property>
</bean>
</property>
</bean>
<bean class="org.archive.modules.deciderules.surt.NotSurtPrefixedDecideRule">
<property name="decision" value="REJECT"/>
<property name="alsoCheckVia" value="true"/>
<property name="surtsSource">
<bean class="org.archive.spring.ConfigString">
<property name="value">
<value>
http://(org,yoursite,
</value>
</property>
</bean>
</property>
</bean>
</list>
</property>
</bean>

I asked a related question in Crawling rules in heritrix, how to load embedded content? and came up with a solution there. Later I found this post as well. I am submitting my solution here as well:
Note: I know the question is old so it was most likely made for an older heritrix version. I am using 3.4
<bean id="scope" class="org.archive.modules.deciderules.DecideRuleSequence">
<property name="rules">
<list>
<bean class="org.archive.modules.deciderules.AcceptDecideRule" />
<bean class="org.archive.modules.deciderules.NotMatchesListRegexDecideRule">
<property name="decision" value="REJECT"/>
<property name="regexList">
<list>
<value>.*site\.domain/path/.*</value>
</list>
</property>
</bean>
<bean class="org.archive.modules.deciderules.HopsPathMatchesRegexDecideRule">
<property name="decision" value="ACCEPT"/>
<property name="regex" value="(E|X)" />
</bean>
<!-- Below are some of the "standard" rules set up on a fresh job, it behaves the same with and without them when it comes to not loading embedded stuff -->
<bean class="org.archive.modules.deciderules.TooManyHopsDecideRule">
<!-- <property name="maxHops" value="20" /> -->
</bean>
<!-- ...and REJECT those with suspicious repeating path-segments... -->
<bean class="org.archive.modules.deciderules.PathologicalPathDecideRule">
<!-- <property name="maxRepetitions" value="2" /> -->
</bean>
<!-- ...and REJECT those with more than threshold number of path-segments... -->
<bean class="org.archive.modules.deciderules.TooManyPathSegmentsDecideRule">
<!-- <property name="maxPathDepth" value="20" /> -->
</bean>
<!-- ...but always ACCEPT those marked as prerequisitee for another URI... -->
<bean class="org.archive.modules.deciderules.PrerequisiteAcceptDecideRule">
</bean>
<!-- ...but always REJECT those with unsupported URI schemes -->
<bean class="org.archive.modules.deciderules.SchemeNotInSetDecideRule">
</bean>
</list>
</property>
</bean>
Adjust <value>.*site\.domain/path/.*</value> to match you site, and path if any.
You can also adjust <property name="regex" value="(E|X)" /> where E|X can be just E if you just want the known included things in the page, like images, css etc. X is a bit experimental for trying things found in javascript files as well.

Spring/Hibernate: New entity-classes in dependend project

I have an (eclipse-)project which uses spring and hibernate to map some entities:
<bean id="mySessionFactory"
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="myDataSource" />
<property name="annotatedClasses">
<list>
<value>Book</value>
<value>Language</value>
<value>Person</value>
<!-- ... -->
</list>
</property>
<property name="hibernateProperties">
<!-- ... -->
</property>
</bean>
Now I have an project which depends on this projects and includes the above mentioned bean in its own applicationContext.xml but must add some own entities.
I do not want to add those entity to the bean-definition of the primary project, because they do not belong there.
How can I extend the bean mySessionFactory with some new annotatedClasses to be used only in the dependend project?

Try using...
<property name="packagesToScan">
<list>
<value>com.acme.domain1</value>
<value>com.acme.domain2</value>
</list>
<property>
This is very like Spring's component-scan feature
http://forum.springsource.org/showthread.php?134056-packagesToScan-in-LocalContainerEntityManagerFactoryBea-doesn-t-work-if

Upgrading to springframework.scheduling.concurrent?

As of Spring 3.0 the ScheduledTimerTask is deprecated and I can't understand how to upgrade to org.springframework.scheduling.concurrent.
<bean id="timerFactoryBean" class="org.springframework.scheduling.timer.TimerFactoryBean">
<property name="scheduledTimerTasks">
<list>
<ref bean="onlineTimeSchedule" />
</list>
</property>
</bean>
<bean id="onlineTimeSchedule" class="org.springframework.scheduling.timer.ScheduledTimerTask">
<property name="timerTask" class="com.example.OnlineTimerTask" />
</property>
<property name="period" value="60000" />
<property name="delay" value="1000" />
</bean>
Where the OnlineTimerTask extends java.util.TimerTask. It's simple task which publishes a message to publisher every minute. I checked the documentation, but nothing.. I can't understand which way to use from the concurrent package and which suits the best.
Also I want to turn this xml into #Bean in Java.
EDIT: So I tried to implement the xml with #Bean and #Configuration instead and here is what I got.
#Configuration
public class ContextConfiguration {
#Bean
public ScheduledExecutorFactoryBean scheduledExecutorFactoryBean() {
ScheduledExecutorFactoryBean scheduledFactoryBean = new ScheduledExecutorFactoryBean();
scheduledFactoryBean.setScheduledExecutorTasks(new ScheduledExecutorTask[] {onlineTimeSchedule()});
return scheduledFactoryBean;
}
#Bean
public ScheduledExecutorTask onlineTimeSchedule() {
ScheduledExecutorTask scheduledTask = new ScheduledExecutorTask();
scheduledTask.setDelay(1000);
scheduledTask.setPeriod(60000);
scheduledTask.setRunnable(new OnlineTimerTask());
return scheduledTask;
}
}
Will the code above be correct replacement for xml? Will in my case the setScheduledExecutorTasks work properly? I mean will the referencing to the same bean instance, if onlineTimeSchedule() is called more than once, will work here?
scheduledFactoryBean.setScheduledExecutorTasks(new ScheduledExecutorTask[] {onlineTimeSchedule()});

Use org.springframework.scheduling.concurrent.ScheduledExecutorFactoryBean in place of org.springframework.scheduling.timer.TimerFactoryBean and use org.springframework.scheduling.concurrent.ScheduledExecutorTask in place of org.springframework.scheduling.timer.ScheduledTimerTask. You will need to adjust the property names and values as needed but, that should be pretty self evident.
Optionally, you could refactor your com.example.OnlineTimerTask to not extend java.util.TimeTask as the ScheduledTimerTask only requires a runnable.

Spring 4 configuration - Below configuration working after spring migration from 3.2.x to 4.6.x
<bean id="schedulerTask"
class="org.springframework.scheduling.support.MethodInvokingRunnable">
<property name="targetObject" ref="springJmsListnerContainer" />
<property name="targetMethod" value="execute" />
</bean>
<bean id="timerTask" class="org.springframework.scheduling.concurrent.ScheduledExecutorTask">
<property name="runnable" ref="schedulerTask" />
<property name="delay" value="100" />
<property name="period" value="60000" />
</bean>
<bean class="org.springframework.scheduling.concurrent.ScheduledExecutorFactoryBean">
<property name="scheduledExecutorTasks">
<list>
<ref bean="timerTask" />
</list>
</property>
</bean>

The answer is - add one "runnable" field
<bean id="scheduledExecutorTask"
class="org.springframework.scheduling.concurrent.ScheduledExecutorTask">
<!-- wait 10 milli seconds before starting repeated execution -->
<property name="delay">
<value>10</value>
</property>
<!-- run every 1 second -->
<property name="period">
<value>1000</value>
</property>
<property name="runnable">
<ref bean="checkInvokingTask"/>
</property>
</bean>